I am happy to announce the launch of one of my first independent websites: paster
Paster is a very simple app for collaborative development, where you can paste your code and share it with anyone you would like. It supports syntax highlighting of over 40 languages, and I plan to add more as I can. It was written in PHP as you might be able to figure out after using it and was written in about a day and a half. So, go forth and paste!
Personally, I’ve always been interested in distributed computing. It’s one subject I’ve done a lot of reading on. After seeing a bunch of posts on various Google blogs, such as this one, I’ve decided I want to work on making my own little cloud. I know I don’t have 500,000+ computers to work with like Google, but I do have 5 (or so…I think I can scrap up a few more out of my extra parts).
I’m starting to plan out everything but this would consist of a “manager” computer, and then slave computers to do the actual processing. I’m going to be building a web server specifically for this purpose and using some undecided DBMS (I’m thinking about SQLite).
At least to start with, each request will only be handled by one computer, although I do plan to eventually spread the processing of one request over multiple computers. I’d like to start with smaller goals. The database however will store information in sort of a “striping” manner, where each database computer will have the same structure, data will be separated over multiple computers, but not mirrored. There will be an application inside the manager to layer on top of SQLite (or whatever I decide on) to manage the returned results from all the combined computers.
Again, I’m going to start small and start with a basic web server that serves basic HTML, and then move up and support a dynamic language (Most likely Python or PHP, not sure which yet though, or hell, maybe both). After I get all this working I will make it distributed between computers and reporting to its “manager”.
The only problem with this plan is that it seems like there would be a bottle neck on the “manager” computer. I guess eventually I’ll have to support several of those, and use a load balancer to switch traffic between them.
I haven’t decided what language I’m going to use yet, but I’m leaning towards Python. One might argue that something of this magnitude should be done in a language like C or C++, but the problem is I think those languages would make something like this too complicated, and would require far more code to do less. Personally I think python would be great for the job, however I’m still deciding because I might also want to use C#.
Will definitely keep updating as I work through the project. Feel free to make suggestions on how I should go about things.
I was thinking the other day. How would I do something on the internet and make my activity completely untraceable. The simplest form of the answer was to scatter where the packets come from.
So the idea of the packet scatter network was born. Anyone could sign up and host a server and there would be a large list of servers that the program would use. what the client would do is grab all outgoing packets and scatter them amongst the servers in the list, attaching a forwarding address. when each packet gets to it a server it is then sent on to its final destination, but anyone trying to trace the packets would find anywhere from 2 to thousands of sources.
I have absolutely no idea if this is even vaugley possible. But its and idea. It seems like it would only work for UDP because those are not ordered and such like TCP. But I’m not sure. I’ll ponder on it more I guess. Just an idea.
Ah, the library grows once again. I either need less books, or a bigger shelf. Its even starting to bow from the weight. And its out of room. Damn….
Web Requests are amazingly easy in C# – All you need is this:
WebRequest myWebRequest = WebRequest.Create("http://www.google.com");
WebResponse myWebResponse = myWebRequest.GetResponse();
Stream ReceiveStream = myWebResponse.GetResponseStream();
StreamReader readStream = new StreamReader(ReceiveStream);
String Response = readStream.ReadToEnd();
Returns everything from wherever you request. So simple. Need I explain more?
int main()
{
double x = 0.0, sum = 0.0;
int count = 0;
printf( "\t--- Calculate Averages ---\n" );
printf("\nEnter some numbers:\n"
"(Type a letter to end you input)\n" );
while ( scanf("%lf", &x ) == 1)
{
sum += x;
count++;
}
if (count == 0)
{
printf("No Input Data!\n");
}
else
printf("The Average of your numbers is %.2f\n", sum/count);
return 0;
}
This is a basic loop that lets you put in as many numbers as you want to, and then a letter to get the average of all said numbers. Very simple, but shows good use of basic constructs in C
Here is just a basic example for how to use threads.
using System;
using System.Threading; // <-- if you dont want to include the entire Threading namespace, just prefix your threading object with System.Threading.[object]
namespace ThreadingExample
{
class ThreadingExample
{
public void Main(String[] args)
{
Console.WriteLine("Starting...");
Thread myThread = new Thread(myThreadedMethod); // <-- remember, don't put the () after the method you want running on that thread!
myThread.Start();
while (true)
{
// do stuff here
}
}
public void myThreadedMethod()
{
while(true)
{
//do other stuff here too
}
}
}
}
Now, both of those while loops will be running forever simultaneously. Like i said, this is just a simple example. Threading can get extremely complicated. If you want to know about threading with a method that needs parameters, you should go look through the MSDN library on it. It has an -ok- example. and some explanation I believe.
If I could find my camera, I would have some funny pictures to go along with this. but I don’t.
You know you’re a geek when:
Well, I could go on probably but i think I’ll stop here. 95% of these apply to me. I guess I’m a geek. Who else is?
As of yesterday, another programmer has joined the Fizzure project. This makes me very happy but at the same time, a little worried.
I am very happy about having another programmer to help me on this project for a bunch of reasons. It gives me someone to help me out when I am stuck on a bug. Someone to bounce my ideas off of and see what they think. Someone to do another portion of the programming. It is a huge help in a lot of ways.
However, at the same time, I am a little worried. I’ve never really worked with someone in a team on a large project before. I have to figure out the best way to keep code and things organized well between both people.
I am thinking of using Mercurial to help manage the code. Luckily, the project is easily split into 2 quite separate parts – the Server and the Client. I will be working on the server, and the other programmer will be working on the client. Both of us wanted to almost start over from a code perspective because what I had written was pretty messy. I think it was quite a good idea. I’ve looked at his basic re-design of the client, and it looks pretty good. We are hoping to implement a skinning system for the client so people can make the program look according to their own likes.
I was not happy with the way the server was organized before so I decided to completely redo the structure. The server is going to be a distributed system so that it will be handled by several computers instead of just overloading one. I’m trying to make it an easily scalable and efficient distributed system. We’ll see how it turns out.
I haven’t spent too much time working with the new addition to the team, but the time we have spent has been great. We discuss the best way to do the current thing that we are working on instead of just going right to the first idea that pops into our heads. This, in the end, I think will cause a much more efficient and well organized program. We’ll see how everything turns out.
Alright, so as I have stated before, Distributed Computing (DC) is one of my biggest areas of interest. I am starting to explore and experiment with it now that I have started to get the hang of network programming.
Unfortunately, I can’t think of any tasks that are worthy of being distributed that I can experiment with! Boo~
I mean I have one idea. I program that finds prime numbers and looks for patterns in distance between each prime number and the prime number before it, and maybe even drawing a 3D graph of it if i can figure out how to do that. Prolly just a 2D graph…
But anyways, I just cant think of things that would really require intense processing power. Technically it doesn’t have to require THAT much, just enough that it would be faster to do on a few computers instead of one. Please leave me any ideas you might have!
