I am happy to announce the launch of one of my first independent websites: paster
Paster is a very simple app for collaborative development, where you can paste your code and share it with anyone you would like. It supports syntax highlighting of over 40 languages, and I plan to add more as I can. It was written in PHP as you might be able to figure out after using it and was written in about a day and a half. So, go forth and paste!
I got kind of bored today, and wrote a pretty simple web crawler with python and it turned out to be less than 50 lines. It doesn’t store output, I’ll leave that up to anyone who wants to use the code, because, well, theres just too many ways to choose from. Right now you pass it a starting link as a parameter and it will crawl forever untill it runs out of links. But that is not a likely condition. So here ya go. Have fun. Feel free to ask questions
import sys
import re
import urllib2
import urlparse
tocrawl = set([sys.argv[1]])
crawled = set([])
keywordregex = re.compile('<meta\sname=["\']keywords["\']\scontent=["\'](.*?)["\']\s/>')
linkregex = re.compile('<a\s*href=[\'|"](.*?)[\'"].*?>')
while 1:
try:
crawling = tocrawl.pop()
print crawling
except KeyError:
raise StopIteration
url = urlparse.urlparse(crawling)
try:
response = urllib2.urlopen(crawling)
except:
continue
msg = response.read()
startPos = msg.find('<title>')
if startPos != -1:
endPos = msg.find('</title>', startPos+7)
if endPos != -1:
title = msg[startPos+7:endPos]
print title
keywordlist = keywordregex.findall(msg)
if len(keywordlist) > 0:
keywordlist = keywordlist[0]
keywordlist = keywordlist.split(", ")
print keywordlist
links = linkregex.findall(msg)
crawled.add(crawling)
for link in (links.pop(0) for _ in xrange(len(links))):
if link.startswith('/'):
link = 'http://' + url[1] + link
elif link.startswith('#'):
link = 'http://' + url[1] + url[2] + link
elif not link.startswith('http'):
link = 'http://' + url[1] + '/' + link
if link not in crawled:
tocrawl.add(link)
ok so this is a little code snippet. Lets say I have 10 text files in a folder and I want to add my ASCII signature to the end of all of them. I can put them in an array, and use a Foreach loop to do it to all of them at once. Heres how:
public static void write(String[] files)
{
foreach (String s in files)
{
FileStream fs = new FileStream(s, FileMode.Open, FileAccess.Write);
String[] line = {" ..jjLLtt \r\n",
" GGffiiGGLL \r\n",
" iiDD ,,EE,, \r\n",
" ttGG EEii \r\n",
" ;;EE.. ;;EEtt \r\n",
" LLEEffGGEEtt \r\n",
" ;;;;..KK;; \r\n",
" ;;,, ;;KK.. ;;;; \r\n",
" ..KKDD ii,,;;EEtt KKEE \r\n",
" ..LLtt LLDDDDtt LLjj \r\n"};
for (int i = 0; i < line.Length; i++)
{
Byte[] buffer = System.Text.Encoding.ASCII.GetBytes(line[i]);
try
{
fs.Write(buffer, 0, line[i].Length);
}
catch (Exception e)
{
// Do Nothing
}
}
fs.Close();
}
Now unfortunately because it gets rid of all the tabs, you can see what it actually looks like, but you can see it here
The parameter for the write method is an array of strings. The strings in this case happen to be file names. So the method will take all the filenames into it, and for each one, open them in a filestream, and using the filestream for each one, will write the ASCII signature in the file, and then close the filestream.
Finally, something to make it easier to develop programs for all platforms. I have been so sick of being stuck using the .NET framework, only running on Windows. But Mono has come to the rescue.
Mono provides the necessary software to develop and run .NET client and server applications on Linux, Solaris, Mac OS X, Windows, and Unix. Sponsored by Novell, the Mono open source project has an active and enthusiastic contributing community and is positioned to become the leading choice for development of Linux applications.
As you can see, for any programmer using a .NET language, this is an amazing tool. This allows us to not have to think so much about which programming language is the best to use. In my case, that is especially important because it is generally more like “what programming language should I learn to enable me to make this?”
int main()
{
double x = 0.0, sum = 0.0;
int count = 0;
printf( "\t--- Calculate Averages ---\n" );
printf("\nEnter some numbers:\n"
"(Type a letter to end you input)\n" );
while ( scanf("%lf", &x ) == 1)
{
sum += x;
count++;
}
if (count == 0)
{
printf("No Input Data!\n");
}
else
printf("The Average of your numbers is %.2f\n", sum/count);
return 0;
}
This is a basic loop that lets you put in as many numbers as you want to, and then a letter to get the average of all said numbers. Very simple, but shows good use of basic constructs in C
Here is just a basic example for how to use threads.
using System;
using System.Threading; // <-- if you dont want to include the entire Threading namespace, just prefix your threading object with System.Threading.[object]
namespace ThreadingExample
{
class ThreadingExample
{
public void Main(String[] args)
{
Console.WriteLine("Starting...");
Thread myThread = new Thread(myThreadedMethod); // <-- remember, don't put the () after the method you want running on that thread!
myThread.Start();
while (true)
{
// do stuff here
}
}
public void myThreadedMethod()
{
while(true)
{
//do other stuff here too
}
}
}
}
Now, both of those while loops will be running forever simultaneously. Like i said, this is just a simple example. Threading can get extremely complicated. If you want to know about threading with a method that needs parameters, you should go look through the MSDN library on it. It has an -ok- example. and some explanation I believe.
Heres a little bit of code for making random strings. I’m sure someone will find it useful. I used to use it a lot in web development. Not sure how useful it would be for software development but use it if you’d like to 
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace Ruf
{
public static class StringGnerator
{
public static Char[] charArray = { ‘-’, ‘a’, ‘b’, ‘c’, ‘d’, ‘e’, ‘f’, ‘g’, ‘h’, ‘i’, ‘j’, ‘k’, ‘l’, ‘m’, ‘n’, ‘p’, ‘q’, ‘r’, ’s’, ‘t’, ‘u’, ‘v’, ‘w’, ‘x’, ‘y’, ‘z’, ‘1′, ‘2′, ‘3′, ‘4′, ‘5′, ‘6′, ‘7′, ‘8′, ‘9′, ‘0′, ‘_’ };
public static string RandomStr()
{
String returns = “”;
Random r = new Random();
for (int i = 0; i < 11; i++)
{
returns += “” + charArray[(int)(r.NextDouble() + 1.0 * 38.0)];
}
return returns;
}
public static string RandomStr(int Length)
{
String returns = “”;
Random r = new Random();
for (int i = 0; i < Length; i++)
{
returns += “” + charArray[(int)(r.NextDouble() + 1.0 * 38.0)];
}
return returns;
}
public static string RandomStr(int Seed, int Length)
{
String returns = “”;
Random r = new Random(Seed);
for (int i = 0; i < Length; i++)
{
returns += “” + charArray[(int)(r.NextDouble() + 1.0 * 38.0)];
}
return returns;
}
}
}
I’ve been working on Fizzure A LOT recently. I made a FizzSrvLight that is not a distributed system like the regular one, which therefore allowed me to write one effectively in about 3 hours. On the way I decided to make a few of my own methods and then realized, hey these can be used in other projects too!
So I made a class library (.dll – Dynamically Linked Library ) with a few methods that have to do with TCP Data transmition. The most important of which is the Send method that I made. Now this is really only useful for the client. Anyway, heres the snippet:
public static void Send(TcpClient Client, String Command)
{
Console.WriteLine("Opening Server Stream");
NetworkStream n = Client.GetStream();
String send = Command;
String receive = null;
byte[] msg = System.Text.Encoding.ASCII.GetBytes(send);
n.Write(msg, 0, msg.Length);
Console.WriteLine("SENT: {0}", send);
}
this method is meant for console programs, but if you are using a GUI all you really need to do is delete the Console.WriteLines()’s in there and replace it with wherever you want the output.
Hope this is helpful to everyone!
Ok, this is just a quick snippet of code I wrote to get a working server up. Obviously theres more commands I could put in there in plenty of different ways, but I really just wanted to keep things simple for now. This took me about 2 hours.
This snippet is the main body of code that controls everything. If you go through it and read you’ll see that I made a struct to hold the information on files named File, in the namespace Structure. So you would access it by saying in this [MainNamespace].Structure.File; or you can just use Structure.File. I’ll paste the code for the struct at the end.
I didn’t leave too many comments because I used a lot of Writelines to tell me what it was doing, and for debugging purposes. Those kind of tell you what things do what.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
using System.Text;
namespace FizzSrvLight
{
class Program
{
static void Main(string[] args)
{
Console.WriteLine(“FizzSrvLight :: Non-Distributed Fizzure Serving Capabilities”);
System.Threading.Thread.Sleep(1000);
Console.Write(“Loading…”);
Console.WriteLine(“!”);
Console.WriteLine(“Initiating Server Variables…”);
System.Net.IPAddress localaddr = System.Net.IPAddress.Parse(“127.0.0.1″);
Console.WriteLine(“Constructing Server Objects…”);
System.Net.Sockets.TcpListener MainServer = new System.Net.Sockets.TcpListener(localaddr, 9000);
Console.WriteLine(“Starting Server…”);
MainServer.Start();
Byte[] bytes = new Byte[1024];
String data = null;
String send = null;
while (true)
{
Console.WriteLine(“Waiting for connection…”);
// Accept Requests
System.Net.Sockets.TcpClient client = MainServer.AcceptTcpClient();
Console.WriteLine(“Client Connected!”);
// Clear Buffers
data = null;
send = null;
// Get Stream Object for reading and writing
System.Net.Sockets.NetworkStream stream = client.GetStream();
int i;
// Initialize File Holder
System.Collections.ArrayList CurrentFiles = new System.Collections.ArrayList();
// Loop to recieve all data sent from client
while ((i = stream.Read(bytes, 0, bytes.Length)) != 0)
{
// Clear buffers again
data = null;
send = null;
string message = “OK”;
// Get data as string
data = System.Text.Encoding.ASCII.GetString(bytes, 0, i);
Console.WriteLine(“FIZZ_RCV: {0}”, data);
String[] command = data.Split(‘ ‘);
// Insert Possible Commands Here
if (command[0] == “FIZZ_ADDFILE”)
{
FizzSrvLight.Structure.File file = new FizzSrvLight.Structure.File(command[1], command[2], command[3], command[4], command[5], command[6]);
CurrentFiles.Add(file);
}
else if (command[0] == “FIZZ_RMVFILE”)
{
FizzSrvLight.Structure.File file = new FizzSrvLight.Structure.File(command[1], command[2], command[3], command[4], command[5], command[6]);
CurrentFiles.Remove(file);
}
else if (command[0] == “FIZZ_AUTH”)
{
string username = command[1];
string password = command[2];
}
else
{
Console.WriteLine(“FIZZ_INVALID_INPUT”);
Console.WriteLine(“Error Handled”);
message = “ERROR”;
}
send = message;
byte[] msg = System.Text.Encoding.ASCII.GetBytes(send);
// Send back an OK response;
stream.Write(msg, 0, msg.Length);
Console.WriteLine(“FIZZ_SND: ” + message);
}
System.Threading.Thread.Sleep(1000);
}
}
}
}
Now, time for the struct.
namespace FizzSrvLight
{
namespace Structure
{
public struct File
{
public string FileName;
public string FilePath;
public string FileType;
public string SharedBy;
public string IPAddress;
public string Blacklist;
public File(string name, string path, string type, string user, string ipaddr, string blacklisted)
{
FileName = name;
FilePath = path;
FileType = type;
SharedBy = user;
IPAddress = ipaddr;
Blacklist = blacklisted;
}
}
}
}
Well, there you have it. A very simple TcpListener Serve. Obviously theres better ways to do it but this is pretty simple, straight forward, and just all around easy. Please leave comments if you find bugs in it or see errors or even if you just don’t understand what some of it does.
Everyone has used one. Mostly because everyone has used Internet Explorer. But everyone has gotten one of those programs that is just slow and a total resource hog. We all know and hate them. You feel even worse, though, when you realize you have created one yourself. I was pretty happy when I posted last because I had gotten a connection between the server and the client and everything was working pretty well.
Unfortunately, now I realize that, even though it does not use a whole lot of memory, the server uses 99% of my CPU. And I have a decent CPU as well. I mean, I know why it does it but i guess i have to figure out a way around it. Heres the code snippet that I’m pretty sure causes the problems:
while (true) {
if (server.Pending()) {
server.AcceptSocket();
Console.Out.WriteLine("Connection Accepted");
}
}
As you can see, the code just creates an infinite loop that checks for connections. Unfortunately this causes it to check about 2 billion times a second. which obviously…would use all of your processors resources. I’m trying to use a timer that checks every second, but I’m having trouble implementing it in a console application. Maybe I will change the server to a GUI application. Might be easier to do a lot of things. I guess I’ll have to ponder it for a while
EDIT: The problem has been fixed! Hooray! With a little bit of help from my good friend Windy, Just putting System.Threading.Thread.Sleep(1000); at the end brought the CPU usage down to 00% :] And it even brought down the memory usage a little! Hooray!
