First a little background to help explain some of the terms, etc. "Python" is a language, similar to how "Java" is a language; unlike Java wherein the language is also relatively synonymous with the actual implementation of that language, Python has multiple implementations. If you've run python(1) from the command line, you're most likely running the CPython implementation of the Python language, in effect, Python implemented in C. Other implementations of Python exist, like Jython (implemented on top of the Java virtual machine), PyPy (Python implemented in Python), and IronPython (Python implemented on top of the .NET CLR).

I was talking with some of the guys from the #mono channel on GIMPNet about IronPython versus CPython as far as performance is concerned and I decided that I would refine my testing (using pybench) for more similar versions of the respective implementations, in as controlled of an environment as possible.

I ran pybench.py on a "quiet" (i.e. not-busy) machine sitting in a remote datacenter not too far from Novell, the machine is a Pentium III (i386) based machine running openSUSE 10.3. Since IronPython reports it's "implementation version" as Python 2.4.0, I decided to build and run CPython 2.4 against it. IronPython is running on top of the recently released Mono 1.2.6 which I also built from source (I got IronPython from the IPCE package in YaST however). pybench reported the various implementation details for both as such:

CPython

       Implementation: 2.4.4


       Executable:     /home/tyler/basket/bin/python


       Version:        2.4.4


       Compiler:       GCC 4.2.1 (SUSE Linux)


       Bits:           32bit


       Build:          Dec 18 2007 23:00:48 (#1)


       Unicode:        UCS2

IronPython

       Implementation: 2.4.0


       Executable:     /usr/lib/IPCE/ipy.exe


       Version:        2.4.0


       Compiler:       .NET 2.0.50727.42


       Bits:           32bit


       Build:           (#)


       Unicode:        UCS2

IronPython did alright, but it got pretty thrashed on a lot of the benchmarks. Unfortunately it's hard to tell whether it's Mono getting beaten up, or whether it's IronPython itself that's losing the battle here, running similar tests on the .NET 2.0 CLR would be beneficial but not something I am curious enough to boot a Windows virtual machine for. Regardless, here are the results, I've highlighed the rows where IronPython performs better than CPython.

Test	Minimum Run-time			Average Run-time
	CPython	IronPython	Diff	CPython	IronPython	Diff
BuiltinFunctionCalls:	448ms	357ms	+25.4%	450ms	405ms	+11.0%
BuiltinMethodLookup:	530ms	1329ms	-60.1%	536ms	1390ms	-61.4%
CompareFloats:	380ms	129ms	+194.3%	381ms	132ms	+187.7%
CompareFloatsIntegers:	377ms	93ms	+306.1%	378ms	97ms	+291.2%
CompareIntegers:	436ms	160ms	+172.5%	437ms	161ms	+170.6%
CompareInternedStrings:	425ms	443ms	-4.1%	426ms	445ms	-4.3%
CompareLongs:	360ms	292ms	+23.3%	361ms	293ms	+23.0%
CompareStrings:	423ms	330ms	+28.0%	423ms	337ms	+25.6%
CompareUnicode:	377ms	243ms	+54.7%	377ms	245ms	+54.2%
ConcatStrings:	726ms	9452ms	-92.3%	823ms	10071ms	-91.8%
ConcatUnicode:	711ms	5687ms	-87.5%	756ms	6039ms	-87.5%
CreateInstances:	508ms	761ms	-33.2%	518ms	815ms	-36.4%
CreateNewInstances:	451ms	3475ms	-87.0%	458ms	3581ms	-87.2%
CreateStringsWithConcat:	473ms	2650ms	-82.1%	475ms	2833ms	-83.2%
CreateUnicodeWithConcat:	482ms	1008ms	-52.1%	508ms	1092ms	-53.4%
DictCreation:	405ms	2944ms	-86.2%	407ms	3057ms	-86.7%
DictWithFloatKeys:	552ms	934ms	-40.9%	553ms	944ms	-41.5%
DictWithIntegerKeys:	423ms	1118ms	-62.2%	426ms	1137ms	-62.5%
DictWithStringKeys:	413ms	1186ms	-65.1%	414ms	1317ms	-68.6%
ForLoops:	412ms	189ms	+118.5%	413ms	217ms	+90.7%
IfThenElse:	372ms	128ms	+191.8%	374ms	141ms	+165.8%
ListSlicing:	311ms	4033ms	-92.3%	315ms	4230ms	-92.6%
NestedForLoops:	488ms	349ms	+39.7%	489ms	382ms	+28.1%
NormalClassAttribute:	430ms	1080ms	-60.2%	432ms	1104ms	-60.9%
NormalInstanceAttribute:	401ms	427ms	-6.1%	404ms	442ms	-8.7%
PythonFunctionCalls:	393ms	302ms	+30.1%	402ms	352ms	+14.3%
PythonMethodCalls:	478ms	643ms	-25.7%	536ms	673ms	-20.3%
Recursion:	547ms	158ms	+245.9%	659ms	159ms	+313.6%
SecondImport:	476ms	1383ms	-65.6%	481ms	1432ms	-66.4%
SecondPackageImport:	501ms	1425ms	-64.8%	503ms	1482ms	-66.1%
SecondSubmoduleImport:	589ms	1916ms	-69.3%	592ms	1990ms	-70.2%
SimpleComplexArithmetic:	475ms	729ms	-34.9%	476ms	758ms	-37.3%
SimpleDictManipulation:	424ms	1009ms	-58.0%	427ms	1020ms	-58.2%
SimpleFloatArithmetic:	416ms	455ms	-8.7%	422ms	480ms	-12.0%
SimpleIntFloatArithmetic:	345ms	161ms	+113.8%	346ms	162ms	+112.9%
SimpleIntegerArithmetic:	345ms	161ms	+114.7%	345ms	161ms	+113.9%
SimpleListManipulation:	346ms	497ms	-30.4%	350ms	501ms	-30.1%
SimpleLongArithmetic:	402ms	1120ms	-64.1%	403ms	1130ms	-64.3%
SmallLists:	417ms	1693ms	-75.4%	421ms	1717ms	-75.5%
SmallTuples:	450ms	3839ms	-88.3%	453ms	3915ms	-88.4%
SpecialClassAttribute:	431ms	1104ms	-60.9%	432ms	1133ms	-61.8%
SpecialInstanceAttribute:	608ms	423ms	+43.8%	610ms	437ms	+39.5%
StringMappings:	443ms	2255ms	-80.3%	448ms	2311ms	-80.6%
StringPredicates:	503ms	1058ms	-52.5%	504ms	1066ms	-52.7%
StringSlicing:	527ms	2880ms	-81.7%	562ms	3008ms	-81.3%
TryExcept:	418ms	21ms	+1905.2%	418ms	39ms	+985.6%
TryRaiseExcept:	587ms	6670ms	-91.2%	591ms	6733ms	-91.2%
TupleSlicing:	390ms	1817ms	-78.5%	397ms	1863ms	-78.7%
UnicodeMappings:	362ms	1323ms	-72.7%	365ms	1347ms	-72.9%
UnicodePredicates:	438ms	860ms	-49.0%	439ms	912ms	-51.8%
UnicodeProperties:	400ms	0ms	n/a	401ms	0ms	n/a
UnicodeSlicing:	624ms	2491ms	-75.0%	666ms	2638ms	-74.7%

The results are disappointing but not all that surprising, especially with regards to string manipulation. I attempted to run the same pybench.py tool on top of Jython but Jython doesn't appear to support the "platform" module, so I don't have a really good baseline for "managed/virtual machine-based Python implementations" right now. However, given the lack of evidence otherwise, I'll just go ahead and assume IronPython blew the doors off of Jython :). In general though this isn't the be-all end-all benchmark for IronPython, especially on Mono, but it does give a nice hint of where some improvements could be made both in the Mono runtime and IronPython. I'll have to run the benchmarks again with the newer versions of both implementations of Python to see where they're improving or degrading but by all means don't let this deter you from checking out IronPython! I'll be writing up a few code samples over the next couple weeks that I hope will be helpful to those "unenlightened" among us; dynamic languages on the CLR, what has the world come to.

I was talking to Dennis about quitting Palantir and coming to work for my startup which has no funding, and no time, but lots of brilliant ideas, when I realized I don't have a name for the startup yet.

So effectively immediately my I'm naming my startup TY-Combinator and wouldn't you know it! We're currently accepting angel funding, demon funding, and picnic baskets filled with those little sandwiches cut into triangles.

Also effectively immediately, I'm still going to work at Slide.

I've spent the week tweaking and adjusting my lighttpd configuration to where it cooperates better with Mono's FastCGI server, and I finally feel confident enough with the configuration to share.

Around thursday morning or so (maybe it was wednesday) the site was spewing so many 500 errors that somebody who I'm not sure I know where I know them from, emailed me saying "dude, site's broke." After checking the error logs, I found a lot of errors that were all like this:

fcgi-server re-enabled:  0 /tmp/fastcgi-mono-server


backend is overloaded; we'll disable it for 2 seconds and 


send the request to another backend instead: reconnects: 0 load: 130





fcgi-server re-enabled:  0 /tmp/fastcgi-mono-server


backend is overloaded; we'll disable it for 2 seconds and 


send the request to another backend instead: reconnects: 0 load: 130

After diagnosing the problem and kicking the server again, I decided that a couple of tips on the wiki page for Mono's FastCGI & Lighttpd had done me in, the first being about the FastCGI handler's max-procs configuration variable:Fortunately Urlenco.de doesn't really need any session information, so I did what Emeril and Apache admins are both familiar with doing, I kicked it up a notch (to about 10). After kicking the server one more time, this time with "max-procs" > 10 I watched the load on my little 1U server spike up to 30. While every terminal I had became so sluggish I could barely interact with the machine, I managed to open up "top(1)" and see what processses were royally screwing my machine. Turns out it was 10 instances of Mono, all trying to digest an ASP.NET site at once, all competing for the meager resources available. It seems that the Mono FastCGI server will process and compile your entire ASP.NET web application as soon as the FastCGI server is bootstrapped and accepting requests. Fortunately pushing new code to the site gets updated on the next HTTP request, so the number of times you'll have to kick (i.e. restart) the Lighttpd server should be minimal and you won't have to incur the huge performance penalty that often (I've since changed max-procs to 4).

I also went against some of the other advice on the wiki page By effectively passing every single request off to the Mono FastCGI Server you can avoid exposing some internal ASP.NET resources that should be interpreted and not sent over the wire, this seems to be poor practice as far as Lighttpd and FastCGI are concerned. Lighttpd is a very good, high performance HTTP server and should be allowed to do it's job, whereas FastCGI servers merely serve as a means for executing server-side pages, returning markup, etc. To avoid passing every single request off to the FastCGI server, I merely setup the FastCGI handler for .aspx pages and then mapped other ASP.NET extensions to that handler as was appropriate:

fastcgi.map-extensions = (


        ".asmx"   => ".aspx",


        ".ashx"   => ".aspx",


        ".asax"   => ".aspx",


        ".ascx"   => ".aspx",


        ".soap"   => ".aspx",


        ".rem"    => ".aspx",


        ".axd"    => ".aspx",


        ".cs"     => ".aspx",


        ".config" => ".aspx",


        ".dll"    => ".aspx"


)

The base configuration for one of my virtual hosts (Urlenco.de) turned out something like this:

$HTTP["host"] == "urlenco.de" {


 fastcgi.server = (


  ".aspx" => ((


   "socket" => "/tmp/fastcgi-mono-server",


   "bin-path" => "/usr/local/bin/fastcgi-mono-server2",


   "bin-environment" => (


    "MONO_FCGI_APPLICATIONS" => "/:/serv/www/domains/urlenco.de/htdocs",


    "MONO_FCGI_LOGLEVELS" => "Standard", #All", #Debug",


    "MONO_FCGI_LOGFILE" => "/var/log/lighttpd/mono.log",


   ),      


   "max-procs" => 4,


   "check-local" => "disable"


  ))      


 )               


}        

Specifying the "application path" is somewhat of a pain, as now I more or less need a separate FastCGI configuration, which means they'll also need separate FastCGI servers, so another virtual host in the configuration (pineapple.monkeypox.org) has the following setup:

$HTTP["host"] == "pineapple.monkeypox.org" {


 fastcgi.server = ( 


  ".aspx" => ((


   "socket" => "/tmp/fastcgi-mono-server-pineapple",


   "bin-path" => "/usr/local/bin/fastcgi-mono-server2",


   "bin-environment" => (


    "MONO_FCGI_APPLICATIONS" => "/:/serv/www/domains/pineapple.monkeypox.org/htdocs",


    "MONO_FCGI_LOGLEVELS" => "Standard", #All", #Debug",


    "MONO_FCGI_LOGFILE" => "/var/log/lighttpd/mono.log",


   ),  


   "max-procs" => 1,


   "check-local" => "disable"


  ))  


 )   


}

Since the virtual host pineapple.monkeypox.org barely runs any ASP.NET code at all, I decided to only give it one Mono FastCGI process. Also of note is that the "socket" is different from the other FastCGI handler, if you try to use the same socket, the first Mono FastCGI process will take it over and both FastCGI handlers will return the same code, returned from the first handler.

Feel free to bug me with any questions, this is my first foray into using Lighttpd and I'm already pleased as punch with it (compared to Apache) but there are certainly some caveats and bits of black magic involved with Mono, FastCGI and Lighttpd. That said, it still feels less sticky than running Apache 2 and mod_mono (not that they're not great and all). Hopefully web traffic will increase and give me a good test-bed for figuring out "the right stuff" to scale Mono on Lighttpd.


Scary thought isn't it? :)

Earlier this week I noticed that the Facebook home page would not stop loading, in the sense that the entire page would load and render, but one resource would continue to load. As I popped open Safari's Activity Monitor I found that the "one resource" was a server-generated image that was effectively streaming to my browser window, since the server would not stop sending data for the file.

Curious as to what the image was, I downloaded it via Safari, which downloads to the "Downloads" folder which has a convenient "stack" icon in the dock. If you're not familiar with "stacks" in Leopard, they essentially are a nifty way to navigate to folders straight from the dock, and also offer an iconic preview of the most-used/latest item in the folder.

Unknowingly, this image that I had downloaded was completely corrupted, but I had just downloaded it to the "Downloads" folder, and the Dock started to try to render a preview in the Dock of the corrupted image. Doing so set off a looping chain-reaction that was a wonderful sight to see, and ended up in a hard-restart of the machine as I couldn't get control of it. First the Dock crashed, following the Dock, Finder restarted and then Spaces crashed entirely. Sitting looking at a Dock that kept restarting and crashing and Spaces that had completely abandoned 5 other "spaces" full of windows, and an unresponsive Finder I made like a Windows ME user and rebooted my machine.

After the machine started up again I got to the Downloads folder and deleted the image before the cycle could start again and managed to restore the machine to a usable state again.

According to Apple, Mac OS X Version 10.5.1 is a full-fledged release, but it still feels like a release candidate depending on the day of the week, the amount of sunshine outside, or any one of a large number of arbitrary variables.

During the nigh 12 hour break I had between regular work over Thanksgiving, I spent about four hours writing a little utility that I wanted to use instead of TinyURL, and found a fantastic domain name for it too: Urlenco.de. I also wanted to use the opportunity to explore using Npgsql, the .NET connector for PostgreSQL, which was a very pleasant experience after using the MySQL .NET connector (part of the pleasant experience was using PostgreSQL itself, of course). Another new thing to explore was the FastCGI support for Mono/ASP.NET, I'll be sure to jot down my experiences with Mono's FastCGI support in a later post since my brain is too fried to talk about it coherently in detail.

The most important part of the entire project was further refining my rapid-development process for Mono and ASP.NET so I can do quick little projects like this and push them to a live webserver in a matter of hours instead of days (of time I don't have). This mostly consists of boiler-plate project templates for some basic database code, page templates, and a NAnt build script that facilitates the building and testing of the site using xsp2 on localhost. Nothing spectacular, just having a toolkit of necessities to take from one project to the next, especially when time is at such a premium, is a minor but important difference from how I work now as opposed to how I used to work (when I had expendable time).

One of my favorite parts of the entire Urlenco.de project was setting up a Urlenco.de API for both encoding (tiny'ing) and decoding (untiny'ing) URLs to and from Urlenco.de, all in under 10 minutes after a suggestion from my friend Dennis at Palantir. After another suggestion, I also wrote a Urlenco.de stats page using the

I hope everybody enjoyed their stay this past week in Madrid for the Mono Summit 2007.

Unfortunately, it's been too hectic a month to take the week off and go to Madrid, so I'm incredibly jealous of all of you. Grumble.

I really don't have much that I can say about this, I came into the office after leaving my Mac on (as per usual) for about 12 hours and found that I was out of space on my startup disk, out of all available system memory, and things were crashing left and right.

What the fuck right?

Well, after I recovered the system enough to pop open "Activity Monitor" I found the exact culprit.

Sergio, one of our talented web monkeys, sent an email out today that started with "OMFGBBQ!"

As it turns out, Sergio is a much more religious reader of Penny Arcade than the rest of us (a public shaming and revocation of some geek cards is in order) since he was the first to notice this:


Click to view the image fullsize


Hell yes.


As a side note, I have Sergio to thank for the sweet drag-and-drop interface on the Top Friends edit page and now for bringing some Gabe and Tycho love to our attention.

As part of what consists of my day/night/weekend job, developing Facebook applications like Top Friends, I spent a lot of time on Facebook (mostly losing games of Scrabulous to other developers). Since I spend anywhere between 20 and 30 hours a day on Facebook, I see a lot of Facebook's ads, and in particular, Facebook's "Flyers".

The concept at it's most basic level is a novel one, allow posting a flyer, similar to stapling a "Free Couch" flyer to a bulletin board, except on Facebook. In practice however, they suck. They suck bad. Really bad. I have a much higher respect for advertisers that can come up with ads that are either intriguing, or at the very least, not absolutely painful to see.

Over the past week I've been quietly taking screenshots of the absolutely worst Flyers that I've seen that have brought me close to sending a flaming bag of poo down to Palo Alto. Think about the lame kind of spam you get in your inbox, that's about the level that Facebook's Flyers seem to be, except I can't fix it with aggressive spamassasin rules.









Isn't this supposed to be targeted? These all seem to target single, stupid, bi-curious, poor, gullible, and desperate people, and I'm pretty sure I only fall in, at most, three of those categories.

Seriously, what the fuck.

When I originally wrote the Facebook demo application "Why are you awesome?" I wrote it in PHP4 in about 3 hours and hated myself for every one of those miserable 180 minutes. Since then however, I've been slowly and methodically working on a new, JSON-based, Facebook client library (Mono.Facebook.Platform) specifically to bring together some of the aspects of pyfacebook, the PHP client, and the Facebook Toolkit that I like (implementation progress can be found in the NOTES). After getting some of the key Facebook calls implemented to support "Why are you awesome?" I figured I might as well give it a whirl and see if a "real" application would work on top of the library (it does).

Thus far, all that were needed as far as library calls were:

• feed.publishActionOfUser
• notifications.send
• fql.query
• profile.setFBML

A couple of the things I've found thus far in my work have been, that writing a library that you have to use forces you to think about what you add and what you remove a lot more and focus on simplicity and extensibility; secondly, JSON is much faster, meaning I can do things with the Mono.Facebook.Platform library that I couldn't with the XML-based PHP4/5 library. Operations like fetching the user IDs of all 700 friends of mine complete in a timely fashion under the JSON library, whereas they typically timeout with the XML-based libraries.


The Mono.Facebook.Platform library isn't even alpha, it's in negative greek letters right now, there's not enough of the API implemented, and it doesn't handle errors very well at all, so don't use it. When it's finished however, I intend to support over 90% of the Facebook calls, and offer it up as a faster, viable option, for ASP.NET developers on Windows and on Mono.

Of course if you want to check out "Why are you awesome?", head on over to the application page and install it.