Seth Goldstein of SocialMedia (who I think are a competitors of ours) has organized a Facebook Developers Meetup/Miniconference of sorts this Wednesday from 12-5 p.m. at Fenwick & West here in San Francisco, which I will be attending. I'm not completely sure on what I'll be talking about, I'm thinking about discussing some of the issues with scaling on the Facebook platform, or the pitfalls of choosing FBML vs. HTML for developing your Facebook application. There's going to be lots of market-speak I'm assuming ("OMFG VIRALITY!") so I'd like to bring a bit more of the technical side of developing applications that can explode on the Facebook platform.

What I end up talking about is completely open to discussion, so if there's anything you'd like me to talk about, feel free to leave me a comment or drop me a mail at tyler [at] slide [diggitydot] com.



While I am not one to discuss a lot about our "strategy" or "pixie dust" or however we've done what we've done, I do know how a lot of what we done works, and his Maxness permitting, i'll be able to impart some of the <fb:wisdom/> I've accumulated upon my fellow application developers. If interest warrants it, I might start to share some of my sicker (read: awesome) FBML hacks on this blog, but since I'm a big nobody in the world of blogs, it might be a waste of time, so we'll see.

Regardless, if you can make it this wednesday come on by and say "howdy"

I promise I probably don't bite.

---

+1, Ego-boost

Just in case you aren't familiar with Facebook's Platform, or what's exactly going on, I figured I'd cite why Slide, and in turn why I am (sort of) credible to talk on the subject.
Slide has done a good job (in my absurdly biased opinion) on the Facebook Platform with some of the top applications such as:

I thought I'd dork around a little bit with recording videos featuring the Neo1973, running OpenMoko. This video is running an older version of OpenMoko (2007.1) so it isn't current. It's also worth noting, to use the phone, you don't have to do a full shutdown and bootup, you should really only ever need to do either if you're a developer, or when you first receive the phone.
Hello Moko.

I bit the bullet last thursday and started playing with some of the 2007.2 OpenMoko snapshot images, 2007.2 denoting the next iteration of the software. Whereas my previous photos were from 2007.1, the interface was a bit weak, and wasn't much to be proud of. The direction that the interface is now heading in is not only sleek, but much more usable. One of the most important changes, in my opinion, is the addition of acceleration-incluenced scrolling throughout the interface. The new scrolling allows you to do a quick swipe with your finger and have the interface scroll quickly and then slow to a stop, as if your scroll had momentum behind it.

OpenMoko Boot screen

2007.1	2007.2

OpenMoko Desktop

2007.1	2007.2

OpenMoko Applications

2007.1	2007.2

OpenMoko Dialer

2007.1	2007.2

Overall things are progressing quite nicely, but as a developer I feel that I'm teetering between bricking my OpenMoko phone, and making beautiful music with it, either way, I'm along for the ride. The last picture I'll leave you with is one from when my friend whurley and I met up this past week since he was in town representing BMC at Linux World. whurley was one of the organizers of the first iPhoneDevCamp and is a proud owner of an iPhone, regardless, he thought the OpenMoko phone was pretty cool :)





You can find all my OpenMoko photos on Flickr

At times I feel as if I am plugged directly into the internet, almost like an NSA wiretap on AT&T's backbone, silently sniffing along reading packets until something throws up a red flag. This specifically applies to both Python, and .NET/Mono related bloggings, in which a fellow I know, Chris Messina, posted something titled "WordPressMU: Making a smart platform choice" which, not surprisingly, threw up a red flag. Chris and I tend not to see eye-to-eye on a lot of things, most notably, microformats, along with Ruby on Rails, and some of the other "Web 2.0" style technologies/idealogies that Chris has embraced, while I stand back and look on, casually remarking "OMGWTFBBQ" every now and then.

Chris opens the post with the following, in reference to a client of his:Their current website is built in .NET and they’re getting to the point where things are about to start getting set in stone in terms of scaling and overall architecture and it kinda freaked me out that they’d continue down this path using a platform that I think offers little when it comes to organic community-building or much in the way of “doing web things right”.

Disclaimer: I'm not an expert on scaling, I just get yelled at when my code doesn't scale

Chris goes on to mention Ruby on Rails, Django, and WordPressMU, deciding on the third as the best option for building a people-powered Web 2.0 community on. Some of the reasons for this are employment, open source, web standards, community, scalability, politics, and a few others that don't matter. While Ruby on Rails, Django on mod_python, and WordPressMU on PHP are all good platforms to build upon, his complete dismissal of .NET (and in turn Mono) is completely unfounded, and in most cases, blatantly incorrect.

Examples

Looking at some popular sites around the internet, you can get a feel for exactly what it takes to scale:

• MySpace
  • OS: Windows Server
  • Platform: ColdFusion/.NET
  • Database: MS SQL
  Originally, they were able to scale with ColdFusion, and have since switched over to .NET, making MySpace one of the largest sites running .NET with the largest MS-SQL installations on the planet.
• Facebook
  • OS: Linux
  • Platform: PHP
  • Database: MySQL
  Facebook has scaled with a combination of MySQL and PHP, with a good amount of customization of their internal build of PHP, and memcached running to ensure database calls are kept to a minimum.
• Slide
  • OS: Linux
  • Platform: Python
  • Database: MySQL
  
  Okay, I'm throwing us in there for fun, but we've scaled with MySQL and mod_python with a hefty dose of secret sauce :)
• Yahoo!
  • OS: FreeBSD
  • Platform: PHP
  • Database: MySQL
  Yahoo! obviously has to scale to serve a gigantic portion of the internet, and they're running PHP, MySQL with C++ extensions written where they need to be in order to acheive extra speed.
• Wikipedia
  • OS: Linux
  • Platform: PHP
  • Database: MySQL
  Wikpedia.org has scaled with MySQL, PHP along with some Lucene indexing servers and memcached servers running around to improve read times from their database servers


• Microsoft
  • OS: Windows Server
  • Platform: .NET
  • Database: MS SQL
  Microsoft tends to eat their own dog food with most of their web sites and portals, running on .NET with MS-SQL, and according to Quantcast have 3 of the top 5 sites on the internet.
• Apple
  • OS: Linux/Solaris/Mac OS X
  • Platform: WebObjects/PHP
  • Database: MySQL
  Apple runs a mix of WebObjects (Java), PHP, and god knows what else on varying platforms (Solaris, Linux, Mac OS X). I'm not 100% on exactly what's going on inside the web team at 1 Infinite Loop anymore, but they seem to be able to scale already with what they've got.

Counter-points

Employment
One of the points made is that it's easier to find PHP developers, as opposed to Python, or Ruby developers, which probably is true. However, .NET developers are definitely going to be more prominent, but most developers worth your employment, especially at a startup, are going to need to be able to pick up new frameworks and technologies quickly.

Open Source
I will agree that having an open source platform to build on is a good idea, but certainly not a deal breaker for .NET, or whichever platform you choose to use. Starting a community, or a web business in general doesn't matter if you can't get your product out the door as soon as possible. Nobody cares how "open" you are in your development process, if you can't ship.

Web standards
Citing Channel 9 as an example of how (somehow) the .NET platform doesn't adhere to web standards and "open data formats" is one of the most ludicrous arguments I've ever seen. You can generate valid JSON, XML, SOAP, and XHTML from any platform, even mod_perl!

Community
Chris makes the argument that somehow his experience in dealing with the WordPress community extrapolates to developing an actual product, and that if you're going to build a community-oriented site, you better use a platform that your community will approve of! Hint, it doesn't matter. 95% of your users probably won't care what you run, as long as they have the product to use.

Economics
The points about economics are certainly valid, in that it's far easier to find hosts and sysadmins familiar with PHP than with Rails or Django (Python), That doesn't mean that it scales however, once you get past a couple of million users, you need people who know what they're doing, with dedicated hardware to help your web application scale.

Scalibility
Talking about how you feel scaling is absolutely absurd. You will feel pain, that's what happens we you have to scale. Standing back, and looking at the code you've worked insanely hard on, and trying to figure out how to make that faster is painful, regardless of platform. If I've consulted with somebody about how to scale my architecture and they say "well, that doesn't feel right" without citing sources, strategies, or reasons, I'm going to find somebody else, or I'm going to fall on my face when the time comes to scale.

Politics
Just a quote:
However, I think people familiar with modern web design would agree with me that WordPress/PHP, Django or Rails are all superior choices over .NET when it come to the politics of technology development. In terms of openness, being forward-thinking and in terms of community outlook, any of these choices are going to net you a very different kind of response. Being keen to what each choice says about you is key to making a wise decision.
First error, is asking people involved in web design how you want to scale, you should probably ask people involved in systems architecture. Politics don't exist when you need to scale, or ship product, it's that simple. What gets the job done, the fastest, with the greatest net result.

End game

Chris' general ignorance of some of the features of ASP.NET 2.0, and his zealotry when it comes to buzzwords like "community, forward-thinking, people-powered, Web 2.0" and their ilk doesn't surprise me, since he's not a developer. For example, in ASP.NET 2.0 you can have asynchronous pages, just like you can have interlaced GIF images, that progressively load, you can have pages that progressively load, instead of needing the server to fully generate the entire page before it's piped back to the client. Of course, none of this matters since architecture is the biggest hurdle when it comes to scaling, not platform. The more important question to answer before you toss out your existing code base in favor of a more buzzword compliant platform are:

• How can you more efficiently handle database queries?
• Can you cut out unnecessary database queries?
• Can you switch over to a newer version (MySQL 4 and 5 I'm looking at you) of your database to improve performance?
• Will it be effective to add a caching solution like memcached between your web farm and your database servers?
• What can be relegated into progessive page loads either via asynchronous pages in ASP.NET 2.0, or through the use of AJAX back to your web servers to retrieve more data instead of forcing a new page load?
• Is this problem simply caused by not having enough servers?

Citing zero empirical evidence, not counting some useless benchmarks (scaling is far more case-by-case than doing benchmarkable operations), and going with whatever "the cool kids" are using is the quick road to failure. All of the sites I mentioned above have people whose job is to sit around all day and figure out how to squeeze more performance out of their architecture and help the sites grow with their userbase. The trick to optimization is rarely a complete rewrite, or any one trick, it's about finding where the bottlenecks are, and doing whatever possible to minimize those.

As a final note, all the platforms referenced above just spit out pages. That's it. It's how you form the output that determines how "community friendly" or aesthetically pleasing the final product is. It's all fair game between the <html> tags :)

I'm not going to try to explain too much here, but I received my Neo 1973, the OpenMoko-based Smart phone today. While it's a developer preview, it's incredibly exciting. So far I've been able to use my Cingular SIM card, if you use AT&T/Cingular, you can check if yours is supported on the wiki. I can run the general built in suite of applications without too much trouble, I also made a phone call, which worked! Unfortunately however the latest build that I have on my Neo doesn't have sound properly working, which sagacis from the #openmoko channel on Freenode is helping me with currently. I'm a bit over-excited so I'll let the images do the rest of the talking for me.

Meet OpenMoko

Some Assembly Required

Welcome to OpenMoko

Yes, It Does

Browsing around OpenMoko

Dialing with OpenMoko

Sizing up OpenMoko

Running applications "on" your desktop

In general, it's a bit slow, but the developer preview is probably about half as powerful as the planned public-released version. so I'm not exceedingly worried about that. This phone is just all around cool, and will hopefully be a fun device to carry around with me. You can check up on the OpenMoko community by adding Planet OpenMoko to the feed reader of your choice. This is just the beginning.

What goes better with Sunday evening boredom better than some Tcl scripting and package management? I know! Nearly nothing, my sentiments exactly. After stumbling across a guide or two, I decided I should give creating an IronPython MacPort a try.

I'm not sure on the utility of this minor hack, since the Mono Framework installer includes IronPython 1.1a1. I decided to give it a whirl nonetheless and came up with a pretty simple IronPython Portfile, although there is one thing missing, the IronMath and IronPython DLLs are not properly installed in the GAC. I'm a bit hazy on whether I should be installing the DLLs using gacutil(1) or whether I should just copy things over to the $(PREFIX)/lib/mono/gac and be done with it.. Therefore the crux of my "work" was just creating a sane Makefile, since the IronPython makefile that's distributed on CodePlex uses "csc" and has some pretty nasty syntax.

You can check out my Portfile and copy of IronPython 1.1 from anonymous subversion by:

Clicking over here
or
Checking out the code with:
svn co svn://svn.geekisp.com/bleep/trunk/IronPython

I'm debating whether or not I should submit the port to MacPorts for inclusion in their next release, mostly because I'm not completely sure if the Mono port includes IronPython or not. If you've got the Mono port installed, feel free to let me know if it installed IronPython, or if you feel like offering up a Makefile.mono patch that properly installs the assemblies into the GAC. I'm a bit new to packaging up open source projects for Mac OS X properly, so any tips would be helpful.

My RT ticket #3824 was finally filled late last week, so I should be receiving my OpenMoko-based (developer preview) mobile phone sometime in the next couple days.

As far as getting Mono functional on top of the device, it seems that the OpenEmbedded project and their BitBake build tool.

I'm wondering how easy the device will be to develop for in terms of getting Mono, IronPython, etc running on top of it, but if it really is the fully-featured Linux-based phone it's reported to be, this could turn out to be lots of fun (regardless of whether or not I can actually use it as a replacement for my miserable Cingular phone).

No matter how good of a source control system you use, branching can always cause loads of problems, and even painful merge days. At Slide, and other companies I've worked with that use Subversion, branching has been particularly painful because Subversion just doesn't maintain branch history like some commercial source control systems (like Perforce), in fact "branching" is quite literally making a copy of the trunk in Subversion.

Depending on the lifespan of a particular branch, and how often you refresh your branch from the mainline branch, merging back down to the main branch can go from bad, to even worse. Interestingly enough, running an `svn diff $BRANCH $MAIN` will give you the correct changes on the appropriate files that have been modified in the branch, where as running an `svn merge -r $START:$END $BRANCH ./` (whereas ./ is your working copy of main) can break in tremendously painful ways causing mis-merges and unintended rollbacks of previous changesets.

While merging down to the main branch today I decided to mix and match both svn diff and svn merge such that I would only merge changes down to the main branch that had been modified in my development branch, ensuring that nothing was changed on the main/stable branch that wasn't intended. The end result was a Python script that would execute the appropriate commands and merge the files one by one from one branch to the other, allowing the developer or QA engineer to check each file before commencing the merge.

After some serious tweaking and a couple of test branches in Slide's Subversion repository, merge-safe.py was born. The script is a bit hackish right now in that it executes svn(1) instead of using Py-Subversion bindings (which haven't ever worked the way I had hoped). There is definitely room for improvement as well, but the basic flow is there such that merge-safe.py will diff the two branches and aggregate a list of files that have been modified since the branch was originally cut from the main/stable branch, then iterate through the file list and either merge (if the file has been edited) or copy (if the file has been added) to the main branch as is necessary.

The script should always be run from the base directory of your working copy of the mainline branch, so if your main working copy is in /home/tyler/slide/main, this script could be run from that directory like:
python ~/scripts/merge-safe.py -h
In general I think the script is easy to use, but I also wrote it so I'm open to suggestions for improvement or ideas on how to more efficiently merge branches together with Subversion.

You can check the code out with:
svn co svn://svn.geekisp.com/bleep/trunk/svnutils/

Usage
ccnet% python qa/svnutil/merge-safe.py -h
Usage:

The merge-safe script should help you, the lowly startup employee
more effectively merge one branch to another by examining which files have changed, and merge/copy those to the destination branch.

Examples:
Do a dry-run of merging from $SRC to $DST where r1002 is the starting branch of $SRC and r1050 is the last revision to merge from $SRC

%> python some/dir/merge-safe.py -s $SRC -d $DST -r 1002:1050 --dry-run

Do an interactive merge from $SRC to $DST
%> python some/dir/merge-safe.py -s $SRC -d $DST -r 1002:1050 -i

Usage: $prog [options]


Options:
-h, --help show this help message and exit
-s SOURCE, --source=SOURCE
The source branch to merge from
-d DEST, --dest=DEST The destination branch to merge to
-i, --interactive Enable merging interactively on each file
--dry-run Run with --dry-run enabled
-r REVISION, --revision=REVISION
Specify the revisions separated by a colon (i.e. -r
100:104)

I've been wanting to play with IronPython for a very long time, but never really got around to it since most of my days are either consumed with Python or Mono to some capacity, but never both.

Despite my initial instinct to flee in terror after looking over some of the IronPython examples I found on various blogs, I decided it would at the very least be worth an install just to check out the interpreter, and to see how well it performs on top of Mono.
ccnet% ipy
IronPython 1.0.2467 on .NET 2.0.50727.42
Copyright (c) Microsoft Corporation. All rights reserved.
>>>

Scary! But familiar, so I forged ahead undaunted, wanting to start hashing some strings, I figured I'd import the md5 module and get to work.

ccnet% ipy
IronPython 1.0.2467 on .NET 2.0.50727.42
Copyright (c) Microsoft Corporation. All rights reserved.
>>> import md5
Traceback (most recent call last):
File md5, line unknown, in Initialize
File hashlib, line unknown, in Initialize
File hashlib, line unknown, in __get_builtin_constructor
ImportError: No module named _md5
>>>

Alright, so there are still some holes in the IronPython bridge into Python, but this is fine by me, I can call into .NET code! One other thing that seemed to be missing was the 'select' built-in module, which in turn made my little 'telnetlib' based project fall on it's head.

I run an icecast2 server on my workstation, so I can just tune in with my MacBook Pro, and get whatever stream is being served up by the server. The source for the icecast2 server is a script using Liquidsoap which allows for shuffling, bumps, and a couple of other things to make my music-listening experience better. One of the nice things about Liquidsoap is that it has a telnet interface, so I can glean meta-data about what's playing, or control the playlist through the telnet interface. With this telnet interface in tow, I set out to hack up a Windows Forms and IronPython-based controller for already scripted radio station. And thus, my little IronRadio Controller was born:

Unfortunately, I couldn't use Python's native "telnetlib" so I rolled my own IronTelnet class that would permit basic reads and writes to the telnet server, but other than that, the IronRadio Controller is mostly WIndows Forms code and some events cobbled together. The interface is unfortunately poor, as I don't have an Interface Builder for WIndows Forms, let alone IronPython-based Forms (not to mention I could care less about spit-and-polish for anything in X11.app).

The source for the script can be found here, and will require IronPython and Mono to run (or .NET if you're on a Windows machine).

I'm still trying to figure out if I can use IronPython with mod_mono to replace fighting with mod_python, but there are no guarantees as to whether that will work or be worth the trouble.

A few nights ago, not knowing what I should hack on and thinking about Andreia's post about her progress embedding XulRunner in a Mono-based Windows Forms application, I remembered something I had heard about vaguely a few months ago (really vaguely, like remembering to turn the oven off after you've already been on vacation for two weeks). Songbird, a Gecko-based media player being touted as the possible Firefox for media applications, and as their site says:

Songbird™ is a desktop Web player, a digital jukebox and Web browser mash-up. Like Winamp, it supports extensions and skins feathers. Like Firefox®, it is built from Mozilla®, cross-platform and open source.

Feeling curious, bored, and a bit sadistic, I decided I'd give it a whirl on my MacBook Pro. How bad could it be? Besides the fact that they call themselves a "desktop web player" which means god-knows-what, it can't be that terrible, they have an über leet black interface!

It was bad. Very bad.

Songbird is at version 0.2.5 and is marked as a developer preview, and as a developer, I didn't like the preview. I would however, recommend trying out Songbird with a grain of salt in that they have to battle with Mozilla's notoriously bad source code and maintaining cross-platform capability across numerous architectues for Linux, Windows and Mac OS X. Regardless, on to the pictures!

Starting up Songbird for the first time

Any application that first presents the user with useless legalese is not off to a good start, especially when Songbird has such an uphill battle to fight against WinAmp and iTunes. I'll subtract a point for the annoyance, but it's a recoverable error for 0.2.5


Agreeing to use the software

YIKES! One cannot stress this enough with any software that embeds the GPL, or any other software source license with their released distributions (looking at you OpenOffice.org). Are you out of your damn minds?! Minus five points!

Phew, almost lost it there, now that I've agreed to promise not to violate the GPL when I play songs or "play the web on my desktop" as I inferred from their website, I can continue to get to actually use the software.

Setting up Songbird

Before you get to playing music, you definitely need to go through a first run setup, just like iTunes does, to set some initial presets. Nothing out of the ordinary here, but I do appreciate the checkbox to turn off reporting my usage or setup information, +1.

Downloadin' ur extenzionz!

I was a bit taken aback by the fact that I needed to download the extensions that one would assume were already bundled, since they're selected in the installation process. Given the immaturity of the project, these extensions could have drastically changed since they bundled and pushed the release, so no point change, I can understand their decision even if I don't agree with it.

The images get a bit bigger from here on out, so they'll be linked and pop up in an inline image window thanks to a derivative of Lightbox.

Importing iTunes Library
Fortunately they prefill the textbox with the full path to my iTunes library, otherwise I might not be able to find it myself, assuming I'm a normal user. No points awarded for common sense :)

Importing iTunes Library, seriously
I'm a bit confused, I thought I just imported my iTunes library? Unfortunately Songbird can grab the meta-data properly from my iTunes.xml library file, but can't seem to find the actual music! Therefore, the user needs to select their actual iTunes music folder, or whereever they actually store their music! Adding another step between me, and hearing my commemorative "10th Anniversary of The Wiggles" tracks is definitely not a good thing, -1.

Now to my music!
Not so fast cowboy, first, Songbird needs to tell me that some of my tracks failed to import. Does it mean my AAC music? Or my iTunes Store purchases? Wait, or does it mean my OGG music? Oh well, something failed somewhere. Now that that's done, you don't go straight to your music you just spent about 3 clicks too many importing, you go to some silly media-homepage in Songbird's internal browser. I don't care! I know what Songbird is! I just downloaded and installed it! All I wanted was to hear some rocking good kid's tunes from "The Wiggles." Minus three points for standing between me and my music.

Menus galore!
No really, menus!
Menus and sidebars!
The menus for most of Songbird are reasonably well done, but they have some fetish for including images and non-standard things in their interface. For example, at least on Mac OS X, I can't think of a single application outside of Safari's "History" menu that includes little icons in their menus to let you know exactly what you're reading really is what it says it is. Their side navigation bar also includes, just like Firefox does, bookmarks for crap I don't care about. They're trying to follow some conventions, but not conventions for my platform, a measely one point awarded.

Browsing the library
Fortunately, they weren't able to convolute the most basic function of the media player, playing and browsing through media. +2.

Mini-birds
More mini-birds
One thing they definitely have over iTunes, is their minimized mode is much more compact, and definitely sleeker, +2.

Browsing mp3 blogs
Browsing "custom" mp3 blogs
Essentially, a podcast, I think? Not sure if they're trying to coin the phrase "mp3 blog", but giving me a more direct access to sites that offer feeds is certainly nice, their hybrid browser + playlist interface is pretty interesting as well, +1 for trying something different.

Trying to use it as a browser
In the above image, I decided I should try to click on the links inside the Songbird browser, and I'm not sure if it's a good thing that it opened up my default browser of choice, or absolutely retarded. Given that I work at software company I am sure that this tiny behavioral decision must have taken forever, I can't even make up my own mind about it, might as well give them a pity point since I know that some poor developer probably had to switch this back and forth 8 times before this shipped.



Ultimately, as a developer and big proponent of open source, I can certainly appreciate what they're trying to do with Songbird. On the other hand, as a user of computers in general, I don't care for what they've acheived thus far. Making great software takes a HUGE investment of both time and money, looking over the pond of media players to iTunes for an example, which took over 7 years to get to where it is today and it still sucks (sort of). As a Mac developer, I would recommend ditching their insane reliance on Firefox's codebase and use WebKit as their internal browser and strive for more native interfaces, but that's just being picky now isn't it?

Final Score: -2 :(