Perhaps the title is a bit too much ego stroking, yes, I did write the fastest Python module for decoding JSON strings and encoding Python objects to JSON. I didn’t however write the parser behind the scenes.

Over the summer I discovered “Yet Another JSON Library” on GitHub, written by Lloyd Hilaiel, jonesing for a Saturday afternoon project I started the “py-yajl” project to see if I could implement a Python C module atop Lloyd’s marvelous parsing library. After tinkering with the project for a while I got a working prototype building (learning how to define custom types in Python along the way) and let the project stagnate as my weekend ended and the workweek resumed.

A little over a week ago “autodata”, another GitHub user, sent me a “Pull Request” with some minor changes to make py-yajl build cleaner on amd64; my interest in the project was suddenly reignited, amazing what a little interest can do for motivation. Over the 10 days following autodata’s pull request I discovered that a former colleague of mine and fellow GitHub user “teepark” had forked the project as well, working on Python 3 support. Going from zero to two people interested in the project, I quickly converted the code from a stagnant, borderline embarrassing, dump of C code into a leak-free, swift JSON library for Python. Not one to miss out on the fun, I pinged Lloyd who quickly became as enamored with making py-yajl the best Python JSON module available, he forked the project and almost immediately sent a number of pull requests my way with further optimizations to py-yajl such as:

• Swapping out the use of Python lists to a custom pointer stack for maintaining internal state
• Accelerating parsing and handling of Number objects
• Pruning a few memory leaks here and there

Thanks to mikeal’s JSON post and jsonperf.py script, Lloyd and I could both see how py-yajl was stacking up against cjson, jsonlib, jsonlib2 and simplejson; things got competitive. Below are the most recent jsonperf.py results with py-yajl v0.1.1:

json.loads:         6470.22037ms
simplejson.loads:   202.21063ms  
yajl.loads:         145.32621ms
cjson.decode:       102.44788ms

json.dumps:         2309.15286ms
cjson.encode:       276.49586ms   
simplejson.dumps:   201.59785ms
yajl.dumps:         161.00153ms

Over the coming days or weeks (as time permits) I’m planning on adding JSON stream parsing support, i.e. parsing a stream of data as it’s coming in off a socket or file object, as well as a few other miscellaneous tasks.

Given the nature of GitHub’s social coding dynamic, py-yajl got off the ground as a project but Yajl itself gained an IRC channel (#yajl on Freenode) and a mailing list (yajl@librelist.com). To date I have over 20 unique repositories on GitHub (i.e. authored by me) but the experience around Yajl has been the most exciting and finally proved the “social coding” concept beneficial to me.

In addition to RSS feeds, one of my favorite sources of reading material is the Git mailing list; I’m not really active, I simply enjoy reading the discussions around code and the best solutions for certain problems. If you read the list long enough, you’ll start to appreciate the time and attention the Git core developers (spearce, peff and junio (a.k.a. gitster)) put into cultivating the code and in cultivating new contributors. Of all the open source projects I watch to one extent or another, Git is very effective at bringing in new contributors and getting their contributions vetted for inclusion.

If you’re a heavy Git user (like me) you can certainly see the results of their tireless efforts, Junio’s (git.git’s maintainer) in particular. I highly recommend checking out his Amazon wishlist to thank him for his efforts.

See parts 1, 2 and 3

My journey at Slide comes to an end today, when I leave this evening I will once again return to being a free agent (if only for two days). Some of my coworkers have casually referred to my writings over the past few days of my “memoirs”, which isn’t too far off to be honest. When I leave Slide this evening, my employment will have accounted for roughly 10% of my entire life and 50% of my adult life. Writing my side of the story down, to some extent, has been more about telling a story to myself and less about telling it to anybody else (apologies). So much of my time spent at Slide has been in a state of controlled choas that at times it’s hard for me to remember when things were done, who was doing them and the order in which they happened.

The two questions I’ve invariably gotten since I gave my notice and subsequently started writing this series of posts have been: why are you leaving and where are you going? My reasons for leaving are irrelevant, I will say that if I could take the people I’ve been working with at Slide with me, I would. I’ve learned such an incredible amount from those that I’ve worked with, both technical and non-technical, while at Slide. Whenever I would pitch friends on the idea of joining Slide, my take-home point was always “when you join Slide, you will not be the smartest person there.” I feel lucky that I was given the opportunity to “come of age” as a young engineer in a company of so many tremendously talented individials, given the chance for a do-over, I would still play my cards the way I’ve played them. I joined Slide a punk kid from Texas, I’m leaving Slide a slightly-more-learned punk kid from Texas.

As to where I am going, after an extended vacation of Saturday and Sunday, I will be joining my second startup ever (Slide’s a pretty good first time out) on Monday. When I started looking at other companies I had a couple of criteria, I wanted to join a smaller team (Slide’s upwards of a hundred people or so) and I had to really like who I would be working with. Tristan, Jesse, Can (John) and the team from Apture fit both criteria. I’m not going to go into detail about what I’m really going to be doing there or where Apture is going as a company. I will say that after two and a half years of working and studying at Slide, I’m looking forward to employing what I’ve learned and continuing my education at Apture.

See you on the beaches of the world.

Continuing on from part 1 and part 2

Prior to joining Slide, a friend of mine “whurley” had nicknamed me the “Angry Young Man” which I promptly put on my first set of business cards (my current business cards list my title as “Meta-Chief Platform Architect, Enterprise Edition”, I received them after mentioning a failed poaching attempt by LinkedIn to Max); when Top Friends went dark on Facebook, I was a little more than an “angry young man.”

Given my close involvement with the product, the amount of sleepless nights working on it, the actions against Top Friends felt personal to me, regardless of the posturing between Slide and Facebook’s executives. As hours turned into days offline, it became clear to me that the suspension of the application was far less about our privacy hole and far more about Facebook making an example out of Top Friends to the rest of the platform development community. The message was heard loud and clear by the majority of the developers that I knew, this is not your platform, these are not your users and you will play by our rules or we will wipe you from the face of the site. Building on the platform was not only no more fun, it was also a risky business decision.

At the time of the suspension, Keith and I had already started discussing what a “TopFriends.com” might look like, as the signals of platform instability for applications were already being sent. When Top Friends went offline, I prepared a few page outline for Max and Keith detailing “my vision” for what Top Friends would become, I was convinced by that time that its future lie as a social network unto itself, rather than a network contained bu another network (yo dawg..). Not content to simply be “vanity and personal expression” inside of Facebook, I wanted Top Friends to become a separate entity by itself, your VIP club on the internet, at one point there was even executive support for the drawing of users away into a destination site for Top Friends. When the seven days of suspension were over and Top Friends came back online, Slide’s strategy shifted drastically. Our new mission for TF on Facebook was to “get as close to Facebook as you can,” we were to integrate into a user’s experience as much as conceivably possible. Previously we had wanted to run as far away from Facebook as we could, taking our users with us, but the fear that was enstilled by the application suspension caused us to rethink that stance and push Top Friends to be a squeeky clean platform citizen, while we contemplated a possible exodus for FunSpace and SuperPoke!.

Around this time in Slide’s history I became quite jaded and cynical with regards to the platform, Top Friends had been neutered by Facebook, and my notion of what Top Friends should have been was neutered by Max. Regardless, we still had plenty of work that needed to be done to try to succeed with our new strategy. Months prior, Tony Hsieh (not the Zappos guy) the original Top Friends PM had failed to win the visa lottery and moved back to China, leaving TF without a product manager for some time. While we continued to look for senior PM to take on the role, I had to play both product and engineering manager (with help in both places every now and again). Quite the twist of fate for me, I had often poked fun at PMs at Slide, once creating a powerpoint (one should speak the language) titled “PM Flowchart”. The presentation consisted of one slide, with a fairly simple state diagram on it, one block labeled “Write Spec” had an arrow pointing to another block labeled “Bitch.” which pointed back at “Write Spec”. Suffice to say, product managers and I usually had a tenuous relationship.

Passionate about the product to begin with, I started meeting more and more often with Max and Keith to discuss product strategy for TF, in between doing my “real job” of Engineering Manager. Some meetings Keith and Max would square off and I would sit back and watch, other times Keith and I squared off against Max, I rarely took Max’s side against Keith’s though. Not that I always disagreed with Max, but he was at a slight disadvantage in these discussions, Keith and I generally shared a lot of fundamental ideas of what TF should be, stemming from months of discussing the product by his desk before he ever “officially” worked with the project. The transition over a year and a half from quivering in fear as the director of engineering cursed at me on Dave’s house phone, to arguing with the CEO about the product he pitched me on, was surreal to say the least. How I didn’t get fired is either a testament to my charm or Max’s patience.

In fall of 2008, when Seema finally joined as the Top Friends product manager, not only was I more than ready to relinquish the post, Top Friends was in the midst of an identity crisis. Our “facebook zerg rush” strategy of getting closer and closer to the platform played out as you might of expected (hindsight and all), Facebook redesigned the profile, changed viral communications channels and did a lot of things that were likely good for Facebook, but terrible for applications. TF had a lot of momentum on the “old profile” thanks to users dragging the TF profile box all the way up on their profiles. When Facebook rolled out their new profile which put applications not in the backseat, but in the way-back seat, the strategy of “be lovey dovey with Facebook” started to break down, they weren’t being lovey dovey back.

Times were also changing outside of Top Friends at Slide, the SuperPoke! Pets product was starting to take off and actually make money directly from users. This was important! Users, giving us money, for pixels! Brilliant! Being a much more reliable revenue stream than the advertising oriented model that FunSpace, SuperPoke! and Top Friends had been built around, Pets quickly became the “top” product at Slide. With ad revenue drying up for Top Friends, we were tasked with experimenting with virtual currency (like Pets) and ultimately “premium items” (like Pets) within Top Friends. It seemed almost as if Top Friends was changing visions, strategies and directions on a bi-weekly basis. One week we were building virtual currency experiments with “Top Dollars”, the next, virtual economy experiments with an “Own your friends’ profiles” feature, the next, premium virtual goods with “Top Gifts”. As the “Top Friends guy” and the manager of the engineering team, I was so confused and disoriented about what we actually did and where we were actually heading, I didn’t stand a chance at convincing Paul, Geoff and Jason of it.

2008 winding down, the writing was on the wall, Top Friends was not going to live long, at least the Top Friends Team wasn’t. We had gained a reputation of being very self-sufficient and competent, but with that autonomy came uncertainty from outsiders. I regularly had to remind coworkers that I was a Slide engineer, not a Top Friends engineer, regardless of the TF team’s internal view of itself as a “microstartup.” When we failed to meet goals set out for us, it was decided that the staff behind Top Friends were too valuable to spend time on a failing product.

Jason, Paul and Seema went to start a new project, while Geoff and I, together since the desktop client days, joined the Server/Infrastructure team. My personal “love” for Top Friends had all but dissolved by this point, I was sick of Top Friends, I was sick of Facebook, I was sick of policy, I didn’t care all that much about the product anymore. The breaking up of the team though, was crushing. As far war metaphors go, the TF team was a small rag-tag group of guerrillas, capable of taking large projects and finishing them in record time. We often talked about what we did as “playing jazz music” because our work had an improvisational style, but the trust and understanding of where we all fit into the act, allowed us to tackle large tasks in stride; that was all over though. The dream team was broken up.

My time on the server team at Slide is unfortunately a boring story of working with stellar engineers capable of writing solid code and deploying it without incident. As exciting as wood filler “this worked out just fine, the end.” After years of frenzy with Top Friends and the Facebook platform, my first project for the server team took three weeks to build, was pushed without a hitch and has only required two minor updates since. With my nose to the grindstone building services and scalable architecture, I went months without particularly concerning myself with “product direction”, company strategy and their ilk. The closest I would come to application development would be jumping up into application code to fix bugs, all the while cursing app developers’ laziness while conveniently forgetting how often I was guilty of the same offense in my tenure with Top Friends.

When I finally stuck my head back up, near the end of the summer, I started to realize that I was working at a different company than I remember joining. Slide had grown tremendously and changed direction once again. Since stepping back from the front-lines, I had changed and Slide had changed too.

It was about time Slide and I started seeing other people.

Continue on to the end

When I finished up writing part 1 of my journey at Slide yesterday, I had just recounted becoming “the Top Friends guy”, savvy readers might have noted that I had not moved off of Dave’s couch at the time. I am uncertain whether it is a record to be proud of, but I held the position of “the guy on Dave’s couch” for two months. With the leadup to the “F8” conference I didn’t have a whole lot of time to find an apartment, Dave being an all around nice guy and amazing cook, wasn’t helping my motivation to leave either. That said, I’m a delightful house guest, honest.

Shortly after the initial successes of the Top Eight product, and the launch of “FunWall” (renamed “FunSpace” later), Slide quickly converted the desktop client team to the “Facebook Team” with 4-5 engineers hacking on Facebook applications to capitalize as quickly as possible on the wild-west nature of the platform at the time. We subsequently launched another couple apps, such as “My Questions” an application that allowed you to poll your friends (likely our most “useful” application). I ended up writing another application alongside Top Eight called “Fortune Cookie”, contrasted to My Questions, it was probably our most useless application. The application was absolutely brilliant (Mike and Max get credit here again), the profile box for the application was a picture of a fortune cookie with a fortune overlaid. Brilliant. If/when the user clicked through to the application’s canvas page, they were met with a simple grid of checkboxes and friends’ faces, checkboxes checked with a giant blue button that said “Invite your Friends!”.

Never underestimate the power of “Select All”, Fortune Cookie exploded, alongside our “Magic 8 Ball” application (guess what that was), it spread through the Facebook ecosystem like an epidemic. By mid June Top Eight was renamed Top Friends after we bumped the number of “top friends” you could list from 8 to 24 (innovation!); with the power of an intrisicly simple value-proposition to users, 24 friend tiles and “Select All”, Top Friends held the rank of #1 application on Facebook. Following Top Friends was iLike, a major initial success, with Fortune Cookie pulling in third place. Further down the list were a couple of familiar applications: Free Gifts, created by Zach Allia, a Northwestern student at the time and a regular on the #facebook channel on Freenode; Rock You!’s “X Me” application was likely one of the first acquisitions on the Facebook platform, after being created by a student who joined the #facebook channel frantically asking for help as his server was crumbling under the load of pure virality, and SuperPoke! an application created by a then part-time Microsoft employee and two friends.

The first couple weeks of the Facebook platform were sheer insanity, determined to one-up our competitors Rock You!, Slide acquired SuperPoke! and the three engineers that wrote it, Nik, Will and Jonathan. Slide was determined to own the market of “virtually do virtual things to your virtual friends on Facebook”. In short order the SuperPoke team moved down from Seattle to join the “Facebook Team” in Slide’s office at 2nd and Howard, Jon went to the metrics team (being a PhD and all) while Nik and Will shared a desk and started learning Python to port SuperPoke! over to Slide’s stack to allow it to scale faster and better than could have been possible on the PHP/MySQL stack it used at the time. Prior to joining Slide, the SuperPoke! application icon was some picture of a goat Nik had plucked from the internets, by joining Slide they had access to real designers, not goats from Google Image Search. Slide’s most senior designer, Johnnie, can be credited with helping define the brand that would ultimately be synonymous with the Facebook platform and Slide: the SuperPoke sheep. While SuperPoke! and X Me battled it out for 4th and 5th place in the application rankings, journalists started writing articles discussing the Facebook platform, in both positive and negative light, without fail mentiong the absurdity of “throwing sheep” at your friends. I always got the impression that Mark Zuckerberg would have considered the Facebook platform successful when IBM ported Lotus Notes to it, being a “utility fetishist”, I can only imagine how “delighted” he must have been with the top applications on the platform being the likes of Top Friends, Fortune Cookie, Horoscopes, Graffiti, X Me and SuperPoke!.

After the SP guys had joined Slide, Facebook hosted a mid-morning event at their Palo Alto office to help kickstart some developer relations and have top application developers do some lightning-round style presentations. The meeting starting at 9, it was only logical that Nik, Will, Max and I meet at the Slide offices at 8:45; we piled into Max’s BMW M3 (a gorgeous car, I highly recommend it) and sped southwards from San Francisco on the 101. Despite driving between 90-100mph through rush-hour traffic towards Palo Alto, we arrived fashionably late; walking in during a presentation, Dave McClure announced to the whole room “Slide has arrived.”

Roll around in an M3 enough, have people announce your arrival enough and you too will feel like a Web 2.0 rockstar. Being the “Top Friends guy”, I certainly had a bit of an ego going, I still kind of do, but I’m far more modest now about being a complete badass.

The summer of 2007 was mostly a blur, the majority of my “workdays” ended up being 14-16 hours usually ending with Geoff, Sergio, Kasey and I drinking into the wee hours of the morning, pushing code and smoking on the fire escape (building management didn’t really care for that part). The night before the iPhone launched, a bunch of Sliders had arranged to wait in line in shifts at Apple’s Market St store (we were third in line). Given my schedule at the time, I worked most of the night and then manned the 4-7 a.m. shift in line. I didn’t even want an iPhone but Tony, the product manager we hired for Top Friends, and I hung out on the sidewalk, smoked fancy cigars and watched the streets get cleaned. My (now) fianceé was still in Texas finishing up with school, so I had nothing to do but hang out, drink, smoke, write code, push the site and sleep every now and again. My apartment, right in the middle of the colorful Tenderloin district, only served as a place to shower and crash. For the duration of my lease, I didn’t own any dishes and rarely had anything in the fridge other than left-over pizza and Cokes.

By the latter part of 2007 we hired Keith Rabois to be the VP of Business Development, presumably to help us ink deals with big important companies about big important things (with big important sacks of money). Initially, I hadn’t a clue what the hell Keith did, other than walk around in his shiney silk shirts talking on his fancy iPhone, loud enough to hear across the office. The layout of Slide’s office was such that on one end was the open floorplan engineering “pit” and on the other end, separated by a ping pong table and a copy machine were the “non-engineers”. The ping pong table was usually as far as I went. At some point, I don’t remember exactly when, I started consulting with Keith on product related matters. He had this chair by his desk, so I would stroll over, plop down and gab for longer than he probably had time for about subjects ranging from the latest Facebook gossip to long-term strategy; Keith’s involvement with Top Friends would only increase from then moving forward.

By the beginning of 2008, the Facebook platform wasn’t fun anymore. Too many emails contained the words “policy” and “violation” and often dastardly combinations of the two. At the same time, Slide had upped its commitment to Top Friends hiring Jason, who I had known for some time from the #facebook IRC channel, his compatriot Paul, and assigning Geoff, a senior QA engineer who had put up with my shit on the client team since I joined the company months earlier. I was promoted to Engineering Lead and shortly thereafter to Engineering Manager. My role had changed dramatically, no longer simply just a monkey coding like there was no tomorrow, I now had people I had to be accountable to, all the miserable hacks I had thrown into Top Friends in the previous 8 months I had to sheepishly explain to Jason and Paul, mentioning from time to time how I could do it better given the time.

Jason and Paul being hired and assigned to my team was likely the luckiest thing that ever happened to me at Slide, overnight I went from a hard-working “army of one” to part of a team of four hard-working bone crushers with an incredible drive to succeed. In a few short months we had shipped an “Awards” feature, built out a “Top Friends Profile” and started pushing our way back to the top.

In June, a reporter for CNet reported on a hole in the Top Friends Profile that allowed a user to view information about other users they could not have otherwise seen. The reporter used this an instrumental piece of a larger article bashing Facebook on their privacy record and the openness of the Faceobok platform. When Keith texted me that night, I rushed home and pushed a fix for the hole within the hour, went to dinner by myself and had the worst Pad Thai I’ve ever eaten, watching the exchange of emails between Slide’s and Facebook’s executive team on my Blackberry.

Top Friends had tens of millions of users and with the flick of a switch, Facebook took Top Friends offline.

Continue on to part 3 and the end

As some of you may, or may not know, this friday October 23rd will be my final day as a Slide employee. With my journey at Slide nearing its completion, I wanted to document some of how I’ve gotten here and where I’ve started, if for nobody other than myself.

Officially I started at Slide April 2nd, 2007, though my journey to Slide started far earlier. At the end of my fourth semester at Texas A&M my then girlfriend, now fianceé and I decided we were through with College Station and to move to San Antonio; most Texans would consider this a lateral move at best. I had every intention of resuming my studies at UTSA following a brief stint at San Antonio College clearing up pre-requisites with a slightly lower price tag. By the end of fall semester it had become clear that I wasn’t cut out for college, I stopped attending and focused full time on software. At the time most of my experience and contacts were through the Mac development community, primarily via IRC on the Freenode network and developer mailing lists for various open source projects. Through my involvement in the Bonjour mailing lists and work with the API, I had at one point impressed Bonjour’s original inventor Stuart Cheshire enough to land an interview at Apple for the Core OS group, working on Bonjour.

Sitting on the Continental flight out to San Jose, I practiced writing network services using BSD sockets and pouring over as much C as I could possibly manage, all told I likely wrote around 3 multicast service/client pairs on that flight. What I wasn’t prepared for was the “computer science” nature of the interview; I bombed it with my rudimentary algorithms knowledge and lack of experience working with C on a day to day basis. Fortunately, my last interviewer of the day was Ernie Prabhakar then a product or marketing manager for the Core OS group; Ernie indulged me in a very interesting conversation about Apple’s position in the open source universe, product direction, etc. Despite bombing the technical portions of the interview, I suppose Ernie saw enough enthusiasm in me to refer me to Dave Morin at Facebook (the two worked together at some point).

Those that know Dave Morin understand that the man wields a Jobsian reality distortion field, even via email 1500 miles away in Texas I felt the power of the field and was drawn to Facebook. While I was ultimately disappointed to not have landed my then-dream-job at Apple, I was incredibly excited to be flying back to Silicon Valley to interview at Facebook. When I mentioned to daver (Dave Young) on IRC that I would be flying back out to see the nice folks at Facebook in Palo Alto, he also arranged an interview at Slide the day after.

A number of factors likely lead to my failure to excite my interviewers at Facebook, not having a Facebook account for one didn’t help, I also think I uttered “fuck” under my breath once or twice while sketching out problems on a whiteboard. Considering my interview was done mostly in “the game room” due to a scheduling error, I didn’t think I was being too unprofessional. As one could assume, my interview with Slide went substantially better, I accepted an offer to join Slide as a junior software engineer working on their now defunct desktop application(s).

My start date was set for April 2nd, within three weeks I had terminated the lease on my apartment in San Antonio, tossed, sold or otherwise gave away the majority of my belongings and furniture and packed my VW Jetta to the brim and drove west. I didn’t particularly have a plan other than “show up, get to work” (I was 21, how young and foolish), so I crashed on daver’s couch while I settled in and started searching for an apartment.

My early days at Slide were all about getting up to speed on Python (Slide’s language of choice) and ActionScript 2 (Slide’s only option for Flash at the time); I started helping with the Windows client, mostly in the spagetti-driven Flash-based screensaver product. Towards the beginning of May, Jeremiah (then Director of Engineering, now CTO) and Bobby (another engineer) were working with some preview APIs from what would ultimately become the Facebook platform. For whatever reason I started working on trying to incorporate some data from Facebook into our desktop client (optimal synergy, etc) and became the third engineer working at Slide on the Facebook platform in its infancy. As May came to completion, we (Slide) were invited to “F8” to unveil some of the applications for their new platform we had built.

Donning my trusty brown cordoruy sport jacket, dark blue Slide t-shirt, I helped man Slide’s booth presenting some of our apps: SlideShows and YouTube Skins (both products turned out to be utter failures). As the business/presentation portion part of F8 wound down, I grabbed a 19” monitor and told Max and Jeremiah that I wanted to stay for the hackathon but didn’t have any idea what to hack on (being a desktop developer and all). Max leaned in and muttered “Top Friends” on his way out, leaving me to set up shop with the only dual screen setup in the hackathon, at a lonely table by myself (I hadn’t figured out how to socialize at that point). Coming from a desktop background, I hadn’t a clue what I was doing, I could barely figure out how to get pages working on Slide’s infrastructure, let alone all this FBML, FQL and JavaScript malarkey.

Fortunately in the days following the hackathon, I was able to enlist the help of Sergio, the best web front-end engineer Slide had to offer to help me create a grid of drag and droppable images along with some other pieces of front-end to make the application palatable. All said and done, if I remember correctly, Top Eight launched less than a week after the platform did, my first “big” project at Slide. Originally I couldn’t get database resources for the app, so I stashed the friends list inside of the profile FBML and then would subsequently retrieve back from Facebook when I needed it, using regular expressions (had help with that too) to pull the list of Facebook user IDs out; that hacked up solution lasted for all of maybe 30 minutes on live as soon as everyone saw how god-awful slow it was.

Day three of Top Eight, I learned what “viral meant”. My parents had neglected to pay their phone bill, taking my “family plan” number out with it, meaning I couldn’t receive the frantic calls from Jeremiah as I slept-in that morning. Turns out by giving the Top Eight a callback URL with Facebook that hit “www.slide.com” was proving impossible to load balance, resulting in a couple hours of site issues for the rest of Slide, as Top Eight skyrocketed hundreds of thousands of users in a single day. I awoke that morning to pounding on Dave’s door (I was still on their couch), opening it I saw Carey (another desktop developer at Slide) who said “your phone’s off.” Not my preferred way to wake up, but it sufficed. I sheepishly called Jeremiah on Dave’s house phone.

Jeremiah was pissed. Not “who ate the rest of my hummus” pissed, righteously pissed, at me. Here I was, living on a friend’s (who I met on the internets) couch, without a proper mailing address trying to figure out how this startup thing worked, and Jeremiah was furious with me. If there were such a thing as an “ideal time” for an earthquake, I would have gladly accepted that as an alternative.

Once the smoke cleared and tempers cooled, we looked at some of the installation and growth numbers of Top Eight during the previous 6 hours; I had found myself a new job at Slide. From that day forth, I was “the Top Friends guy.”

Continue on to part 2, part 3 and the end

Last week @admc, despite being a big proponent of Windmill, needed to use WatiN for a change. WatiN has the distinct capability of being able to work with Internet Explorer’s HTTPS support as well as frames, a requirement for the task at hand. As adorable as it was to watch @admc, a child of the dynamic language revolution, struggle with writing in C# with Visual Studio and the daunting “Windows development stack,” the prospect of a language shift at Slide towards C# on Windows is almost laughable. Since Slide is a Python shop, IronPython became the obvious choice.

Out of an hour or so of “extreme programming” which mostly entailed Adam watching as I wrote IronPython in his Windows VM, IronWatin was born. IronWatin itself is a very simple test runner that hooks into Python’s “unittest” for creating integration tests with WatiN in a familiar environment.

I intended IronWatin to be as easy as possible for “native Python” developers, by abstracting out updates to sys.path to include the Python standard lib (adds the standard locations for Python 2.5/2.6 on Windows) as well as adding WatiN.Core.dll via clr.AddReference() so developers can simply import IronWatin; import WatiN.Core and they’re ready to start writing integration tests. When using IronWatin, you create test classes that subclass from IronWatin.BrowserTest which takes care of setting up a browser (WatiN.Core.IE/WatiN.Core.FireFox) instance to a specified URL, this leaves your runTest() method to actually execute the core of your test case.

Another “feature”/design choice with IronWatin, was to implement a main() method specifically for running the tests on a per-file basis (similar to unittest.main()). This main method allows for passing in an optparse.OptionParser instance to add arguments to the script such as “–server” which are passed into your test classes themselves and exposed as “self.server” (for example). Which leaves you with a fairly straight-forward framework with which to start writing tests for the browser itself:

#!/usr/bin/env ipy

The import of IronWatin will add a reference to WatiN.Core.dll

and update sys.path to include C:\Python25\Lib and C:\Python26\Lib

so you can import from the Python standard library

import IronWatin

import WatiN.Core as Watin import optparse

class OptionTest(IronWatin.BrowserTest): url = ‘http://www.github.com’

def runTest(self):
    # Run some Watin commands
    assert self.testval

if name == ‘main’: opts = optparse.OptionParser() opts.add_option(‘–testval’, dest=’testval’, help=’Specify a value’) IronWatin.main(options=opts) </code>

Thanks to IronPython, we can make use of our developers’ and QA engineers’ Python knowledge to get the up and running with writing integration tests using WatiN rapidly instead of trying to overcome the hump of teaching/training with a new language.

Deployment Notes: We’re using IronPython 2.6rc1 and building WatiN from trunk in order to take advantage of some recent advances in their Firefox/frame support. We’ve not tested IronWatin, or WatiN at all for that matter, anywhere other than Windows XP.

This weekend I finally got around to downloading IronPython 2.6rc1 to test it against the upcoming builds of Mono 2.6 preview 1 (the version numbers matched, it felt right). Additionally in the land of Mono, I’ve been toying around with the IKVM project as of late, as a means of bringing some legacy Java code that I’m familiar with onto the CLR. As I poked in one xterm (urxvt actually) with IKVM and with IronPython in another, a lightbulb went off. What if I could mix different languages in the same runtime; wouldn’t that just be cool as a cucumber? Turns out, it is.

After grabbing a recent release (0.40.0.1) of IKVM, I whipped up a simple Test.java file:

I compiled Test.java to Test.class then to Test.dll with ikvmc (note: this is using JDK 1.6); in short, Java was compiled to Java bytecode and then to CIL:

javac Test.java
mono ikvm-0.40.0.1/bin/ikvmc.exe -target:library -out:Test.dll Test.class

Once you have a DLL, it is fairly simple to import that into an IronPython script thanks to the clr module IronPython provides. It is important to note however, that IKVM generated DLLs will try to load other DLLs at runtime (IKVM.Runtime.dll for example) so these either need to be installed in the GAC or available in the directory your IronPython script is running in.

Here’s my sample test IronPython file, using the unittest module to verify that the compiled Java code is doing what I expect it to:

When I run the IronPython script, everything “just works”:

% mono IronPython-2.6/ipy.exe IkvmTest.py  
.
----------------------------------------------------------------------
Ran 1 test in 0.040s

OK
% 

While my Test.java is a fairly tame example of what is going on here, the underlying lesson is an important one. Thanks to the Mono project’s CLR and the advent of the DLR on top of that we are getting closer to where “language” and “runtime” are separated enough to not be interdependent (as it is with CPython), allowing me (or you) to compile or otherwise execute code written in multiple languages on a common (language) runtime.

That just feels good.

Once upon a time I was lucky enough to take an “Intro to C++” class taught by none other than Bjarne Stroustrop himself, while I learned a lot of things about what makes C++ good and sucky at the same time, he also taught a very important lesson: great engineers are lazy. It’s fairly easy to enumerate functionality in tens of hundreds of lines of poorly organized, inefficient code, but (according to Bjarne) it’s the great engineers that are capable of distilling that functionality into it’s most succinct form. I’ve since taken this notion of being “ultimately lazy” into my professional career, making it the root answer for a lot of my design decisions and choices: “Why bother writing unit tests?” I’m too lazy to fire up the whole application and click mouse buttons, and I can only do that so fast; “Why do you only work with Vim in GNU/screen?” I can’t be bothered to set up a new slew of terminals when I switch machines, and so on down the line.

Earlier this week I found another bit of manual work that I shouldn’t be doing and should be lazy about: building. The local build is something that’s common to every single software developer regardless of language, Slide being a Python shop, we have a bit more subtle of a “build”, that is to say, developers implicitly run a “build” when they hit a page in Apache or a test/script. I found myself constantly switching between two terminal windows, one with my editor (Vim) and one for running tests and other scripts.

Being an avid Hudson user, I decided I’d give the File system SCM a try. Very quickly I was able to set up Hudson to poll my working directory and watch for files to change every minute, and then run a “build” with some tests to go with it. Now I can simply sit in Vim all day and write code, only context-switching to commit changes.

Setting up Hudson for local continuous integration is quite simple, by visiting hudson-ci.org you can download hudson.war which is a fully self contained runnable version of Hudson, you can start it up locally with java -jar hudson.war. Once it’s started, visit http://localhost:8080 and you’ve find yourself smack-dab in the middle of a fresh installation of Hudson.

First things first, you’ll need the File System SCM plugin from the Hudson Update Center (left side bar, “Manage Hudson” > “Manage Plugins” > “Available” tab)

After installing the plugin, you’ll need to restart Hudson, then you can create your job, configuring the File System SCM to poll your working directory:

Of course, add the necessary build steps to build/test your software as well, and you should be set for some good local continuous integration. Once the job is saved, the job will poll your working directory for files to be modified and then copy things over to the job’s workspace for execution.

After the job is building, you can hook up the RSS feed (http://localhost:8080/rssLatest) to Growl or some other form of desktop notifier so you don’t even have to move your eyes to know whether your local build succeeded or not (I use the “hudsonnotify” script for Linux/libnotify below).

By automating this part of my local workflow with Hudson I can take advantage of a few things:

• I no longer need to context switch to run my tests
• I can make use of Hudson’s nice UI for visually inspecting test results as they change over time
• I have near-instant feedback on the validity of the changes I’m making

The only real downside I can think of is no longer having any excuse for checking in code that “breaks the build”, but in the end that’s probably a good thing.

Instead of relying on commits, you can get near-instant feedback on your changes before you even get things going far enough to check them in, tightening the feedback loop on your changes even further, very-very continuous integration. Your mileage may vary of course, but I recommend giving it a try.

hudsonnotify.py