Howdy!

Welcome to my blog where I write about software development, cycling, and other random nonsense. This is not the only place I write, you can find more words I typed on the Buoyant Data blog, Scribd tech blog, and GitHub.

5 tips for traveling with Tux

After running a Linux laptop for a number of years and having mostly negative travel experiences from messing something up along the way, this holiday season I think I’ve finally figured out how to optimally travel with a Linux notebook. The following tips are some of the lessons I’ve had to learn the hard way through trial and error over the course of countless flights spanning a few years.

Purchase a small laptop or netbook

Far and away the best thing I’ve done for my travel experience thus far has been the purchase of my new Thinkpad X200 (12.1”). My previous laptops include a MacBook Pro (15”), a Thinkpad T43 (14”) and a Thinkpad T64 (14”). Invariably I have the same problems with all larger laptops, their size is unwieldy in economy class and their power consumption usually allows me very little time to get anything done while up in the air. Being 6’4” and consistently cheap, I’m always in coach, quite often on redeye flights where the passenger in front of me invariably leans their seat back drastically reducing my ability to open a larger laptop and see the screen. With a 12” laptop or a netbook (I’ve traveled with an Eee PC in the past as well) I’m able to open the screen enough to see it clearly and actually type comfortbaly on it. Additionally, the smaller screen and size of the laptop means less power consumption, allowing me to use it for extended periods of time.

Use a basic window manager

Personally, I prefer XMonad, but I believe any simplistic window manager will save a noticable number of cycles compared to the Gnome and KDE “desktop environments.” Unlike Gnome, for example, XMonad does not run a number of background daemons to help provide a “nice” experience in the way of applets, widgets, panels and menus.

Disable unneeded services and hardware

Reducing power consumption is a pretty important goal for me while traveling with a Linux laptop, I love it when I have sufficient juice to keep myself entertained for an entire cross-country flight. Two of the first things I disable before boarding a plane are Wireless and Bluetooth via the NetworkManager applet that I run. If I’m on a redeye, I’ll also set my display as dark as possible which not only saves power but also eye strain. It’s also important to make sure your laptop is running its CPU in “power-save” mode, which means the clockspeed of the chip is reduced, allowing you to save even more power. Finally I typically take a look at htop(1) to see if there are any unneeded processes taking up cycles/memory that I either don’t need or don’t intend to use for the flight. The flight I’m currently on (Miami to San Francisco) I discovered that Chrome was churning some unnecessary cycles and killed it (no web browsing on American Airlines).

Use an external device for music/video

If you’re like me, you travel with a good pair of headphones and a desire to not listen to babies crying on the plane. I find a dedicated device purely for music can help avoid wasting power on music since most devices can play for 12-40 hours depending on the device. It’s generally better (in my opinion) to use your $100 iPod for music and your $2000 computer for computing, that might just be personal bias though.

Load applications you’ll need ahead of time

I generally have an idea of what I want to do before I board a plane, I have a project that I’d like to spend some time hacking on or something I want to write out or experiment with. Having a “game plan” before I get onto the plane means I can load up any and all applications while plugged in at the airport. This might be a minor power saver but after I’ve lowered the CPU clockspeed and disabled some services, I certainly don’t want to wait around for applications to load up while I sit idly in coach.

Update: As Etni3s from reddit points out, powertop(1) is a pretty handy utility for watching power consumption.

As I write this article, I’m probably an hour into my five and half hour flight and the battery monitor for my X200 is telling me I have an estimated eight hours of juice left.

I’m proud to say, Tux is my copilot.

Read more →

Using Cheetah templates with Django

Some time ago after reading a post on Eric Florenzano’s blog about hacking together support for Cheetah with Django, I decided to add “proper” support for Cheetah/Django to Cheetah v2.2.1 (released June 1st, 2009). At the time I didn’t use Django for anything, so I didn’t really think about it too much more.

Now that I work at Apture, which uses Django as part of its stack, Cheetah and Django playing nicely together is more attractive to me and as such I wanted to jot down a quick example project for others to use for getting started with Cheetah and Django. You can find the django_cheetah_example project on GitHub, but the gist of how this works is as follows.

Requires

Getting Started

For all intents and purposes, using Cheetah in place of Django’s templating system is a trivial change in how you write your views.

After following the Django getting started documentation, you’ll want to create a directory for your Cheetah templates, such as Cheetar/templates. Be sure to touch __init__.py in your template directory to ensure that templates can be imported if they need to.

Add your new template directory to the TEMPLATE_DIRS attribute in your project’s settings.py.

Once that is all set up, utilizing Cheetah templates in Django is just a matter of a few lines in your view code: import Cheetah.Django

def index(req):
    return Cheetah.Django.render('index.tmpl', greet=False)</code>

Note: Any keyword-arguments you pass into the Cheetah.Django.render() function will be exposed in the template’s “searchList”, meaning you can then access them with $-placeholders. (i.e. $greet)

With the current release of Cheetah (v2.4.1), there isn’t support for using pre-compiled Cheetah templates with Django (it’d be trivial to put together though) which means Cheetah.Django.render() uses Cheetah’s dynamic compilation mode which can add a bit of overhead since templates are compiled at runtime (your mileage may vary).

Read more →

Pyrage: from toolbox import hammer

Those that have worked with my directly know I’m a tad obsessive when it comes to imports in Python. Once upon a time I had to write some pretty disgusting import hooks to solve a problem and got to learn first-hand how gnarly Python’s import subsystem can be. I have a couple coding conventions that I follow when I’m writing Python for my own personal projects that typically follows:

  • “strict” system imports first (i.e. import time)
  • “from” system imports second (i.e. from eventlet import api)
  • “local” imports (import mymodule)
  • local “from” imports (from mypackage import module)

In all of these sections, I like to list things alphabetically as well, just to make sure that at no point are modules ever doubley-imported. This results in code that looks clean (in my humblest of opinions): #!/usr/bin/env python import os import sys from eventlet import api

import app.util
from app.models import account

## Etc.</code>

A module importing habit that absolutely drives me up the wall, I was introduced to and told “don’t-do-that” by Dave: importing symbols from modules; in effect: from MySQLdb import IntegrityError. I have two major reasons for hating the importing of symbols, the first one is that it messes with your module’s namespace. If the symbol import above were in a file called “foo.py”, the foo module would then have the member foo.IntegrityError. Additionally, it makes the code more difficult to understand when you flatten the module’s namespace out; 500 lines down in the file if you see acct_m = AccountManager() as a developer new to the file you’ll have to go up to the top and figure out where the hell AccountManager is actually coming from to understand how it works.

As code with these sort of symbol-level imports ages, it becomes more and more frustrating to deal with, if I need OperationalError in my module now I have three options:

  • Update the line to say: from MySQLdb import IntegrityError, OperationalError
  • Add import MySQLdb and just refer to IntegrityError and MySQLdb.OperationalError
  • Add import MySQLdb and update all references to IntegrityError

I’ve seen code in open source projects that have abused the symbol imports so badly that an import statement look like: from mod import CONST1, CONST2, CONST3, SomeError, AnotherClass (ad infinium).

I think poor import style is a good indicator of how one can expect the rest of the Python code to look, I cannot recall a single instance where I’ve looked at a Python module with gross import statements and clean classes and functions. from MySQLdb import IntegrityError, OperationalError, MySQLError, ProgrammingError, \ NotSupportedError, InternalError

PYRAGE!

Read more →

One year of Cheetah

While working at Slide I had a tendency to self-assign major projects, not content with things being “good-enough” I tended to push and over-extend myself to improve the state of Slide Engineering. Sometimes these projects would fail and I would get uncomfortably close to burning myself out, other times, such as the migration from Subversion to Git, turned out to be incredibly rewarding and netted noticable improvements in our workflow as a company.

One of my very first major projects was upgrading our installation of Cheetah from 1.0 to 2.0, at the time I vigorously hated Cheetah. My distain of the templating system stemmed from using a three year old version (that sucked to begin with) and our usage of Cheetah which bordered between “hackish” and “vomitable.” At this point in Slide’s history, the growth of the Facebook applications meant there was going to be far less focus on the Slide.com codebase which is where some of the more egregious Cheetah code lived; worth noting that I never “officially” worked on the Slide.com codebase. When I successfully convinced Jeremiah and KB that it was worth my time and some of their time to upgrade to Cheetah 2.0 which offered a number of improvements that we could make use of, I still held some pretty vigorous hatred towards Cheetah. My attitude was simple though, temporary pain on my part would alleviate pain inflicted on the rest of the engineering team further down the line. Thanks to fantastic QA by Ruben and Sunil, the Cheetah upgrade went down relatively issue free, things were looking fine in production and everybody went back to their regularly scheduled work.

Months went by without me thinking of Cheetah too much until late 2008, Slide continued to write front-end code using Cheetah and developers continued to grumble about it. Frustrated by the lack of development on the project, I did the unthinkable, I started fixing it. Over the Christmas break, I used git-cvsimport(1) to create a git repository from the Cheetah CVS repo hosted with SourceForge and I started applying patches that had circulated on the mailing list. By mid-March I had a number of changes and improvements in my fork of Cheetah and I released “Community Cheetah”. Without project administrator privileges on SourceForge, I didn’t have much of a choice but to publish a fork on GitHub. Eventually I was able to get a hold of Tavis Rudd, the original author of Cheetah who had no problem allowing me to become the maintainer of Cheetah proper, in a matter of months I had gone from hating Cheetah to fulfilling the oft touted saying “it’s open source, fix it!” What was I thinking.

Thanks in part to git and GitHub’s collaborative/distributed development model patches started to come in and the Cheetah community for all intents and purposes “woke up.” Over the course of the past year, Cheetah has seen an amazing number of improvements, bugfixes and releases. Cheetah now properly supports unicode throughout the system, supports @staticmethod and @classmethod decorators, supports use with Django and now supports Windows as a “first-class citizen”. While I committed the majority of the fixes to Cheetah, five other developers contributed fixes:

In 2008, Cheetah saw 7 commits and 0 releases, while 2009 brought 342 commits and 10 releases; something I’m particularly proud of. Unforunately since I’ve left Slide, I no longer use Cheetah in a professional context but I still find it tremendously useful for some of my personal projects.

I am looking forward to what 2010 will bring for the Cheetah project, which started in mid-2001 and has seen continued development since thanks to a number of contributors over the years.

Read more →

Pyrage: Generic Exceptions

Earlier while talking to Ryan I decided I’d try to coin the term “pyrage” referring to some frustrations I was having with some Python packages. The notion of “pyrage” can extend to anything from a constant irritation to a pure “WTF were you thinking!” kind of moment.

Not one to pass up a good opportunity to bitch publicly, I’ll elaborate on some of my favorite sources of “pyrage”, starting with generic exceptions. While at Slide, one of the better practices I picked up from Dave was the use of specifically typed exceptions to specific errors. In effect: class Connection(object): ## Pretend this object has "stuff" pass

class InvalidConnectionError(Exception):
    pass
class ConnectionConfigurationError(Exception):
    pass
    
def configureConnection(conn):
    if not isinstance(conn, Connection):
        raise InvalidConnectionError('configureConnection requires a Connection object')
    if conn.connected:
        raise ConnectionConfigurationError('Connection (%s) is already connected' % conn)
    ## etc </code>

Django, for example, is pretty stacked with generic exceptions, using builtin exceptions like ValueError and AttributeError for a myriad of different kinds of exceptions. urllib2’s HTTPError is good example as well, overloading a large number of HTTP errors into one exception leaving a developer to catch them all, and check the code, a la: try: urllib2.urlopen('http://some/url') except urllib2.HTTPError, e: if e.code == 503: ## Handle 503's special pass else: raise

Argh. pyrage!

Read more →

Get out there and buy (me) stuff

A weekend or two ago I sat down and created an Amazon wishlist of stuff that I would like to purchase. In the past I’ve found Amazon wishlists an ideal way of saying thanks to a number of folks in the open source community whose work I value, at one point I didn’t desire any new gadgets and went through a large number of gift cards buying goodies for some of my favorite hackers on Amazon.

On the off chance you appreciate pictures of Fatso, Cheetah templates, @hudsonci on Twitter, py-yajl, Urlenco.de, tweets on a regular basis, watching me talk in a brown corduroy jacket or just plain have money to burn, below is my very own Amazon wishlist.

Holy Wishlist Batman!

Get out there and buy (me) stuff, our economy depends on it.

Read more →

Python/JSON Eat/Drink-up in San Francisco

A few weeks ago when I started working more on py-yajl I discovered that there are actually a number of Python developers who work(ed) with JSON parsers in the Bay Area. As luck would have it Lloyd, the author of Yajl, is going to be in town next weekend for Add-on-Con, time to meet up and have some beers! I’ve invited the authors of: simplejson, jsonlib, jsonlib2 and a few other Python hackers in the Bay Area that I know of.

If you’re in San Francisco this Saturday (Dec. 12th) and loves you some Python, don’t hesitate to swing by 21st Amendment around 1pm-ish to join us!

Read more →

Server-side image transforms in Python

While working at Slide, I became enamored with the concept of cooperative threads (coroutines) and the in-house library built around greenlet to implement coroutines for Python. As an engineer on the “server team” I had the joy of working in a coro-environment on a daily basis but now that I’m “out” I’ve had to find an alternative library to give me coroutines: eventlet. Interestingly enough, eventlet shares common ancestry with Slide’s internal coroutine implementation like two different species separated thousands of years ago by continental drift (a story for another day).

A few weekends ago, I had a coroutine itch to scratch one afternoon: an eventlet-based image server for applying transforms/filters/etc. After playing around for a couple hours “PILServ” started to come together. One of the key features I wanted to have in my little image server project was the ability to not only pass the server a URL of an image instead of a local path but also to “chain” transforms in a jQuery-esque style. Using segments of the URL as arguments, a user can arbitrarily chain arguments into PILServ, i.e.:

http://localhost:8080/flip/filter(blur)/rotate(45)/resize(64x64)/<url to an image>

At the end of the evening I spent on PILServ, I had something going that likely shows off more of the skills of PIL rather than eventlet itself but I still think it’s neat. Below is a sample of some images transformed by PILServ running locally:

Read more →