And now a video of my cat being obnoxious
Behold, the powerful lungs of Fatso:
Welcome to my blog where I write about software
development, cycling, and other random nonsense. This is not
the only place I write, you can find more words I typed on the Buoyant Data blog, Scribd tech blog, and GitHub.
After running a Linux laptop for a number of years and having mostly negative travel experiences from messing something up along the way, this holiday season I think I’ve finally figured out how to optimally travel with a Linux notebook. The following tips are some of the lessons I’ve had to learn the hard way through trial and error over the course of countless flights spanning a few years.
Far and away the best thing I’ve done for my travel experience thus far has been the purchase of my new Thinkpad X200 (12.1”). My previous laptops include a MacBook Pro (15”), a Thinkpad T43 (14”) and a Thinkpad T64 (14”). Invariably I have the same problems with all larger laptops, their size is unwieldy in economy class and their power consumption usually allows me very little time to get anything done while up in the air. Being 6’4” and consistently cheap, I’m always in coach, quite often on redeye flights where the passenger in front of me invariably leans their seat back drastically reducing my ability to open a larger laptop and see the screen. With a 12” laptop or a netbook (I’ve traveled with an Eee PC in the past as well) I’m able to open the screen enough to see it clearly and actually type comfortbaly on it. Additionally, the smaller screen and size of the laptop means less power consumption, allowing me to use it for extended periods of time.
Personally, I prefer XMonad, but I believe any simplistic window manager will save a noticable number of cycles compared to the Gnome and KDE “desktop environments.” Unlike Gnome, for example, XMonad does not run a number of background daemons to help provide a “nice” experience in the way of applets, widgets, panels and menus.
Reducing power consumption is a pretty important goal for me while traveling with a Linux laptop, I love it when I have sufficient juice to keep myself entertained for an entire cross-country flight. Two of the first things I disable before boarding a plane are Wireless and Bluetooth via the NetworkManager applet that I run. If I’m on a redeye, I’ll also set my display as dark as possible which not only saves power but also eye strain. It’s also important to make sure your laptop is running its CPU in “power-save” mode, which means the clockspeed of the chip is reduced, allowing you to save even more power. Finally I typically take a look at htop(1) to see if there are any unneeded processes taking up cycles/memory that I either don’t need or don’t intend to use for the flight. The flight I’m currently on (Miami to San Francisco) I discovered that Chrome was churning some unnecessary cycles and killed it (no web browsing on American Airlines).
If you’re like me, you travel with a good pair of headphones and a desire to not listen to babies crying on the plane. I find a dedicated device purely for music can help avoid wasting power on music since most devices can play for 12-40 hours depending on the device. It’s generally better (in my opinion) to use your $100 iPod for music and your $2000 computer for computing, that might just be personal bias though.
I generally have an idea of what I want to do before I board a plane, I have a project that I’d like to spend some time hacking on or something I want to write out or experiment with. Having a “game plan” before I get onto the plane means I can load up any and all applications while plugged in at the airport. This might be a minor power saver but after I’ve lowered the CPU clockspeed and disabled some services, I certainly don’t want to wait around for applications to load up while I sit idly in coach.
Update: As Etni3s from reddit points out, powertop(1) is a pretty handy utility for watching power consumption.
As I write this article, I’m probably an hour into my five and half hour flight and the battery monitor for my X200 is telling me I have an estimated eight hours of juice left.
I’m proud to say, Tux is my copilot.
Some time ago after reading a post on Eric Florenzano’s blog about hacking together support for Cheetah with Django, I decided to add “proper” support for Cheetah/Django to Cheetah v2.2.1 (released June 1st, 2009). At the time I didn’t use Django for anything, so I didn’t really think about it too much more.
Now that I work at Apture, which uses Django as part of its stack, Cheetah and Django playing nicely together is more attractive to me and as such I wanted to jot down a quick example project for others to use for getting started with Cheetah and Django. You can find the django_cheetah_example project on GitHub, but the gist of how this works is as follows.
For all intents and purposes, using Cheetah in place of Django’s templating system is a trivial change in how you write your views.
After following the Django getting started
documentation, you’ll want to create a directory for your Cheetah templates, such
as Cheetar/templates. Be sure to touch __init__.py in your template
directory to ensure that templates can be imported if they need to.
Add your new template directory to the TEMPLATE_DIRS attribute
in your project’s settings.py.
Once that is all set up, utilizing Cheetah templates in Django is just
a matter of a few lines in your view code:
import Cheetah.Django
def index(req):
return Cheetah.Django.render('index.tmpl', greet=False)</code>
Note: Any keyword-arguments you pass into the Cheetah.Django.render()
function will be exposed in the template’s “searchList”, meaning you can
then access them with $-placeholders. (i.e. $greet)
With the current release of Cheetah (v2.4.1), there isn’t support for using pre-compiled Cheetah templates with Django (it’d be trivial to put together though) which means Cheetah.Django.render() uses Cheetah’s dynamic compilation mode which can add a bit of overhead since templates are compiled at runtime (your mileage may vary).
Those that have worked with my directly know I’m a tad obsessive when it comes to imports in Python. Once upon a time I had to write some pretty disgusting import hooks to solve a problem and got to learn first-hand how gnarly Python’s import subsystem can be. I have a couple coding conventions that I follow when I’m writing Python for my own personal projects that typically follows:
import time)from eventlet import api)import mymodule)from mypackage import module)In all of these sections, I like to list things alphabetically
as well, just to make sure that at no point are modules ever
doubley-imported. This results in code that looks clean (in
my humblest of opinions):
#!/usr/bin/env python
import os
import sys
from eventlet import api
import app.util
from app.models import account
## Etc.</code>
A module importing habit that absolutely drives me up the wall,
I was introduced to and told “don’t-do-that” by Dave: importing
symbols from modules; in effect: from MySQLdb import IntegrityError.
I have two major reasons for hating the importing of symbols, the
first one is that it messes with your module’s namespace. If the
symbol import above were in a file called “foo.py”, the foo module
would then have the member foo.IntegrityError. Additionally, it
makes the code more difficult to understand when you flatten the module’s
namespace out; 500 lines down in the file if you see acct_m = AccountManager()
as a developer new to the file you’ll have to go up to the top and figure
out where the hell AccountManager is actually coming from to understand
how it works.
As code with these sort of symbol-level imports ages, it becomes more and more
frustrating to deal with, if I need OperationalError in my module now I have
three options:
from MySQLdb import IntegrityError, OperationalErrorimport MySQLdb and just refer to IntegrityError and MySQLdb.OperationalErrorimport MySQLdb and update all references to IntegrityErrorI’ve seen code in open source projects that have abused the symbol imports
so badly that an import statement look like: from mod import CONST1, CONST2, CONST3, SomeError, AnotherClass
(ad infinium).
I think poor import style is a good indicator of how one can expect the
rest of the Python code to look, I cannot recall a single instance where I’ve
looked at a Python module with gross import statements and clean classes and functions.
from MySQLdb import IntegrityError, OperationalError, MySQLError, ProgrammingError, \
NotSupportedError, InternalError
PYRAGE!
While working at Slide I had a tendency to self-assign major projects, not content with things being “good-enough” I tended to push and over-extend myself to improve the state of Slide Engineering. Sometimes these projects would fail and I would get uncomfortably close to burning myself out, other times, such as the migration from Subversion to Git, turned out to be incredibly rewarding and netted noticable improvements in our workflow as a company.
One of my very first major projects was upgrading our installation of Cheetah from 1.0 to 2.0, at the time I vigorously hated Cheetah. My distain of the templating system stemmed from using a three year old version (that sucked to begin with) and our usage of Cheetah which bordered between “hackish” and “vomitable.” At this point in Slide’s history, the growth of the Facebook applications meant there was going to be far less focus on the Slide.com codebase which is where some of the more egregious Cheetah code lived; worth noting that I never “officially” worked on the Slide.com codebase. When I successfully convinced Jeremiah and KB that it was worth my time and some of their time to upgrade to Cheetah 2.0 which offered a number of improvements that we could make use of, I still held some pretty vigorous hatred towards Cheetah. My attitude was simple though, temporary pain on my part would alleviate pain inflicted on the rest of the engineering team further down the line. Thanks to fantastic QA by Ruben and Sunil, the Cheetah upgrade went down relatively issue free, things were looking fine in production and everybody went back to their regularly scheduled work.
Months went by without me thinking of Cheetah too much until late 2008, Slide continued to write front-end code using Cheetah and developers continued to grumble about it. Frustrated by the lack of development on the project, I did the unthinkable, I started fixing it. Over the Christmas break, I used git-cvsimport(1) to create a git repository from the Cheetah CVS repo hosted with SourceForge and I started applying patches that had circulated on the mailing list. By mid-March I had a number of changes and improvements in my fork of Cheetah and I released “Community Cheetah”. Without project administrator privileges on SourceForge, I didn’t have much of a choice but to publish a fork on GitHub. Eventually I was able to get a hold of Tavis Rudd, the original author of Cheetah who had no problem allowing me to become the maintainer of Cheetah proper, in a matter of months I had gone from hating Cheetah to fulfilling the oft touted saying “it’s open source, fix it!” What was I thinking.
Thanks in part to git and GitHub’s collaborative/distributed development model patches started to come in and the Cheetah community for all intents and purposes “woke up.” Over the course of the past year, Cheetah has seen an amazing number of improvements, bugfixes and releases. Cheetah now properly supports unicode throughout the system, supports @staticmethod and @classmethod decorators, supports use with Django and now supports Windows as a “first-class citizen”. While I committed the majority of the fixes to Cheetah, five other developers contributed fixes:
In 2008, Cheetah saw 7 commits and 0 releases, while 2009 brought 342 commits and 10 releases; something I’m particularly proud of. Unforunately since I’ve left Slide, I no longer use Cheetah in a professional context but I still find it tremendously useful for some of my personal projects.
I am looking forward to what 2010 will bring for the Cheetah project, which started in mid-2001 and has seen continued development since thanks to a number of contributors over the years.
Earlier while talking to Ryan I decided I’d try to coin the term “pyrage” referring to some frustrations I was having with some Python packages. The notion of “pyrage” can extend to anything from a constant irritation to a pure “WTF were you thinking!” kind of moment.
Not one to pass up a good opportunity to bitch publicly, I’ll elaborate on some of my favorite sources of “pyrage”, starting with generic exceptions. While at Slide, one of the better practices I picked up from Dave was the use of specifically typed exceptions to specific errors. In effect:
class Connection(object):
## Pretend this object has "stuff"
pass
class InvalidConnectionError(Exception):
pass
class ConnectionConfigurationError(Exception):
pass
def configureConnection(conn):
if not isinstance(conn, Connection):
raise InvalidConnectionError('configureConnection requires a Connection object')
if conn.connected:
raise ConnectionConfigurationError('Connection (%s) is already connected' % conn)
## etc </code>
Django, for example, is pretty stacked with generic exceptions, using builtin exceptions like ValueError and AttributeError for a myriad of different kinds of exceptions. urllib2’s HTTPError is good example as well, overloading a large number
of HTTP errors into one exception leaving a developer to catch them all, and check the code, a la:
try:
urllib2.urlopen('http://some/url')
except urllib2.HTTPError, e:
if e.code == 503:
## Handle 503's special
pass
else:
raise
Argh. pyrage!
A weekend or two ago I sat down and created an Amazon wishlist of stuff that I would like to purchase. In the past I’ve found Amazon wishlists an ideal way of saying thanks to a number of folks in the open source community whose work I value, at one point I didn’t desire any new gadgets and went through a large number of gift cards buying goodies for some of my favorite hackers on Amazon.
On the off chance you appreciate pictures of Fatso, Cheetah templates, @hudsonci on Twitter, py-yajl, Urlenco.de, tweets on a regular basis, watching me talk in a brown corduroy jacket or just plain have money to burn, below is my very own Amazon wishlist.
Get out there and buy (me) stuff, our economy depends on it.
A few weeks ago when I started working more on py-yajl I discovered that there are actually a number of Python developers who work(ed) with JSON parsers in the Bay Area. As luck would have it Lloyd, the author of Yajl, is going to be in town next weekend for Add-on-Con, time to meet up and have some beers! I’ve invited the authors of: simplejson, jsonlib, jsonlib2 and a few other Python hackers in the Bay Area that I know of.
If you’re in San Francisco this Saturday (Dec. 12th) and loves you some Python, don’t hesitate to swing by 21st Amendment around 1pm-ish to join us!
Update: Some of this information is out of date. Instead of pushing to the
gerrit master branch I recommend setting up
“replication” and using the
“Submit” button inside of the “Review” page.
While working at Slide, I became enamored with the concept of cooperative threads (coroutines) and the in-house library built around greenlet to implement coroutines for Python. As an engineer on the “server team” I had the joy of working in a coro-environment on a daily basis but now that I’m “out” I’ve had to find an alternative library to give me coroutines: eventlet. Interestingly enough, eventlet shares common ancestry with Slide’s internal coroutine implementation like two different species separated thousands of years ago by continental drift (a story for another day).
A few weekends ago, I had a coroutine itch to scratch one afternoon: an eventlet-based image server for applying transforms/filters/etc. After playing around for a couple hours “PILServ” started to come together. One of the key features I wanted to have in my little image server project was the ability to not only pass the server a URL of an image instead of a local path but also to “chain” transforms in a jQuery-esque style. Using segments of the URL as arguments, a user can arbitrarily chain arguments into PILServ, i.e.:
http://localhost:8080/flip/filter(blur)/rotate(45)/resize(64x64)/<url to an image>
At the end of the evening I spent on PILServ, I had something going that likely shows off more of the skills of PIL rather than eventlet itself but I still think it’s neat. Below is a sample of some images transformed by PILServ running locally:
