Lazily loading attributes.
Posted:I found myself talking to Jason today about the virtues of
hasattr()in Python and "abusing" the dynamic nature of the language which reminded me of some lazy-loading code I wrote a while back. In February I found the need to have portions of the logic behind one of our web applications fetch data once per-request. The nature of the web applications we're building on top of the MySpace, Hi5 and Facebook platforms require some level of network data-access (traditionally via REST-like APIs). This breaks our data access model into the following tiers:
Working with network-centric data resources is difficult in any scenario (desktop, mobile, web) but the particularly difficult thing about network data access in the mod_python-driven request model is that it will be synchronous (mod_python doesn't support "asynchronous pages" like ASP.NET does). This means every REST call to Facebook, for example, is going to block execution of the request handler until the REST request to Facebook's API tier completes.
There is also a network hit (albeit minor) for accessing cached data or data stored in databases. The general idea is that we'll need to have some level of data resident in memory through-out a request that can differ widely from request-to-request.
def request_handler(self, *args, **kwargs):
fb_uid = kwargs.get('fb_sig_user')
print "Fetching the name for %s" % fb_uid
name = facebook.users.getInfo(uid=fb_uid)
### Continue generating the page...
Lazy loading in PythonTo help avoid unnecessary database access or network access I wrote a bit of class-sugar to make this a bit easier and more fail-proof:
This makes developing with network-centric web applications a bit easier, for example, if I have a "friends" lazily-loading attribute off the base "FacebookRequest" class, all developers writing code subclassing FacebookRequest can simply refer to
LazyProgrammer allows for lazily-loaded attributes on the subclasses
of this object. In order to enable lazily-loaded attributes define
"_X_attr_init()" for the attribute "obj.X"
def __getattr__(self, name):
rc = object.__getattribute__(self, '_%s_attr_init')()
setattr(self, name, rc)
self.friendsand feel confident they aren't incurring unnecessary bandwidth hits, and the friends-list fetching code is located in once spot. If one-per-request starts to become too resource intensive as well, it'd be trivial to override the
_friends_attr_init()method to hit a caching server instead of the REST servers first, without needing to change any code "downstream."
setattr()) but you can "abuse" properties in a manner similar to the C# singleton pattern, to get the desired effect:
Admittedly I don't find myself writing Facebook/MySpace/Hi5 applications these days on top of ASP.NET so I cannot say I actually use the class above in production, but conceptually it makes sense.
public class LazySharp
#region "Lazy Members"
_names = null;
#region "Lazy Properties"
if (this._names == null)
this._names = this.SomeExpensiveCall();
Lazy loading attributes I find useful in the more hodge-podge situations, where code and feature-sets have both grown organically over time, they're not for everybody but I figured I'd share anyways.