Lazily loading attributes.

I found myself talking to Jason today about the virtues of getattr(), setattr(), and hasattr() in Python and "abusing" the dynamic nature of the language which reminded me of some lazy-loading code I wrote a while back. In February I found the need to have portions of the logic behind one of our web applications fetch data once per-request. The nature of the web applications we're building on top of the MySpace, Hi5 and Facebook platforms require some level of network data-access (traditionally via REST-like APIs). This breaks our data access model into the following tiers: Dia FTW

Working with network-centric data resources is difficult in any scenario (desktop, mobile, web) but the particularly difficult thing about network data access in the mod_python-driven request model is that it will be synchronous (mod_python doesn't support "asynchronous pages" like ASP.NET does). This means every REST call to Facebook, for example, is going to block execution of the request handler until the REST request to Facebook's API tier completes.




 def request_handler(self, *args, **kwargs):


    fb_uid = kwargs.get('fb_sig_user')


    print "Fetching the name for %s" % fb_uid


    print time.time()


    name = facebook.users.getInfo(uid=fb_uid) 


    ### WAIT-WAIT-WAIT-WAIT-WAIT


    print time.time()      


    ### Continue generating the page...

There is also a network hit (albeit minor) for accessing cached data or data stored in databases. The general idea is that we'll need to have some level of data resident in memory through-out a request that can differ widely from request-to-request.

Lazy loading in Python

To help avoid unnecessary database access or network access I wrote a bit of class-sugar to make this a bit easier and more fail-proof:




class LazyProgrammer(object):


    ''' 


        LazyProgrammer allows for lazily-loaded attributes on the subclasses 


        of this object. In order to enable lazily-loaded attributes define


        "_X_attr_init()" for the attribute "obj.X"


    '''


    def __getattr__(self, name):


        rc = object.__getattribute__(self, '_%s_attr_init')()


        setattr(self, name, rc) 


        return rc

This makes developing with network-centric web applications a bit easier, for example, if I have a "friends" lazily-loading attribute off the base "FacebookRequest" class, all developers writing code subclassing FacebookRequest can simply refer to self.friends and feel confident they aren't incurring unnecessary bandwidth hits, and the friends-list fetching code is located in once spot. If one-per-request starts to become too resource intensive as well, it'd be trivial to override the _friends_attr_init() method to hit a caching server instead of the REST servers first, without needing to change any code "downstream."

Lazy loading in C#

Since C# is not a dynamically-typed language like Python or JavaScript, you can't implement lazily-loaded attributes in the same fashion (calling something like setattr()) but you can "abuse" properties in a manner similar to the C# singleton pattern, to get the desired effect:




using System;


using System.Collections.Generic;





public class LazySharp


{


	#region "Lazy Members"


	private Dictionary _names = null;


	#endregion





	#region "Lazy Properties"


	public Dictionary Names


	{


		get {


			if (this._names == null)


				this._names = this.SomeExpensiveCall();


			return this._names;


		}


	}


	#endregion


}

Admittedly I don't find myself writing Facebook/MySpace/Hi5 applications these days on top of ASP.NET so I cannot say I actually use the class above in production, but conceptually it makes sense.

Lazy loading attributes I find useful in the more hodge-podge situations, where code and feature-sets have both grown organically over time, they're not for everybody but I figured I'd share anyways.