Welcome to my blog where I write about software
development, cycling, and other random nonsense. This is not
the only place I write, you can find more words I typed on the Buoyant Data blog, Scribd tech blog, and GitHub.
mono
Ok, Twitterer almost had a valid excuse, but this is just bloody unnecessary. I wrote a C# (Mono) news bot for twitter last night out of boreom while waiting for a client to finish writing the webservices needed for my project. The Twitterbot is based very loosely on Mario Menti's perl source code, in that I took the tinyurl idea (and that's about it). I wrote the bot with the concept in mind of using one bot to manage all the feeds at once, which does have a slightly undesirable affect of posting the updates for all the feeds at once, but with a bit of tweaking that can be lessened.
The Twitterbot makes use of RSS.NET for all its incoming feed parsing, but uses the standard System.Net.WebRequest class for posting to both twitter and retrieving the proper tinyurl link. The basic structure of the bot is simple, whenever it grabs new items it'll check that feed's last (stored internally) items to prevent duplication of twitter-posts, and then will shorten the title, generate the tinyurl link and finally post the tiny message to twitter. One of the issues I discovered with Google News is that they randomize the story providers for any given story, i.e. a they might push out a story to their feeds about China's shooting down of a satellite but each time the bot updates that feed may return a different title for the story from a different news organization. In order to prevent flooding, the bot currently has a twitter-posting maximum of two per iteration, which combined with the update interval (30 minutes) helps cut down on both duplicate stories, but also spamming the living hell out of twitter.
The file format that the bot reads feeds and twitter information from is also simple:
<feeds>
<!-- & -> & -->
<feed name="Google News" url="http://news.google.com/news?ned=us&topic=h&output=rss" twitter="user" password="pass"/>
</feeds>
I think I am going to release this as open source in the very near future but I want to check with the twitter guys first to take any steps necessary to prevent spamming their goofy little service. I really think my bot would help organizations use twitter as a minimalistic content delivery platform (market-speak!) if they already spit out content in standard RSS formats elsewhere, but I don't want to step on any toes.
A nice and neat little Twitterbot, all in less than 300 lines of code :)
mono
After some recent frustrations using Thread.Abort() and Thread.Join() in C# I adopted another means of cancelling a thread. In the previous iteration of this code, it was hanging on the following code:
myThread.Abort();
if (!myThread.Join(5000)) {
Console.WriteLine("Failed to join secondary thread");
}
Under certain conditions (such as invoking unmanaged code from C#), the function being executed by the thread "myThread" can cause the Thread.Join(uint) to block indefinitely regardless of what timeout value (in microseconds) is passed as an argument to Join().
Instead of using the Thread class' functions for cancelling a thread, it is much more reliable to use a stay-alive boolean instead in your thread. In my code, the thread serves as a basic runloop that iterates a couple of times a second, so I added the following code:
private static readonly object threadLock = new object();
private static volatile bool threadStayAlive = false;
public static void ThreadFunc() {
//...
lock (threadLock) {
if (!threadStayAlive)
return;
}
//...
}
Elsewhere in the code, instead of calling both Thread.Abort() and Thread.Join(uint), I set threadStayAlive to false, and call Thread.Join(uint) to ensure that the thread is properly cancelled before continuing execution. I've tested this for about an hour or so trying to replicate the infinite-blocking that would occur with Thread.Abort() and Thread.Thread.Join(uint) and I've not been able to lock up my application yet with the stay-alive boolean. Checking the boolean periodically inside the thread function will allow the the code to appropriately clean up after itself instead of relying on Thread.Abort() which may interrupt whatever the thread is executing at any given time leaving your application in an inconsistent state. Catching a ThreadAbortException inside the thread function can help prevent inconsistent thread states, but I've found it's not preventing any amount of thread lock-ups like the stay-alive will.
Regardless of the framework, threading is never something to be undertaken lightly; it is absolutely essential to know what your thread should be doing so you don't end up with race conditions or corrupted objects. Thread handling in C# is relatively straightforward, and certainly not as frustrating as pthreads, but still can have the same general threading related bugs.
mono
I neglected to mention it here, but I released some quick code to allow me to write text in OpenGL and C#. It's called FTGL# and hopefully someone besides me will have a use for it.
cocoa
I've been hacking around with more webservices-based applications recently (Twitterer for example) and I've also reminded myself what an utter pain in the ass they can be in Cocoa. With twitter, they make available both JSON and XML-based webservices, which is good since they use basic HTTP authentication for their user-specific webservices (i.e. everything but retrieving the public timeline). The XML-based webservices are more or less straight-forward to hack up with Cocoa, all one really needs to do is write a parser (NSXMLDocument) and then make use of the URL loading classes (NSURLConnection, NSMutableURLRequest) to retrieve and process, or POST data. This method of interacting with "webservices" (more wgetting than anything else) is rudimentary at best, and in my personal opinion, isn't as robust as SOAP webservices are. It does however, work. Regardless of the framework, programming language, or geo-orbital location of the moon, they will work because all you're doing is making an HTTP GET and then parsing the results.
The benefit of SOAP webservices however, is the almost "literal" translation of objects encapsulated into SOAP messages, into runtime objects. In effect, if I have a Person object defined in my WSDL and similarly defined in my application (with a good SOAP framework) the objects should be encoded and decoded appropriately when passed via SOAP. The real power of SOAP can be realized when used with ASP.NET (ick) webservices and .NET clients (see ASP.NET - Mono) where you can relatively quickly and easily build and deploy a services-oriented application. That's .NET/Mono however, my work is dealing with NuSOAP and Cocoa, a less than ideal mix.
To start hashing out a Cocoa-webservices-oriented application, your first stop should be at a local pub, brewery, or mayor's office for a good round of intoxication, only after every tissue in your body is soaked with fine casket-aged whiskey will you be prepared to embark on your journey. It's usually best to start with a complete WSDL, you can then use /Developer/Tools/WSMakeStubs to generate what resembles some Objective-C that you can flesh out to some extent to provide an intermediary layer between your sane Cocoa code and the actual SOAP method calls. Cocoa makes SOAP painful. The stub code revolves around one magical "object" WSMethodInvocationRef. Think of WSMethodInvocationRef like you would a sadistic elf that only pops out of his little elf home to kick you in the groin before shouting "TRY AGAIN IDIOT" and scurrying away, there's a bit of magic involved, but mostly pain. Since the invocations will just return a generic "id" type, the only way to really be sure what your webservice invocation returns is to either call [NSObject respondsToSelector:(SEL)] or just trust that the webservice you called will return what you expect, whether it be an NSString, NSDictionary, NSArray, or NSNumber (NSNumber is what's returned in place of numbers and booleans, it's WebServiceCore's cheap way of boxing those primitives). TRY AGAIN IDIOT.
Something else to note is that things you might expect to be able to use, such as basic HTTP authentication are absolutely non-existent in the magic WebServicesCore black-box. With a URL loading-based webservice (JSON, XML) you can just use the delegate method:
- (void)connection:(NSURLConnection *)connection didReceiveAuthenticationChallenge:(NSURLAuthenticationChallenge *)challenge
And authenticate from there, popping around through the various NSURLConnection delegate methods. I have also experimented with updating the SOAP endpoint to include a URL something like the following:
http://user:password@bleepsoft.com/some/stupid/url
While one might assume the HTTP subsystem hidden behind that magical WSMethodInvocationRef would handle this appropriately, and translate it to the basic HTTP authentication tokens, it just doesn't work. TRY AGAIN IDIOT.
The alternatives are few and far between if you absolutely need to use SOAP webservices for a certain Cocoa project, I am working on converting a framework that Jonathan Wight wrote to use the URL loading classes in Cocoa, but other than that learn to love WSMethodInvocationRef or plead with your web developers to rewrite their existing webservices with REST, JSON, etc. WebServicesCore is an antiquated pain in the ass, and probably hasn't been updated since Steve Jobs was at NeXT.
cocoa
A little birdie chirped into my email wondering what the radar number for my previous gripings about WebServicesCore, and I figured that for all one of the Apple employees that actually read my inane bullshit, behold:
Radar #4945073 ("WebServicesCore lacks support for basic HTTP authentication")
Let's all cross our fingers and hope for a much needed update to WebServicesCore. If all else fails, we can tap our shoes together and wish for a new web services API right?
Most of what's attracted me to the site has been their very simple API, which has allowed me to get an application called "Twitterer" up and running in less than a few hours (1, 2, 3). I've not yet released it (no icon!) but i've been using it for the past two days and am quite pleased with it (if I may say so myself). In fact, if you have read this before, you might notice a recent trend in posts about webservices...ahem.
Twitter allows the masses to finally answer the question "what are you doing?"
To which the masses can reply in a most resounding fashion "nothin really."
media
Now that I've finally figured out where I know Zach Hale from (via Colin it seems) I wanted to link to his mixtape-blog. It''s an idea I've thought about before, but never taken the initiative to do, so golf claps are in order for Zach. Anyways, they're some great mixes with lots of artists you probably haven't been introduced to just yet, check it out over on poorleno.com
mono
I recently posted this: Customer Brief: Autonomic Software, Inc to the bleep consulting site. While I cannot disclose too much about how "we have done what we have done" I can say that I used the Mono runtime to allow for them to deploy their software onto both Mac OS X and Linux.
When deploying at a client site recently, I used Package Maker to build a meta-package to install Mono.framework and their software simultaneously. Using the power of launchd(8) the installer installed a LaunchDaemon job file (.plist) and started the job with launchctl all from within the installer. On the Linux side of things rsync(1) to sync things into the appropriate places and then fire off the init.d script.
Overall it's been quite an interesting experience bringing Mono into the corporate world; it's almost like you're telling somebody something that's too good to be true: "wait, I can run this under .NET on Windows, and Mono on everything else? Really?" Unfortunately because of my NDA, I can't disclose too much about the actual project, but it's certainly proved that not only is Mono "enterprise ready," it is probably a better choice to write and deploy software than most other cross-platform frameworks out there.
media
Many Mac users around the world are having problems using video chat on iChat AV. The problem is routers or so I hear, every time I try to video chat with some ( sometimes video chat works) i get an error message concerning a communication error -8. When typed into google, there are several people with my same problem, why can't iChat love everyone equally, does it just hate mexicans. Why can't I hit Video Chat and it just works, I can just hope the new OS can fix this or someone can help me.
software development
I've been experiencing a kernel panic for the past couple weeks, sporadically, but I've finally come up with a reliable set of reproduction steps (for my set up anyways). I have a nagging feeling it has something to do with the Parallels Kernel Extensions (specifically the pseudo-networking devices).
The basics of my kernel panic are as follows, for purposes of demonstration, let's pretend Mac[0] is the machine you feel like kernel panicing, and Mac[1] is some other machine sitting around causing trouble:
On Mac[0] enable "Personal File Sharing" (i.e. turn on Apple File Sharing)
Using Mac[1], mount an AFP share from Mac[0].
Transfer a large file (ISO, DMG, pr0n.mp4) from Mac[0] to Mac[1].
I've been able to reproduce this at the login screen for Mac[0], all the way up to full interactivity (running iTunes, Xcode, etc). In my office, Mac[0] is a 20" intel iMac, whereas Mac[1] is a 12" PowerBook G4. If I had more machines to test with, I'm sure I'd be able to reproduce it there as well. I find it very unlikely that the Apple drivers are kernel panicing my box (see crash logs at end of post), as Apple's IOKit drivers seem to be very solid, so I'm guessing that it is related to the Parallels kernel extensions (.kext). A brief look at kextstat(8) returns this:
intellian:~ tyler$ kextstat | grep parallels
79 0 0x8d4000 0x5000 0x4000 com.parallels.kext.ConnectUSB (2.5.0) <33 11 6 5 4 3>
91 0 0x8d9000 0x6000 0x5000 com.parallels.kext.Pvsnet (2.2) <5 4 3 2>
101 0 0x6bd000 0x14000 0x13000 com.parallels.kext.hypervisor (2.2) <11 6 5 4 3 2>
102 0 0x9ed000 0xa000 0x9000 com.parallels.kext.vmmain (2.2) <11 6 5 4 3 2>
103 0 0x4a10d000 0x3000 0x2000 com.parallels.kext.Pvsvnic (2.2) <36 4 3>
Regardless of whether or not Parallels is running, to ensure I don't come off as a Parallels-basher (even if I really am), VMWare leaves kernel extensions loaded when VMWare Fusion isn't running as well:
intellian:~ tyler$ kextstat | grep vmware
95 0 0x48fe5000 0x1b000 0x1a000 com.vmware.kext.vmmon (1.0.0d1) <11 5 4 3 2>
99 0 0x48d7f000 0x5000 0x4000 com.vmware.kext.vmioplug (1.0.0d1) <33 19 5 4 3>
100 0 0x48b74000 0x5000 0x4000 com.vmware.kext.vmnet (1.0.0d1) <5 4 3 2>
Anyways, back on topic. Given the inherently cryptic crash logs that a kernel panic will leave behind (if any), it's hard to truly tell what is causing the panic. As much as I like to fantasize about becoming an über 1337 kernel haxx0r, I simply haven't the time to whip out a firewire cable, and use Mac[1] as a debugging console to reproduce and crash my main workstation (Mac[0]).
As a software developer however, I'm a bit annoyed that these virtualization applications (Parallels, VMWare) are leaving KEXTs loaded into kernel space even when they're not running, leaving the door wide open to crashes like this one. Unfortunately, a kernel is only as strong as it's weakest link/kext, if one of the KEXTs crash in the spectacular fashion in which they normally do, they can bring down an entire system, possibly leaving a lone developer in central Texas with no other options than to crack open a beer shortly after lunch.
academia
Found an interesting article (through slashdot) on how we should be educating engineering students. The bird's eye view is that students should come out of school with the ability to continue learning rather than some specific skillset. The slashdot discussion and the article really cover mostly different ground on the subject, with the comments on /. debating the pros and cons of teaching engineers as thinkers or trades-people and the original article focusing more on how the education of thinkers can be done. Of course this isn't limited to engineering, any really good Comp. Sci. program runs the same give-and-take between teaching students to program and teaching them how to solve problems.
I don't really think there's necessarily a right answer in any of this. At this point in time it seems to me like the market for Engineers as workers with a certain skill set is notably present (and I'm going to entirely avoid the debate about whether that is right or wrong), while there is undeniable value in being able to apply knowledge outside of a specific skill set (I'm also going to avoid debate about whether this can be taught).
My own career, thus far, has been a case study in why problem solving is more important than specific skills. I am a graduate of the Computer Engineering Co-op program at the University of Alberta. Of my five terms of "Work Experience" I spent the first three doing PLC programming, circuit diagrams (in AutoCAD), and specification and ordering of parts for industrial control systems. My last two were spent doing data acquisition programming in Visual Basic in a direct precursor to my current position, which is a mix of Windows programming, putting together custom experiments, and heat transfer research. I'm also preparing to begin a M.Sc. degree in Mechanical Engineering.
The common theme in all of this is that none of my day-to-day work heavily involves anything I learned as a skill in my undergraduate degree, though I admit I haven't followed an entirely traditional career progression. I heard a Faculty member whom I have a tremendous amount of respect for answer the question "What qualities do you look for when you're looking for graduate students?" with something like "I like to see how they react to problems outside of their comfort zone." Of course he elaborated more on that, and the point was that it much more important for the student to be able to figure out something than to recite something. He also noted that you can get very different solutions to a problem from someone who solves it using a well-trained skill and someone who solves it with ingenuity, reasoning, and research.
Getting back to the article, I really think the term "meta-skills" is fantastic. While you learn skill in a program (and that's inevitable, most concepts in applied science and engineering are taught through some usable skill) the important thing isn't necessarily the skill or even the concept (which set the "trajectory" talked about in the article), but the ability to turn learning skills and concepts into a skill in itself. I think the article essentially skips the most important part of making that happen for students, telling them directly what they should be gaining. All the hand-on activity in small groups in the world isn't going to help them make that leap if they just see it as another lab assignment in a course that's only marginally (if that) related to the job they hope to have when they graduate.
As for the slashdot discussion, while interesting, I think it misses the point by focusing on theory vs. application. As many comments correctly point out, both theoretical understanding and practical ability are important, but in the context of the articles theories and concepts are simply another skill learned. A high level mathematics theory is every bit as narrow as a single programming language if the person who knows that skill doesn't have the ability to work outside their "comfort zone". Versatility is key. But like I said, the discussion is pretty interesting. I've noticed quite afewinterestingcomments. As a note on that last comment, it is fairly similar to (albeit longer) the engineering program here. All engineers here take a common first year, specialize in years 2-4 (while still taking cross-discipline courses), then have to practice as an E.I.T. for years (a residency of sorts) before being able apply to become a Professional Engineer (which is a term whose use is legally restricted).
opinion
So today I began what I hope is a great experience at Texas A&M University, but as I walk through the freezing cold weather and enter my first class I realize I have entered a different world. This has nothing to do with the amout of work or toughieness of the classes ( liked my bushism there... heh heh), its getting used to saying Howdy. Howdy, who the dues says howdy...., people dont say hi here, they say howdy.... what is with this, plus depending on how you say it howdy can sound very (insert brokeback mountain joke here). Well lets just hope things work out, i may just revolt and just say, hi.
cocoa
Through some twist of fate, I was interviewed on Cocoa Radio. I think I managed to maintain some semblance of competence throughout the interview until some random fellow sat really close to Blake and I and threw my concentration.
It was pretty fun, Blake and I hung out most of the week (I even rescued him from the airport in blue lightning on thursday) so doing the interview was a bit weird, as we talk regularly.
And thus begins my long hard trek into the public consciousness; I'll be super-famous, just like Sting, you'll see.
opinion
On my recent trip back from road trip, I couldn't help but appreciate Chris Rock's commentary on "being white" when he said "I always viewed being white like always having five bucks in your pocket."
Sitting in line at a border patrol station just east of El Paso, I watched a hispanic family in a van get inspected relatively thoroughly, and couldn't help but giggle at the "inspection" that awaited me:
Are you a U.S. citizen? arrrrrrr ...Is that a yes or a no? yes. (-_-)
Have a nice day.
cocoa
As some of you (both rather) may have noticed, the bleepsoft.com homepage is a bit different these days.
In the very near future bleep will be releasing an application called "Emission" that, besides the cool icon, will certainly change how I work with the wide array of people across the globe that I deal with on a daily basis.
As a lot of the network core is still unwritten (yes, it's very network-ey), so I'm not going to disclose too many details, but check out the site, Fernando Lins has done a fantastic job on the icon.
opinion
I suppose I will really never be able to fulfill my childhood dream of working for Apple Computer, Inc. Bummer. The changing of the name from Apple Computer, Inc. to Apple, Inc. is certainly symbolic, but I think carries far more weight in the industry than anybody is really giving credence.
Over five years ago, Apple told us that the Mac was becoming/is the digital media hub. They told us that the Mac was going to be the center of our digital life, and like a dropping a penny into an empty well, nothing happened. A glance of the industry as a whole is almost sickening in terms of the void that just didn't get filled. Apple should not have needed to create the iPod. Apple should not have needed to create the iPhone. To paraphrase what Steve Jobs said to a reporter from CNBC "we create products we want to [need/]use." Apple is slowly learning what John D. Rockefeller learned over a hundred years ago, vertical mergers will make you obscenely rich, or to put it more succinctly in terms of Apple's situation, you cannot trust the rest of the industry "figure it out." The Nomad Jukebox is a decent device, but it doesn't integrate into the rest of my "digital life" like the iPod did when it came out. The Motorola Razr, or the Blackberry are all nice devices, they sleek, they have appeal, but they just suck. The software is miserable, and they don't integrate like the iPhone does/will, so they're doomed to play the second-fiddle that Microsoft is finding itself playing with regards to the Zune.
Moving from Apple Computer to just Apple is a weighty change at least in terms of the company mindset that should have the rest of the industry scared sh#$less. Apple is moving away from just computers to something most Mac OS X users have become familiar with, the experience which they have excelled and building with iPhoto, iMovie, iTunes, and the peripherals to go with them. While Microsoft did beat Apple into the living room with the original XBox (I know a number of people that use their first gen XBoxes as both DVD players and media hubs), appleTV has some catching up to do, but has something Microsoft doesn't (besides the religious fanbase), the "experience" necessary to get every joe and smoe type to not only desire an appleTV, but actually use it.
The iPhone is a good example of Apple's power to look at an existing market and completely change the "level of play" required to compete in that market, and do it in such a way that everybody from Main Street to Wall Street is now paying attention.
That's officially my one, requisite, Macworld 2007 blog posting. I'll finish with this image, courtesy of David Young's blog.
media
Yesterday afternoon at the Flash Fire Facility we got a visit from Rick Mercer. Since it won't be airing for a week or so (and thankfully I won't be in it) I won't yet comment on the stuff we did, but it was pretty cool. It was a lot more fun than some of the other TV spots we've done (local news, Daily Planet). Rick was pretty cool, and needless to say very funny. I'd invite him over for a BBQ anyday.
miscellaneous
Almost exactly 333 miles into this trip, and I find myself sitting at a texas rest area (free wireless!). I stopped here about 50% because of the free wireless, and about 50% because I really wanted to see what it felt like to go to the bathroom in 30 degree temperatures.
Unfortunately, thus far the coolest things I've seen are, something on fire on the side of the road, and the moon. West texas isn't really all that exciting. I better get back to cruising though, El Paso is something like 260 miles from here.
I don't plan on making regular blogging stops, this one just happens to coincide with a more biological deadline.
