The discourse around “AI” has finally started to shift to a very key question: who are the winners and the losers? The fundamental thesis of shoving Large Language Models (LLMs) into everything is that it will increase productivity. Setting aside the question on whether those productivity gains are happening or even possible, the question of who reaps those rewards is starting to become a larger concern by software developers I follow.

I deploy on Friday. I have heard the arguments to the contrary and I still continue to deploy on Friday. The end of the week is as good as any other to deploy! Why wait another two days to realize the value or return-on-investment from the work I have queued up? I feel confident making this judgement call for two important reasons: I am confident in the success of the changes being deployed and I am willing to take ownership for when I’m wrong.

Writing has been a part of my work for a long time, it helps me think and more importantly it helps me share ideas with other developers. Recently a tremendous amount of my time has been spent writing internal design documents, blog posts, and other materials. By the time it has come to personal blogging my words all been spent.

I spend a lot of time looking at and optimizing costs, kind of for anything. I do it as part of my day job, I do it as part of my night job, and of course for my home life as well. Spreadsheets on spreadsheets tracking and analyzing, finding inefficiencies and opportunities for investment. In 2024 an opportunity presented itself to invest in a next generation home battery which I can now unequivocally say is a good investment.

Some recent changes to the primary server in my office left the fans running louder than previously. I do have some noise canceling headphones but I still don’t want to hear the sound of jet fans from the other room, with a little bit of free time I was able to nail down and correct the behavior using ipmitool(1) and magic bytes.

Whenever a system comes offline I try to always make multiple updates at once. When I recently received some new mounting hardware for one of my big FreeBSD servers, I decided to plonk an idle PCI-e device into the chassis while it was out of the rack.

A colleague once told me about their boss whose office door decorated by a single 8x11 piece of paper with “speed wins” scrawled upon it. I didn’t even work for them and I find that motivating. I think about it a lot, particularly when I’m waiting for Rust builds to complete. Speed wins, every second counts, and “why is this so fscking slow!” all run through my mind as units are compiled and linked.

I have recently been on a quest to get more speed from my Rust development environment, and today’s “why is this so fscking slow!” culprit is rust-analyzer.A larger project like delta-rs benefits from IDE-like machinery to help work across a sprawling codebase written in Python and Rust. rust-analyzer helps greatly with that, but it comes with a speed penalty.

Today I launched a new rework of buoyantdata.com thanks to the work of a designer I found in the fediverse! The original “design” of the site was something I had cobbled together with a Jekyll theme I originally ported to Cobalt, but it was always lacking.

Purchasing new hardware to run Linux on used to be so perilous that I would only buy hardware which was at least 6 months out of date. The ecosystem has changed dramatically such that when my Dell XPS experienced a chassis failure, I went to a big box store and came home with a Lenovo Slim 7. I immediately installed a Linux distribution on it and started setting up my new portable workstation without even considering hardware support.

Nothing quite feels like “I made it!” like being published. Which is why I am thrilled to share that Delta Lake: The Definitive Guide is available for purchase, and I kind of helped! I wanted to share a little bit about how my contributions (Chapter 6!) came about, because my entrance into the Delta Lake ecosystem was about as unplanned as my authorship of part of this wonderful book.

This year has been so jam packed full of activities that I forgot to share some videos from Data and AI Summit 2024 this past summer! The annual conference hosted by Databricks has become one of my favorites to meet with other Delta Lake users and developers to discuss the future of large-scale data ingestion and processing. This year however, I overdid it a little bit.

The staggering complexity of modern software makes it impossible for us to truly understand what is happening while our code runs, but when it fails there is always something we can learn. At the beginning of my career we, the industry, generally understood that programs were getting complex. Without hesitation we made things more complex, more distributed, and somehow more coupled. Failure is a “learning opportunity”, and those opportunities are in abundance.

I asked a large language model this question:

I have always been a technology scavenger, picking up cheap or disused computers for parts or tinkering. Last year when I picked up a full-height server cabinet, a new world of rack-mountable junk finally became possible! One lucky Craigslist find ended up being an older 2U IBM xSeries server with 8 drive sleds that was described as “sorta working” by the owner, who was shedding some extra stuff for his move across the country. I accepted the challenge, forked over a few Jacksons, and brought the machine home.

Large language models (LLMs) seem to only be good at two things: summarizing text and making up bullshit. The idea that a general purpose LLM is going to herald a new age of software development efficiency is misleading in most cases bordering on malicious. While there are a number of other recommendations or predictive machine learning models which can improve software development efficiency, LLMs propensity to generate bullshit undermines trust in a way that makes me question their validity at baseline as a software development tool.

I have had the good fortune this year to help a number of organizations develop and deploy native data applications in Python and Rust using a project I helped found: delta-rs. At a high level delta-rs is a Rust implementation of the Delta Lake protocol which offers ACID-like transactions for data lake use-cases. One of the big areas of my focus has been in evaluating and improving performance in highly concurrent runtime environments on AWS.

For a long time after I rebuilt my jails host, I could not restart a certain number of jails due to an “interface already exists” error. For the life of me I could not make sense of it, The services running in the jails were useful but not required so I put off tinkering with it. I thought that I would magically stumble into the solution in my sleep or something equally silly.

My home office has grown in size and for the first time in decades I believe I have a surplus of compute power at my disposal. These computational resources are not in the form of some big beefy machine but a number of smaller machines all tied together by a gigabit network hiding away in a server cabinet. The big problem has become how to effectively utilize all that computational power, I turned to Nomad to orchestrate arbitrary workloads on static and ephemeral (netboot) machines. As the title would suggest, it’s almost good but it still falls frustratingly short for my use-cases.

Dependency management in the Rust ecosystem is fairly mature from my perspective, with crates.io, Cargo, and some cultural norms around semantic versions, I feel safer with dependencies in Rust than I have in previous toolchains. It’s far from perfect however, and this question helps highlight one of the quirks of how Rust dependency management does or does not work, depending on your perspective: