One foggy morning a few weeks ago, I received a disk usage alert courtesy of the Jenkins project’s infrastructure on-call rotation. In every infrastructure ever, disk usage alerts seem to be the most common alert to crop up, something somewhere is not properly cleaning up after itself. This time, the alert was from our own Jenkins environment. The logging filesystem wasn’t the problem, the filesystem hosting JENKINS_HOME was perilously close to running out of space. The local time, about 6:20 in the morning, and yours truly was quietly furious at the back of a bus headed into San Francisco for the day.

One of the first pain points many organizations endure when scaling Jenkins is the rapid accumulation of artifacts on their master’s filesystem. Artifacts are typically built packages such as .jar, .tar.gz, or .img files, which are useful to persist after a Pipeline Run has completed for later review as necessary. The problem that manifests over time, is quite predictable, archived artifacts incur significant disk usage on the master’s filesystem and the network traffic necessary to store and serve the artifacts becomes a non-trivial problem for the availability of the Jenkins master.

In this post I would like to share a handy little workaround for returning to Google Hangouts, despite Google Meet. Having narrowly escaped working at Google via acquisitions twice, I have stood by and watched as the Ad Words money-pipe funded rewrite after boondoggle after rewrite. When Google announced “Google Meet” earlier this year as an “enterprise-friendly version” of Google Hangouts, I was annoyed, but not surprised.

After learning how to build my first terrible website, in ye olden days, perhaps the second useful thing I ever really learned was to run multiple websites on a single server using Apache VirtualHosts. The novelty of being able to run more than one application on a server was among the earliest things I recall being excited about. Fast forward to the very different deployment environments we have available today, and I find myself excited about the same basic kinds of things. Today I thought I would share how one can implement a concept similar to Apache’s VirtualHosts across Namespaces in Kubernetes.

This research was funded by CloudBees as part of my work in the CTO’s Office with the vague guideline of “ask interesting questions and then answer them.” It does not represent any specific product direction by CloudBees and was performed with Jenkins, rather than CloudBees products, and Kubernetes 1.8.1 on Azure.

Months ago Microsoft announced Azure Container Instances (ACI), which allow for rapidly provisioning containers “in the cloud.” When they were first announced, I played around with them for a bit, before realizing that the pricing for running a container “full-time” was almost 3x what it would cost to deploy that container on an equitable Standard A0 virtual machine. Since then however, Azure has added support for a “Never” restart policy, which opens the door for using Azure Container Instances for arbitrary task execution.

A couple weeks ago I boarded a plane at the always-adorable Charles M. Schulz Sonoma County Airport en route to Seattle to participate in a Microsoft Azure OpenDev Event. Thanks to my pal Ken Thompson, who recently joined Microsoft as a product marketing manager for their Open Source DevOps team, I was invited to talk about all things Jenkins with a dash of Azure.

2018 will be the sixth year for the Testing/Automation dev room at FOSDEM. This room is about creating better software through a focus on testing and automation at all layers of the stack. From creating libraries and end-user applications all the way down to packaging, distribution and deployment. Testing and automation is not isolated to a single toolchain, language or platform, there is much to learn and share regardless of background!

The traffic on the Bay Bridge connecting San Francisco to Oakland is one of the most congested routes of traffic in all of Northern California. Somehow it gets even worse on Saturday and Sunday. One weekend, a few years ago, I was driving my wife and some of the women from her soccer team back to Berkeley, from a game in San Francisco’s Golden Gate Park. On the east side of the bridge, before inching onto I-580N, I was pretty pissed off, and half-joking half-frustrated shook back-and-forth at the steering wheel “GAHHHHHHHHHHHH.” The woman sitting behind me, who was certainly the “funny one” of the group, put her hand on my arm and gently said “Tyler, this is your reality now.”

The insanely strong gusts of wind would not stop clattering the tin roof panels over the back patio. Begrudgingly, I awoke, dressed, and tried to secure the roof panels before the neighbors got too ornery. Stepping up the ladder, I noticed an orange glow north of the house. Just after midnight, I had not heard any sirens, I jumped into the car on the assumption that one of those houses by the park was burning and had not yet been reported.

Quite possibly my favorite part about working on open source infrastructure is that I can share as much as I want! Contrary to corporate infrastructures, where most of it is closed source and hidden away, open source project infrastructure is by its very nature open. From a pedagogic standpoint, this allows me to teach others without needing to create contrived demonstrations or examples, but we can instead refer to the real code being used to deploy the Jenkins project.

Over the past decade two things have become increasingly clear: practically every modern industry is part of “the software industry,” in one way or another, and “the software industry” is rife with shortcuts and technical debt. Working in an Operations or Systems Administration capacity provides a front-row seat to many of these dysfunctional behaviors. But it’s not just sysadmins, many developers are also called to engage in or allow: half-baked product launches, poor-quality code deployments, or subpar patch lifecycle management.

“For vegetables, your best bet is to get some drip lines ‘cause you don’t want to get water on the leaves” said the helpful employee at a local farm supply store. I have heard this “advice” numerous times over the past few years, and it gets a little deeper under my skin each time I hear it. Like most advice handed out in this fashion, there’s a kernel of truth hiding somewhere behind layers of indirection associated with such old wive’s tales.

I have tremendous difficulty with decommissioning electronics. I only recently stopped using my Galaxy Nexus, an almost five year old cell phone. Earlier this year I recycled a 32-bit x86-based Thinkpad T41, only because its overheating issues made it impractical to continue running workloads. And up until today, the lowest powered device actively running a Unix in my office, was a 266Mhz AMD Geode-based Soekris.

Practically every professional developer can name a great, and a terrible, manager they have worked with in the past. Good Engineering Managers are kind of like the bass line in a song, you might not notice them when they’re there, but something will definitely sound wrong if they’re absent. For one reason or another, I have somehow ended up leading a team or acting as an Engineering Manager at each of the four companies I have worked for over the past decade. As time has progressed, I have become increasingly aware of “management” as a skill, rather than some intristic talent. A skill which can be practiced, honed, and improved upon.

To say that I’m an expert gardener would be an extraordinary stretching of the truth; capable, yes, expert, not even close. While I tend to focus on what crops fail outright, or produce lower-than-desired yields, my neighbors and some of the folks I know online seem to be impressed with my results.

Yesterday I participated in a very fun and productive Docker Hack Day, wherein a few folks (myself included) spent the day hacking on porting Docker to FreeBSD. After which, I had a nice relaxing beer (or two) on my boat/train rides home and enjoyed one of my favorite past-times: shit-posting on Twitter.

Jenkins World 2017 is nearly upon us, and I must admit that I am not actually presenting a talk at the event. I haven’t been speaking as much this year, compared to 2016, since I have been very busy building the Evangelism team at CloudBees. Three new hires and one transfer later and it’s already time for Jenkins World! Eep!

Azure has started to grow on me. I could imagine myself, a couple years ago, lamenting their poor non-Windows support, clumsy user interfaces (and APIs), and overall “beta dog” performance. Fortunately for cloud users like myself, Microsoft is hungry, and has heavily invested in Azure, becoming very competitive in a very short amount of time. One aspect of Azure I didn’t expect to like however, was their web UI. If you’re already familiar with the AWS web dashboard, you’re probably accustomed to…low expectations, so just about any web interface designed later than 2008 would be an improvement in comparision. Fortunately for me (and you if you use Azure), the Azure “blade UI” was designed more recently, and was clearly created by a team of thoughtful UI designers rather than engineers.

I consider myself one of the world’s foremost experts in terrible ways to use Jenkins, partially because my brain is awash with awful ideas, but also because I have been around the project long enough to see hundreds of different “clever” (ab)uses of Jenkins. Today I thought I would share something I came up with a few weeks ago which, to date, might be one of my more deplorable creations.