Today I fell into a rabbit hole around user and process permissions and had to learn how sudo actually worked. For the program I was working on I set out to figure out how to perform something akin to a “user swap” when launching subprocesses. On its face it seems simple enough, my program runs with user id 1000 and I wish to shunt child processes over to run as another user id. sudo can do it, so why can’t I? “For reasons” is the answer.

For some recent web application projects like dotdotvote and riverbank I reached for Tide and built them in Rust. I have a lot of reasons for liking Tide, not the least of which is that it is reminiscient of Sinatra in the Ruby ecosystem. Perusing the internet today I noticed this really great blog series by Javier Viola which will walk you through the full process of developing a real application with Tide.

This weekend I learned some unfamiliar behaviors with the way Handlebars handles nested variable scopes. I typically use Handlebars via the handlebars-rust implementation which aims to maintain nearly one to one compatibility with the JavaScript implementation. They have block scope helpers such as #each and #with, both of which create an inner scope for variable resolution. Unfortunately, the syntax can be quite unintuitive for accessing outer scope once in those nested scopes.

Vim can be used as an IDE of sorts for Rust by using a variety of plugins, that don’t always play nicely together. A few weeks ago while I was hacking on some Rust and these errors started showing up inline. Blaring red text basically as soon as I was done typing half-finished thoughts.

“Serverless” is sometimes referred to as “cgi-bin” which isn’t entirely fair as it’s somewhere between cgi-bin and FastCGI. Somewhere along the way both faded from memory. While goofing off last weekend wondered to myself: is FastCGI still useful? Unlike the classic cgi-bin approach where a script or program was executed for each individual request, FastCGI is a binary protocol which allows for longer lived processes serving multiple requests. It continues to be used in the PHP community but seems to have largely fallen out of favor. Nonetheless I decided to tinker a little bit with FastCGI in Rust.

Last week as part Delta Hack 2021 I hosted a live coding stream to introduce new-comers to Rust or delta-rs to both Rust and contributing to the project. I have been enjoying writing code live on Twitch which is typically more akin to pairing with the fourth wall. This session was different in that I was intentionally start at a more foundational level: getting started with Rust itself.

Investing the time to learn FreeBSD jails has led to a dramatic increase in the number of services I run in my “home lab.” Jails, which I have written about before, are effectively a lightweight quasi-virtualization technique which I use to create multiple software-defined networks to segment workloads. Jails have allowed me to change my “home lab” dramatically, allowing me to reduce the number of machines and increase hardware utilization. For now, the days of stacking machines, dangling Raspberry Pis, or hiding laptops on the shelf are all gone. Almost all my needs have been consolidated into a single FreeBSD machine running on a 4 year old used workstation.

Ransomware is the most significant and dangerous evolution of computer-based crime I have seen, and it’s going to get worse. Ransomware attacks have compromised oil pipelines, hospitals, and beef. While they’re nothing new over the past two years, targets have become increasingly high-profile and the adverse impacts of ransomware have similarly become more dire. Based on my read of the reports and incident reviews, these attacks seem to largely be affecting physical infrastructure assets: workstations, servers sitting in closets, and small-scale data center operations. Given this trend, it might be easy conclude that running in AWS, Azure, or Google Cloud offers some level of protection. I strongly doubt it, and I think ransomware is about to get worse.

Everything went fine. Social media is awash with anecdotes about getting vaccinated, side effects, and opinions about other people becoming vaccinated. I recently crossed the waiting threshold after my last dose and felt I should share my experiences as well.

I want smart appliances, not the current commonly understood “smart” Internet-of-Things (IoT) appliances, but smart in actually useful ways. A couple years ago I had solar panels installed. The “smart” I want is pretty simple: I want devices that know about surplus energy. Devices which have lower power idle modes that can kick into more productive usage when solar power is bountiful. Generally IoT devices seem to be almost everything I don’t want. I don’t need devices that listen in on my conversations, track my data, and open up security holes in my home network. In this post I want to outline what I do want from “smart” appliances in my home.

Deutsch ist nicht meine Muttersprache. Trotzdem höre ich seit Jahren deutschsprachige Podcasts. In den letzten paar Monaten habe ich Jung und Naiv gefunden und kann es euch nur empfehlen. Der Podcast-Feed ist eine Mischung aus Bundespressekonferenzen, der Hans Jessen Show, und Interviews mit verschiedenen Leute in der oder aus der Politik. Viele von diesen Interviews sind wahnsinnig lang und ich habe selten ein paar Stunden Zeit, wo ich die Kopfhörer rein stecken kann um ein einziges Gespräch am Stück zu hören. Deswegen habe ich mir in den vergangenen Wochen in kleinen Teilen dieses Interview mit Dieter Nuhr angehört.

This week is Data and AI Summit at which I am speaking, but I’m already looking past it at Delta Hack: 2021. My interest in using Delta Lake has been steadily increasing since we started delta-rs and I’m finding more and more use-cases for our recently introduced writer support. Delta Hack (#deltahack) will be an opportunity to really dive into some more complex problems that cannot be solved in an hour or two after work.

I generally default to using JSON for data interchange but there are still a myriad of formats of XML out there, for which I have created the xmltojson crate. I originally wrote this one night to help me get an XML dataset into JSON so that I could use PostgreSQL’s JSONB column type, but I only recently published it to crates.io since it may be useful for others.

Working over SSH on a big beefy remote machine is a great way to extend the life of any laptop, but presents challenges when developing network-based services. Fortunately OpenSSH has a “port forwarding” feature which has been around for a number of years. Port forwarding allows the user to tunnel a port from the remote machine back to their local machine, in essence allowing you to access a remote service bound to port 8000 on your own localhost:8000. When I first learned about this, I would fiddle around with my ssh invocations or hardcode a list of potential ports forwarded in my ~/.ssh/config. If I was working on a new service that needed a port not yet forwarded, I would disconnect, add it to the list of ports in my config file, and then reconnect. That was until my pal nibz (nibalizer) showed me how to dynamically add port forwards to an already existing session.

Creating Pre-signed S3 URLs in Rust took me a little more brainpower than I had anticipated, so I thought I would share how to generate them using Rusoto. Pre-signed URLs allow the creation of purpose built URLs for fetching or uploading objects to S3, and can be especially useful when granting access to S3 objects to mobile or web clients. In my use-case, I wanted the clients of my web service to be able to access some specific objects from a bucket.

Investing in the development of delta-rs is one of the longer shots I have taken recently and with my upcoming Data and AI Summit talk on the subject, I wanted to share some back story on the project. As I have mentioned before Delta Lake is a key component of Scribd’s data platform. We selected Delta Lake for many reasons, including that it is a open and vendor neutral project. The power of Delta Lake has opened up countless opportunities for data and for the past year I have seen the potential for many more.

Entering into the data platform space with a lot of experience in more traditional production operations is a lot of fun, especially when you ask questions like “what if X goes horribly wrong?” My favorite scenario to consider is: “how much damage could one accidentally cause with our existing policies and controls?” At Scribd we have made Delta Lake a cornerstone of our data platform, and as such I’ve spent a lot of time thinking about what could go wrong and how we would defend against it.

Data partitioning is one of the principles to utilize when developing large data sets, but do you know what that actually means for the storage format? I didn’t! Many “big data” storage systems such as HDFS, S3, and Azure Data Lake Storage all are effectively a file system. This past year or so, I’ve become much more familiar with Delta Lake and kind of just assumed that data partitioning was something being done at the transaction log level. Turns out I guessed wrong.

This weekend I finally got around to building a little Rust “full text search engine” based on the educational post written by my Scribd colleague Bart: titled Building a full-text search engine in 150 lines of Python code. Bart did a great job writing an accessible post which introduced some common search concepts using Python, my objective wasn’t necessarily to write something faster or better but to use the exercise as Rust practice. My day job is no longer writing code so the opportunity for a problem with fixed scope which would work out my Rust muscles was too good to pass up. In this post I want to share some things which I’ve learned in the process of duplicating Bart’s work.

I am have always been a fan of podcasts, but have never had really any good way to share the interesting things I am listening to. A couple weeks ago I struck upon an idea that seems so bafflingly simple in retrospect: I could just host my own podcast feed.