This weekend I finally got around to building a little Rust “full text search
engine” based on the educational post written by my
Scribd colleague Bart:
titled Building a full-text search engine in 150 lines of Python
code.
Bart did a great job writing an accessible post which introduced some common
search concepts using Python, my objective wasn’t necessarily to write
something faster or better but to use the exercise as Rust practice. My day
job is no longer writing code so the opportunity for a problem with fixed scope
which would work out my Rust muscles was too good to pass up. In this post I
want to share some things which I’ve learned in the process of duplicating
Bart’s work.
I am have always been a fan of podcasts, but have never had really any good way
to share the interesting things I am listening to. A couple weeks ago I struck
upon an idea that seems so bafflingly simple in retrospect: I could just host
my own podcast feed.
As a comprehensive operating system FreeBSD never ceases to impress me, the
recent iterations of FreeBSD
Jails as an example have been an
absolute joy to use. The introduction of the
vnet(9)
network subsystem has completely transformed what I had originally thought
about software-defined networking. My previous exposure to the concept of
software-defined
networking was
through both OpenStack and Docker, two very
different approaches to the broad domain of “SDN”. FreeBSD’s vnet system has
resonated most strongly with me and has allowed me some measure of success in
deploying real production-grade virtualized networks.
Bridging data types between the database and a programming language is such a
foundational feature of most database-backed applications that many developers
overlook it, until it doesn’t work. For many of my Rust-based applications I
have been enjoying sqlx which strikes
the right balance between “too close to the database”, working with raw cursors
and buckets of bytes, and “too close to the programming language”, magic object
relational mappings. It reminds me a lot of what I wanted Ruby Object
Mapper to be back when it was called “data mapper.” sqlx
can do many things, but it’s not a silver bullet and it errs on the side of
“less magic” in many cases, which leaves the developer to deal with some
trade-offs. Recently I found myself with just such a trade-off: mapping a Uuid such that I could do IN queries.
WebTorrent
is one of the most novel uses of some modern browser technologies that I have
recently learned about. Using WebRTC is able to implement a truly
peer-to-peer data transport on top of support offered by existing browsers. I
came across WebTorrent when I was doing some research on what potential future
options might exist for more scalable distribution of free and open source
libraries and applications. In this post, I want to share some thoughts and
observations I jotted down while considering WebTorrent.
I am a big fan of the open web and although I have enjoyed
Twitter the platform has regressed in dramatic
form and function since I first adopted it. I remember Twitter actively
avoided building a walled garden with fantastic APIs and RSS feeds open to
the public. Much of the popularity of the platform hinged upon the incredible
third party applications and integrations developers like me built in the first
five-ish years of its existence. Over time the site has strayed from open APIs
and standards, and while I still enjoy Twitter, I want some more flexibility
which is why you can now subscribe to my microblog with any
RSS-capable client.
The quantity of things I need to keep track of or be responsible for has
exploded in the past few years, so much so that I have had to really focus on
organizing my “personal knowledgebase.” When I originally tried to spend some
time improving my information management system, I found numerous different
services offering to improve my productivity and to help me keep track of
everything. Invariably many of these tools were web apps. In order to quickly and
productively work with information, a <textarea/> in a web page is the choice
of just about last resort. I recently revisited
Vimwiki and have been quite satisfied both by
my productivity boost and the benefits that come with having raw
text to work with. The best benefit: easy synchronization of notes with Nextcloud.
Every now and again I’ll encounter a silly problem, fix it, forget about it,
and then later run into the exact same problem again. Today’s example is a
confusing error I encountered when reverse-proxying a
Tide application with
Nginx. In the Tide application, I was greeted with an ever-so-descriptive error:
Nginx is a fantastic web server and reverse proxy to use
with Let’s Encrypt, but when dealing with multiple
domains it can be a bit tedious to configure. I have been moving services into
more FreeBSD jails as I alluded to in my previous
post, among them the
general Nginx proxy jail which I have serving my HTTP-based services. Using
Let’s Encrypt for TLS, I found myself declaring multiple server blocks inside
my virtual host configurations to handle the apex domain (e.g.
dotdotvote.com), the www subdomain, and vanity domains (e.g.
dotdot.vote). With the help Membear and MTecknology in the #nginx
channel on Freenode, I was able to refactor multiple
largely redundant server blocks into one.
In the modern era of highly connected software, I have been trying to “offline”
as many of my personal services as I can. The ideal scenario being a service
running in an environment where it cannot reach other nodes on the network, or
in some cases even route back to the public internet. To accomplish this I have
been working with FreeBSD
jails a
quite a bit, creating a service per-jail in hopes of achieving high levels of
isolation between them. This approach has a pretty notable problem at first
glance: if you need to install software from remote sources in the jail, how do
you keep it “offline”?