The web is getting faster but feeling slower, something which I have
complained about loudly on Twitter but now
some folks have put together data to back it
up. The web is
simultaneously a medium to transmit documents (e.g. an article) and an
application platform (e.g. Jira). Anecdotally it seems to me like far too many
publishers think of the web only as the latter. There are more and more
websites which require significant JavaScript or other multimedia resources to
render what ends up being a few paragraphs of text. If you don’t believe me,
just visit the website for your local television news station with
NoScript turned on. In my own way, I have been
resisting this push by keeping this blog as barebones as neceessary to present
the content you’re reading now. On a whim, I recently took this idea a little
bit further by deploying a Gopher site (viewable over HTTP via a proxy).
You never really appreciate disk space until it’s all gone. This morning I
noticed that my laptop had come perilously close to exhausting all its
available disk space. Oops! Normally I would prune some Docker images with
docker system prune -f but this time around I couldn’t blame Docker, the
wasted space was due to cargo,
critical part of the Rust development toolchain.
I stumbled across a crate which implemented string parsing that I
wished to incorporate into some of my serde.rs
deserialization code. Unfortunately the crate in question,
cron does not implement the
#[derive(Deserialize)] macro on its Schedule, so I needed to fiddle with
one of serde’s “field attributes” in order to move forward: deserialize_with.
The Real-time Data Platform is one
of the fun things we have been building at Scribd since I joined in 2019. Last
month I was fortunate enough to share some of our approach in a presentation at
Spark and AI Summit titled: “The revolution will be streamed.” At a high level,
what I had branded the “Real-time Data Platform” is really: Apache
Kafka, Apache Airflow,
Structured streaming with Apache Spark, and a
smattering of microservices to help shuffle data around. All sitting on top of
Delta Lake which acts as an incredibly versatile and useful
storage layer for the platform.
Delivering a simple payload to IRC is an ideal use-case for Function-based
cloud computing. Last year when GitHub discontinued their service for pushing
notifications into IRC channels, I had a perfect situation to couch-hack with a
library I had recently discovered: Azure Functions for
Rust
The async/await keywords in modern Rust make building high-throughput
daemons pretty straightforward, but as I learned that doesn’t necessarily mean
“easy.” Last month on the Scribd tech blog wrote
about a daemon named hotdog which we
deployed into production: Ingesting production logs with
Rust. In
this post, I would like to write about some of the technical challenges I
encountered getting the performance tuned for this
async-std based Rust application.
Much of the information I read during the day, not counting e-mail, comes from
my RSS reader: Newsboat. Whenever I see an interesting
blog post on Twitter or elsewhere, I habitually subscribe the author’s RSS
feed. I recently stumbled across an interesting RSS feed which wasn’t served
over HTTP, leading me to wonder: how can I subscribe?
I discovered today that since version 8.1, Vim apparently supports spawning a
terminal from within the Vim editor. This is a handy little feature that could
make life easier for checking documentation, running tests, and so on.
I have been writing a lot ofRust lately
and as a consequence I have had to get a lot better at writing unit tests. As
if testing along weren’t tricky enough, almost everything I am writing takes
advantage of async/await and is running on top of the
async-std runtime.
Suddengly managing a remote engineering team may seem like a daunting situation,
one which many people are suddently finding themselves in as tech companies
institute sudden “work-from-home” policies in response to the Corona virus.
If you find yourself in this situation don’t
panic. Managing remotely is not
significantly different than managing in-person, and your already existing good
management and communication habits will greatly help. Nonetheless, I thought I
might be able to help newly remote managers by hosting an open office
hours,
with the first experimental session yesterday in the afternoon PST.