I have described the general mission of the team as follows:

The Ruby Infrastructure team will help Scribd adopt major ecosystem improvements such as Sorbet, new Rails versions, and interpreter releases. Measure and optimize performance across the thousands of requests per second served by Ruby at Scribd. Create libraries that encapsulate common Ruby application patterns and approaches. Open high quality pull requests to improve upstream projects like Sidekiq, Rails, and Ruby itself.

Ruby at Scribd is serious business. We run one of the largest Rails deployments on the internet (hi GitHub!) and need more focused effort on scaling it from a technology and organization standpoint. The Ruby ecosystem has also matured greatly over the past 10 years and every couple of months there are new improvements which Scribd can adopt.

The Ruby Infrastructure team is intended to be the group of people which make sure that all our Ruby and Rails applications are performing well, scaling, and are easy to develop and deploy.

To give you a better idea of what this team will do, here are some of the projects which I have in mind:

Simplify with Aurora

We have over 7TB of online relational data which, for historical reasons, is spread across a number of master-replica clusters. Migrating these databases to, and adopting RDS/Aurora looks very promising. The advertised read-performance and dataset storage scalability may allow us to consolidate the database infrastructure and allow us to delete swaths of complex database magic in the applications.

All that code for switching up database connections or delegating reads to read-replicas may disappear behind the curtains of Aurora. We certainly need to do some investigation here, but this is a pristine example of that grey area where the Ruby Infrastructure team will excel.

Web Socketin’

Enabling Web Sockets on smaller applications is trivially easy these days. For larger sites like Scribd.com a large number of variables need to be considered: do we terminate sockets in Rails? What will an incredibly high connection count do to our existing app infrastructure? How will application developers write code which supports Web Sockets and their existing request flows? Does our app host capacity plan change dramatically as a result?

Seemingly mundane requests like “can we enable web sockets?” from application developers or product managers, at the scale of billions of requests per month, can have far reaching implications that the Ruby Infrastructure team is poised perfectly to answer.

Efficient Host Sizing

Our current infrastructure is at times over-provisioned. The specifics I won’t get into in this post, but there are a lot of low-hanging fruit in understanding our existing application footprints and then sizing our infrastructure around them appropriately. Whether we’re talking about understanding or improving our memory utilization, or becoming more elastic around CPU utilization. Building an overall understanding of how these Ruby applications perform, how to tune them, and how to structure their resource usage is going to be one of the frequently re-evaluated projects for Ruby Infrastructure.

There are a myriad of other interesting projects which will crop up once a couple Ruby Infrastructure Engineers join the company. Like the other teams in Platform Engineering, this team will be entirely remote which means we can hire the most qualified people we’re able to find, from nearly anywhere.

I’m excited to see the upstream pull requests, RailsConf presentations, and blog posts that we’re going to be able to share once we start solving problems together!

The session consists of two segments, historically with one focusing on the capabilities of the language itself and the second segment focusing on the community built around the scripting language.

Instead of choosing to create a presentation, I took the road less traveled and created a series of live coding demos to demonstrate the utility of JRuby.

The collection of demos that I created can be found in this demo repository. For all of my demos I used Pry as a live JRuby interpreter to load and execute Java and Ruby code on the fly, which I hope made for a compelling presentation.

The following are the demos worth mentioning here:

ascii table from jar

In this demo I had grabbed some random Java code from the internet to print ascii-only formatted tables, and wrote some Ruby glue to make use of it. Copying and pasting this code into a running Pry session and you’ll get a nice table drawn in your console.

[16] pry(main)> puts make_table(headers, mails)
+-----------------------------+--------------------------+
|           Subject           |           From           |
+-----------------------------+--------------------------+
|         NOT SPAM WE PROMISE | sirspamsalot@hotmail.com |
| Important work related info |     yourboss@example.com |
|       Don't forget the milk |         spouse@family.io |
+-----------------------------+--------------------------+
=> nil
[17] pry(main)>

anonymous classes

This demo demonstrates how JRuby merges the use of anonymous classes in Java with Ruby’s concept of blocks. Instead of creating an anonymous class that implements the Runnable interface, we can just use pass a block of code to be executed inside of a java.lang.Thread.

“turtles”

This might be my favorite demo because it demonstrates one of the more compelling (to me) features of JRuby, it’s embeddability. In the “turtles” demo, I’ve got an entirely embedded and separate Ruby runtime environment, nested within an already existing Ruby runtime.

The result is that I can create isolated Ruby environments within the same JVM, which opens the door to all kinds of interesting sandboxing applications.

[44] pry(main)> require './turtles/first'
=> true
[45] pry(main)> puts "Our original JRuby has version: #{JavaOne::VERSION}".colorize(:magenta)
Our original JRuby has version: 1
=> nil
[46] pry(main)> puts

=> nil
[47] pry(main)> # Now let's load a the same module with a different version inside the runtime environment
[48] pry(main)> evaler.eval(runtime, "require './turtles/second'"); nil
=> nil
[49] pry(main)> evaler.eval(runtime, "puts \"Our embedded JRuby has version: \#{JavaOne::VERSION}\".colorize(:green)"); nil
Our embedded JRuby has version: 2
=> nil
[50] pry(main)>

jruby-gradle

The final demo that I gave is a little self-promiting, in that I demonstrated the capabilities of the jruby-gradle project. A project which allows you to use the Gradle tool for managing projects, obviating the need for Bundler, JBundler, Warbler and RVM.

During the demo I showed the build.gradle file which references a Ruby gem and demonstrated how to build a self-contained, self-executing .jar file with the tool:

% ./gradlew shadowJar

Then executed it:

% java -jar build/libs/jruby-gradle-example-all.jar
REQUIRED RELATIVE
Hello from JRuby built with Gradle!
%

That is a jar file, running Java and Ruby code all from within the file, pretty spiffy if you ask me!

Predictably, JRuby didn’t win the Script Bowl, unpredictably Clojure did. The event was enjoyable, but for me my favorite part was creating the demos and honing my pitch for JRuby, which I’m sure I’ll have the opportunity to reuse in the future.

My colleague Ian Smith and I will be presenting a talk at JRubyConf EU 2014 titled:

Building a scalable messaging fabric with JRuby and Storm.

The abstract can be found below, but since this is my personal blog I’d like to expand a bit more on why we’re giving the talk.

Last year my team and I deployed ActiveMQ in order to provide an inter-service messaging bus between a new product we were developing and some legacy infrastructure at Lookout. Alongside ActiveMQ we deployed a number of different worker daemons to consume and dispatch messages. These were JRuby-based processes that polled a STOMP connection for new messages and then performed some amount of “work.”

In the process of building out all of this infrastructure we learned a couple of important lessons:

• ActiveMQ scales like MySQL. That is to say it kind of doesn’t scale, the most effective way to scale out a number of ActiveMQ brokers is to perform some amount of application-layer “sharding” across message brokers. We did not do ActiveMQ any favors in attempting to scale it out effectively, so minus one point us.
• Many of our problems became less of shoveling work from a single app server into a work-queue, where tools like Sidekiq excel, and more about complex message routing and processing, which Storm was built specifically to solve for.
• If you provide a good stream of events, everybody, everybody will want to take part in it. One of the unexpected results of deploying ActiveMQ and some messaging facilities was that nearly every team in the company then wanted to take part in the event streams. This wouldn’t have been a problem if we had planned to become the “messaging team” or even had a roadmap to scale out the deployment and managing of messaging infrastructure with ActiveMQ+STOMP. Alas, we did not, and quickly ran into scaling challenges both technically and organizationally.

In our talk, Ian and I will discuss some of these lessons coming from our “old messaging infrastructure” along with some of the important lessons that we’ve learned moving to the newer system. Naturally, neither Storm nor Kafka are silver bullets for scalability, but I believe that they’re the right path to moving towards a broader Lookout-wide messaging fabric.

As previously mentioned, this talk will be at JRubyConf EU which will be held August 1st in Potsdam, Germany. We hope to see you there!

Abstract

As Lookout has grown the number of backend Ruby services the need for reliable, asynchronous service-to-service messaging has gone from “nice to have” to “absolute requirement.”

Our first attempts included some names you may be familiar with: ActiveMQ, Resque, Redis, and Sidekiq. As our infrastructure grew, we found we were reinventing many of the concepts “Storm,” an open source real-time computation/stream-processing system, was specifically designed to handle such as message routing, durability, streaming from multiple inputs, delivering to multiple outputs, etc.

In this talk we’ll cover how we built a scalable service-to-service messaging fabric on top of Storm and Kafka, with JRuby playing a starring role.

A couple of the posts I’ve written which may interest you are:

• Integration Testing with Foreman

  At Lookout we find ourselves building more and more APIs and backend services these days. Naturally we would like to be certain that everything will work fine and dandy once it has been deployed. The reality of building out a service-oriented architecture is that you not only have to expect failure to happen, you have to plan and test for it.

• Continuous Deployment for Ruby Gems

  With the above workflow and these conventions in place, we’ve reaped a couple of benefits, the most obvious one has been the severely reduced time it takes for new gems to be created and incorporated into production systems. At a higher level, we’ve gained more confidence in our gems with this emphasis on testing, traceability and code review baked into the process from the start.

The blog is still fairly young, and we’re getting more developers used to writing blog posts for it. You can subscribe to the atom feed here or just follow @LookoutEng on Twitter.

The impossible has happened. Not only have I written a plugin for Jenkins, I’ve released it. An event I’ve long avoided has finally come to pass, mostly thanks to the fantastic Ruby plugin development support developed largley by Charles Lowell.

The Vagrant plugin is still fairly new, and “beta” tested. Thus far I’ve modeled it after the fantastic Android Emulator plugin in that it will bring up a Vagrant VM for the duration of the job, and then destroy it afterwards.

As you can see, the Vagrant plugin adds a couple new build steps to the configure page. One thing to note is that the provisioning is operated as a separate step, instead of bundled in with the boot of the Vagrant machine. This is purposeful, to allow any additional set up of the environment prior to running Puppet or Chef.

The plugin does of course expect that you have a functional VirtualBox environment on the slave you’re running the job on, if you don’t, Vagrant will explode with a fantastic stack-trace and probably burn your house down.

In the near future

I’m thinking about bundling vagrant-snap with the plugin to allow fancy snapshotting support, but I’m not sure of a good definite usecase for that just yet.

If there’s anything you’d like to see, please file a request on the GitHub issues page.

You can find the plugin:

• in your Jenkins “Manage Plugins” page :)
• on GitHub
• on the Jenkins wiki

Enjoy!

NOTE: Be sure to grab version 0.1.2 or later!

For a rather long time, I’ve used clang on my Linux machine as the default value of the $CC environment variable. I’ve simply found it to be a faster and more friendly compiler to work with than the default gcc.

When I first came to Ruby, I tried building Ruby 1.8.7 with clang and it failed utterly. After bitching on Twitter, as I’m prone to doing, Joe Damato pointed out that Ruby 1.8 isn’t really valid C. Instead it is a hodge podge of macros, C code, assembly and tears.

I later found out that Ruby 1.9 seems to cope with being built with clang properly, so I built a couple of 1.9 variants with RVM for my personal projects.

Everything was all fine and good until I tried building a native extension for a 1.9 Ruby:

make
compiling em.cpp
cc1plus: warning: command line option ‘-Wdeclaration-after-statement’ is valid for C/ObjC but not for C++ [enabled by default]
cc1plus: error: unrecognized command line option ‘-Wshorten-64-to-32’
cc1plus: warning: command line option ‘-Wimplicit-function-declaration’ is valid for C/ObjC but not for C++ [enabled by default]
make: *** [em.o] Error 1

I lost hours trying to figure out what the hell was going on here. I was convinced for hours that EventMachine (the gem being built here) had a bug in it. Then I was convinced that RVM had a bug in it, but I finally landed on Ruby’s build system itself.

I should point out that the warning shorten-64-to-32 only exists in patched versions of compilers shipped with Mac OS X. It certainly doesn’t exist on any compiler on Linux, so where the hell did this flag come from?

Somehow, and arriving at this “somehow” took me a number of miserable hours, the difference between using clang(1) and gcc(1) to build Ruby resulted in a single nonsense CXXFLAGS value which didn’t come back to bite me until I tried to build a native extension.

How the shit does Ruby get built!?

In the post I mention how installations of anything lower than Ruby 1.9.2 need special treatment in order to make ri usable on a day-to-day basis.

For my side-projects, I’ve been using Ruby 1.9.2, so imagine my surprise when I ran the following command:

-> % ri Random
Nothing known about Random
-> %

In order to manage swapping between Ruby 1.8 for work and Ruby 1.9 for play, I’ve been using the tool rvm to manage local installations of multiple rubies.

As [bad] luck would have it, rvm install ruby-1.9.2 by default doesn’t generate any of the standard library documentation, which will leave you with a marginally useful ri as shown above.

The remedy for this is quite simple, re-run the installation command with the --docs flag:

-> % rvm install ruby-1.9.2 --docs
  ... # configuring
  ... # installing
  Attempting to generate ri documentation...
  Install of ruby-1.9.2-p290 - #complete 
-> %

Ta-da! Proper local documentation for the standard library.

-> % ri Random
Random < Object

# --- etc

-> %

The only downside I’ve found, thus far for Cucumber comes down to the use of the term “cukes” to describe Cucumber-based tests.

The word “cuke” or “cukes” simply has an unpleasant, wretched feeling to it. It sounds like something unpleasant to get on your shoes. I really do not like this word.

As such, I will henceforth refer to Cucumber-based tests as “gherkins” instead.

I look forward to writing plenty of posts in the future about creating high quality gherkins for browser testing and cross-system testing, stay tuned!

(I sure have been complaining about names a lot lately haven’t I?)

I’ve slightly changed my opinions since then, mostly because I can no longer afford to avoid JOINs. Sometimes harsh reality is the best cure for young idealism.

Anywho, what was my point again about joins?

Ah yes, joins in ActiveRecord. While ActiveRecord is a very solid ORM gem for Ruby, it is not without it’s quirks. Most of which come down to “magic” behavior or defaults you’ve assumed incorrectly. As much as I would like to pontificate on the odd method_missing magic ActiveRecord developers seem to love, this post is more about the latter.

In general, JOINs involve two tables, JOINing together the tables typically for a nested query of some form or fashion. For an example, let’s use this make believe query:

    SELECT `guests`.* FROM `guests` JOIN `plates` \
        ON plates.guest_id = guests.id WHERE \
        (guests.favorite = 'sushi' OR plates.content = 'sushi');

There are five basic JOINs in SQL land, and in this context they mean:

• LEFT JOIN: Find all the rows in guests, even if they don’t have a corresponding plates row, then apply sub-query.
• RIGHT JOIN: Find all rows in plates, even if they don’t have a corresponding guests row, then apply sub-query.
• INNER JOIN: Find rows only if guests has a corresponding plates row.
• STRAIGHT JOIN: Find your way to the DBA’s desk and receive your punishment.
• NATURAL JOIN: I’ll be honest, I don’t fully understand what a natural join does.

Back in ActiveRecord land, let’s imagine I want to create a named_scope, which in Rails 3 has been deprecated in favor of just scope, I’d write code something like this:

    class Guest < ActiveRecord::Base
      has_one :plate
      # ...
      named_scope :with_food, lambda { |food| {
                    :conditions => ["guests.favorite = ? OR plates.content = ?", food, food],
                    :joins => :plate
                }}
    end

Underneath the hood, ActiveRecord generates a query almost exactly like the one above with one subtle difference. “JOIN” is instead an “INNER JOIN” which means my query will not return a Guest object for any guests who do not already have a plate object.

The “solution” is to use a LEFT JOIN, which is unfortunately rather gnarly. There may be a better way to perform alternate JOINs in ActiveRecord, but I don’t yet of one:

    class Guest < ActiveRecord::Base
      has_one :plate
      # ...
      named_scope :with_food, lambda { |food| {
                    :conditions => ["guests.favorite = ? OR plates.content = ?", food, food],
                    :joins => "LEFT JOIN `plates` ON plates.guest_id = guests.id"
                }}
    end

Basically if you want to use anything other than a simple INNER JOIN, you’ve got to enter it in yourself. At a certain point ActiveRecord throws up its hands and says “Look, buddy, I have no idea what you’re trying to do here. Enter your own damn SQL.”

Probably offensive

I basically spent about 30 minutes typing borderline or completely NSFW terms into the rubygems.org search box to come up with this list. It’s a little longer than 10 gems, I didn’t anticipate finding so many “gems” (so to speak).

For slang or singular terms, I will include the definition according to Google, of the term to the right of the gem.

• lolita - A sexually precocious young girl
• tranny Short form of transmission in the automotive sense; … A transgender person. (the second link listed was to tranny/shemale porn)
• asset_hat (pun on “ass hat”, clever)
• sex_it_up
• womanizer a man who likes many women and has short sexual relationships with them.
• trollop A woman perceived as sexually disreputable or promiscuous.
• hash_dealer
• ball_gag A gag is sometimes worn during some bondage and BDSM roleplays.
• clitt
• tramp (debatable whether is closer “Lady and the Tramp” or “Tramp Stamp”)
• texticle (thanks for the reminder @scottmlikens)

Tips for Naming a Gem:

• Can you make an incredibly obtuse pun on the name of the library you’re wrapping?
• Grab a photo album from your childhood, find a picture of your bedroom, what’s the name of the first thing you see?
• Are you coding in between bong rips? Choose a word that sounds the funniest to your stoner friends.
• Are you British? What is the local slang term for “slut?”
• Go to 4chan, pick the 7th word of a random post to /b/
• Are you a white guy from Memphis who loves Asian culture? Use the Japanese name of the nearest household appliance.

Just plain stupid

While searching for blatantly offensive gem names, I found a number of other “wonderfully” named gems that don’t offend so much as they confuse. There was such a plethora of simple nouns which can make searching for documentation exceptionally difficult for novice users.

• cocaine
• girlfriend (okay, the name is appropriate, but seriously, what the?)
• smurf
• tit
• analdiffist
• nokogiri, from my understanding a Nokogiri is a saw?
• miniskirt
• hoe
• vegas
• loofah
• mandy
• home_run
• hansel
• vacation
• whoa

I don’t have much of an idea what half of these gems do, and I can’t be troubled to dig into what the use case is for “cocaine”, “vegas” or “whoa”, at least from the Ruby perspective that is.