war story: caching

There was that one time I used strace and a Ruby script to bypass a really long step in a build pipeline. The trick was figuring out the inputs and outputs by running the process under strace and utilizing the output from strace to compute some hashes. The core of the script was a utility class and some convenience methods for computing hashes by shelling out to find, tar, and shasum Continue reading

finger exercises: pipes, bytes, and fibers

In which I try to figure out how to pack and unpack bytes over an in-process pipe so that I can use it in some future message framing protocol for a worker pool. There will be a guest appearance by Fiber to simplify the parsing of messages in a non-blocking manner. Continue reading

software hygiene: encrypt your secrets

Most of my projects have a Rakefile because common tasks should be expressed in code instead of english and Rake is a great way to codify those common tasks. One thing that I have seen developers do is check-in secret tokens into their repositories in plaintext. I have done this as well. It is the simplest thing to do but it is terrible practice so to atone for my past sins and get others to not check-in secret tokens here is some code I now use to handle secret tokens. Adapt to your own workflow accordingly. Continue reading

optimizing spot instance allocation

There is surprisingly little information on how to optimize costs using the AWS spot instance market. There are several services that provide management on top of the spot market if you have an architecture that supports an interruptible workload but very little in the way of how to go about doing it yourself other than surface level advice on setting up autoscaling groups. To remedy the situation here’s an outline of how I’ve solved part of the problem for CI (continuous integration) type of workload Continue reading

How does Ruby do X?

The answer can usually be found with ruby -rdebug or ruby -rtracer. There is also the trick with RUBYOPT. If you are executing something with bundler then you can run RUBYOPT="-rdebug" bundle exec ${command} and you will be dropped in the debugger as usual.

production grade logging

Logging to sockets is better than logging to files. It allows for more flexibility in terms of log rotation and data integrity. I started looking around for examples of this but everything these days when it comes to logging is built for the enterprise. The actual skeleton of what all those enterprise systems are doing is quite simple. In fact it is so simple that you can do it in less than 30 lines of code in most high level languages. Here’s the skeleton for a logging server in Ruby: Continue reading

production grade ruby interpreter deployment process

rvm, chruby, rbenv, etc. do not belong in a production environment. Even if you are deploying and co-hosting applications that require different versions of ruby those tools still do not belong in a production environment. All those tools are strictly for development environments.

Binary shims and other hacks have no place in a production environment. Ideally you have one user per application that has the proper profile for setting up PATH to point to the right version of ruby which has been compiled and deployed wholesale ahead of time. This is actually quite simple and is in fact a one time operation if you do it right and package the binary bits with an RPM or Debian package. Heck, even a tar file would work if you’re willing to have some extra deployment logic and these days you can use any number of devops tools like chef and ansible to codify the initial production environment setup as well. Continue reading