varnish, nginx, and S3

A while back I wrote proxy for accessing an S3 bucket that transparently handles decrypting objects before serving them. It combines some things I’v already talked about in what I consider interesting ways.

Disclaimer: The proxy is in no way production worthy. There are much better S3 proxies out there with much better documentation. This post is just to document a trick that I came up with to avoid using a database. Continue reading

basic infrastructure patterns

Some basic patterns I’ve noticed come up over and over again while working on build/CI/deployment related things.


The pipeline is the bedrock on which pretty much everything else is built. A properly designed pipeline takes well-defined inputs and produces well-defined outputs Continue reading

infrastructure as a database

The more I work with infrastructure as code the more I dislike it. I think that’s because fundamentally it’s the wrong metaphor. Code is terrible, it’s brittle and constantly breaking. Not to mention versioning and dependency issues. Even with immutable infrastructure you still have pretty much the same problems. I’ve used pretty much every configuration management tool that is out there and found them lacking. Continue reading

well-typed vignettes

I present to you some well-typed programs along with associated run-time and compile-time errors for pondering the meaning of well-typed. The context was me trying to understand the meaning of the quote “Well-typed programs cannot go wrong”. The exercise was illuminating for me but when I started writing explanations they got a little too philosophical so I’m just going to present the code and the errors. Continue reading

a day in the life of a programmer

The Ideal

You wake up full of ideas about new features and capabilities that can theoretically be added to the product you are working on. Several of these new features generalize existing functionality and get rid of corner cases that have been getting consistently flagged as bugs by users. The features and their underpinnings have been incubating for a while now and you’ve been working on sketches, mock-ups, and prototype code over the last few weekends. Continue reading

file system as a database

So it turns out that mv is an atomic operation if the source and destination are on the same device. This means that transforming and shuttling data between files can in theory be done in a transactional manner at the cost of some extra space. Here is an example workflow that came up at work: Continue reading

making chroots

Recently I had to build a chroot for CentOS 6.4 and I kept getting RPM DB errors. The errors were because I was building the chroot on a CentOS 7 box. After banging my head against the wall for a while I discovered that once yum is installed in the chroot it is much better to change into the chroot, rebuild the RPM DB to fix the errors, and then install all further packages from within the chroot. The finished chroot can be used as-is or imported into Docker.

broken by design

So it turns out making “broken things” and then “fixing” them by heeding the opinion of some person higher up in the social/corporate hierarchy is nothing new

According to Vasari Pier Soderini (the Mayor of Florence) was standing beneath the statue as it was put into place. The Mayor complained to Michelangelo that the nose was too thick. Michelangelo tricked Soderini by climbing the statue with a chisel and some marble dust concealed in his hand, pretending to work on the nose and sending down a shower of dust, he asked Soderini if it was improved, the Mayor replied “I like it better, you have given it life”.

I wish I could say that this kind of trickery is unnecessary in software but I would be lying.