finger exercises: pipes, bytes, and fibers

In which I try to figure out how to pack and unpack bytes over an in-process pipe so that I can use it in some future message framing protocol for a worker pool. There will be a guest appearance by Fiber to simplify the parsing of messages in a non-blocking manner. Continue reading

cloud patterns: baking VMs with Packer

When managing a cloud infrastructure there are foundational components that pretty much all your VMs will need. For those foundational components instead of installing them at run time as the VM is starting you should use Packer to just put them into a VM image once and for all. This is a good practice in general because if done right it will reduce startup time and lead to a more efficient and consistent fleet for your cloud infrastructure. Continue reading

software hygiene: encrypt your secrets

Most of my projects have a Rakefile because common tasks should be expressed in code instead of english and Rake is a great way to codify those common tasks. One thing that I have seen developers do is check-in secret tokens into their repositories in plaintext. I have done this as well. It is the simplest thing to do but it is terrible practice so to atone for my past sins and get others to not check-in secret tokens here is some code I now use to handle secret tokens. Adapt to your own workflow accordingly. Continue reading

optimizing spot instance allocation (part deux)

See part 1 for context. This post explains the control/planner strategy I finally settled on to minimize the cost and make things as stable as possible to mitigate workflow disruptions. Continue reading