War story: sources, sinks, archives, and SSH sessions

Sometimes the right way to think about programming problems is as a sequence of transformations between sources and sinks. I present some Go code for streaming a tar archive from a remote host to the local filesystem. The standard library has everything we need and we are just going to put it together. Copy and adapt to your own needs as necessary. Continue reading

How to hire

tldr; You are not as good at hiring as you think and more than likely will never get any better at hiring so you should use triplebyte.com, interviewing.io, or The Recurse Center. If giants like Google and Microsoft get it wrong with an entire army of statisticians and HR people to sift through the data what hope do you have of getting it right? Seriously, just use one of those three companies and save all those hours for running your actual business. Continue reading

ruby dsl tricks: reifying references

Ruby is great for writing DSLs because it has first class support for two of the most important ingredients of DSLs, contexts and code blocks. With the proper use of instance_eval the same block of code can be evaluated in various contexts to have different kinds of effects but most often what we want to do is evaluate the code block in the “freest” possible context to create an AST (abstract syntax tree). I’m almost certain there is a connection here with initial and terminal algebras in category theory but someone smarter than me will have to chase that analogy. Today I’m just going to demonstrate how to reify references so that we can support cyclic structures in our DSL. Continue reading

finger exercises: pipes, bytes, and fibers

In which I try to figure out how to pack and unpack bytes over an in-process pipe so that I can use it in some future message framing protocol for a worker pool. There will be a guest appearance by Fiber to simplify the parsing of messages in a non-blocking manner. Continue reading

optimizing spot instance allocation (part deux)

See part 1 for context. This post explains the control/planner strategy I settled on to minimize the cost and make things as stable as possible to mitigate disruptions caused by VMs being re-allocated. Continue reading