How to hire

tldr; You are not as good at hiring as you think and more than likely will never get any better at hiring so you should use triplebyte.com, interviewing.io, or The Recurse Center. If giants like Google and Microsoft get it wrong with an entire army of statisticians and HR people to sift through the data what hope do you have of getting it right? Seriously, just use one of those three companies and save all those hours for running your actual business. Continue reading

optimizing spot instance allocation (part deux)

See part 1 for context. This post explains the control/planner strategy I settled on to minimize the cost and make things as stable as possible to mitigate disruptions caused by VMs being re-allocated. Continue reading

good design through abstraction lowering

Many times when I run into buggy software and peek under the hood I notice that the design is convoluted and confused. There are modules with cross-cutting concerns, there is no obvious flow for the inputs and outputs, and there is no set of core abstractions that everything else builds on.

Now compare this to something that is well designed. There is usually a core set of abstractions and associated operations for building up larger computations to accomplish a goal and the rest of the software is built upon those core components. Parser combinators and optimizing compilers are good examples of such software. The flow of the computation is obvious because there are explicit and clear steps that transform high level abstractions and operations into successively lower level ones until everything is expressed by the core set of abstractions. So to understand the whole you just need to understand the core components and how they are combined. There is no reason more general applications can not be built this way. In fact most enterprise architectures with their layered approach are trying to do exactly this but fall short for one reason or another.

refactor by partial evaluation

Some time ago I read an article on proggit that made an analogy between compression and clean design. Basically if you see a lot of repetition then you factor out that structure and re-use it the same way a compression algorithm takes out common patterns and re-uses them to more compactly represent some blob of data. Sometimes though it is not the structure or the common patterns that make things hard to understand but instead it is too much generality and indirection. So how do you solve that problem? Continue reading

continuous integration for infrastructure

If you are using the cloud for you infrastructure then you need to version it the same way you version your code and deployment artifacts. Lately, I’ve been using packer to generate AMIs and I couldn’t be happier with how things are working out. I now have a consistent environment whenever I want to experiment with something and when I discover something worthwhile that I think should be a basic functionality of all environments I need then I check that into the repository that contains the packer templates and regenerate all the AMIs. This means I can go back to any point in time and spin up an environment exactly as it existed at that time. Infrastructure is no longer something special. It is now just another artifact in a software development pipeline and you should treat it that way.

abstract machines all the way down

Programming is not about writing code. Syntax changes over time but the fundamental nature of programming remains the same. The true nature of programming is taking abstract machines and putting them together to perform a task. Programming at the end of the day is about those abstract machines and their connections. This is why type theory and category theory should be essential components of any programmer’s education. Once you start viewing programming through those lenses it is impossible to go back to the syntax oriented view of programming. Here are some books and various other resources I have found useful during my education: Understanding Computation: From Simple Machines to Impossible Programs, An Introduction to Mathematical Reasoning, Types and Programming Languages, Nathan’s University, Functional Programming Principles in Scala, Applied Cryptography, Artificial Intelligence for Robotics, r/programming.

simple monitoring with xinetd

Whenever you are running servers behind load balancers you need to perform health checks for marking servers up and down. If you are lucky then those health checks are built into the application code itself but more often than not you need to hack something together on top of existing legacy application code. In those instances xinetd and a simple script can do the trick. Continue reading

fixing hiring problems

As long as I’m figuring things out here’s another one for you. The Silicon Valley technology worker crunch is a self-inflicted wound. Looking for programmers with X years of experience makes no sense and if this is your criteria for hiring technology workers then you are going to fail miserably. Continue reading

schizophrenic companies

I have finally figured out why all big companies are so schizophrenic. By schizophrenic I mean if you are at a certain level of the hierarchy then all the movements around you seem completely out of whack. You see different groups working against each other instead of cooperatively developing cool technologies and delivering value to the customer. The reason for all of it is what I’m calling the “portfolio theory of management”. Continue reading