How to hire

tldr; You are not as good at hiring as you think and more than likely will never get any better at hiring so you should use,, or The Recurse Center. If giants like Google and Microsoft get it wrong with an entire army of statisticians and HR people to sift through the data what hope do you have of getting it right? Seriously, just use one of those three companies and save all those hours for running your actual business. Continue reading

optimizing spot instance allocation (part deux)

See part 1 for context. This post explains the control/planner strategy I settled on to minimize the cost and make things as stable as possible to mitigate disruptions caused by VMs being re-allocated. Continue reading

good design through abstraction lowering

Many times when I run into buggy software and peek under the hood I notice that the design is convoluted and confused. There are modules with cross-cutting concerns, there is no obvious flow for the inputs and outputs, and there is no set of core abstractions that everything else builds on.

Now compare this to something that is well designed. There is usually a core set of abstractions and associated operations for building up larger computations to accomplish a goal and the rest of the software is built upon those core components. Parser combinators and optimizing compilers are good examples of such software. The flow of the computation is obvious because there are explicit and clear steps that transform high level abstractions and operations into successively lower level ones until everything is expressed by the core set of abstractions. So to understand the whole you just need to understand the core components and how they are combined. There is no reason more general applications can not be built this way. In fact most enterprise architectures with their layered approach are trying to do exactly this but fall short for one reason or another.

refactor by partial evaluation

Some time ago I read an article on proggit that made an analogy between compression and clean design. Basically if you see a lot of repetition then you factor out that structure and re-use it the same way a compression algorithm takes out common patterns and re-uses them to more compactly represent some blob of data. Sometimes though it is not the structure or the common patterns that make things hard to understand but instead it is too much generality and indirection. So how do you solve that problem? Continue reading

continuous integration for infrastructure

If you are using the cloud for you infrastructure then you need to version it the same way you version your code and deployment artifacts. Lately, I’ve been using packer to generate AMIs and I couldn’t be happier with how things are working out. I now have a consistent environment whenever I want to experiment with something and when I discover something worthwhile that I think should be a basic functionality of all environments I need then I check that into the repository that contains the packer templates and regenerate all the AMIs. This means I can go back to any point in time and spin up an environment exactly as it existed at that time. Infrastructure is no longer something special. It is now just another artifact in a software development pipeline and you should treat it that way.

fixing hiring problems

As long as I’m figuring things out here’s another one for you. The Silicon Valley technology worker crunch is a self-inflicted wound. Looking for programmers with X years of experience makes no sense and if this is your criteria for hiring technology workers then you are going to fail miserably. Continue reading

schizophrenic companies

I have finally figured out why all big companies are so schizophrenic. By schizophrenic I mean if you are at a certain level of the hierarchy then all the movements around you seem completely out of whack. You see different groups working against each other instead of cooperatively developing cool technologies and delivering value to the customer. The reason for all of it is what I’m calling the “portfolio theory of management”. Continue reading

turing complete frameworks

I have recently noticed something that I’m dubbing “the turing complete framework”. It is not a good thing if your framework is “turing complete”. Examples of such frameworks are AngularJS, Meteor, Ember.js, ExtJS, Rails, Django, etc. All my examples are from the web development world because that is what I’m most familiar with but I’m sure every domain has their own set of “turing complete frameworks”. Continue reading

productivity and aesthetically pleasing code

When people say one programmer is more productive than another what they really mean is that the more productive programmer in general writes code with better structure. Part of that structure is following the conventions of the language and the community and so the code is easier to follow and reason about. But there is also another component of the structure that is harder to quantify and describe, Continue reading