See part 1 for context. This post explains the control/planner strategy I settled on to minimize the cost and make things as stable as possible to mitigate disruptions caused by VMs being re-allocated. Continue reading
Many times when I run into buggy software and peek under the hood I notice that the design is convoluted and confused. There are modules with cross-cutting concerns, there is no obvious flow for the inputs and outputs, and there is no set of core abstractions that everything else builds on.
Now compare this to something that is well designed. There is usually a core set of abstractions and associated operations for building up larger computations to accomplish a goal and the rest of the software is built upon those core components. Parser combinators and optimizing compilers are good examples of such software. The flow of the computation is obvious because there are explicit and clear steps that transform high level abstractions and operations into successively lower level ones until everything is expressed by the core set of abstractions. So to understand the whole you just need to understand the core components and how they are combined. There is no reason more general applications can not be built this way. In fact most enterprise architectures with their layered approach are trying to do exactly this but fall short for one reason or another.
Some time ago I read an article on proggit that made an analogy between compression and clean design. Basically if you see a lot of repetition then you factor out that structure and re-use it the same way a compression algorithm takes out common patterns and re-uses them to more compactly represent some blob of data. Sometimes though it is not the structure or the common patterns that make things hard to understand but instead it is too much generality and indirection. So how do you solve that problem? Continue reading
If you are using the cloud for you infrastructure then you need to version it the same way you version your code and deployment artifacts. Lately, I’ve been using packer to generate AMIs and I couldn’t be happier with how things are working out. I now have a consistent environment whenever I want to experiment with something and when I discover something worthwhile that I think should be a basic functionality of all environments I need then I check that into the repository that contains the packer templates and regenerate all the AMIs. This means I can go back to any point in time and spin up an environment exactly as it existed at that time. Infrastructure is no longer something special. It is now just another artifact in a software development pipeline and you should treat it that way.
As long as I’m figuring things out here’s another one for you. The Silicon Valley technology worker crunch is a self-inflicted wound. Looking for programmers with X years of experience makes no sense and if this is your criteria for hiring technology workers then you are going to fail miserably. Continue reading
I have finally figured out why all big companies are so schizophrenic. By schizophrenic I mean if you are at a certain level of the hierarchy then all the movements around you seem completely out of whack. You see different groups working against each other instead of cooperatively developing cool technologies and delivering value to the customer. The reason for all of it is what I’m calling the “portfolio theory of management”. Continue reading
I have recently noticed something that I’m dubbing “the turing complete framework”. It is not a good thing if your framework is “turing complete”. Examples of such frameworks are AngularJS, Meteor, Ember.js, ExtJS, Rails, Django, etc. All my examples are from the web development world because that is what I’m most familiar with but I’m sure every domain has their own set of “turing complete frameworks”. Continue reading
When people say one programmer is more productive than another what they really mean is that the more productive programmer in general writes code with better structure. Part of that structure is following the conventions of the language and the community and so the code is easier to follow and reason about. But there is also another component of the structure that is harder to quantify and describe, Continue reading
Do you ever wonder why almost every library out there is a convoluted mess of ill conceived abstractions with ill defined composition semantics? I do all the time and I think I have finally figured out why. Continue reading
Many programmers are obsessed with being seen as smart and productive. One of the reasons for this obsession is Continue reading