optimizing spot instance allocation

There is surprisingly little information on how to optimize costs using the AWS spot instance market. There are several services that provide management on top of the spot market if you have an architecture that supports an interruptible workload but very little in the way of how to go about doing it yourself other than surface level advice on setting up autoscaling groups. To remedy the situation here’s an outline of how I’ve solved part of the problem for CI (continuous integration) type of workload Continue reading

refactor by partial evaluation

Some time ago I read an article on proggit that made an analogy between compression and clean design. Basically if you see a lot of repetition then you factor out that structure and re-use it the same way a compression algorithm takes out common patterns and re-uses them to more compactly represent some blob of data. Sometimes though it is not the structure or the common patterns that make things hard to understand but instead it is too much generality and indirection. So how do you solve that problem? Continue reading

simple in-memory store with consistent reads

Problem Statement

Suppose you want to write a simple in-memory JSON store with an equally simple socket based protocol. You want this in-memory store to support parallel and consistent reads. By “parallel reads” what I mean is if 10 clients request to read data from the store then no client should be blocking any other client. By “consistent reads” what I mean is when a client requests some data from the store there is absolutely no way that client gets half of the data before a write and half of it after a write and there is also some kind of ordering for reads and writes. In other words, if we have an array “[1,2,3,4]” that corresponds to the key “ints” in our JSON store then the following sequence of events is impossible: Continue reading

simple monitoring with xinetd

Whenever you are running servers behind load balancers you need to perform health checks for marking servers up and down. If you are lucky then those health checks are built into the application code itself but more often than not you need to hack something together on top of existing legacy application code. In those instances xinetd and a simple script can do the trick. Continue reading

turing complete frameworks

I have recently noticed something that I’m dubbing “the turing complete framework”. It is not a good thing if your framework is “turing complete”. Examples of such frameworks are AngularJS, Meteor, Ember.js, ExtJS, Rails, Django, etc. All my examples are from the web development world because that is what I’m most familiar with but I’m sure every domain has their own set of “turing complete frameworks”. Continue reading

productivity and aesthetically pleasing code

When people say one programmer is more productive than another what they really mean is that the more productive programmer in general writes code with better structure. Part of that structure is following the conventions of the language and the community and so the code is easier to follow and reason about. But there is also another component of the structure that is harder to quantify and describe, Continue reading

programmer types

There are many posts on programmer types but this one is mine.

Most programmers are individually smart but once you put them in a group they become dumb because all their smarts are diverted to figuring out how to show everyone else that they are smart. Once in a while though you come across programmers that are not just smart individually but also smart when you put them in a group. This is rare and I don’t know how it happens but such programmers are usually the easiest to work with and in general just get things done. If you want to advance your career then seek out the programmers that are not just smart individually but also smart in groups because those guys will have your back.