finger exercises: pipes, bytes, and fibers

In which I try to figure out how to pack and unpack bytes over an in-process pipe so that I can use it in some future message framing protocol for a worker pool. There will be a guest appearance by Fiber to simplify the parsing of messages in a non-blocking manner. Continue reading

optimizing spot instance allocation (part deux)

See part 1 for context. This post explains the control/planner strategy I settled on to minimize the cost and make things as stable as possible to mitigate disruptions caused by VMs being re-allocated. Continue reading

optimizing spot instance allocation

There is surprisingly little information on how to optimize costs using the AWS spot instance market. There are several services that provide management on top of the spot market if you have an architecture that supports an interruptible workload but very little in the way of how to go about doing it yourself other than surface level advice on setting up autoscaling groups. To remedy the situation here’s an outline of how I’ve solved part of the problem for CI (continuous integration) type of workload Continue reading

refactor by partial evaluation

Some time ago I read an article on proggit that made an analogy between compression and clean design. Basically if you see a lot of repetition then you factor out that structure and re-use it the same way a compression algorithm takes out common patterns and re-uses them to more compactly represent some blob of data. Sometimes though it is not the structure or the common patterns that make things hard to understand but instead it is too much generality and indirection. So how do you solve that problem? Continue reading

simple monitoring with xinetd

Whenever you are running servers behind load balancers you need to perform health checks for marking servers up and down. If you are lucky then those health checks are built into the application code itself but more often than not you need to hack something together on top of existing legacy application code. In those instances xinetd and a simple script can do the trick. Continue reading

fixing hiring problems

As long as I’m figuring things out here’s another one for you. The Silicon Valley technology worker crunch is a self-inflicted wound. Looking for programmers with X years of experience makes no sense and if this is your criteria for hiring technology workers then you are going to fail miserably. Continue reading

how to be a better programmer

If you want to become a better programmer then stop thinking in code. Programming is ultimately about solving problems with a predetermined set of abstractions. How these abstractions are expressed syntactically is irrelevant. So you should be practicing how to glue abstractions together instead of chasing the latest idioms and design patterns and incorporating them into your latest framework. This is why lisp programmers, and functional programmers in general, are better programmers. Functional languages force you to practice thinking with abstractions instead of low-level details and so over time functional programmers become better problem solvers because every time they sit down to program they are practicing better mental habits.