Be kinder to your fellow human beings. If you’re a nerd then like Joscha Bach you probably have an extra degree of freedom when it comes to normative belief systems and the signals regular people use to gauge how good/bad they are is probably lost on you. I know it is lost on me because the hyper-parameters in my neural nets are configured slightly differently. My empathy networks are more subdued than the average. Fortunately if you’re not a psychopath there is plenty of time to re-adjust those parameters.
P.S.: I’m an atheist. I just think “godpseed” sounds cool.
There is surprisingly little information on how to optimize costs using the AWS spot instance market. There are several services that provide management on top of the spot market if you have an architecture that supports an interruptible workload but very little in the way of how to go about doing it yourself other than surface level advice on setting up autoscaling groups. To remedy the situation here’s an outline of how I’ve solved part of the problem for CI (continuous integration) type of workload Continue reading →
The author couldn’t be more wrong even if he tried. My current project is using GLPK to do some basic mixed integer programming to optimize AWS spot instance allocation. If I had not taken linear algebra, calculus, and a few courses in linear programming the idea would not even have crossed my mind that I could use mixed integer programming to solve the spot allocation problem and that’s just the first half.
The second half can be considered a problem in control theory because it requires taking the new allocations and gracefully transitioning from the old set of allocations. You can go even further and say that the whole thing would be even better if I understood more about stochastic processes and could potentially model the spot market and make predictions ahead of time to simplify the control problem and get ahead of the price fluctuations.
Saying all you need is Excel and 8th grade is in the words of one famous physicist “not even wrong”. If you’re in an engineering discipline then the more math you know the better.
This recently came up at work: capture the STDERR/STDOUT from another process and ship it over a socket somewhere else for archiving and viewing purposes. It was trickier than I expected so I’m writing it down here for next time. Continue reading →
A while back I wrote proxy for accessing an S3 bucket that transparently handles decrypting objects before serving them. It combines somethings I’v already talked about in what I consider interesting ways.
Disclaimer: The proxy is in no way production worthy. There are much better S3 proxies out there with much better documentation. This post is just to document a trick that I came up with to avoid using a database. Continue reading →