Book notes

Designing Data-Intensive Applications: Ch 6. – Partitioning

After a certain point your data will become a bottleneck either because of size or access patterns and at that point you’ll need to partition the data. Although there are a few ways to do that and several open source solutions for partitioning massive data sets each has its own downsides that will need to be carefully considered for specific use cases. Continue reading

Book notes

Designing Data-Intensive Applications: Ch2. – Data models and query languages

Like data structures there are many ways to represent and query persistent data. The main models for representing persistent data are relational, document oriented, graphical, and hierarchical. Although that last one seems to be all but dead because of the onerous burden it places on the programmer to maintain and query the data. Relational is still king after 30 or so years and document oriented and graphical are on the rise and shine in domains with specific modeling requirements. Continue reading

War story: sources, sinks, archives, and SSH sessions

Sometimes the right way to think about programming problems is as a sequence of transformations between sources and sinks. I present some Go code for streaming a tar archive from a remote host to the local filesystem. The standard library has everything we need and we are just going to put it together. Copy and adapt to your own needs as necessary. Continue reading

Re-editable code

Once in a while when I have an idea I look around to see if others have the same idea. Sometimes they do and sometimes they don’t. Most of the time the majority opinion is almost the exact opposite. But every so often I’m vindicated Continue reading

How to hire

tldr; You are not as good at hiring as you think and more than likely will never get any better at hiring so you should use,, or The Recurse Center. If giants like Google and Microsoft get it wrong with an entire army of statisticians and HR people to sift through the data what hope do you have of getting it right? Seriously, just use one of those three companies and save all those hours for running your actual business. Continue reading