Book notes

Designing Data-Intensive Applications: Ch2. – Data models and query languages

Like data structures there are many ways to represent and query persistent data. The main models for representing persistent data are relational, document oriented, graphical, and hierarchical. Although that last one seems to be all but dead because of the onerous burden it places on the programmer to maintain and query the data. Relational is still king after 30 or so years and document oriented and graphical are on the rise and shine in domains with specific modeling requirements. Continue reading

War story: sources, sinks, archives, and SSH sessions

Sometimes the right way to think about programming problems is as a sequence of transformations between sources and sinks. I present some Go code for streaming a tar archive from a remote host to the local filesystem. The standard library has everything we need and we are just going to put it together. Copy and adapt to your own needs as necessary. Continue reading

Re-editable code

Once in a while when I have an idea I look around to see if others have the same idea. Sometimes they do and sometimes they don’t. Most of the time the majority opinion is almost the exact opposite. But every so often I’m vindicated Continue reading

How to hire

tldr; You are not as good at hiring as you think and more than likely will never get any better at hiring so you should use,, or The Recurse Center. If giants like Google and Microsoft get it wrong with an entire army of statisticians and HR people to sift through the data what hope do you have of getting it right? Seriously, just use one of those three companies and save all those hours for running your actual business. Continue reading

cobbling together code

A while back I wrote an implementation of SHA256 in Dart using generators and byte buffer views. Going back and looking at the implementation I can’t really make sense of it. It is not obvious why I index and shuffle the byte buffers one way instead of some other way.

So what is the lesson here? Writing code is always easier than understanding it. If after putting it aside for a while I can’t figure out why I made certain decisions then what hope is there of someone else figuring it out?

software engineering principles to live by

  1. There are three ways to solve problems: bring the problem closer to an existing solution, bring an existing solution closer to the problem, invalidate the problem by changing all your underlying assumptions.
  2. It is impossible to maintain large codebases in dynamic languages. Anyone that thinks otherwise hasn’t yet written enough dynamic code.
  3. Be proud of your work but do not be your work. Be critical of the code but not the person that wrote the code.
  4. The tools must always be subordinate to human intentions and not the other way around.
  5. Capabilities are always better than features. Capabilities can be composed. Features can only be contextualized.

encrypt and check your secrets into git

When it comes to managing secret tokens whatever you do someone will come out and say you’re doing it wrong and propose an alternate solution that on the surface seems to be better but upon closer scrutiny is susceptible to the same attack vectors. The secure solutions is always some kind of custom solution for storing secrets that provides an audit trail. The audit trail means whenever a secret is accessed you have a log entry of it somewhere. Which is nice because when stuff leaks you can in theory trace it back to some specific person. Unless that person was hacked and they were used as a patsy. So today let’s compare storing encrypted secrets in git or some custom solution that is not git. Continue reading