Software hostility pt. 2

So today I had an idea about figuring out what is what on my laptop and immediately I hit a brick wall: nothing on my computer has provenance. What do I mean by that? I mean there is no way to trace anything on the filesystem in a way that has semantic meaning. Linux can’t tell me where each file came from, why it is there, what other programs rely on that file, in what way they rely on that file, is the file a meaningful component in some other collection of files, and basically a million other questions that have no sensible answers. Continue reading

Software hostility

All the tools I use are actively user hostile but I’ve gotten used to all the hostility and have basically developed Stockholm syndrome. Every so often I have moments of clarity and I wonder if we are insane for having gotten to this state of affairs without wondering if there was a better way. Consider the following example. Continue reading

Cybernetics of software

Recently I was thinking about how to properly structure software systems to reduce the cognitive burden on the people using and managing those systems as much as possible. The more I dug into the idea the more it became obvious that this is already a solved problem but there isn’t enough mindshare around the solution. The proper solution already exists under the heading of hierarchical planning and process control. An example from a domain I’m familiar with to demonstrate: build systems, release engineering, and in general, infrastructure software. Continue reading

Passion narrative considered harmful

I’ve gotten tired of seeing articles on programming blogs mentioning the word “passion” in the opening paragraph so I’m going to explain why you should ignore all such articles and focus on the mechanics of programming. To become good at programming you just need to practice. Programming is a skill and passion is not a prerequisite for practicing a skill. Continue reading

Book notes

Designing Data-Intensive Applications: Ch 6. – Partitioning

After a certain point your data will become a bottleneck either because of size or access patterns and at that point you’ll need to partition the data. Although there are a few ways to do that and several open source solutions for partitioning massive data sets each has its own downsides that will need to be carefully considered for specific use cases. Continue reading

Book notes

Designing Data-Intensive Applications: Ch2. – Data models and query languages

Like data structures there are many ways to represent and query persistent data. The main models for representing persistent data are relational, document oriented, graphical, and hierarchical. Although that last one seems to be all but dead because of the onerous burden it places on the programmer to maintain and query the data. Relational is still king after 30 or so years and document oriented and graphical are on the rise and shine in domains with specific modeling requirements. Continue reading

War story: sources, sinks, archives, and SSH sessions

Sometimes the right way to think about programming problems is as a sequence of transformations between sources and sinks. I present some Go code for streaming a tar archive from a remote host to the local filesystem. The standard library has everything we need and we are just going to put it together. Copy and adapt to your own needs as necessary. Continue reading

Re-editable code

Once in a while when I have an idea I look around to see if others have the same idea. Sometimes they do and sometimes they don’t. Most of the time the majority opinion is almost the exact opposite. But every so often I’m vindicated Continue reading