Software hostility

All the tools I use are actively user hostile but I’ve gotten used to all the hostility and have basically developed Stockholm syndrome. Every so often I have moments of clarity and I wonder if we are insane for having gotten to this state of affairs without wondering if there was a better way. Consider the following example.

You are designing an event processing pipeline and in order to make things efficient you batch the incoming events in memory before processing them. This makes sense if you consider things from a database perspective because each event transformation can be considered a kind of transaction and transactions have non-trivial overhead. In fact, databases I think are the most humane systems we have developed so far because everything else is actively trying to poke your eye out. Now where was I? Right, batching events in memory for efficiency. What happens when the process that is keeping the events in memory crashes? You lose all those events. The operating system flushes everything into the void.

Consider the hostility of this design decision. There was some work you wanted the computer to do efficiently and for one reason or another people in the past made decisions that actively work against your intentions. This is an insane state of affairs. Now in order to side-step this insanity you have to set up replication, some kind of locking mechanism so that you don’t do the work twice, recovery mechanisms in case all your replicated processes crash, recovery mechanisms for when the locking mechanism crashes, and you need to make each of those things in turn robust enough so that when the operating system flushes everything into the void you can recover gracefully. Why couldn’t the operating system do something else? Why couldn’t the operating system snapshot the state of the process at the time of the crash and let you recover the relevant bits by mapping the uncrashed state into a new process? Why isn’t this a standard part of every operating system? Why do all programmers everywhere have to re-implement replication and transactional semantics in their processes every time they want to do something that requires graceful degradation and efficiency?

Programming is hell because we made it that way. It didn’t have to be this way. Things could have been different.