Dynamics, Aliasing, and Projection in Theory Space

constructing theories from “projections”

The comments on that twitter thread don’t do a good enough job of explaining what is going on in that picture and why it’s more meaningful than appears at first sight and thought.

When I first saw that I thought about the sampling theorem: to recover a signal from its samples we must sample it at twice the rate of the highest frequency. If our sampling rate is lower than what the theorem says it should be then we can’t reconstruct the original signal with full fidelity and we get non-unique solutions, i.e. aliasing. This idea extends to basically any domain.

I like to think of the sampling theorem as a projection of a signal onto some lower dimensional representation. The sampling theorem says that specific lower dimensional representations are enough to recover the higher dimensional object that is “casting” the lower dimensional shadow. Now what happens if we extend this idea to theories and the physical/non-physical phenomenon they try to explain?

Theories themselves can be thought of as living in some theory space and as humans we are very good at working with first order theories like first order logic. First order theories are 1 dimensional projections of higher dimensional theories. The unfortunate truth is that our brains are small so we don’t really have good intuitions for higher order theories like we do for first order theories. The way we deal with this problem is by projecting higher order theories onto lower order theories and then tacking on information about the projection and our guess about where the projection came from via a meta-theory. The issue though, like in the sampling theorem, is that we have aliasing in theory space as well. The meta-theory gives us a guess about what part of theory space the projection came from but it’s just a guess because we don’t have enough information and also have no idea how we would reason about the higher dimensional information even if we had it. So ideally what we would do is stitch together all the projections to get a better representation and theory but this again starts to look very much like higher order reasoning and the complexity overwhelms our brain.

There is no conclusion or punchline. Enjoy the animation.