The Physics of Intelligence

This missing discourse of AI

This research note is part of the Mediocre Computing series

Yesterday, I was doing some thing thinking about the differences between the everyday and AI senses of the word attention, inspired by an analogy to renewable energy, and I sketched this diagram to think about it:

This morning, I was delighted to see a very similar-looking diagram in Gordon Brander’s essay, Feedback is All You Need, in his excellent newsletter Subconscious, which you should subscribe to if you’re interested in computing themes.

The similarity is more than cosmetic. Gordon’s diagram is drawn from the cybernetics literature, while mine is a natural sort of diagram to draw if you think in control theory terms (for those unaware of the connection, control theory is to cybernetics as computer science is to AI). These sorts of block diagrams, with flows into and out of boxes, and feedback loops, are the natural way to visualize systems if you’re interested in questions of boundary conditions, stability, scaling, signals, and information flows. They are a natural outcome of trying to think about engineered systems from a physics perspective, and trying to do things like write down equations that describe the natural behavior of artificial systems.

We need more people thinking about AI in this way because in my opinion there is a missing physics-style discourse in AI. There are strong philosophy and engineering discourses, but no physics discourse. This is a problem because when engineers mainline philosophy questions in engineering frames without the moderating influence of physics frames, you get crackpottery. This is why the field is being over-run by crackpots, and increasingly at risk of complete theocratic capture by priests, as I argued last week. Crackpot priests are what you get when there aren’t enough physicists mediating between philosophers and engineers. I’ll argue this point in a future newsletter, but here’s a preliminary thread on Farcaster. For now, let’s table that interesting topic and focus on what it means to investigate the physics of intelligence.

You will notice, firstly, that I did not say the physics of artificial intelligence. Six months ago, I might have used that more qualified phrase. But I think it has been adequately demonstrated in the last six months that at least intelligence (if not subtler notions like consciousness or sentience that may or may not be well-posed) is not substrate dependent. The physics of intelligence is no more about silicon semiconductors or neurotransmitters than the physics of flight is about feathers or aluminum.

These low-level substrates constrain but do not define the physics in either case. When you analyze the flight characteristics of an airplane or a bird, you might quickly check the strength to weight ratio of bone-and-feather composites or aluminum, but then you move on talking about wing geometry, lift versus drag, and so on. Concepts that still belong in physics, but at a different level of abstraction.

Flight is actually a very good reference phenomenon for thinking about intelligence, since it too is a property of biological organisms that we reproduced with non-living machines that work on similar, but not identical principles. Understanding the physics of flight in a way that’s agnostic to the differences between birds and aircraft is a similar problem to that of understanding the physics of intelligence, whether realized with silicon or neurons. Interestingly, even though aerospace engineering is a mature discipline today, the physics of flight is still actually quite mysterious. As an aerospace engineer, I have the standard engineering understanding of it, but the physics understanding is more demanding, and less complete.

But getting back to intelligence, thinking carefully about the concept of a wing, and the role it plays in flight, as we will see, sheds interesting light on the concept of attention. Attention is the focus of one of the six basic questions about the physics of intelligence that I’ve been thinking about. Here is my full list:

1. What is attention, and how does it work?

2. What role does memory play in intelligence?

3. How is intelligence related to information?

4. How is intelligence related to spacetime?

5. How is intelligence related to matter?

6. How is intelligence related to energy and thermodynamics?

The first three are obviously at the “physics of intelligence” level of abstraction, just as “wing” is at the “physics of flight” level of abstraction. The last three get more abstract, and require some constraining, but there are already some good ideas floating around on how to do the constraining.

Obviously, we don’t have good answers, let alone validated and dispositive ones, to any of these questions. But I think I have intriguing clues in hand for each that I’m finding productive to think about. In this essay, I want to share some initial thoughts on the first three questions, which are somewhat easier to grok, and (very briefly) preview my thinking on the last three, which get much harder. I’ll cover those in detail in a future issue at a TBD date, since my thinking on them is still very early-stage.

An important note. We are not talking about the physics of computation in general. There are well known approaches to these questions for the broader category of computation (which to some extent is just an alternative way of talking about physics). I’ll mention these in passing in my discussion, but computation and intelligence are not synonymous or co-extensive.

To first order, I think of intelligences as embodied systems that are good at certain kinds of computation, and are situated in the universe in specific persistent ways, characterized by particular boundary conditions (which my cartoon diagram above gestures at). To talk about intelligence, it is necessary, but not sufficient, to talk about computation. You also have to talk about the main aspects of embodiment: spatial and temporal extent, materiality, bodily integrity maintenance in relation to environmental forces, and thermodynamic boundary conditions. My six questions get at those things.

The 6 questions above can also be asked about computation in general, and the answers constrain, but do not specify, answers to the same questions in relation to intelligence.

With those caveats out of the way, let’s dive in.