Stepbystep logic

Humans and AI are both neural systems, but they solve different learning problems because they have opposite constraints: humans have huge capacity and little data; big neural nets have smaller capacity and massive data.

These are some of my notes from listening to this interview with Geoffrey Hinton on Sunday. It's well worth a watch to hear the godfather of AI explain how the AI models of today work.

• The human brain has about 100 trillion connections (synapses), but a person only lives for a few billion seconds, so we get relatively few experiences compared with our capacity.
• Large AI models have far fewer connections than us (around 1% of our synapses, roughly a trillion parameters), but they are trained on thousands of times more data than a single human ever sees.
• Backpropagation is very good at compressing huge amounts of experience into a limited number of connections, which suits AI's "lots of data, not as many connections" regime.
• Humans face the opposite regime: "lots of connections, not much data", so our brains must extract as much as possible from each experience, likely using learning mechanisms that are not plain backpropagation.

Scaling and selfgenerated data

• When engineers made neural nets bigger and fed them more data, performance improved in a predictable way, as long as they didn't run out of data.
• For many tasks, you eventually hit a wall because there is only so much highquality human data available; beyond that, making models bigger gives diminishing returns.
• Some systems dodge this limit by generating their own data through selfplay or simulation, like AlphaGo/AlphaZero playing games against themselves to keep improving far beyond human experts.

Applying this to language models

• Today's language models mainly learn by imitating humans: they predict the next word in humanwritten text, analogous to mimicking expert moves in Go; this caps performance at roughly "expert level".
• A more powerful approach is for models to use reasoning over their own beliefs expressed in language: derive new conclusions, detect inconsistencies, and then adjust either their beliefs or their reasoning process.
• If a model can iteratively notice "my beliefs contradict each other, so something must change", it effectively generates its own learning signals, just like selfplay generates its own game data.

Intelligence, creativity, and experience

• With selfgenerated "mental data" of this kind, language models could become much more intelligent than humans, at least in reasoning and knowledge organisation.
• However, creating works that deeply matter to humans (for example, a Shakespearelevel novel) might still require experiences similar to ours, such as embodiment and genuine exposure to human life.

Mortality and digital minds

• Human minds are "analog" in the sense that when the brain dies, its exact synaptic configuration is gone, so our knowledge dies with us.
• Digital intelligences are different: you can save their parameters, destroy all hardware, then later rebuild hardware and reload those weights so the same mind "comes back to life".
• In this sense, digital systems have a practical form of resurrection: their identity and knowledge can be paused and restarted indefinitely, unlike human beings.

Now, the question is: Can language models do the same thing?

Currently, models learn by predicting the next word in human text, just as AlphaGo once mimicked experts. To surpass humans, language models would need to generate and test their own beliefs, not just copy ours.

A model could take what it already believes (its internal "knowledge"), reason about it, and detect inconsistencies - e.g., "If A and B are true, then C should be true... but I don't believe C." Resolving such conflicts would let the model correct itself and grow smarter - without more external data.

This would be the next stage of AI learning:

• Internally generate reasoning questions (selfplay for thought).
• Detect contradictions.
• Revise internal models through reasoning.

That approach blends deep learning's pattern recognition with symbolic reasoning's logic - creating a selfimproving system. Models like Gemini are already believed to be moving in this direction.