Uploaded April 2025 | Updated September 2026, 1 week ago
Jeff Beck spent years studying computational neuroscience with Alexandre Pouget, Peter Latham, and Wei Ji Ma before founding Noumenal Labs. In this conversation with Tim Scarfe, he argues that language models are fundamentally limited because they manipulate symbols without the physical grounding that gives those symbols meaning.
Beck walks through the Bayesian brain hypothesis — the idea that our brains maintain probabilistic models of the world, built from direct sensory experience. Language, he says, is just a thin lossy summary of that richer internal model. He points to Markus Meister's work showing that human information output runs at roughly 10 bits per second, a tiny fraction of what comes in.
The conversation gets interesting around the question of whether AI systems can genuinely understand anything. Beck's position is clear: until a language model produces something genuinely novel — not recombined from training data — he won't call it intelligent. The second half turns to alignment — Beck argues that you cannot separate someone's beliefs from their reward function just by observing their behavior, a claim with serious implications for AI safety.
SPONSOR MESSAGES:
***
Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich.
Goto tufalabs.ai
***
---
TIMESTAMPS:
00:00:00 Bayesian brain and neural computation foundations
00:02:00 Belief-reward entanglement and AI alignment
00:02:19 Sponsor: Tufa Labs
00:02:45 Visual cortex discovery and the Hubel-Wiesel experiment
00:05:00 Theory of mind tests and ChatGPT limitations
00:08:50 The sum and product riddle: pattern recognition vs reasoning
00:09:30 Systems engineering and object decomposition
00:12:20 Vicarious experience and grounded understanding
00:15:20 Neural coding and choice of generative model
00:17:30 Line-of-sight legibility in AI reasoning
00:18:50 Language as lossy compression of cognition
00:22:50 10 bits per second: information processing bottleneck
00:25:00 Why language models cannot substitute for understanding
00:28:00 Scientific abstraction and idealization
00:29:40 Markov blankets and system partitioning
00:33:20 Scientific realism, noise, and the limits of models
00:36:00 Free energy principle as mathematical framework
00:38:40 Black-box prediction vs legible explanation
00:41:00 Reward functions and the impossibility of alignment
00:45:00 Modeling beliefs as prerequisite for value inference
---
REFERENCES:
paper:
[00:00:15] Bayesian inference in neural computation (Ma, Beck, Latham, Pouget)
nature.com/articles/nn1790
[00:01:50] Noumenal Labs research paper
arxiv.org/html/2502.13161v1
[00:05:25] LLM performance on theory of mind tasks (Kosinski)
arxiv.org/abs/2302.02083
[00:16:25] Bayesian Mechanics (Ramstead, Sakthivadivel, Heins et al.)
arxiv.org/abs/2205.11543
[00:17:10] Building Human-like Communicative Intelligence (Dubova)
arxiv.org/abs/2201.02734
[00:20:45] Why do we live at 10 bits/s? (Meister, Zheng)
sciencedirect.com/science/article/abs/pii/S0896627324008080
[00:28:30] Markov blankets in biological systems (Friston)
royalsocietypublishing.org/doi/10.1098/rsif.2017.0792
[00:32:15] Critique of Gabor patches in neuroscience (Tsao)
pmc.ncbi.nlm.nih.gov/articles/PMC9564096
[00:35:55] MaxEnt and MaxCal principles (Presse et al.)
journals.aps.org/rmp/abstract/10.1103/RevModPhys.85.1115
[00:40:08] Reward is Enough (Silver, Singh, Precup, Sutton)
sciencedirect.com/science/article/pii/S0004370221000862
[00:45:40] Modeling Human Beliefs about AI Behavior (Lang, Forre)
arxiv.org/pdf/2502.21262
website:
[00:01:50] Noumenal Labs (Jeff Beck)
noumenal.ai
[00:02:19] Tufa AI Labs
tufalabs.ai
[00:22:25] Steven Piantadosi
https://colala.berkeley.edu/people/piantadosi/
[00:23:35] Mad Libs (Stern, Price)
en.wikipedia.org/wiki/Mad_Libs
[00:31:05] Mathematical Platonism (Linnebo, SEP)
https://plato.stanford.edu/entries/platonism-mathematics/
book:
[00:25:25] The Brain Abstracted (Chirimuuta)
https://mitpress.mit.edu/9780262548045/the-brain-abstracted/
---
LINKS:
Full Transcript: app.rescript.info/share/75dfad07d8cad3ed6ccd03253aa006be
Download PDF transcript: app.rescript.info/api/public/sessions/981caa3aeba083df/pdf
Extended version on patreon:
patreon.com/posts/jeff-beck-125455115
Jeff Beck spent years studying computational neuroscience with Alexandre Pouget, Peter Latham, and Wei Ji Ma before founding Noumenal Labs. In this conversation with Tim Scarfe, he argues that language models are fundamentally limited because they manipulate symbols without the physical grounding that gives those symbols meaning.
Beck walks through the Bayesian brain hypothesis — the idea that our brains maintain probabilistic models of the world, built from direct sensory experience. Language, he says, is just a thin lossy summary of that richer internal model. He points to Markus Meister's work showing that human information output runs at roughly 10 bits per second, a tiny fraction of what comes in.
The conversation gets interesting around the question of whether AI systems can genuinely understand anything. Beck's position is clear: until a language model produces something genuinely novel — not recombined from training data — he won't call it intelligent. The second half turns to alignment — Beck argues that you cannot separate someone's beliefs from their reward function just by observing their behavior, a claim with serious implications for AI safety.
SPONSOR MESSAGES:
***
Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich.
Goto tufalabs.ai
***
---
TIMESTAMPS:
00:00:00 Bayesian brain and neural computation foundations
00:02:00 Belief-reward entanglement and AI alignment
00:02:19 Sponsor: Tufa Labs
00:02:45 Visual cortex discovery and the Hubel-Wiesel experiment
00:05:00 Theory of mind tests and ChatGPT limitations
00:08:50 The sum and product riddle: pattern recognition vs reasoning
00:09:30 Systems engineering and object decomposition
00:12:20 Vicarious experience and grounded understanding
00:15:20 Neural coding and choice of generative model
00:17:30 Line-of-sight legibility in AI reasoning
00:18:50 Language as lossy compression of cognition
00:22:50 10 bits per second: information processing bottleneck
00:25:00 Why language models cannot substitute for understanding
00:28:00 Scientific abstraction and idealization
00:29:40 Markov blankets and system partitioning
00:33:20 Scientific realism, noise, and the limits of models
00:36:00 Free energy principle as mathematical framework
00:38:40 Black-box prediction vs legible explanation
00:41:00 Reward functions and the impossibility of alignment
00:45:00 Modeling beliefs as prerequisite for value inference
---
REFERENCES:
paper:
[00:00:15] Bayesian inference in neural computation (Ma, Beck, Latham, Pouget)
nature.com/articles/nn1790
[00:01:50] Noumenal Labs research paper
arxiv.org/html/2502.13161v1
[00:05:25] LLM performance on theory of mind tasks (Kosinski)
arxiv.org/abs/2302.02083
[00:16:25] Bayesian Mechanics (Ramstead, Sakthivadivel, Heins et al.)
arxiv.org/abs/2205.11543
[00:17:10] Building Human-like Communicative Intelligence (Dubova)
arxiv.org/abs/2201.02734
[00:20:45] Why do we live at 10 bits/s? (Meister, Zheng)
sciencedirect.com/science/article/abs/pii/S0896627324008080
[00:28:30] Markov blankets in biological systems (Friston)
royalsocietypublishing.org/doi/10.1098/rsif.2017.0792
[00:32:15] Critique of Gabor patches in neuroscience (Tsao)
pmc.ncbi.nlm.nih.gov/articles/PMC9564096
[00:35:55] MaxEnt and MaxCal principles (Presse et al.)
journals.aps.org/rmp/abstract/10.1103/RevModPhys.85.1115
[00:40:08] Reward is Enough (Silver, Singh, Precup, Sutton)
sciencedirect.com/science/article/pii/S0004370221000862
[00:45:40] Modeling Human Beliefs about AI Behavior (Lang, Forre)
arxiv.org/pdf/2502.21262
website:
[00:01:50] Noumenal Labs (Jeff Beck)
noumenal.ai
[00:02:19] Tufa AI Labs
tufalabs.ai
[00:22:25] Steven Piantadosi
https://colala.berkeley.edu/people/piantadosi/
[00:23:35] Mad Libs (Stern, Price)
en.wikipedia.org/wiki/Mad_Libs
[00:31:05] Mathematical Platonism (Linnebo, SEP)
https://plato.stanford.edu/entries/platonism-mathematics/
book:
[00:25:25] The Brain Abstracted (Chirimuuta)
https://mitpress.mit.edu/9780262548045/the-brain-abstracted/
---
LINKS:
Full Transcript: app.rescript.info/share/75dfad07d8cad3ed6ccd03253aa006be
Download PDF transcript: app.rescript.info/api/public/sessions/981caa3aeba083df/pdf
Extended version on patreon:
patreon.com/posts/jeff-beck-125455115
