The Free Energy Principle approach to Agency @MachineLearningStreetTalk

Updated September 2026, 1 week ago

The Free Energy Principle approach to Agency

The AI Progress Chart Everyone Is Misreading — Beth Barnes & David Rein

Genuine understanding is the worlds most valuable commodity right now

There are monsters in your LLM. (Murray Shanahan)

$We need AIs with PHYSICAL experience (Jeff Beck) Jeff Beck spent years studying computational neuroscience with Alexandre Pouget, Peter Latham, and Wei Ji Ma before founding Noumenal Labs. In this conversation with Tim Scarfe, he argues that language models are fundamentally limited because they manipulate symbols without the physical grounding that gives those symbols meaning. Beck walks through the Bayesian brain hypothesis — the idea that our brains maintain probabilistic models of the world, built from direct sensory experience. Language, he says, is just a thin lossy summary of that richer internal model. He points to Markus Meisters work showing that human information output runs at roughly 10 bits per second, a tiny fraction of what comes in. The conversation gets interesting around the question of whether AI systems can genuinely understand anything. Becks position is clear: until a language model produces something genuinely novel — not recombined from training data — he wont call it intelligent. The second half turns to alignment — Beck argues that you cannot separate someones beliefs from their reward function just by observing their behavior, a claim with serious implications for AI safety. SPONSOR MESSAGES: *** Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich. Goto https://tufalabs.ai/ *** TIMESTAMPS: 00:00:00 Bayesian brain and neural computation foundations 00:02:00 Belief-reward entanglement and AI alignment 00:02:19 Sponsor: Tufa Labs 00:02:45 Visual cortex discovery and the Hubel-Wiesel experiment 00:05:00 Theory of mind tests and ChatGPT limitations 00:08:50 The sum and product riddle: pattern recognition vs reasoning 00:09:30 Systems engineering and object decomposition 00:12:20 Vicarious experience and grounded understanding 00:15:20 Neural coding and choice of generative model 00:17:30 Line-of-sight legibility in AI reasoning 00:18:50 Language as lossy compression of cognition 00:22:50 10 bits per second: information processing bottleneck 00:25:00 Why language models cannot substitute for understanding 00:28:00 Scientific abstraction and idealization 00:29:40 Markov blankets and system partitioning 00:33:20 Scientific realism, noise, and the limits of models 00:36:00 Free energy principle as mathematical framework 00:38:40 Black-box prediction vs legible explanation 00:41:00 Reward functions and the impossibility of alignment 00:45:00 Modeling beliefs as prerequisite for value inference REFERENCES: paper: [00:00:15] Bayesian inference in neural computation (Ma, Beck, Latham, Pouget) https://www.nature.com/articles/nn1790 [00:01:50] Noumenal Labs research paper https://arxiv.org/html/2502.13161v1 [00:05:25] LLM performance on theory of mind tasks (Kosinski) https://arxiv.org/abs/2302.02083 [00:16:25] Bayesian Mechanics (Ramstead, Sakthivadivel, Heins et al.) https://arxiv.org/abs/2205.11543 [00:17:10] Building Human-like Communicative Intelligence (Dubova) https://arxiv.org/abs/2201.02734 [00:20:45] Why do we live at 10 bits/s? (Meister, Zheng) https://www.sciencedirect.com/science/article/abs/pii/S0896627324008080 [00:28:30] Markov blankets in biological systems (Friston) https://royalsocietypublishing.org/doi/10.1098/rsif.2017.0792 [00:32:15] Critique of Gabor patches in neuroscience (Tsao) https://pmc.ncbi.nlm.nih.gov/articles/PMC9564096/ [00:35:55] MaxEnt and MaxCal principles (Presse et al.) https://journals.aps.org/rmp/abstract/10.1103/RevModPhys.85.1115 [00:40:08] Reward is Enough (Silver, Singh, Precup, Sutton) https://www.sciencedirect.com/science/article/pii/S0004370221000862 [00:45:40] Modeling Human Beliefs about AI Behavior (Lang, Forre) https://www.arxiv.org/pdf/2502.21262 website: [00:01:50] Noumenal Labs (Jeff Beck) https://www.noumenal.ai/ [00:02:19] Tufa AI Labs https://tufalabs.ai/ [00:22:25] Steven Piantadosi https://colala.berkeley.edu/people/piantadosi/ [00:23:35] Mad Libs (Stern, Price) https://en.wikipedia.org/wiki/Mad_Libs [00:31:05] Mathematical Platonism (Linnebo, SEP) https://plato.stanford.edu/entries/platonism-mathematics/ book: [00:25:25] The Brain Abstracted (Chirimuuta) https://mitpress.mit.edu/9780262548045/the-brain-abstracted/ LINKS: Full Transcript: https://app.rescript.info/share/75dfad07d8cad3ed6ccd03253aa006be Download PDF transcript: https://app.rescript.info/api/public/sessions/981caa3aeba083df/pdf Extended version on patreon: https://www.patreon.com/posts/jeff-beck-125455115$