![AI HAS A BODY PROBLEM... [Dr. Maxwell Ramstead] @MachineLearningStreetTalk](https://i.ytimg.com/vi/jsIt2sTB_vo/maxresdefault.jpg)
Uploaded June 2025 | Updated September 2026, 1 week ago
This episode features Dr. Maxwell Ramstead and Jason Fox both from Noumenal discussing why current AI approaches fall short for real-world applications and what's needed for true physical AI.
The guests argue that today's AI systems, including large language models, are fundamentally "stuck in data space" - they only process patterns in data rather than understanding the physical world that generates that data.
Maxwell uses Plato's Cave as a powerful metaphor: like prisoners seeing only shadows on a wall, LLMs interact with representations of reality (text, images) rather than reality itself.
Rather than building monolithic models, Noumenal is creating a compositional system - essentially a "marketplace of models" where specialized AI components can be dynamically combined and deployed to robots.
Sponsor messages:
========
Tufa AI Labs are hiring for ML Engineers and a Chief Scientist in Zurich/SF. They are top of the ARCv2 leaderboard!
tufalabs.ai
========
TRANSCRIPT:
app.rescript.info/public/share/DVaCBhYF3Y-1Q2kA2N5-YkV4kOGBNZc5KLrDwntLznU
noumenal.ai
https://x.com/mjdramstead
scholar.google.ca/citations?user=ILpGOMkAAAAJ&hl=fr
https://x.com/jasongfox?lang=en-GB
TOC
Opening & Context
00:00:00 - Opening Hook: Why Create a Physical AI Company?
00:01:59 - Sponsor: Tufa AI Labs
00:02:30 - Guest Introductions: Maxwell Ramstead & Jason Fox
00:05:18 - Noumenal Background
Core Problems with Current AI
00:09:30 - The Embodiment Problem: Why Bodies Matter
00:10:15 - LLMs Lack Physical Grounding
00:12:00 - AI Stuck in Plato's Cave
00:16:15 - Language as Wrong Compression for Physics
00:17:22 - The Exhaustion of Static Datasets
00:19:54 - Humans as the Grounding for LLMs
Philosophical Foundations
00:28:00 - Fractured vs. Deep Understanding
00:32:15 - Defining "Real": When You Bump Into Things
00:37:00 - Emergence: Weak vs. Strong Causal Power
00:41:45 - The Free Energy Principle Explained
00:44:15 - Constraints: How the Universe Builds Things
Objects, Intelligence & Grounding
00:46:15 - What Is an Object? From Data to Physics
00:51:00 - Learning Primitives & Predictive Grip
00:55:58 - "There Is No General Intelligence"
01:00:15 - The Human-AI Feedback Loop
01:03:08 - The Irony of LLM Specialization
01:06:05 - LLMs as Tools vs. Autonomous Agents
01:08:45 - Hallucinating Capabilities: The Third Leg Problem
The Noumenal Solution
01:09:00 - A Marketplace of Specialized Models
01:13:45 - Dynamic Skill Loading: "Phone a Friend"
01:16:15 - Learning from Brain Evolution
01:18:00 - Business Model Critique: Why OpenAI Won't Work
01:22:30 - The Physical Dataset Problem
Implementation & Future
01:22:30 - Community-Driven Data Collection
01:24:45 - Jim Fan's Physical Turing Test
01:26:30 - Enterprise vs. Consumer Models
01:27:22 - Docker for Robotics: The Technical Architecture
01:30:12 - Reproducibility in Learning Systems
01:32:00 - Closing Thoughts
This episode features Dr. Maxwell Ramstead and Jason Fox both from Noumenal discussing why current AI approaches fall short for real-world applications and what's needed for true physical AI.
The guests argue that today's AI systems, including large language models, are fundamentally "stuck in data space" - they only process patterns in data rather than understanding the physical world that generates that data.
Maxwell uses Plato's Cave as a powerful metaphor: like prisoners seeing only shadows on a wall, LLMs interact with representations of reality (text, images) rather than reality itself.
Rather than building monolithic models, Noumenal is creating a compositional system - essentially a "marketplace of models" where specialized AI components can be dynamically combined and deployed to robots.
Sponsor messages:
========
Tufa AI Labs are hiring for ML Engineers and a Chief Scientist in Zurich/SF. They are top of the ARCv2 leaderboard!
tufalabs.ai
========
TRANSCRIPT:
app.rescript.info/public/share/DVaCBhYF3Y-1Q2kA2N5-YkV4kOGBNZc5KLrDwntLznU
noumenal.ai
https://x.com/mjdramstead
scholar.google.ca/citations?user=ILpGOMkAAAAJ&hl=fr
https://x.com/jasongfox?lang=en-GB
TOC
Opening & Context
00:00:00 - Opening Hook: Why Create a Physical AI Company?
00:01:59 - Sponsor: Tufa AI Labs
00:02:30 - Guest Introductions: Maxwell Ramstead & Jason Fox
00:05:18 - Noumenal Background
Core Problems with Current AI
00:09:30 - The Embodiment Problem: Why Bodies Matter
00:10:15 - LLMs Lack Physical Grounding
00:12:00 - AI Stuck in Plato's Cave
00:16:15 - Language as Wrong Compression for Physics
00:17:22 - The Exhaustion of Static Datasets
00:19:54 - Humans as the Grounding for LLMs
Philosophical Foundations
00:28:00 - Fractured vs. Deep Understanding
00:32:15 - Defining "Real": When You Bump Into Things
00:37:00 - Emergence: Weak vs. Strong Causal Power
00:41:45 - The Free Energy Principle Explained
00:44:15 - Constraints: How the Universe Builds Things
Objects, Intelligence & Grounding
00:46:15 - What Is an Object? From Data to Physics
00:51:00 - Learning Primitives & Predictive Grip
00:55:58 - "There Is No General Intelligence"
01:00:15 - The Human-AI Feedback Loop
01:03:08 - The Irony of LLM Specialization
01:06:05 - LLMs as Tools vs. Autonomous Agents
01:08:45 - Hallucinating Capabilities: The Third Leg Problem
The Noumenal Solution
01:09:00 - A Marketplace of Specialized Models
01:13:45 - Dynamic Skill Loading: "Phone a Friend"
01:16:15 - Learning from Brain Evolution
01:18:00 - Business Model Critique: Why OpenAI Won't Work
01:22:30 - The Physical Dataset Problem
Implementation & Future
01:22:30 - Community-Driven Data Collection
01:24:45 - Jim Fan's Physical Turing Test
01:26:30 - Enterprise vs. Consumer Models
01:27:22 - Docker for Robotics: The Technical Architecture
01:30:12 - Reproducibility in Learning Systems
01:32:00 - Closing Thoughts
![AI HAS A BODY PROBLEM... [Dr. Maxwell Ramstead]](https://i.ytimg.com/vi/jsIt2sTB_vo/mqdefault.jpg "AI HAS A BODY PROBLEM... [Dr. Maxwell Ramstead]
This episode features Dr. Maxwell Ramstead and Jason Fox both from Noumenal discussing why current AI approaches fall short for real-world applications and whats needed for true physical AI.
The guests argue that todays AI systems, including large language models, are fundamentally stuck in data space - they only process patterns in data rather than understanding the physical world that generates that data.
Maxwell uses Platos Cave as a powerful metaphor: like prisoners seeing only shadows on a wall, LLMs interact with representations of reality (text, images) rather than reality itself.
Rather than building monolithic models, Noumenal is creating a compositional system - essentially a marketplace of models where specialized AI components can be dynamically combined and deployed to robots.
Sponsor messages:
Tufa AI Labs are hiring for ML Engineers and a Chief Scientist in Zurich/SF. They are top of the ARCv2 leaderboard!
https://tufalabs.ai/
TRANSCRIPT:
http://app.rescript.info/public/share/DVaCBhYF3Y-1Q2kA2N5-YkV4kOGBNZc5KLrDwntLznU
https://www.noumenal.ai/
https://x.com/mjdramstead
https://scholar.google.ca/citations?user=ILpGOMkAAAAJ&hl=fr
https://x.com/jasongfox?lang=en-GB
TOC
Opening & Context
00:00:00 - Opening Hook: Why Create a Physical AI Company?
00:01:59 - Sponsor: Tufa AI Labs
00:02:30 - Guest Introductions: Maxwell Ramstead & Jason Fox
00:05:18 - Noumenal Background
Core Problems with Current AI
00:09:30 - The Embodiment Problem: Why Bodies Matter
00:10:15 - LLMs Lack Physical Grounding
00:12:00 - AI Stuck in Platos Cave
00:16:15 - Language as Wrong Compression for Physics
00:17:22 - The Exhaustion of Static Datasets
00:19:54 - Humans as the Grounding for LLMs
Philosophical Foundations
00:28:00 - Fractured vs. Deep Understanding
00:32:15 - Defining Real: When You Bump Into Things
00:37:00 - Emergence: Weak vs. Strong Causal Power
00:41:45 - The Free Energy Principle Explained
00:44:15 - Constraints: How the Universe Builds Things
Objects, Intelligence & Grounding
00:46:15 - What Is an Object? From Data to Physics
00:51:00 - Learning Primitives & Predictive Grip
00:55:58 - There Is No General Intelligence
01:00:15 - The Human-AI Feedback Loop
01:03:08 - The Irony of LLM Specialization
01:06:05 - LLMs as Tools vs. Autonomous Agents
01:08:45 - Hallucinating Capabilities: The Third Leg Problem
The Noumenal Solution
01:09:00 - A Marketplace of Specialized Models
01:13:45 - Dynamic Skill Loading: Phone a Friend
01:16:15 - Learning from Brain Evolution
01:18:00 - Business Model Critique: Why OpenAI Wont Work
01:22:30 - The Physical Dataset Problem
Implementation & Future
01:22:30 - Community-Driven Data Collection
01:24:45 - Jim Fans Physical Turing Test
01:26:30 - Enterprise vs. Consumer Models
01:27:22 - Docker for Robotics: The Technical Architecture
01:30:12 - Reproducibility in Learning Systems
01:32:00 - Closing Thoughts")
![Neural Networks Are Elastic Origami! [Prof. Randall Balestriero]](https://i.ytimg.com/vi/l3O2J3LMxqI/mqdefault.jpg "Neural Networks Are Elastic Origami! [Prof. Randall Balestriero]
Professor Randall Balestriero joins us to discuss neural network geometry, spline theory, and emerging phenomena in deep learning, based on research presented at ICML. Topics include the delayed emergence of adversarial robustness in neural networks (grokking), geometric interpretations of neural networks via spline theory, and challenges in reconstruction learning. We also cover geometric analysis of Large Language Models (LLMs) for toxicity detection and the relationship between intrinsic dimensionality and model control in RLHF.
SPONSOR MESSAGES:
***
CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments.
https://centml.ai/pricing/
Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. Are you interested in working on reasoning, or getting involved in their events?
Goto https://tufalabs.ai/
***
Show notes and transcript: https://www.dropbox.com/scl/fi/3lufge4upq5gy0ug75j4a/RANDALLSHOW.pdf?rlkey=nbemgpa0jhawt1e86rx7372e4&dl=0
TOC:
[00:00:00] Introduction
1. Neural Network Geometry and Spline Theory
[00:01:41] 1.1 Neural Network Geometry and Spline Theory
[00:07:41] 1.2 Deep Networks Always Grok
[00:11:39] 1.3 Grokking and Adversarial Robustness
[00:16:09] 1.4 Double Descent and Catastrophic Forgetting
2. Reconstruction Learning
[00:18:49] 2.1 Reconstruction Learning
[00:24:15] 2.2 Frequency Bias in Neural Networks
3. Geometric Analysis of Neural Networks
[00:29:02] 3.1 Geometric Analysis of Neural Networks
[00:34:41] 3.2 Adversarial Examples and Region Concentration
4. LLM Safety and Geometric Analysis
[00:40:05] 4.1 LLM Safety and Geometric Analysis
[00:46:11] 4.2 Toxicity Detection in LLMs
[00:52:24] 4.3 Intrinsic Dimensionality and Model Control
[00:58:07] 4.4 RLHF and High-Dimensional Spaces
5. Conclusion
[01:02:13] 5.1 Neural Tangent Kernel
[01:08:07] 5.2 Conclusion
REFS:
[00:01:35] Balestriero/Humayun – Deep network geometry & input space partitioning
https://arxiv.org/html/2408.04809v1
[00:03:55] Balestriero & Paris – Linking deep networks to adaptive spline operators
https://proceedings.mlr.press/v80/balestriero18b/balestriero18b.pdf
[00:13:55] Song et al. – Gradient-based white-box adversarial attacks
https://arxiv.org/abs/2012.14965
[00:16:05] Humayun, Balestriero & Baraniuk – Grokking phenomenon & emergent robustness
https://arxiv.org/abs/2402.15555
[00:18:25] Humayun – Training dynamics & double descent via linear region evolution
https://arxiv.org/abs/2310.12977
[00:20:15] Balestriero – Power diagram partitions in DNN decision boundaries
https://arxiv.org/abs/1905.08443
[00:23:00] Frankle & Carbin – Lottery Ticket Hypothesis for network pruning
https://arxiv.org/abs/1803.03635
[00:24:00] Belkin et al. – Double descent phenomenon in modern ML
https://arxiv.org/abs/1812.11118
[00:25:55] Balestriero et al. – Batch normalization’s regularization effects
https://arxiv.org/pdf/2209.14778
[00:29:35] EU – EU AI Act 2024 with compute restrictions
https://www.lw.com/admin/upload/SiteAttachments/EU-AI-Act-Navigating-a-Brave-New-World.pdf
[00:39:30] Humayun, Balestriero & Baraniuk – SplineCam: Visualizing deep network geometry
https://openaccess.thecvf.com/content/CVPR2023/papers/Humayun_SplineCam_Exact_Visualization_and_Characterization_of_Deep_Network_Geometry_and_CVPR_2023_paper.pdf
[00:40:40] Carlini – Trade-offs between adversarial robustness and accuracy
https://arxiv.org/abs/1902.06705
[00:44:55] Balestriero & LeCun – Limitations of reconstruction-based learning methods
https://raw.githubusercontent.com/mlresearch/v235/main/assets/balestriero24b/balestriero24b.pdf
[00:47:20] Balestriero & LeCun – Spectral analysis of neural network learning
https://proceedings.neurips.cc/paper_files/paper/2022/file/aa56c74513a5e35768a11f4e82dd7ffb-Paper-Conference.pdf
[00:49:45] He et al. – MAE: Masked Autoencoders for self-supervised learning
https://arxiv.org/abs/2111.06377
[00:54:50] Balestriero et al. – Geometric analysis of LLM layers for toxicity detection
https://arxiv.org/abs/2309.12312
[00:59:35] Balestriero et al. – Superior toxicity detection via geometric features
https://arxiv.org/html/2312.01648v2
[01:04:45] UofT ML – Self-attention control & context length effects
https://arxiv.org/abs/2310.04444
[01:11:55] Roberts – Foundations of deep learning theory
https://arxiv.org/abs/2106.10165
[01:15:40] Balestriero & Cha – Kolmogorov GAM Networks via spline partition theory
https://arxiv.org/pdf/2501.00704
[01:16:40] Various – Graph Kolmogorov-Arnold Networks (GKAN) extension
https://www.nature.com/articles/s41598-024-85083-8")
![The ARC Prize 2024 Winning Algorithm [Daniel Franzen and Jan Disselhoff]](https://i.ytimg.com/vi/mTX_sAq--zY/mqdefault.jpg "The ARC Prize 2024 Winning Algorithm [Daniel Franzen and Jan Disselhoff]
SPONSOR MESSAGES:
***
CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. Check out their super fast DeepSeek R1 hosting!
https://centml.ai/pricing/
Daniel Franzen and Jan Disselhoff the ARChitects won the ARC Prize 2024 with co-researcher David Hartmann, achieving a remarkable 53.5% accuracy on the ARC challenge using a 12-billion parameter language model. Filmed at Tufa Labs in Zurich, they walk through their solution architecture in detail: how they tokenized grid-based visual puzzles as text and fed them directly into an LLM, why test-time training on evaluation examples gave a major score boost, and how depth-first search over token probabilities outperformed greedy and beam search for solution generation.
The conversation gets into the surprising computational capabilities of language models on spatial reasoning tasks. The team found that LLMs could handle 2D grid reasoning without explicit positional encodings, that symmetry augmentations served as a powerful validation mechanism rather than a training aid, and that the models second-best solutions were often conceptually correct just wrong in a specific detail like rotation direction. They also discuss why their fine-tuned 12B model outperformed much larger foundation models, the role of LoRA in preventing catastrophic forgetting during test-time training, and what the entropy distribution of their search trees reveals about how the model represents uncertainty across different task types.
REFERENCES:
Paper:
[00:01:00] The ARChitects: Winning ARC Prize 2024 Solution
https://github.com/da-fr/arc-prize-2024/blob/main/the_architects.pdf
[00:03:38] Robustness of Analogical Reasoning in LLMs
https://arxiv.org/html/2411.14215
[00:14:58] Search Methods in Language Models
https://arxiv.org/html/2408.00724v2
[00:22:28] GPT-4 Code Solutions for ARC (50% SOTA)
https://redwoodresearch.substack.com/p/getting-50-sota-on-arc-agi-with-gpt
[00:53:08] Overcoming Catastrophic Forgetting
https://www.pnas.org/doi/10.1073/pnas.1611835114
[00:53:58] LoRA: Low-Rank Adaptation of Large Language Models
https://arxiv.org/abs/2106.09685
Tool:
[00:07:48] Re-ARC Dataset Generator
https://github.com/michaelhodel/re-arc
LINKS:
Full Transcript: https://app.rescript.info/share/57e5d773f2d0b195cbce7eee1f53aef2
Download PDF transcript: https://app.rescript.info/api/public/sessions/7772acbf1f11f44b/pdf
Daniel Franzen
https://github.com/da-fr
REFS
[00:01:05] Winning ARC 2024 solution using 12B param model, Franzen, Disselhoff, Hartmann
https://github.com/da-fr/arc-prize-2024/blob/main/the_architects.pdf
[00:07:50] Re-ARC dataset generator for ARC task variations, Michael Hodel
https://github.com/michaelhodel/re-arc
[00:22:30] GPT-4 guided code solutions for ARC tasks, Ryan Greenblatt
https://redwoodresearch.substack.com/p/getting-50-sota-on-arc-agi-with-gpt")
