Uploaded December 2025 | Updated November 2026, 4 days ago
We often think of Large Language Models (LLMs) as all-knowing, but as the team reveals, they still struggle with the logic of a second-grader. Why can’t ChatGPT reliably add large numbers? Why does it "hallucinate" the laws of physics? The answer lies in the architecture. This episode explores how *Category Theory*—an ultra-abstract branch of mathematics—could provide the "Periodic Table" for neural networks, turning the "alchemy" of modern AI into a rigorous science.
In this deep-dive exploration, *Andrew Dudzik*, *Petar Velichkovich*, *Taco Cohen*, *Bruno Gavranović*, and *Paul Lessard* join host *Tim Scarfe* to discuss the fundamental limitations of today’s AI and the radical mathematical framework that might fix them.
---
Key Insights in This Episode:
* *The "Addition" Problem:* *Andrew Dudzik* explains why LLMs don't actually "know" math—they just recognize patterns. When you change a single digit in a long string of numbers, the pattern breaks because the model lacks the internal "machinery" to perform a simple carry operation.
* *Beyond Alchemy:* *Tim Scarfe* argues that deep learning is currently in its "alchemy" phase—we have powerful results, but we lack a unifying theory. Category Theory is proposed as the framework to move AI from trial-and-error to principled engineering. [00:13:49]
* *Algebra with Colors:* To make Category Theory accessible, the guests use brilliant analogies—like thinking of matrices as *magnets with colors* that only snap together when the types match. This "partial compositionality" is the secret to building more complex internal reasoning. [00:09:17]
* *Synthetic vs. Analytic Math:* *Paul Lessard* breaks down the philosophical shift needed in AI research: moving from "Analytic" math (what things are made of) to "Synthetic" math (how things behave and relate to one another). [00:23:41]
* *The 4D Carry:* In a mind-blowing conclusion, the team discusses how simple algorithmic tasks, like "carrying the one" in addition, actually relate to complex geometric structures like *Hopf Fibrations*. [00:39:30]
---
Why This Matters for AGI
If we want AI to solve the world's hardest scientific problems, it can't just be a "stochastic parrot." It needs to internalize the rules of logic and computation. By imbuing neural networks with categorical priors, researchers are attempting to build a future where AI doesn't just predict the next word—it understands the underlying structure of the universe.
---
TIMESTAMPS:
00:00:00 The Failure of LLM Addition & Physics
00:01:26 Tool Use vs Intrinsic Model Quality
00:03:07 Efficiency Gains via Internalization
00:04:28 Geometric Deep Learning & Equivariance
00:07:05 Limitations of Group Theory
00:09:17 Category Theory: Algebra with Colors
00:11:25 The Systematic Guide of Lego-like Math
00:13:49 The Alchemy Analogy & Unifying Theory
00:15:33 Information Destruction & Reasoning
00:18:00 Pathfinding & Monoids in Computation
00:20:15 System 2 Reasoning & Error Awareness
00:23:31 Analytic vs Synthetic Mathematics
00:25:52 Morphisms & Weight Tying Basics
00:26:48 2-Categories & Weight Sharing Theory
00:28:55 Higher Categories & Emergence
00:31:41 Compositionality & Recursive Folds
00:34:05 Syntax vs Semantics in Network Design
00:36:14 Homomorphisms & Multi-Sorted Syntax
00:39:30 The Carrying Problem & Hopf Fibrations
---
REFERENCES:
Company:
Model:
[00:01:05] Veo
https://deepmind.google/models/veo/
[00:01:10] Genie
https://deepmind.google/blog/genie-3-a-new-frontier-for-world-models/
Paper:
[00:04:30] Geometric Deep Learning Blueprint
arxiv.org/abs/2104.13478
[00:16:45] AlphaGeometry
https://deepmind.google/blog/alphageometry-an-olympiad-level-ai-system-for-geometry/
[00:16:55] AlphaCode
arxiv.org/abs/2203.07814
[00:17:05] FunSearch
nature.com/articles/s41586-023-06924-6
[00:37:00] Attention Is All You Need
arxiv.org/abs/1706.03762
[00:43:00] Categorical Deep Learning
arxiv.org/abs/2402.15332
Thanks to ly3xqhl8g9 on our Discord server for the draft show review!
We often think of Large Language Models (LLMs) as all-knowing, but as the team reveals, they still struggle with the logic of a second-grader. Why can’t ChatGPT reliably add large numbers? Why does it "hallucinate" the laws of physics? The answer lies in the architecture. This episode explores how *Category Theory*—an ultra-abstract branch of mathematics—could provide the "Periodic Table" for neural networks, turning the "alchemy" of modern AI into a rigorous science.
In this deep-dive exploration, *Andrew Dudzik*, *Petar Velichkovich*, *Taco Cohen*, *Bruno Gavranović*, and *Paul Lessard* join host *Tim Scarfe* to discuss the fundamental limitations of today’s AI and the radical mathematical framework that might fix them.
---
Key Insights in This Episode:
* *The "Addition" Problem:* *Andrew Dudzik* explains why LLMs don't actually "know" math—they just recognize patterns. When you change a single digit in a long string of numbers, the pattern breaks because the model lacks the internal "machinery" to perform a simple carry operation.
* *Beyond Alchemy:* *Tim Scarfe* argues that deep learning is currently in its "alchemy" phase—we have powerful results, but we lack a unifying theory. Category Theory is proposed as the framework to move AI from trial-and-error to principled engineering. [00:13:49]
* *Algebra with Colors:* To make Category Theory accessible, the guests use brilliant analogies—like thinking of matrices as *magnets with colors* that only snap together when the types match. This "partial compositionality" is the secret to building more complex internal reasoning. [00:09:17]
* *Synthetic vs. Analytic Math:* *Paul Lessard* breaks down the philosophical shift needed in AI research: moving from "Analytic" math (what things are made of) to "Synthetic" math (how things behave and relate to one another). [00:23:41]
* *The 4D Carry:* In a mind-blowing conclusion, the team discusses how simple algorithmic tasks, like "carrying the one" in addition, actually relate to complex geometric structures like *Hopf Fibrations*. [00:39:30]
---
Why This Matters for AGI
If we want AI to solve the world's hardest scientific problems, it can't just be a "stochastic parrot." It needs to internalize the rules of logic and computation. By imbuing neural networks with categorical priors, researchers are attempting to build a future where AI doesn't just predict the next word—it understands the underlying structure of the universe.
---
TIMESTAMPS:
00:00:00 The Failure of LLM Addition & Physics
00:01:26 Tool Use vs Intrinsic Model Quality
00:03:07 Efficiency Gains via Internalization
00:04:28 Geometric Deep Learning & Equivariance
00:07:05 Limitations of Group Theory
00:09:17 Category Theory: Algebra with Colors
00:11:25 The Systematic Guide of Lego-like Math
00:13:49 The Alchemy Analogy & Unifying Theory
00:15:33 Information Destruction & Reasoning
00:18:00 Pathfinding & Monoids in Computation
00:20:15 System 2 Reasoning & Error Awareness
00:23:31 Analytic vs Synthetic Mathematics
00:25:52 Morphisms & Weight Tying Basics
00:26:48 2-Categories & Weight Sharing Theory
00:28:55 Higher Categories & Emergence
00:31:41 Compositionality & Recursive Folds
00:34:05 Syntax vs Semantics in Network Design
00:36:14 Homomorphisms & Multi-Sorted Syntax
00:39:30 The Carrying Problem & Hopf Fibrations
---
REFERENCES:
Company:
Model:
[00:01:05] Veo
https://deepmind.google/models/veo/
[00:01:10] Genie
https://deepmind.google/blog/genie-3-a-new-frontier-for-world-models/
Paper:
[00:04:30] Geometric Deep Learning Blueprint
arxiv.org/abs/2104.13478
[00:16:45] AlphaGeometry
https://deepmind.google/blog/alphageometry-an-olympiad-level-ai-system-for-geometry/
[00:16:55] AlphaCode
arxiv.org/abs/2203.07814
[00:17:05] FunSearch
nature.com/articles/s41586-023-06924-6
[00:37:00] Attention Is All You Need
arxiv.org/abs/1706.03762
[00:43:00] Categorical Deep Learning
arxiv.org/abs/2402.15332
Thanks to ly3xqhl8g9 on our Discord server for the draft show review!

![AI in 2024 - efficiency over model size (Nick Jakobi)
This video is sponsored by Cohere
Nick Jakobi, Director of Product for the modeling team at Cohere, discusses the practical realities of deploying large language models in enterprise environments. The conversation covers how Cohere differentiates in a crowded LLM market through enterprise-focused capabilities like retrieval-augmented generation (RAG), tool use, and multilingual support, rather than competing purely on model scale.
Jakobi offers a candid perspective on the economics of model selection: most enterprise customers gravitate toward the smallest model that solves their use case, making cost and latency as important as raw capability. He describes the emerging token game where model providers compete on price-performance ratios, and argues that making medium-sized models cheaper and faster may be more commercially valuable than pushing the frontier of model intelligence.
The discussion extends into broader questions about how generative AI is reshaping education, workforce dynamics, and information ecosystems. Jakobi shares personal reflections on whether traditional education remains relevant when AI tools are becoming ubiquitous, and offers a measured take on AI safety that centers on the practical challenges of disinformation and autonomous agency rather than speculative existential risks.
REFERENCES:
Company:
[00:00:00] Cohere
https://cohere.com/
Paper:
[00:00:00] Episode Shownotes
https://www.dropbox.com/scl/fi/sjwqllqh1ughuu8ismonk/NickJakobi2.pdf?rlkey=d20wydhhala2hmh4yaoo55wyy&st=unso2w6i&dl=0
[00:00:30] RAG Paper
https://arxiv.org/abs/2005.11401
Person:
[00:00:20] Nick Jakobi
https://www.linkedin.com/in/nickjakobi/
Product:
[00:14:50] Cohere Command R
https://cohere.com/command
Tool:
[00:20:50] LMSYS Chatbot Arena
https://chat.lmsys.org/
LINKS:
Full Transcript: https://app.rescript.info/share/d694e87868aa6d0272fe26e0a7b4de87
Download PDF transcript: https://app.rescript.info/api/public/sessions/72c99a6b30e9b8d0/pdf AI in 2024 - efficiency over model size (Nick Jakobi)](https://i.ytimg.com/vi/BJMi_dAwE8k/mqdefault.jpg)


![Test-Time Adaptation: A New Frontier in AI [Dr. Jonas Hübotter]
Jonas Hübotter, PhD student at ETH Zurichs Institute for Machine Learning, discusses his groundbreaking research on test-time computation and local learning. He demonstrates how smaller models can outperform larger ones by 30x through strategic test-time computation and introduces a novel paradigm combining inductive and transductive learning approaches.
Using Bayesian linear regression as a surrogate model for uncertainty estimation, Jonas explains how models can efficiently adapt to specific tasks without massive pre-training. He draws an analogy to Google Earths variable resolution system to illustrate dynamic resource allocation based on task complexity.
The conversation explores the future of AI architecture, envisioning systems that continuously learn and adapt beyond current monolithic models. Jonas concludes by proposing hybrid deployment strategies combining local and cloud computation, suggesting a future where compute resources are allocated based on task complexity rather than fixed model size.
This research represents a significant shift in machine learning, prioritizing intelligent resource allocation and adaptive learning over traditional scaling approaches.
SPONSOR MESSAGES:
CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments.
https://centml.ai/pricing/
Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on ARC and AGI, they just acquired MindsAI - the current winners of the ARC challenge. Are you interested in working on ARC, or getting involved in their events? Goto https://tufalabs.ai/
Transcription, references and show notes PDF download:
https://www.dropbox.com/scl/fi/cxg80p388snwt6qbp4m52/JonasFinal.pdf?rlkey=glk9mhpzjvesanlc14rtpvk4r&st=6qwi8n3x&dl=0
Jonas Hübotter
https://jonhue.github.io/
https://scholar.google.com/citations?user=pxi_RkwAAAAJ
Transductive Active Learning: Theory and Applications (NeurIPS 2024)
https://arxiv.org/pdf/2402.15898
EFFICIENTLY LEARNING AT TEST-TIME: ACTIVE FINE-TUNING OF LLMS (SIFT)
https://arxiv.org/pdf/2410.08020
TOC:
1. Test-Time Computation Fundamentals
[00:00:00] Intro
[00:03:10] 1.1 Test-Time Computation and Model Performance Comparison
[00:05:52] 1.2 Retrieval Augmentation and Machine Teaching Strategies
[00:09:40] 1.3 In-Context Learning vs Fine-Tuning Trade-offs
2. System Architecture and Intelligence
[00:15:58] 2.1 System Architecture and Intelligence Emergence
[00:23:22] 2.2 Active Inference and Constrained Agency in AI
[00:29:52] 2.3 Evolution of Local Learning Methods
[00:32:05] 2.4 Vapniks Contributions to Transductive Learning
3. Resource Optimization and Local Learning
[00:34:35] 3.1 Computational Resource Allocation in ML Models
[00:35:30] 3.2 Historical Context and Traditional ML Optimization
[00:37:55] 3.3 Variable Resolution Processing and Active Inference in ML
[00:43:01] 3.4 Local Learning and Base Model Capacity Trade-offs
[00:48:04] 3.5 Active Learning vs Local Learning Approaches
4. Information Retrieval and Model Interpretability
[00:51:08] 4.1 Information Retrieval and Nearest Neighbor Limitations
[01:03:07] 4.2 Model Interpretability and Surrogate Models
[01:15:03] 4.3 Bayesian Uncertainty Estimation and Surrogate Models
5. Distributed Systems and Deployment
[01:23:56] 5.1 Memory Architecture and Controller Systems
[01:28:14] 5.2 Evolution from Static to Distributed Learning Systems
[01:38:03] 5.3 Transductive Learning and Model Specialization
[01:41:58] 5.4 Hybrid Local-Cloud Deployment Strategies Test-Time Adaptation: A New Frontier in AI [Dr. Jonas Hübotter]](https://i.ytimg.com/vi/C6sSs6NgANo/mqdefault.jpg)

![The Gap Between Humans and Machines Is [Dr. Max Bartolo]
SPONSOR MESSAGES:
***
Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich.
Dr. Max Bartolo from Cohere discusses the gap between model capabilities and genuine robustness: why next-token prediction can produce impressive results yet still fail on slightly reformulated…
TIMESTAMPS:
00:00:00 Model Reasoning and Consistency Verification
00:03:25 Influence Functions and Distributed Knowledge Analysis
00:10:28 AI Application Development and Model Deployment
00:14:24 AI Alignment and Human Feedback Limitations
00:20:15 Human Evaluation Challenges and Factuality Assessment
00:27:15 Cultural and Demographic Influences on Model Behavior
00:32:43 Adversarial Examples and Model Robustness
00:41:54 DynaBench and Dynamic Benchmarking Approaches
00:50:02 Benchmarking Challenges and Data-Centric Evaluation
00:55:15 Cohere Command A Development Process
01:00:26 Model Quantization and Performance Evaluation
01:05:18 Reasoning Capabilities and Training Progression
01:13:48 Context Windows and Enterprise Applications
REFERENCES:
person:
[00:00:00] Max Bartolo Website
https://www.maxbartolo.com/
company:
[00:00:00] Cohere
https://cohere.com/command
[00:12:10] Command A Model
https://huggingface.co/CohereForAI/c4ai-command-a-03-2025
paper:
[00:03:25] Procedural Knowledge in Pretraining Drives Reasoning in LLMs
https://cohere.com/research/papers/procedural-knowledge-in-pretraining-drives-reasoning-in-large-language-models-2024-11-20
[00:04:15] Influence Functions in Machine Learning
https://arxiv.org/abs/1703.04730
[00:08:05] Studying Large Language Model Generalization with Influence Functions
https://arxiv.org/abs/2308.03296
[00:16:15] Human Feedback is not Gold Standard
https://arxiv.org/abs/2309.16349
[00:27:15] The PRISM Alignment Dataset
https://arxiv.org/abs/2404.16019
[00:32:50] Adversarial Examples Are Not Bugs, They Are Features
https://arxiv.org/abs/1905.02175
[00:43:00] DynaBench: Rethinking Benchmarking in NLP
https://aclanthology.org/2021.naacl-main.324.pdf
[00:50:15] Sara Hooker on Compute Limitations
https://arxiv.org/html/2407.05694v1
[00:53:25] DataPerf: Benchmarks for Data-Centric AI
https://arxiv.org/abs/2207.10062
[01:04:35] DROP: A Reading Comprehension Benchmark
https://arxiv.org/abs/1903.00161
[01:07:05] GSM8k
https://paperswithcode.com/sota/arithmetic-reasoning-on-gsm8k
[01:09:30] ARC-AGI Challenge
https://github.com/fchollet/ARC-AGI
LINKS:
Full Transcript: https://app.rescript.info/share/163bf5e7338685f635fc0b8d6920005c
Download PDF transcript: https://app.rescript.info/api/public/sessions/7167a679366f97f8/pdf
REFS:
[00:03:10] Research at Cohere with Laura Ruis et al., Max Bartolo, Laura Ruis et al.
https://cohere.com/research/papers/procedural-knowledge-in-pretraining-drives-reasoning-in-large-language-models-2024-11-20
[00:04:15] Influence functions in machine learning, Koh & Liang
https://arxiv.org/abs/1703.04730
[00:08:05] Studying Large Language Model Generalization with Influence Functions, Roger Grosse et al.
https://storage.prod.researchhub.com/uploads/papers/2023/08/08/2308.03296.pdf
[00:11:10] The LLM ARChitect: Solving ARC-AGI Is A Matter of Perspective, Daniel Franzen, Jan Disselhoff, and David Hartmann
https://github.com/da-fr/arc-prize-2024/blob/main/the_architects.pdf
[00:12:10] Hugging Face model repo for C4AI Command A, Cohere and Cohere For AI
https://huggingface.co/CohereForAI/c4ai-command-a-03-2025
[00:13:30] OpenInterpreter
https://github.com/KillianLucas/open-interpreter
[00:16:15] Human Feedback is not Gold Standard, Tom Hosking, Max Bartolo, Phil Blunsom
https://arxiv.org/abs/2309.16349
[00:27:15] The PRISM Alignment Dataset, Hannah Kirk et al.
https://arxiv.org/abs/2404.16019
[00:32:50] How adversarial examples arise, Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, Aleksander Madry
https://arxiv.org/abs/1905.02175
[00:43:00] DynaBench platform paper, Douwe Kiela et al.
https://aclanthology.org/2021.naacl-main.324.pdf
[00:50:15] Sara Hookers work on compute limitations, Sara Hooker
https://arxiv.org/html/2407.05694v1
[00:53:25] DataPerf: Community-led benchmark suite, Mazumder et al.
https://arxiv.org/abs/2207.10062
[01:04:35] DROP, Dheeru Dua et al.
https://arxiv.org/abs/1903.00161
[01:07:05] GSM8k, Cobbe et al.
https://paperswithcode.com/sota/arithmetic-reasoning-on-gsm8k
[01:09:30] ARC, François Chollet
https://github.com/fchollet/ARC-AGI
[01:15:50] Command A, Cohere
https://cohere.com/blog/command-a
[01:22:55] Enterprise search using LLMs, Cohere
https://cohere.com/blog/commonly-asked-questions-about-search-from-coheres-enterprise-customers The Gap Between Humans and Machines Is [Dr. Max Bartolo]](https://i.ytimg.com/vi/DL7qwmWWk88/mqdefault.jpg)

![Manhattan Project for AI Safety [Connor Leahy]
Connor Leahy and Gabriel Alfour, AI researchers from Conjecture and authors of The Compendium, join us for a critical discussion centered on Artificial Superintelligence (ASI) safety and governance. Drawing from their comprehensive analysis in The Compendium, they articulate a stark warning about the existential risks inherent in uncontrolled AI development, framing it through the lens of intelligence domination—where a sufficiently advanced AI could subordinate humanity, much like humans dominate less intelligent species.
SPONSOR MESSAGES:
***
Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich.
Goto https://tufalabs.ai/
***
TRANSCRIPT + REFS + NOTES:
https://www.dropbox.com/scl/fi/p86l75y4o2ii40df5t7no/Compendium.pdf?rlkey=tukczgf3flw133sr9rgss0pnj&dl=0
Transcript (new system): https://app.rescript.info/public/share/FObT5GvZYfnixs-aSAD8px4Je_4_aC1kHYG76bvBBRU
https://www.thecompendium.ai/
https://en.wikipedia.org/wiki/Connor_Leahy
https://www.conjecture.dev/about
https://substack.com/@gabecc
TOC:
1. AI Intelligence and Safety Fundamentals
[00:00:00] 1.1 Understanding Intelligence and AI Capabilities
[00:06:20] 1.2 Emergence of Intelligence and Regulatory Challenges
[00:10:18] 1.3 Human vs Animal Intelligence Debate
[00:18:00] 1.4 AI Regulation and Risk Assessment Approaches
[00:26:14] 1.5 Competing AI Development Ideologies
2. Economic and Social Impact
[00:29:10] 2.1 Labor Market Disruption and Post-Scarcity Scenarios
[00:32:40] 2.2 Institutional Frameworks and Tech Power Dynamics
[00:37:40] 2.3 Ethical Frameworks and AI Governance Debates
[00:40:52] 2.4 AI Alignment Evolution and Technical Challenges
3. Technical Governance Framework
[00:55:07] 3.1 Three Levels of AI Safety: Alignment, Corrigibility, and Boundedness
[00:55:30] 3.2 Challenges of AI System Corrigibility and Constitutional Models
[00:57:35] 3.3 Limitations of Current Boundedness Approaches
[00:59:11] 3.4 Abstract Governance Concepts and Policy Solutions
4. Democratic Implementation and Coordination
[00:59:20] 4.1 Governance Design and Measurement Challenges
[01:00:10] 4.2 Democratic Institutions and Experimental Governance
[01:14:10] 4.3 Political Engagement and AI Safety Advocacy
[01:25:30] 4.4 Practical AI Safety Measures and International Coordination
CORE REFS:
[00:01:45] The Compendium (2023), Leahy et al.
https://pdf.thecompendium.ai/the_compendium.pdf
[00:06:50] Geoffrey Hinton Leaves Google, BBC News
https://www.bbc.com/news/world-us-canada-65452940
[00:10:00] ARC-AGI, Chollet
https://arcprize.org/arc-agi
[00:13:25] A Brief History of Intelligence, Bennett
https://www.amazon.com/Brief-History-Intelligence-Humans-Breakthroughs/dp/0063286343
[00:25:35] Statement on AI Risk, Center for AI Safety
https://www.safe.ai/work/statement-on-ai-risk
[00:26:15] Machines of Love and Grace, Amodei
https://darioamodei.com/machines-of-loving-grace
[00:26:35] The Techno-Optimist Manifesto, Andreessen
https://a16z.com/the-techno-optimist-manifesto/
[00:31:55] Techno-Feudalism, Varoufakis
https://www.amazon.co.uk/Technofeudalism-Killed-Capitalism-Yanis-Varoufakis/dp/1847927270
[00:42:40] Introducing Superalignment, OpenAI
https://openai.com/index/introducing-superalignment/
[00:47:20] Three Laws of Robotics, Asimov
https://www.britannica.com/topic/Three-Laws-of-Robotics
[00:50:00] Symbolic AI (GOFAI), Haugeland
https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence
[00:52:30] Intent Alignment, Christiano
https://www.alignmentforum.org/posts/HEZgGBZTpT4Bov7nH/mapping-the-conceptual-territory-in-ai-existential-safety
[00:55:10] Large Language Model Alignment: A Survey, Jiang et al.
http://arxiv.org/pdf/2309.15025
[00:55:40] Constitutional Checks and Balances, Bok
https://plato.stanford.edu/entries/montesquieu/
[00:59:25] Goodharts Law, Manheim & Garrabrant
https://arxiv.org/pdf/1803.04585
[01:00:25] French Constitution Article 37-1, French National Assembly
https://www2.assemblee-nationale.fr/langues/welcome-to-the-english-website-of-the-french-national-assembly
[01:03:00] Intelligence Enhancement and AI Alignment, Yudkowsky
https://intelligence.org/files/AIPosNegFactor.pdf
[01:14:10] UK Government AI Regulation Response, UK Government
https://www.gov.uk/government/consultations/ai-regulation-a-pro-innovation-approach-policy-proposals/outcome/a-pro-innovation-approach-to-ai-regulation-government-response
[01:20:20] French AI Action Summit, Caroli
https://www.csis.org/analysis/frances-ai-action-summit
[01:20:45] Holden Karnofsky Joins Anthropic, Fortune Magazine
https://fortune.com/2025/02/13/anthropic-hired-president-daniela-amodei-husband-ai-safety-responsible-scaling/
[01:25:35] Kill Switch Mechanisms, IEEE/ISO
https://arxiv.org/html/2410.22151v1 Manhattan Project for AI Safety [Connor Leahy]](https://i.ytimg.com/vi/Dt1ySXYTGuA/mqdefault.jpg)
![He Co-Invented the Transformer. Now: Continuous Thought Machines [Llion Jones / Luke Darlow]
The Transformer architecture (which powers ChatGPT and nearly all modern AI) might be trapping the industry in a localized rut, preventing us from finding true intelligent reasoning, according to the person who co-invented it. Llion Jones and Luke Darlow, key figures at the research lab Sakana AI, join the show to make this provocative argument, and also introduce new research (CTM) which might lead the way forwards.
We speak about Inventors Remorse & The Trap of Success Despite being one of the original authors of the famous Attention Is All You Need paper that gave birth to the Transformer, Llion explains why he has largely stopped working on them. He argues that the industry is suffering from success capture—because Transformers work so well, everyone is focused on making small tweaks to the same architecture rather than discovering the next big leap.
**SPONSOR MESSAGES START**
—
Build your ideas with AI Studio from Google - http://ai.studio/build
—
Tufa AI Labs is hiring ML Research Engineers https://tufalabs.ai/
—
cyber•Fund https://cyber.fund/?utm_source=mlst is a founder-led investment firm accelerating the cybernetic economy
Hiring a SF VC Principal: https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal#content?utm_source=mlst
Submit investment deck: https://cyber.fund/contact?utm_source=mlst
—
**END**
The Spiral Problem – Llion uses a striking visual analogy to explain what current AI is missing. If you ask a standard neural network to understand a spiral shape, it solves it by drawing tiny straight lines that just happen to look like a spiral. It fakes the shape without understanding the concept of spiraling. They argue that todays AI models are similar—they are incredible at mimicking intelligent answers without having an internal process of thinking.
Introducing the Continuous Thought Machine (CTM) Luke Darlow deep dives into their solution: a biology-inspired model that fundamentally changes how AI processes information.
The Maze Analogy: Luke explains that standard AI tries to solve a maze by staring at the whole image and guessing the entire path instantly. Their new machine walks through the maze step-by-step.
Thinking Time: This allows the AI to ponder. If a problem is hard, the model can naturally spend more time thinking about it before answering, effectively allowing it to correct its own mistakes and backtrack—something current Language Models struggle to do genuinely.
The pair discuss the culture of Sakana AI, which is modeled after the early days of Google Brain/DeepMind. Llion nostalgically recalls that the Transformer wasnt born from a corporate mandate, but from random people talking over lunch about interesting problems.
https://sakana.ai/
https://x.com/YesThisIsLion
https://x.com/LearningLukeD
TRANSCRIPT:
https://app.rescript.info/public/share/crjzQ-Jo2FQsJc97xsBdfzfOIeMONpg0TFBuCgV2Fu8
TOC:
00:00:00 - Stepping Back from Transformers
00:00:43 - Introduction to Continuous Thought Machines (CTM)
00:01:09 - The Changing Atmosphere of AI Research
00:04:13 - Sakana’s Philosophy: Research Freedom
00:07:45 - The Local Minimum of Large Language Models
00:18:30 - Representation Problems: The Spiral Example
00:29:12 - Technical Deep Dive: CTM Architecture
00:36:00 - Adaptive Computation & Maze Solving
00:47:15 - Model Calibration & Uncertainty
01:00:43 - Sudoku Bench: Measuring True Reasoning
REFS:
Why Greatness Cannot be planned [Kenneth Stanley]
https://www.amazon.co.uk/Why-Greatness-Cannot-Planned-Objective/dp/3319155237
https://www.youtube.com/watch?v=lhYGXYeMq_E
The Hardware Lottery [Sara Hooker]
https://arxiv.org/abs/2009.06489
https://www.youtube.com/watch?v=sQFxbQ7ade0
Continuous Thought Machines [Luke Darlow et al / Sakana]
https://arxiv.org/abs/2505.05522
https://sakana.ai/ctm/
https://youtu.be/5X9cjGLggv0 great walkthrough of algo by Yacine Mahdid
LSTM: The Comeback Story? [Prof. Sepp Hochreiter]
https://www.youtube.com/watch?v=8u2pW2zZLCs
Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis [Kumar/Stanley]
https://arxiv.org/pdf/2505.11581
Intelligent Matrix Exponentiation [Thomas Fischbacher] (Spiral reference)
https://arxiv.org/abs/2008.03936
A Spline Theory of Deep Networks [Randall Balestriero]
https://proceedings.mlr.press/v80/balestriero18b/balestriero18b.pdf
https://www.youtube.com/watch?v=86ib0sfdFtw
https://www.youtube.com/watch?v=l3O2J3LMxqI
On the Biology of a Large Language Model [Anthropic, Jack Lindsey et al]
https://transformer-circuits.pub/2025/attribution-graphs/biology.html
The ARC Prize 2024 Winning Algorithm [Daniel Franzen and Jan Disselhoff] “The ARChitects”
https://www.youtube.com/watch?v=mTX_sAq zY
Neural Turing Machine [Graves]
https://arxiv.org/pdf/1410.5401
Adaptive Computation Time for Recurrent Neural Networks [Graves]
https://arxiv.org/abs/1603.08983
Sudoko Bench [Sakana]
https://pub.sakana.ai/sudoku/ He Co-Invented the Transformer. Now: Continuous Thought Machines [Llion Jones / Luke Darlow]](https://i.ytimg.com/vi/DtePicx_kFY/mqdefault.jpg)
