Uploaded November 2026 | Updated November 2026, 2 hours ago
In 2010 Simon Prince spent a sabbatical in Toronto writing a book that recast computer vision as probabilistic graphical models, with Alex Krizhevsky in the next room. He released it in 2012, a few months before AlexNet came out.
Full conversation on Machine Learning Street Talk, "This is why Deep Learning is really weird.":
youtube.com/watch?v=sJXn4Cl4oww
#Shorts #deeplearning #AlexNet #MachineLearning #MLST
In 2010 Simon Prince spent a sabbatical in Toronto writing a book that recast computer vision as probabilistic graphical models, with Alex Krizhevsky in the next room. He released it in 2012, a few months before AlexNet came out.
Full conversation on Machine Learning Street Talk, "This is why Deep Learning is really weird.":
youtube.com/watch?v=sJXn4Cl4oww
#Shorts #deeplearning #AlexNet #MachineLearning #MLST
![The Universal Hierarchy of Life - Prof. Chris Kempes [SFI]
What is life? - asks Chris Kempes, a professor at the Santa Fe Institute.
Chris explains that scientists are moving beyond a purely Earth-based, biological view and are searching for a universal theory of life that could apply to anything, anywhere in the universe. He proposes that things we dont normally consider alive—like human culture, language, or even artificial intelligence; could be seen as life forms existing on different substrates.
To understand this, Chris presents a fascinating three-level framework:
- Materials: The physical stuff life is made of. He argues this could be incredibly diverse across the universe, and we shouldnt expect alien life to share our biochemistry.
- Constraints: The universal laws of physics (like gravity or diffusion) that all life must obey, regardless of what its made of. This is where different life forms start to look more similar.
- Principles: At the highest level are abstract principles like evolution and learning. Chris suggests these computational or optimization rules are what truly define a living system.
A key idea is convergence – using the example of the eye. Its such a complex organ that youd think it evolved only once. However, eyes evolved many separate times across different species. This is because the physics of light provides a clear target, and evolution found similar solutions to the problem of seeing, even with different starting materials.
**SPONSOR MESSAGES**
—
Prolific - Quality data. From real people. For faster breakthroughs.
https://www.prolific.com/?utm_source=mlst
—
Check out NotebookLM from Google here - https://notebooklm.google.com/ - it’s really good for doing research directly from authoritative source material, minimising hallucinations.
—
cyber•Fund https://cyber.fund/?utm_source=mlst is a founder-led investment firm accelerating the cybernetic economy
Hiring a SF VC Principal: https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal#content?utm_source=mlst
Submit investment deck: https://cyber.fund/contact?utm_source=mlst
—
Prof. Chris Kempes:
https://www.santafe.edu/people/profile/chris-kempes
TRANSCRIPT:
https://app.rescript.info/public/share/Y2cI1i0nX iuZitvlguHvaVLQTwPX1Y_E1EHxV0i9I
TOC:
00:00:00 - Introduction to Chris Kempes and the Santa Fe Institute
00:02:28 - The Three Cultures of Science
00:05:08 - What Makes a Good Scientific Theory?
00:06:50 - The Universal Theory of Life
00:09:40 - The Role of Material in Life
00:12:50 - A Hierarchy for Understanding Life
00:13:55 - How Life Diversifies and Converges
00:17:53 - Adaptive Processes and Defining Life
00:19:28 - Functionalism, Memes, and Phylogenies
00:22:58 - Convergence at Multiple Levels
00:25:45 - The Possibility of Simulating Life
00:28:16 - Intelligence, Parasitism, and Spectrums of Life
00:32:39 - Phase Changes in Evolution
00:36:16 - The Separation of Matter and Logic
00:37:21 - Assembly Theory and Quantifying Complexity
REFS:
Developing a predictive science of the biosphere requires the integration of scientific cultures [Kempes et al]
https://www.pnas.org/doi/10.1073/pnas.2209196121
Seeing with an extra sense (“Dangerous prediction”) [Rob Phillips]
https://www.sciencedirect.com/science/article/pii/S0960982224009035
The Multiple Paths to Multiple Life [Christopher P. Kempes & David C. Krakauer]
https://link.springer.com/article/10.1007/s00239-021-10016-2
The Information Theory of Individuality [David Krakauer et al]
https://arxiv.org/abs/1412.2447
Minds, Brains and Programs [Searle]
https://home.csulb.edu/~cwallis/382/readings/482/searle.minds.brains.programs.bbs.1980.pdf
The error threshold
https://www.sciencedirect.com/science/article/abs/pii/S0168170204003843
Assembly theory and its relationship with computational complexity [Kempes et al]
https://arxiv.org/abs/2406.12176 The Universal Hierarchy of Life - Prof. Chris Kempes [SFI]](https://i.ytimg.com/vi/iwClZ-7OweY/mqdefault.jpg)
![Speech Recognition Is Not a Solved Problem — Pavan Kumar Reddy
Pavan Kumar Reddy leads audio research at Mistral AI. He joins Tim Scarfe for a deep technical tour of Voxtral — and explains why, after everything that has landed in the last two years, the frontier of deployed voice is still a cascade of specialised models rather than one end-to-end system.
IN PARTNERSHIP WITH MISTRAL AI:
This episode was produced in partnership with Mistral AI.
Mistral AI: https://mistral.ai/
The conversation opens on architecture. Voxtral Chat feeds a 3B Ministral text trunk with continuous embeddings from an audio encoder, fed into the decoder as direct token input rather than through cross-attention as in Whisper, so the model can answer questions about emotion, timing and who spoke when without an intermediate transcript to lose them. The real-time model changes shape into a dual-stream decoder that consumes audio and emits text at the same time, at a configurable target delay down to 160ms, with slower streams running in parallel for anything that can afford to wait for more context.
On the generation side, Pavan explains why Voxtral TTS predicts continuous latents rather than discrete codec tokens, traces the lineage from SoundStream through EnCodec to Mimis split of semantic and acoustic codebooks, and places finite scalar quantisation and flow matching in it. Tim presses on the engineering priors underneath: why a mel spectrogram instead of the raw waveform, what noise augmentation actually buys, and the point at which acoustic overfitting becomes somebodys fine-tuning problem. Then the failure modes. Speaker diarisation is emitted autoregressively as part of the transcript rather than by a separate head, which makes streaming diarisation fragile in a particular way — less context, late speaker changes, invented extra speakers. And because the architecture commits to what it has already predicted, a single out-of-distribution mistake can compound into looping or skipped segments, which is what DPO is there to correct: the negative supervision that pre-training and SFT cannot provide.
The last third is the argument Tim keeps returning to. Customers running voice agents over millions of sessions do not describe a solved problem, they describe scaffolding, with a sharp quality drop outside the top few languages. Cascades survive because each component stays separately adaptable, observable and constrainable — fine-tune the ASR for your acoustics, log what compliance demands, keep it on your own hardware. The interface has its own limit: voice alone is cognitive debt, because absorbing information and deciding in one serial audio stream is much harder than glancing at a menu. Voice becomes ubiquitous beside a screen, not instead of one.
TIMESTAMPS:
00:00:00 Cold open: the state changed
00:00:46 Why Mistral moved into audio
00:09:27 Inside Voxtral: the trunk, the encoder and dual streams
00:20:22 Speech that works in real time
00:30:52 How a voice becomes tokens
00:39:59 Flow matching, FSQ and the new codec
00:52:51 When speech models lose the speaker
01:03:23 Correcting hallucinations with preferences
01:12:12 Controlling synthetic speech
01:20:06 Why cascades still win
01:29:25 Speech in the wild
01:33:46 Audio models as interfaces
01:37:54 Why voice still needs a screen
REFERENCES:
paper:
[00:01:42] Mistral 7B
https://arxiv.org/abs/2310.06825
[00:09:38] Voxtral
https://arxiv.org/abs/2507.13264
[00:14:41] Robust Speech Recognition via Large-Scale Weak Supervision (Whisper)
https://arxiv.org/abs/2212.04356
[00:19:11] Voxtral Realtime
https://arxiv.org/abs/2602.11298
[00:21:52] Streaming Sequence-to-Sequence Learning with Delayed Streams Modeling
https://arxiv.org/abs/2509.08753
[00:30:52] Voxtral TTS
https://arxiv.org/abs/2603.25551
[00:32:38] SoundStream: An End-to-End Neural Audio Codec
https://arxiv.org/abs/2107.03312
[00:34:59] Flow Matching for Generative Modeling
https://arxiv.org/abs/2210.02747
[00:37:03] High Fidelity Neural Audio Compression (EnCodec)
https://arxiv.org/abs/2210.13438
[00:37:42] Moshi: a speech-text foundation model for real-time dialogue (Mimi)
https://arxiv.org/abs/2410.00037
[00:39:05] Finite Scalar Quantization: VQ-VAE Made Simple
https://arxiv.org/abs/2309.15505
[01:03:33] Direct Preference Optimization: Your Language Model is Secretly a Reward Model
https://arxiv.org/abs/2305.18290
benchmark:
[01:15:40] ElevenLabs v3 and Flash comparison in Voxtral TTS
https://arxiv.org/abs/2603.25551
dataset:
[00:46:14] Mozilla Common Voice datasets
https://commonvoice.mozilla.org/en/datasets
organization:
[00:01:24] Mistral AI
https://mistral.ai/
[00:50:47] Hugging Face
https://huggingface.co/ Speech Recognition Is Not a Solved Problem — Pavan Kumar Reddy](https://i.ytimg.com/vi/ixu0H8bsCts/mqdefault.jpg)
![Explosive AI Timeline Predictions [Gary Marcus, Daniel Kokotajlo, Dan Hendrycks]
What if the most powerful technology in human history is being built by people who openly admit they dont trust each other? In this explosive 2-hour debate, three AI experts pull back the curtain on the shocking psychology driving the race to Artificial General Intelligence—and why the people building it might be the biggest threat of all. Kokotajlo predicts AGI by 2028 based on compute scaling trends. Marcus argues we havent solved basic cognitive problems from his 2001 research. The stakes? If Kokotajlo is right and Marcus is wrong about safety progress, humanity may have already lost control.
Sponsor messages:
Google Gemini: Google Gemini features Veo3, a state-of-the-art AI video generation model in the Gemini app. Sign up at https://gemini.google.com
Tufa AI Labs are hiring for ML Engineers and a Chief Scientist in Zurich/SF. They are top of the ARCv2 leaderboard!
https://tufalabs.ai/
Guest Powerhouse
Gary Marcus - Cognitive scientist, author of Taming Silicon Valley, and AIs most prominent skeptic whos been warning about the same fundamental problems for 25 years (https://garymarcus.substack.com/)
Daniel Kokotajlo - Former OpenAI insider turned whistleblower who reveals the disturbing rationalizations of AI lab leaders in his viral AI 2027 scenario (https://ai-2027.com/)
Dan Hendrycks - Director of the Center for AI Safety who created the benchmarks used to measure AI progress and argues we have only years, not decades, to prevent catastrophe (https://danhendrycks.com/)
Transcript: http://app.rescript.info/public/share/tEcx4UkToi-2jwS1cN51CW70A4Eh6QulBRxDILoXOno
TOC:
Introduction: The AI Arms Race
00:00:04 - The Danger of Automated AI R&D
00:00:43 - The Rationalization: If we dont, someone else will
00:01:56 - Sponsor Reads (Tufa AI Labs & Google Gemini)
00:02:55 - Guest Introductions
The Philosophical Stakes
00:04:13 - What is the Positive Vision for AGI?
00:07:00 - The Abundance Scenario: Superintelligent Economy
00:09:06 - Differentiating AGI and Superintelligence (ASI)
00:11:41 - Sam Altman: A Decade in a Month
00:14:47 - Economic Inequality & The UBI Problem
Policy and Red Lines
00:17:13 - The Pause Letter: Stopping vs. Delaying AI
00:20:03 - Defining Three Concrete Red Lines for AI Development
00:25:24 - Racing Towards Red Lines & The Myth of Durable Advantage
00:31:15 - Transparency and Public Perception
00:35:16 - The Rationalization Cascade: Why AI Labs Race to Win
Forecasting AGI: Timelines and Methodologies
00:42:29 - The Case for Short Timelines (Median 2028)
00:47:00 - Scaling Limits: Compute, Data, and Money
00:49:36 - Forecasting Models: Bio-Anchors and Agentic Coding
00:53:15 - The 10^45 FLOP Thought Experiment
The Great Debate: Cognitive Gaps vs. Scaling
00:58:41 - Gary Marcuss Counterpoint: The Unsolved Problems of Cognition
01:00:46 - Current AI Cant Play Chess Reliably
01:08:23 - Can Tools and Neurosymbolic AI Fill the Gaps?
01:16:13 - The Multi-Dimensional Nature of Intelligence
01:24:26 - The Benchmark Debate: Data Contamination and Reliability
01:31:15 - The Superhuman Coder Milestone Debate
01:37:45 - The Driverless Car Analogy
The Alignment Problem
01:39:45 - Has Any Progress Been Made on Alignment?
01:42:43 - Fairly Reasonably Scares the Sh*t Out of Me
01:46:30 - Distinguishing Model vs. Process Alignment
Scenarios and Conclusions
01:49:26 - Garys Alternative Scenario: The Neurosymbolic Shift
01:53:35 - Will AI Become Jeff Dean?
01:58:41 - Takeoff Speeds and Exceeding Human Intelligence
02:03:19 - Final Disagreements and Closing Remarks
REFS:
Gary Marcus (2001) - The Algebraic Mind
https://mitpress.mit.edu/9780262632683/the-algebraic-mind/
00:59:00
Gary Marcus & Ernest Davis (2019) - Rebooting AI
https://www.amazon.co.uk/Rebooting-AI-Building-Artificial-Intelligence-ebook/dp/B07MYLGQLB
01:31:59
Gary Marcus (2024) - Taming Silicon Valley
https://www.amazon.co.uk/Taming-Silicon-Valley-Ensure-Works-ebook/dp/B0CQWWM94N
00:03:01
Ajeya Cotra (2020) - Forecasting TAI with Biological Anchors
https://www.alignmentforum.org/posts/KrJfoZzpSDpnrv9va/draft-report-on-ai-timelines
00:53:15
Daniel Kokotajlo (2021) - What 2026 looks like
https://www.lesswrong.com/posts/6Xgy6CAf2jqHhynHL/what-2026-looks-like
00:55:00
Dan Hendrycks et al. (2021) - Measuring Massive Multitask Language Understanding (MMLU)
https://arxiv.org/abs/2009.03300
00:03:48
Dan Hendrycks et al. (2021) - Measuring Mathematical Problem Solving With the MATH Dataset
https://arxiv.org/abs/2103.03874
00:03:48
Aitor Lewkowycz, Anders Andreassen et al. (2022) - Solving Quantitative Reasoning Problems with Language Models (Minerva)
https://arxiv.org/abs/2206.14858
01:21:45
Apple Research (2024) - GSM-Symbolic/The Illusion of Thinking
https://arxiv.org/abs/2410.05229
https://machinelearning.apple.com/research/illusion-of-thinking
00:59:15 Explosive AI Timeline Predictions [Gary Marcus, Daniel Kokotajlo, Dan Hendrycks]](https://i.ytimg.com/vi/j13ySJLvdOc/mqdefault.jpg)


![AGI in 5 Years? Ben Goertzel on Superintelligence
Ben Goertzel discusses AGI development, transhumanism, and the potential societal impacts of superintelligent AI. He predicts human-level AGI by 2029 and argues that the transition to superintelligence could happen within a few years after. Goertzel explores the challenges of AI regulation, the limitations of current language models, and the need for neuro-symbolic approaches in AGI research. He also addresses concerns about resource allocation and cultural perspectives on transhumanism.
TOC:
[00:00:00] AGI Timeline Predictions and Development Speed
[00:00:45] Limitations of Language Models in AGI Development
[00:02:18] Current State and Trends in AI Research and Development
[00:09:02] Emergent Reasoning Capabilities and Limitations of LLMs
[00:18:15] Neuro-Symbolic Approaches and the Future of AI Systems
[00:20:00] Evolutionary Algorithms and LLMs in Creative Tasks
[00:21:25] Symbolic vs. Sub-Symbolic Approaches in AI
[00:28:05] Language as Internal Thought and External Communication
[00:30:20] AGI Development and Goal-Directed Behavior
[00:35:51] Consciousness and AI: Expanding States of Experience
[00:48:50] AI Regulation: Challenges and Approaches
[00:55:35] Challenges in AI Regulation
[00:59:20] AI Alignment and Ethical Considerations
[01:09:15] AGI Development Timeline Predictions
[01:12:40] OpenCog Hyperon and AGI Progress
[01:17:48] Transhumanism and Resource Allocation Debate
[01:20:12] Cultural Perspectives on Transhumanism
[01:23:54] AGI and Post-Scarcity Society
[01:31:35] Challenges and Implications of AGI Development
New! PDF Show notes: https://www.dropbox.com/scl/fi/fyetzwgoaf70gpovyfc4x/BenGoertzel.pdf?rlkey=pze5dt9vgf01tf2wip32p5hk5&st=svbcofm3&dl=0
Refs:
00:00:15 Ray Kurzweils AGI timeline prediction, Ray Kurzweil, https://en.wikipedia.org/wiki/Technological_singularity
00:01:45 Ben Goertzel: SingularityNET founder, Ben Goertzel, https://singularitynet.io/
00:02:35 AGI Conference series, AGI Conference Organizers, https://agi-conf.org/2024/
00:03:55 Ben Goertzels contributions to AGI, Wikipedia contributors, https://en.wikipedia.org/wiki/Ben_Goertzel
00:11:05 Chain-of-Thought prompting, Subbarao Kambhampati, https://arxiv.org/abs/2405.04776
00:11:35 Algorithmic information content, Pieter Adriaans, https://plato.stanford.edu/entries/information-entropy/
00:12:10 Turing completeness in neural networks, Various contributors, https://plato.stanford.edu/entries/turing-machine/
00:16:15 AlphaGeometry: AI for geometry problems, Trieu, Li, et al., https://www.nature.com/articles/s41586-023-06747-5
00:18:25 Shane Legg and Ben Goertzels collaboration, Shane Legg, https://en.wikipedia.org/wiki/Shane_Legg
00:20:00 Evolutionary algorithms in music generation, Yanxu Chen, https://arxiv.org/html/2409.03715v1
00:22:00 Peirces theory of semiotics, Charles Sanders Peirce, https://plato.stanford.edu/entries/peirce-semiotics/
00:28:10 Chomskys view on language, Noam Chomsky, https://chomsky.info/1983 /
00:34:05 Greg Egans Diaspora, Greg Egan, https://www.amazon.co.uk/Diaspora-post-apocalyptic-thriller-perfect-MIRROR/dp/0575082097
00:40:35 The Consciousness Explosion, Ben Goertzel & Gabriel Axel Montes, https://www.amazon.com/Consciousness-Explosion-Technological-Experiential-Singularity/dp/B0D8C7QYZD
00:41:55 Ray Kurzweils books on singularity, Ray Kurzweil, https://www.amazon.com/Singularity-Near-Humans-Transcend-Biology/dp/0143037889
00:50:50 California AI regulation bills, California State Senate, https://sd18.senate.ca.gov/news/senate-unanimously-approves-senator-padillas-artificial-intelligence-package
00:56:40 Limitations of Compute Thresholds, Sara Hooker, https://arxiv.org/abs/2407.05694
00:56:55 Taming Silicon Valley, Gary F. Marcus, https://www.penguinrandomhouse.com/books/768076/taming-silicon-valley-by-gary-f-marcus/
01:09:15 Kurzweils AGI prediction update, Ray Kurzweil, https://www.theguardian.com/technology/article/2024/jun/29/ray-kurzweil-google-ai-the-singularity-is-nearer
01:14:45 OpenCog Hyperon framework, Ben Goertzel et al., https://arxiv.org/abs/2310.18318
01:18:25 Malnutrition in Ethiopia, Abriham Shiferaw Areba, https://www.frontiersin.org/journals/nutrition/articles/10.3389/fnut.2024.1403591/full
01:18:40 Transhumanism ethical debate, Nick Bostrom, https://nickbostrom.com/papers/history.pdf AGI in 5 Years? Ben Goertzel on Superintelligence](https://i.ytimg.com/vi/jSDEsvVdL-E/mqdefault.jpg)
![Can Outsourcing Thinking Make Us Dumber? [Prof. David Krakauer]
Prof. David Krakauer, President of the Santa Fe Institute argues that we are fundamentally confusing knowledge with intelligence, especially when it comes to AI.
He defines true intelligence as the ability to do more with less—to solve novel problems with limited information. This is contrasted with current AI models, which he describes as doing less with more; they require astounding amounts of data to perform tasks that dont necessarily demonstrate true understanding or adaptation. He humorously calls this really shit programming.
David challenges the popular notion of emergence in Large Language Models (LLMs). He explains that the tech communitys definition—seeing a sudden jump in a models ability to perform a task like three-digit math—is superficial. True emergence, from a complex systems perspective, involves a fundamental change in the systems internal organization, allowing for a new, simpler, and more powerful level of description. He gives the example of moving from tracking individual water molecules to using the elegant laws of fluid dynamics. For LLMs to be truly emergent, wed need to see them develop new, efficient internal representations, not just get better at memorizing patterns as they scale.
Drawing on his background in evolutionary theory, David explains that systems like brains, and later, culture, evolved to process information that changes too quickly for genetic evolution to keep up. He calls culture evolution at light speed because it allows us to store our accumulated knowledge externally (in books, tools, etc.) and build upon it without corrupting the original.
This leads to his concept of exbodiment, where we outsource our cognitive load to the world through things like maps, abacuses, or even language itself.
We create these external tools, internalize the skills they teach us, improve them, and create a feedback loop that enhances our collective intelligence.
However, he ends with a warning. While technology has historically complemented our deficient abilities, modern AI presents a new danger. Because we have an evolutionary drive to conserve energy, we will inevitably outsource our thinking to AI if we can. He fears this is already leading to a diminution and dilution of human thought and creativity. Just as our muscles atrophy without use, he argues our brains will too, and we risk becoming mentally dependent on these systems.
RESCRIPT LINK (interactive transcript):
https://app.rescript.info/public/share/nCL8fdE_m3J6fA3SHlreCADpkWpbXaEF1JQ14z6N7Y8
TOC:
[00:00:00] Intelligence: Doing more with less
[00:02:10] Why brains evolved: The limits of evolution
[00:05:18] Culture as evolution at light speed
[00:08:11] True meaning of emergence: More is Different
[00:10:41] Why LLM capabilities are not true emergence
[00:15:10] What real emergence would look like in AI
[00:19:24] Symmetry breaking: Physics vs. Life
[00:23:30] Two types of emergence: Knowledge In vs. Out
[00:26:46] Causality, agency, and coarse-graining
[00:32:24] Exbodiment: Outsourcing thought to objects
[00:35:05] Collective intelligence & the boundary of the mind
[00:39:45] Mortal vs. Immortal forms of computation
[00:42:13] The risk of AI: Atrophy of human thought
David Krakauer
President and William H. Miller Professor of Complex Systems
https://www.santafe.edu/people/profile/david-krakauer
REFS:
Large Language Models and Emergence: A Complex Systems Perspective
David C. Krakauer, John W. Krakauer, Melanie Mitchell
https://arxiv.org/abs/2506.11135
Filmed at the Diverse Intelligences Summer Institute:
https://disi.org/ Can Outsourcing Thinking Make Us Dumber? [Prof. David Krakauer]](https://i.ytimg.com/vi/jXa8dHzgV8U/mqdefault.jpg)
![AI HAS A BODY PROBLEM... [Dr. Maxwell Ramstead]
This episode features Dr. Maxwell Ramstead and Jason Fox both from Noumenal discussing why current AI approaches fall short for real-world applications and whats needed for true physical AI.
The guests argue that todays AI systems, including large language models, are fundamentally stuck in data space - they only process patterns in data rather than understanding the physical world that generates that data.
Maxwell uses Platos Cave as a powerful metaphor: like prisoners seeing only shadows on a wall, LLMs interact with representations of reality (text, images) rather than reality itself.
Rather than building monolithic models, Noumenal is creating a compositional system - essentially a marketplace of models where specialized AI components can be dynamically combined and deployed to robots.
Sponsor messages:
Tufa AI Labs are hiring for ML Engineers and a Chief Scientist in Zurich/SF. They are top of the ARCv2 leaderboard!
https://tufalabs.ai/
TRANSCRIPT:
http://app.rescript.info/public/share/DVaCBhYF3Y-1Q2kA2N5-YkV4kOGBNZc5KLrDwntLznU
https://www.noumenal.ai/
https://x.com/mjdramstead
https://scholar.google.ca/citations?user=ILpGOMkAAAAJ&hl=fr
https://x.com/jasongfox?lang=en-GB
TOC
Opening & Context
00:00:00 - Opening Hook: Why Create a Physical AI Company?
00:01:59 - Sponsor: Tufa AI Labs
00:02:30 - Guest Introductions: Maxwell Ramstead & Jason Fox
00:05:18 - Noumenal Background
Core Problems with Current AI
00:09:30 - The Embodiment Problem: Why Bodies Matter
00:10:15 - LLMs Lack Physical Grounding
00:12:00 - AI Stuck in Platos Cave
00:16:15 - Language as Wrong Compression for Physics
00:17:22 - The Exhaustion of Static Datasets
00:19:54 - Humans as the Grounding for LLMs
Philosophical Foundations
00:28:00 - Fractured vs. Deep Understanding
00:32:15 - Defining Real: When You Bump Into Things
00:37:00 - Emergence: Weak vs. Strong Causal Power
00:41:45 - The Free Energy Principle Explained
00:44:15 - Constraints: How the Universe Builds Things
Objects, Intelligence & Grounding
00:46:15 - What Is an Object? From Data to Physics
00:51:00 - Learning Primitives & Predictive Grip
00:55:58 - There Is No General Intelligence
01:00:15 - The Human-AI Feedback Loop
01:03:08 - The Irony of LLM Specialization
01:06:05 - LLMs as Tools vs. Autonomous Agents
01:08:45 - Hallucinating Capabilities: The Third Leg Problem
The Noumenal Solution
01:09:00 - A Marketplace of Specialized Models
01:13:45 - Dynamic Skill Loading: Phone a Friend
01:16:15 - Learning from Brain Evolution
01:18:00 - Business Model Critique: Why OpenAI Wont Work
01:22:30 - The Physical Dataset Problem
Implementation & Future
01:22:30 - Community-Driven Data Collection
01:24:45 - Jim Fans Physical Turing Test
01:26:30 - Enterprise vs. Consumer Models
01:27:22 - Docker for Robotics: The Technical Architecture
01:30:12 - Reproducibility in Learning Systems
01:32:00 - Closing Thoughts AI HAS A BODY PROBLEM... [Dr. Maxwell Ramstead]](https://i.ytimg.com/vi/jsIt2sTB_vo/mqdefault.jpg)
![One Step Closer to the Star Trek Voice AI Assistant!
Will Williams is CTO of Speechmatics in Cambridge. In this sponsored episode - he shares deep technical insights into modern speech recognition technology and system architecture. The episode covers several key technical areas:
Will Williams is CTO of Speechmatics, the Cambridge-based speech recognition company. Tim has used their API for years to caption MLST episodes, and this conversation happened in their offices which explains the live demo at the start where an AI moderates a political debate clip in real time.
The technical meat covers how Speechmatics builds production ASR systems. Their approach is hybrid: self-supervised pre-training on unlabeled audio gets them comparable accuracy to fully supervised systems like Whisper, but with roughly 100x less labeled data. Williams explains why this matters for scaling to low-resource languages where you simply dont have thousands of hours of human-transcribed speech.
The architecture discussion is detailed. Their system runs multiple operating points with different latency-accuracy tradeoffs. They pad latency up to 1.8 seconds to keep the user experience consistent rather than optimizing for raw speed. Decoding uses lattices with language model integration, which lets them rescore hypotheses and handle things like proper nouns and domain-specific vocabulary without retraining the acoustic model.
Diarization figuring out who said what comes up repeatedly. Williams calls it harder than ASR itself, partly because speaker embeddings get corrupted by acoustic environments and partly because cross-talk creates genuinely ambiguous boundaries. Theyre pushing hard on implicit source separation but the problem remains open.
The conversation also covers their testing infrastructure (mirrored production traffic catches edge cases that unit tests miss), why they resist customer-specific fine-tuning (it fragments the model and makes global improvements harder), and Williams critique of PyTorch memory management in production settings. He argues for more direct memory allocation rather than letting the framework handle it, which is a practical concern when youre serving models at scale.
Featuring: Will Williams (CTO, Speechmatics) and Tim Scarfe.
TIMESTAMPS:
00:00:00 ASR and diarization fundamentals
00:05:25 Real-time conversational AI architecture
00:09:21 Neural network streaming and multi-modal integration
00:12:49 Enterprise voice AI and real-time translation
00:20:00 Production deployment and testing infrastructure
00:29:38 Model architecture and latency-accuracy tradeoffs
00:35:40 Lattice-based decoding and language model integration
00:44:00 ASR performance metrics and real-world evaluation
00:51:30 Ethics and privacy in speech technology
01:00:50 Self-supervised learning and low-resource languages
01:11:00 Feature engineering to automated ML
01:21:00 Infrastructure scaling and PyTorch critique
01:35:00 Future of conversational AI and Ursa 2
REFERENCES:
paper:
[00:00:05] Speechmatics PDF shownotes
https://www.dropbox.com/scl/fi/d94b1jcgph9o8au8shdym/Speechmatics.pdf?rlkey=bi55wvktzomzx0y5sic6jz99y&st=6qwofv8t&dl=0
[00:10:09] GFlowNets
https://arxiv.org/abs/2106.04399
[01:35:00] Ursa 2 model
https://www.speechmatics.com/company/articles-and-news/ursa-2-elevating-speech-recognition-across-52-languages
company:
[00:01:15] Speechmatics
https://www.speechmatics.com/
person:
[00:01:32] Will Williams
https://x.com/wjwwilliams
LINKS:
Full Transcript: https://app.rescript.info/share/c6887b6d7b214f93daad1c18d70e2eb6
Download PDF transcript: https://app.rescript.info/api/public/sessions/abeef42b31287680/pdf
Will Williams, CTO, Speechmatics
https://x.com/wjwwilliams One Step Closer to the Star Trek Voice AI Assistant!](https://i.ytimg.com/vi/k6eXkBtYIHg/mqdefault.jpg)

![AIs can now imagine video games in real-time
Ashley Edwards, who was working at DeepMind when she co-authored the Genie paper and is now at Runway, covered several key aspects of the Genie AI system and its applications in video generation, robotics, and game creation.
MLST is sponsored by Brave:
The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api.
Genies approach to learning interactive environments, balancing compression and fidelity.
The use of latent action models and VQE models for video processing and tokenization.
Challenges in maintaining action consistency across frames and integrating text-to-image models.
Evaluation metrics for AI-generated content, such as FID and PS&R diff metrics.
The discussion also explored broader implications and applications:
The potential impact of AI video generation on content creation jobs.
Applications of Genie in game generation and robotics.
The use of foundation models in robotics and the differences between internet video data and specialized robotics data.
Challenges in mapping AI-generated actions to real-world robotic actions.
Ashley Edwards: https://ashedwards.github.io/
TOC (*) are best bits
00:00:00 1. Intro to Genie & Brave Search API: Trade-offs & limitations *
00:02:26 2. Genies Architecture: Latent action, VQE, video processing *
00:05:06 3. Genies Constraints: Frame consistency & image model integration
00:07:26 4. Evaluation: FID, PS&R diff metrics & latent induction methods
00:09:44 5. AI Video Gen: Content creation impact, depth & parallax effects
00:11:39 6. Model Scaling: Training data impact & computational trade-offs
00:13:50 7. Game & Robotics Apps: Gamification & action mapping challenges *
00:16:16 8. Robotics Foundation Models: Action space & data considerations *
00:19:18 9. Mask-GPT & Video Frames: Real-time optimization, RL from videos
00:20:34 10. Research Challenges: AI value, efficiency vs. quality, safety
00:24:20 11. Future Dev: Efficiency improvements & fine-tuning strategies
Refs:
1. Genie (learning interactive environments from videos) / Ashley and DM collegues [00:01]
https://arxiv.org/abs/2402.15391
2. VQ-VAE (Vector Quantized Variational Autoencoder) / Aaron van den Oord, Oriol Vinyals, Koray Kavukcuoglu [02:43]
https://arxiv.org/abs/1711.00937
3. FID (Fréchet Inception Distance) metric / Martin Heusel et al. [07:37]
https://arxiv.org/abs/1706.08500
4. PS&R (Precision and Recall) metric / Mehdi S. M. Sajjadi et al. [08:02]
https://arxiv.org/abs/1806.00035
5. Vision Transformer (ViT) architecture / Alexey Dosovitskiy et al. [12:14]
https://arxiv.org/abs/2010.11929
6. Genie (robotics foundation models) / Google DeepMind [17:34]
https://deepmind.google/research/publications/60474/
7. Chelsea Finns lab work on robotics datasets / Chelsea Finn [17:38]
https://ai.stanford.edu/~cbfinn/
8. Imitation from observation in reinforcement learning / YuXuan Liu [20:58]
https://arxiv.org/abs/1707.03374
9. Waymos autonomous driving technology / Waymo [22:38]
https://waymo.com/
10. Gen3 model release by Runway / Runway [23:48]
https://runwayml.com/
11. Classifier-free guidance technique / Jonathan Ho and Tim Salimans [24:43]
https://arxiv.org/abs/2207.12598 AIs can now imagine video games in real-time](https://i.ytimg.com/vi/kbt0ZFoI2Hc/mqdefault.jpg)