Uploaded March 2026 | Updated September 2026, 1 week ago
Dive into the realities of AI-assisted coding, the origins of modern fine-tuning, and the cognitive science behind machine learning with fast.ai founder Jeremy Howard. In this episode, we unpack why AI might be turning software engineering into a slot machine and how to maintain true technical intuition in the age of large language models.
GTC is coming, the premier AI conference, great opportunity to learn about AI. NVIDIA and partners will showcase breakthroughs in physical AI, AI factories, agentic AI, and inference, exploring the next wave of AI innovation for developers and researchers. Register for virtual GTC for free, using my link and win NVIDIA DGX Spark (nvda.ws/4qQ0LMg)
Jeremy Howard is a renowned data scientist, researcher, entrepreneur, and educator. As the co-founder of fast.ai, former President of Kaggle, and the creator of ULMFiT, Jeremy has spent decades democratizing deep learning. His pioneering work laid the foundation for modern transfer learning and the pre-training and fine-tuning paradigm that powers today's language models.
Key Topics and Main Insights Discussed:
- The Origins of ULMFiT and Fine-Tuning
- The Vibe Coding Illusion and Software Engineering
- Cognitive Science, Friction, and Learning
- The Future of Developers
RESCRIPT: app.rescript.info/public/share/BhX5zP3b0m63srLOQDKBTFTooSzEMh_ARwmDG_h_izk
app.rescript.info/api/public/sessions/62d06c0336c567d6/pdf
Jeremy Howard:
https://x.com/jeremyphoward
answer.ai
---
TIMESTAMPS (fixed):
00:00:00 Introduction & GTC Sponsor
00:04:30 ULMFiT & The Birth of Fine-Tuning
00:12:00 Intuition & The Mechanics of Learning
00:18:30 Abstraction Hierarchies & AI Creativity
00:23:00 Claude Code & The Interpolation Illusion
00:27:30 Coding vs. Software Engineering
00:30:00 Cosplaying Intelligence: Dennett vs. Searle
00:36:30 Automation, Radiology & Desirable Difficulty
00:42:30 Organizational Knowledge & The Slope
00:48:00 Vibe Coding as a Slot Machine
00:54:00 The Erosion of Control in Software
01:01:00 Interactive Programming & REPL Environments
01:05:00 The Notebook Debate & Exploratory Science
01:17:30 AI Existential Risk & Power Centralization
01:24:20 Current Risks, Privacy & Enfeeblement
---
REFERENCES:
Blog Post:
[00:03:00] fast.ai Blog: Self-Supervised Learning
fast.ai/posts/2020-01-13-self_supervised.html
[00:13:30] DeepMind Blog: Gemini Deep Think
https://deepmind.google/blog/accelerating-mathematical-and-scientific-discovery-with-gemini-deep-think/
[00:19:30] Modular Blog: Claude C Compiler analysis
modular.com/blog/the-claude-c-compiler-what-it-reveals-about-the-future-of-software
[00:19:45] Anthropic Engineering Blog: Building C Compiler
anthropic.com/engineering/building-c-compiler
[00:48:00] Cursor Blog: Scaling Agents
cursor.com/blog/scaling-agents
[01:05:15] fast.ai Blog: NB Dev Merged Driver
fast.ai/posts/2022-08-25-jupyter-git.html
[01:17:30] Jeremy Howard: Response to AI Risk Letter
normaltech.ai/p/is-avoiding-extinction-from-ai-really
Book:
[00:08:30] M. Chirimuuta: The Brain Abstracted
https://mitpress.mit.edu/9780262548045/the-brain-abstracted/
[00:30:00] Daniel Dennett: Consciousness Explained
amazon.com/Consciousness-Explained-Daniel-C-Dennett/dp/0316180661
[00:42:30] Cesar Hidalgo: Infinite Alphabet / Laws of Knowledge
amazon.com/Infinite-Alphabet-Laws-Knowledge/dp/0241655676
Archive Article:
[00:13:45] MLST Archive: Why Creativity Cannot Be Interpolated
archive.mlst.ai/read/why-creativity-cannot-be-interpolated
Research Study:
[00:24:30] METR Study: AI OS Development
metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study
Paper:
[00:24:45] Fred Brooks: No Silver Bullet
https://www.cs.unc.edu/techreports/86-020.pdf
[00:30:15] John Searle: Minds, Brains, and Programs
cambridge.org/core/journals/behavioral-and-brain-sciences/article/minds-brains-and-programs/DC644B47A4299C637C89772FACC2706A
Research Paper:
[00:13:50] Mathilde Caron et al.: Emerging Properties in Self-Supervised Vision Transformers (DINO)
arxiv.org/abs/2104.14294
[00:25:00] Oxford VGG: Sculptor Identification Paper
robots.ox.ac.uk/~vgg/publications/2012/Arandjelovic12a/arandjelovic12a.pdf
[00:36:30] Anthropic Paper: AI Skill Formation
arxiv.org/pdf/2601.20245
Historical Reference:
[00:36:45] Ebbinghaus: Memory / Spaced Repetition
loc.gov/item/e11000616
Technical Note:
[00:42:45] John Ousterhout: Slope vs Intercept
gist.github.com/gtallen1187/e83ed02eac6cc8d7e185
Video:
[00:59:00] Bret Victor: Inventing on Principle
vimeo.com/906418692
[01:05:00] Joel Grus: I Don't Like Notebooks
youtube.com/watch?v=7jiPeIFXb6U
Dive into the realities of AI-assisted coding, the origins of modern fine-tuning, and the cognitive science behind machine learning with fast.ai founder Jeremy Howard. In this episode, we unpack why AI might be turning software engineering into a slot machine and how to maintain true technical intuition in the age of large language models.
GTC is coming, the premier AI conference, great opportunity to learn about AI. NVIDIA and partners will showcase breakthroughs in physical AI, AI factories, agentic AI, and inference, exploring the next wave of AI innovation for developers and researchers. Register for virtual GTC for free, using my link and win NVIDIA DGX Spark (nvda.ws/4qQ0LMg)
Jeremy Howard is a renowned data scientist, researcher, entrepreneur, and educator. As the co-founder of fast.ai, former President of Kaggle, and the creator of ULMFiT, Jeremy has spent decades democratizing deep learning. His pioneering work laid the foundation for modern transfer learning and the pre-training and fine-tuning paradigm that powers today's language models.
Key Topics and Main Insights Discussed:
- The Origins of ULMFiT and Fine-Tuning
- The Vibe Coding Illusion and Software Engineering
- Cognitive Science, Friction, and Learning
- The Future of Developers
RESCRIPT: app.rescript.info/public/share/BhX5zP3b0m63srLOQDKBTFTooSzEMh_ARwmDG_h_izk
app.rescript.info/api/public/sessions/62d06c0336c567d6/pdf
Jeremy Howard:
https://x.com/jeremyphoward
answer.ai
---
TIMESTAMPS (fixed):
00:00:00 Introduction & GTC Sponsor
00:04:30 ULMFiT & The Birth of Fine-Tuning
00:12:00 Intuition & The Mechanics of Learning
00:18:30 Abstraction Hierarchies & AI Creativity
00:23:00 Claude Code & The Interpolation Illusion
00:27:30 Coding vs. Software Engineering
00:30:00 Cosplaying Intelligence: Dennett vs. Searle
00:36:30 Automation, Radiology & Desirable Difficulty
00:42:30 Organizational Knowledge & The Slope
00:48:00 Vibe Coding as a Slot Machine
00:54:00 The Erosion of Control in Software
01:01:00 Interactive Programming & REPL Environments
01:05:00 The Notebook Debate & Exploratory Science
01:17:30 AI Existential Risk & Power Centralization
01:24:20 Current Risks, Privacy & Enfeeblement
---
REFERENCES:
Blog Post:
[00:03:00] fast.ai Blog: Self-Supervised Learning
fast.ai/posts/2020-01-13-self_supervised.html
[00:13:30] DeepMind Blog: Gemini Deep Think
https://deepmind.google/blog/accelerating-mathematical-and-scientific-discovery-with-gemini-deep-think/
[00:19:30] Modular Blog: Claude C Compiler analysis
modular.com/blog/the-claude-c-compiler-what-it-reveals-about-the-future-of-software
[00:19:45] Anthropic Engineering Blog: Building C Compiler
anthropic.com/engineering/building-c-compiler
[00:48:00] Cursor Blog: Scaling Agents
cursor.com/blog/scaling-agents
[01:05:15] fast.ai Blog: NB Dev Merged Driver
fast.ai/posts/2022-08-25-jupyter-git.html
[01:17:30] Jeremy Howard: Response to AI Risk Letter
normaltech.ai/p/is-avoiding-extinction-from-ai-really
Book:
[00:08:30] M. Chirimuuta: The Brain Abstracted
https://mitpress.mit.edu/9780262548045/the-brain-abstracted/
[00:30:00] Daniel Dennett: Consciousness Explained
amazon.com/Consciousness-Explained-Daniel-C-Dennett/dp/0316180661
[00:42:30] Cesar Hidalgo: Infinite Alphabet / Laws of Knowledge
amazon.com/Infinite-Alphabet-Laws-Knowledge/dp/0241655676
Archive Article:
[00:13:45] MLST Archive: Why Creativity Cannot Be Interpolated
archive.mlst.ai/read/why-creativity-cannot-be-interpolated
Research Study:
[00:24:30] METR Study: AI OS Development
metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study
Paper:
[00:24:45] Fred Brooks: No Silver Bullet
https://www.cs.unc.edu/techreports/86-020.pdf
[00:30:15] John Searle: Minds, Brains, and Programs
cambridge.org/core/journals/behavioral-and-brain-sciences/article/minds-brains-and-programs/DC644B47A4299C637C89772FACC2706A
Research Paper:
[00:13:50] Mathilde Caron et al.: Emerging Properties in Self-Supervised Vision Transformers (DINO)
arxiv.org/abs/2104.14294
[00:25:00] Oxford VGG: Sculptor Identification Paper
robots.ox.ac.uk/~vgg/publications/2012/Arandjelovic12a/arandjelovic12a.pdf
[00:36:30] Anthropic Paper: AI Skill Formation
arxiv.org/pdf/2601.20245
Historical Reference:
[00:36:45] Ebbinghaus: Memory / Spaced Repetition
loc.gov/item/e11000616
Technical Note:
[00:42:45] John Ousterhout: Slope vs Intercept
gist.github.com/gtallen1187/e83ed02eac6cc8d7e185
Video:
[00:59:00] Bret Victor: Inventing on Principle
vimeo.com/906418692
[01:05:00] Joel Grus: I Don't Like Notebooks
youtube.com/watch?v=7jiPeIFXb6U
![We Built Calculators Because Were STUPID! [Prof. David Krakauer]](https://i.ytimg.com/vi/dY46YsGWMIc/mqdefault.jpg "We Built Calculators Because Were STUPID! [Prof. David Krakauer]
Prof. David Krakauer argues that intelligence isnt about knowing more—its about doing more with less.
His core thesis flips our assumptions on their head: while emergence in complex systems is about more is different, intelligence is fundamentally about less is more.
Why Large Language Models Miss the Point
LLMs are more is more, Krakauer explains. Theyre essentially sophisticated libraries that know everything but understand nothing. When a student looks up answers in a library during an exam, we call them a cheater, not intelligent.
Intelligence Across All Life
Perhaps most surprisingly, Krakauer claims that all living things are intelligent—even bacteria. His reasoning: intelligence is the information accumulated through evolutions selective pressures. Its why we can say an elephant is smarter than a worm, but wed never say its more alive. The difference is accumulated capacity.
Three Dimensions of Intelligence
Krakauer proposes intelligence exists in three distinct flavours:
- Strategic Intelligence - Adaptation and survival where viruses actually beat us/
- Inferential Intelligence - Math and computation (where we built calculators because were so bad at it)
- Representational Intelligence - Finding better ways to encode problems (the most uniquely human)
The Soma Cube Insight
Using a Soma Cube puzzle, Krakauer demonstrates something incredible: a four-year-old can solve a combinatorial problem with 15,000 logical clauses—not by computing, but by using physical representation. The physical world does the computing for us. This is intelligence: making impossibly hard problems tractable through clever representation.
This talk was recorded at DISI 2025 https://disi.org/
David Krakauer
President and William H. Miller Professor of Complex Systems
https://www.santafe.edu/people/profile/david-krakauer
Note this version is edited down, you can watch the full version on our Patreon - https://www.patreon.com/posts/masterclass-on-142898847
Watch interview we published with David - https://www.youtube.com/watch?v=jXa8dHzgV8U
TRANSCRIPT:
https://app.rescript.info/public/share/Cs8QFwMw_FiOEROkNPiIf6Y-e_ahAxNMl0oFiL7_4QA
TOC:
00:00:00 - Complexity, Life & Intelligence
00:00:45 - Ptolemy vs Newton
00:01:30 - Purpose of Science
00:02:15 - Feynman on Rules vs Strategies
00:03:00 - Entropy as Unifying Rule
00:03:45 - Intelligence in Physics
00:04:15 - Complex Systems & Broken Symmetries
00:05:00 - Historical Definitions of Intelligence
00:06:15 - Herbert: Capacity to Acquire Capacity
00:06:45 - Problems with Turing Test
00:07:30 - Intelligence vs Knowledge
00:08:00 - LLMs as Libraries
00:08:30 - Humanistic Perspectives
00:09:00 - Unified Theory: Life & Intelligence
00:09:30 - Intensive vs Extensive Properties
00:10:30 - Selection Gradient Analogy
00:11:15 - Universal Intelligence (Bacteria to Elephants)
00:12:30 - Three Dimensions of Intelligence
00:13:00 - Strategic Intelligence & Viruses
00:13:30 - Inferential Intelligence & Calculators
00:14:15 - Encoding & Representation
00:14:30 - Different Paths: Life vs AI
00:15:15 - Soma Cube Problem
00:16:00 - Principle of Materiality
00:17:15 - Embodied Representation
00:17:45 - Intelligence Makes Hard Problems Easy
00:18:00 - Stupidity & Conclusion
REFS:
Papers and Academic Works
More is Different by Phil Anderson [https://www.tkm.kit.edu/downloads/TKM1_2011_more_is_different_PWA.pdf ]
Recent paper with Melanie and John Krakauer on emergence [https://arxiv.org/abs/2506.11135 ]
Formal treatment paper on intelligence/evolution duality from David - Darwinian demons, evolutionary complexity, and information maximization [https://wiki.santafe.edu/images/b/b1/CHAOEH213037110_1.pdf ]
The Unreasonable Effectiveness of Mathematics in the Natural Sciences by Eugene Wigner [https://webhomes.maths.ed.ac.uk/~v1ranick/papers/wigner.pdf ]")
