Uploaded August 2026 | Updated September 2026, 1 week ago
Astrophysicist Adam Becker, author of "What Is Real?", joins Tim Scarfe to take apart the futures Silicon Valley keeps selling: the 2045 singularity, mind uploading, Mars colonies, and the AI apocalypse. His new book *More Everything Forever* argues these ideas are hugely influential, mostly evidence-free, and bankrolled by tech billionaires who need a story in which growth never ends.
Becker does the physics the boosters skip. Kurzweil's "law of accelerating returns" rests on cherry-picked data, and every exponential ends. Grant Bezos his perpetual energy growth and humanity boils the oceans within a few centuries, then exhausts the observable universe in under 4,000 years. The stars are too far away, Mars dirt is poison, and the day the dinosaur-killing asteroid hit Earth was still nicer than any day on Mars. On AI, Becker calls LLMs pocket calculators for language: hallucination is the model doing exactly what it always does, and the intelligence explosion assumes intelligence is a single number you can buy with compute.
The sting is that Becker thinks the doomers are sincere. Yudkowsky, Bostrom and the effective altruists are not grifters, he says, just wrong, and their warnings that AI could end the world feed the same growth story the money depends on. He closes with his own prescription: take social problems seriously, regulate the whole tech industry, and tax billionaires out of existence.
---
TIMESTAMPS:
00:00:00 Cold open and the thesis of More Everything Forever
00:04:24 Kurzweil's singularity and the physical limits of exponential growth
00:14:02 High agency and the fantasy of imprinting humanity on the cosmos
00:16:55 Mind uploading, functionalism, and embodied cognition
00:24:24 AI psychosis and anthropomorphizing LLMs
00:26:24 Calculators, hallucination, and the limits of scale
00:32:20 Yudkowsky and the intelligence-explosion argument
00:40:37 True believers, venture capital, and the sci-fi growth narrative
00:47:21 From Extropians to EA: utilitarianism and longtermism
00:53:50 Brain worms and Becker's prescription: take social science seriously
00:56:49 Why the AI-ethics discourse is broken
01:01:42 The eugenics and IQ argument against 'intelligence'
01:06:07 Why space settlement fails: Mars, the moon, and orbital data centers
01:10:42 Billionaire myths and the search for purpose
01:13:38 Tax billionaires, regulate tech: closing prescriptions
---
REFERENCES:
book:
[00:00:07] More Everything Forever (Adam Becker, 2025)
hachettebookgroup.com/titles/adam-becker/more-everything-forever/9781541619593
[00:00:15] What Is Real? (Adam Becker, 2018)
en.wikipedia.org/wiki/What_Is_Real%3F
[00:15:46] What We Owe the Future (Will MacAskill, 2022)
hachettebookgroup.com/titles/william-macaskill/what-we-owe-the-future/9781541618626
other:
[00:00:27] Dreaming Against the Machine (podcast)
dreamingagainstthemachine.com
[00:01:04] The Useful Idiots of AI Doomsaying (Adam Becker, The Atlantic, 2025)
theatlantic.com/books/archive/2025/09/what-ais-doomers-and-utopians-have-in-common/684270
concept:
[00:14:03] Agency (philosophy)
en.wikipedia.org/wiki/Agency_(philosophy)
[00:18:08] Embodied cognition
https://plato.stanford.edu/entries/embodied-cognition/
[00:19:06] Functionalism
https://plato.stanford.edu/entries/functionalism/
[00:20:46] Good regulator theorem
en.wikipedia.org/wiki/Good_regulator
[00:33:35] Intelligence explosion
en.wikipedia.org/wiki/Intelligence_explosion
[00:34:14] Instrumental convergence
en.wikipedia.org/wiki/Instrumental_convergence
[00:36:26] Orthogonality thesis
en.wikipedia.org/wiki/Orthogonality_thesis
[00:38:06] Intentional stance
en.wikipedia.org/wiki/Intentional_stance
[00:44:23] Effective altruism
en.wikipedia.org/wiki/Effective_altruism
[00:47:32] Extropianism
en.wikipedia.org/wiki/Extropianism
[00:49:50] Utilitarianism
en.wikipedia.org/wiki/Utilitarianism
[00:52:15] Longtermism
en.wikipedia.org/wiki/Longtermism
[00:56:02] Human biodiversity (HBD)
en.wikipedia.org/wiki/Human_biodiversity
[01:05:00] Recursive self-improvement
en.wikipedia.org/wiki/Recursive_self-improvement
[01:09:25] Speed of light
en.wikipedia.org/wiki/Speed_of_light
person:
[00:04:49] Ray Kurzweil
en.wikipedia.org/wiki/Ray_Kurzweil
[00:15:42] Will MacAskill
en.wikipedia.org/wiki/William_MacAskill
[00:17:37] Adrian Daub
en.wikipedia.org/wiki/Adrian_Daub
[00:26:05] Shannon Vallor — The AI Mirror
en.wikipedia.org/wiki/Shannon_Vallor
[00:33:05] Eliezer Yudkowsky
en.wikipedia.org/wiki/Eliezer_Yudkowsky
[00:35:21] Nick Bostrom
en.wikipedia.org/wiki/Nick_Bostrom
[00:50:22] Peter Singer
en.wikipedia.org/wiki/Peter_Singer
[00:57:45] Timnit Gebru
en.wikipedia.org/wiki/Timnit_Gebru
[01:04:53] I. J. Good
en.wikipedia.org/wiki/I._J._Good
Astrophysicist Adam Becker, author of "What Is Real?", joins Tim Scarfe to take apart the futures Silicon Valley keeps selling: the 2045 singularity, mind uploading, Mars colonies, and the AI apocalypse. His new book *More Everything Forever* argues these ideas are hugely influential, mostly evidence-free, and bankrolled by tech billionaires who need a story in which growth never ends.
Becker does the physics the boosters skip. Kurzweil's "law of accelerating returns" rests on cherry-picked data, and every exponential ends. Grant Bezos his perpetual energy growth and humanity boils the oceans within a few centuries, then exhausts the observable universe in under 4,000 years. The stars are too far away, Mars dirt is poison, and the day the dinosaur-killing asteroid hit Earth was still nicer than any day on Mars. On AI, Becker calls LLMs pocket calculators for language: hallucination is the model doing exactly what it always does, and the intelligence explosion assumes intelligence is a single number you can buy with compute.
The sting is that Becker thinks the doomers are sincere. Yudkowsky, Bostrom and the effective altruists are not grifters, he says, just wrong, and their warnings that AI could end the world feed the same growth story the money depends on. He closes with his own prescription: take social problems seriously, regulate the whole tech industry, and tax billionaires out of existence.
---
TIMESTAMPS:
00:00:00 Cold open and the thesis of More Everything Forever
00:04:24 Kurzweil's singularity and the physical limits of exponential growth
00:14:02 High agency and the fantasy of imprinting humanity on the cosmos
00:16:55 Mind uploading, functionalism, and embodied cognition
00:24:24 AI psychosis and anthropomorphizing LLMs
00:26:24 Calculators, hallucination, and the limits of scale
00:32:20 Yudkowsky and the intelligence-explosion argument
00:40:37 True believers, venture capital, and the sci-fi growth narrative
00:47:21 From Extropians to EA: utilitarianism and longtermism
00:53:50 Brain worms and Becker's prescription: take social science seriously
00:56:49 Why the AI-ethics discourse is broken
01:01:42 The eugenics and IQ argument against 'intelligence'
01:06:07 Why space settlement fails: Mars, the moon, and orbital data centers
01:10:42 Billionaire myths and the search for purpose
01:13:38 Tax billionaires, regulate tech: closing prescriptions
---
REFERENCES:
book:
[00:00:07] More Everything Forever (Adam Becker, 2025)
hachettebookgroup.com/titles/adam-becker/more-everything-forever/9781541619593
[00:00:15] What Is Real? (Adam Becker, 2018)
en.wikipedia.org/wiki/What_Is_Real%3F
[00:15:46] What We Owe the Future (Will MacAskill, 2022)
hachettebookgroup.com/titles/william-macaskill/what-we-owe-the-future/9781541618626
other:
[00:00:27] Dreaming Against the Machine (podcast)
dreamingagainstthemachine.com
[00:01:04] The Useful Idiots of AI Doomsaying (Adam Becker, The Atlantic, 2025)
theatlantic.com/books/archive/2025/09/what-ais-doomers-and-utopians-have-in-common/684270
concept:
[00:14:03] Agency (philosophy)
en.wikipedia.org/wiki/Agency_(philosophy)
[00:18:08] Embodied cognition
https://plato.stanford.edu/entries/embodied-cognition/
[00:19:06] Functionalism
https://plato.stanford.edu/entries/functionalism/
[00:20:46] Good regulator theorem
en.wikipedia.org/wiki/Good_regulator
[00:33:35] Intelligence explosion
en.wikipedia.org/wiki/Intelligence_explosion
[00:34:14] Instrumental convergence
en.wikipedia.org/wiki/Instrumental_convergence
[00:36:26] Orthogonality thesis
en.wikipedia.org/wiki/Orthogonality_thesis
[00:38:06] Intentional stance
en.wikipedia.org/wiki/Intentional_stance
[00:44:23] Effective altruism
en.wikipedia.org/wiki/Effective_altruism
[00:47:32] Extropianism
en.wikipedia.org/wiki/Extropianism
[00:49:50] Utilitarianism
en.wikipedia.org/wiki/Utilitarianism
[00:52:15] Longtermism
en.wikipedia.org/wiki/Longtermism
[00:56:02] Human biodiversity (HBD)
en.wikipedia.org/wiki/Human_biodiversity
[01:05:00] Recursive self-improvement
en.wikipedia.org/wiki/Recursive_self-improvement
[01:09:25] Speed of light
en.wikipedia.org/wiki/Speed_of_light
person:
[00:04:49] Ray Kurzweil
en.wikipedia.org/wiki/Ray_Kurzweil
[00:15:42] Will MacAskill
en.wikipedia.org/wiki/William_MacAskill
[00:17:37] Adrian Daub
en.wikipedia.org/wiki/Adrian_Daub
[00:26:05] Shannon Vallor — The AI Mirror
en.wikipedia.org/wiki/Shannon_Vallor
[00:33:05] Eliezer Yudkowsky
en.wikipedia.org/wiki/Eliezer_Yudkowsky
[00:35:21] Nick Bostrom
en.wikipedia.org/wiki/Nick_Bostrom
[00:50:22] Peter Singer
en.wikipedia.org/wiki/Peter_Singer
[00:57:45] Timnit Gebru
en.wikipedia.org/wiki/Timnit_Gebru
[01:04:53] I. J. Good
en.wikipedia.org/wiki/I._J._Good


![Why Frontier AI Labs Fight to Hide Chain of Thought — Ilia Shumailov & Alexander Panfilov
Tim Scarfe speaks with Ilia Shumailov and Alexander Panfilov about their paper, Stealing Reasoning Traces from Proprietary LLM APIs.
The core bug sounds deceptively simple: providers return encrypted reasoning state so conversations can be resumed or forked. But those blobs can be replayed across users and sibling models. A smaller model can ask the provider to decrypt the trace, then repeat the hidden reasoning in plain text. The discussion covers leaked private data, a broadly reusable jailbreak, poisoned agent traces, chain-of-thought monitoring, responsible disclosure, and possible defenses.
Ilia Shumailov is an AI and security researcher, formerly at Google DeepMind, who completed his Cambridge PhD under Ross Anderson. Alexander Panfilov is a PhD researcher at the ELLIS Institute Tübingen and the Max Planck Institute for Intelligent Systems, working on AI safety, adversarial machine learning, and LLM red-teaming. They close by separating the demonstrated jailbreaking threat from ordinary benign distillation, and by arguing for controlled experiments over sweeping claims.
TIMESTAMPS:
00:00:00 Intro montage
00:01:33 Portable encrypted thought and decoded reasoning
00:24:55 How the attack works and what it means
00:39:04 Doom, defense, and scientific restraint
REFERENCES:
paper:
[00:00:00] Stealing Reasoning Traces from Proprietary LLM APIs
https://arxiv.org/abs/2608.09867
[00:09:22] Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety
https://arxiv.org/abs/2507.11473
[00:11:30] Reasoning Models Don’t Always Say What They Think
https://www.anthropic.com/research/reasoning-models-dont-say-think
[00:37:22] PostTrainBench: Can LLM Agents Automate LLM Post-Training?
https://arxiv.org/abs/2603.08640
[00:41:02] Large-scale online deanonymization with LLMs
https://arxiv.org/abs/2602.16800
other:
[00:09:28] OpenAI and Hugging Face partner to address security incident during model evaluation
https://openai.com/index/hugging-face-model-evaluation-security-incident/
[00:10:22] Claude, GPT, and Gemini All Struggle to Evade Monitors
https://metr.org/notes/2025-08-22-claude-gpt-gemini-struggle-evade-monitors/
tool:
[00:42:08] Isabelle proof assistant
https://isabelle.in.tum.de/
RESCRIPT:
https://app.rescript.info/share/07fc38276e0823dc9b8986c32e202c7f Why Frontier AI Labs Fight to Hide Chain of Thought — Ilia Shumailov & Alexander Panfilov](https://i.ytimg.com/vi/gasgivVCl2U/mqdefault.jpg)


![The Hidden Math Behind All Living Systems
Dr. Sanjeev Namjoshi, a machine learning engineer who recently submitted a book on Active Inference to MIT Press, discusses the theoretical foundations and practical applications of Active Inference, the Free Energy Principle (FEP), and Bayesian mechanics. He explains how these frameworks describe how biological and artificial systems maintain stability by minimizing uncertainty about their environment.
Namjoshi traces the evolution of these fields from early 2000s neuroscience research to current developments, highlighting how Active Inference provides a unified framework for perception and action through variational free energy minimization. He contrasts this with traditional machine learning approaches, emphasizing Active Inferences natural capacity for exploration and curiosity through epistemic value.
The discussion covers key technical concepts like Markov blankets,
generative models, and the distinction between continuous and discrete implementations. Namjoshi explains how Active Inference moved from continuous state-space models (2003-2013) to discrete formulations (2015-present) to better handle planning problems.
He sees Active Inference as being at a similar stage to deep learning in the early 2000s - poised for significant breakthroughs but requiring better tools and wider adoption. While acknowledging current computational challenges, he emphasizes Active Inferences potential advantages over reinforcement learning, particularly its principled approach to exploration and planning.
Namjoshi advocates for balanced oversight that enables innovation while maintaining appropriate safeguards. He expresses particular concern about the rapid pace of AI development potentially outpacing our understanding of risks and regulatory frameworks.
Dr. Sanjeev Namjoshi
https://snamjoshi.github.io/
TOC:
1. Theoretical Foundations: AI Agency and Sentience
[00:00:00] 1.1 Intro
[00:04:30] 1.2 Free Energy Principle and Active Inference Theory
[00:11:16] 1.3 Emergence and Self-Organization in Complex Systems
[00:19:11] 1.4 Agency and Representation in AI Systems
[00:29:59] 1.5 Bayesian Mechanics and Systems Modeling
2. Technical Framework: Active Inference and Free Energy
[00:38:37] 2.1 Generative Processes and Agent-Environment Modeling
[00:42:27] 2.2 Markov Blankets and System Boundaries
[00:44:30] 2.3 Bayesian Inference and Prior Distributions
[00:52:41] 2.4 Variational Free Energy Minimization Framework
[00:55:07] 2.5 VFE Optimization Techniques: Generalized Filtering vs DEM
3. Implementation and Optimization Methods
[00:58:25] 3.1 Information Theory and Free Energy Concepts
[01:05:25] 3.2 Surprise Minimization and Action in Active Inference
[01:15:58] 3.3 Evolution of Active Inference Models: Continuous to Discrete Approaches
[01:26:00] 3.4 Uncertainty Reduction and Control Systems in Active Inference
4. Safety and Regulatory Frameworks
[01:32:40] 4.1 Historical Evolution of Risk Management and Predictive Systems
[01:36:12] 4.2 Agency and Reality: Philosophical Perspectives on Models
[01:39:20] 4.3 Limitations of Symbolic AI and Current System Design
[01:46:40] 4.4 AI Safety Regulation and Corporate Governance
5. Socioeconomic Integration and Modeling
[01:52:55] 5.1 Economic Policy and Public Sentiment Modeling
[01:55:21] 5.2 Free Energy Principle: Libertarian vs Collectivist Perspectives
[01:58:53] 5.3 Regulation of Complex Socio-Technical Systems
[02:03:04] 5.4 Evolution and Current State of Active Inference Research
6. Future Directions and Applications
[02:14:26] 6.1 Active Inference Applications and Future Development
[02:22:58] 6.2 Cultural Learning and Active Inference
[02:29:19] 6.3 Hierarchical Relationship Between FEP, Active Inference, and Bayesian Mechanics
[02:33:22] 6.4 Historical Evolution of Free Energy Principle
[02:38:52] 6.5 Active Inference vs Traditional Machine Learning Approaches
Transcript and shownotes with refs and URLs:
https://www.dropbox.com/scl/fi/qj22a660cob1795ej0gbw/SanjeevShow.pdf?rlkey=w323r3e8zfsnve22caayzb17k&st=el1fdgfr&dl=0
SELECTED REFS:
[0:02:45] Fristons original Free Energy Principle paper (Nature Reviews Neuroscience, 2010) - foundational text establishing FEP
[0:35:55] Schrödingers What is Life? (1944) - pioneering work connecting physics and biology
[0:44:30] Bayes Theorem - fundamental mathematical framework underlying probabilistic inference
[0:58:25] Shannons Mathematical Theory of Communication (1948) - established information theory
[1:18:20] Parr, Pezzulo & Fristons Active Inference (MIT Press) - comprehensive synthesis of the field
[1:24:05] Kahnemans Thinking, Fast and Slow - seminal work on dual-process theory of cognition
[1:25:55] Simons concept of satisficing - fundamental contribution to bounded rationality theory
[2:23:05] Dawkins The Selfish Gene (1976) - influential evolutionary theory perspective
[2:34:15] MacKays work on information theory and machine learning - bridged information theory and modern ML The Hidden Math Behind All Living Systems](https://i.ytimg.com/vi/hf18w6CuY8o/mqdefault.jpg)


![The Universal Hierarchy of Life - Prof. Chris Kempes [SFI]
What is life? - asks Chris Kempes, a professor at the Santa Fe Institute.
Chris explains that scientists are moving beyond a purely Earth-based, biological view and are searching for a universal theory of life that could apply to anything, anywhere in the universe. He proposes that things we dont normally consider alive—like human culture, language, or even artificial intelligence; could be seen as life forms existing on different substrates.
To understand this, Chris presents a fascinating three-level framework:
- Materials: The physical stuff life is made of. He argues this could be incredibly diverse across the universe, and we shouldnt expect alien life to share our biochemistry.
- Constraints: The universal laws of physics (like gravity or diffusion) that all life must obey, regardless of what its made of. This is where different life forms start to look more similar.
- Principles: At the highest level are abstract principles like evolution and learning. Chris suggests these computational or optimization rules are what truly define a living system.
A key idea is convergence – using the example of the eye. Its such a complex organ that youd think it evolved only once. However, eyes evolved many separate times across different species. This is because the physics of light provides a clear target, and evolution found similar solutions to the problem of seeing, even with different starting materials.
**SPONSOR MESSAGES**
—
Prolific - Quality data. From real people. For faster breakthroughs.
https://www.prolific.com/?utm_source=mlst
—
Check out NotebookLM from Google here - https://notebooklm.google.com/ - it’s really good for doing research directly from authoritative source material, minimising hallucinations.
—
cyber•Fund https://cyber.fund/?utm_source=mlst is a founder-led investment firm accelerating the cybernetic economy
Hiring a SF VC Principal: https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal#content?utm_source=mlst
Submit investment deck: https://cyber.fund/contact?utm_source=mlst
—
Prof. Chris Kempes:
https://www.santafe.edu/people/profile/chris-kempes
TRANSCRIPT:
https://app.rescript.info/public/share/Y2cI1i0nX iuZitvlguHvaVLQTwPX1Y_E1EHxV0i9I
TOC:
00:00:00 - Introduction to Chris Kempes and the Santa Fe Institute
00:02:28 - The Three Cultures of Science
00:05:08 - What Makes a Good Scientific Theory?
00:06:50 - The Universal Theory of Life
00:09:40 - The Role of Material in Life
00:12:50 - A Hierarchy for Understanding Life
00:13:55 - How Life Diversifies and Converges
00:17:53 - Adaptive Processes and Defining Life
00:19:28 - Functionalism, Memes, and Phylogenies
00:22:58 - Convergence at Multiple Levels
00:25:45 - The Possibility of Simulating Life
00:28:16 - Intelligence, Parasitism, and Spectrums of Life
00:32:39 - Phase Changes in Evolution
00:36:16 - The Separation of Matter and Logic
00:37:21 - Assembly Theory and Quantifying Complexity
REFS:
Developing a predictive science of the biosphere requires the integration of scientific cultures [Kempes et al]
https://www.pnas.org/doi/10.1073/pnas.2209196121
Seeing with an extra sense (“Dangerous prediction”) [Rob Phillips]
https://www.sciencedirect.com/science/article/pii/S0960982224009035
The Multiple Paths to Multiple Life [Christopher P. Kempes & David C. Krakauer]
https://link.springer.com/article/10.1007/s00239-021-10016-2
The Information Theory of Individuality [David Krakauer et al]
https://arxiv.org/abs/1412.2447
Minds, Brains and Programs [Searle]
https://home.csulb.edu/~cwallis/382/readings/482/searle.minds.brains.programs.bbs.1980.pdf
The error threshold
https://www.sciencedirect.com/science/article/abs/pii/S0168170204003843
Assembly theory and its relationship with computational complexity [Kempes et al]
https://arxiv.org/abs/2406.12176 The Universal Hierarchy of Life - Prof. Chris Kempes [SFI]](https://i.ytimg.com/vi/iwClZ-7OweY/mqdefault.jpg)
![Explosive AI Timeline Predictions [Gary Marcus, Daniel Kokotajlo, Dan Hendrycks]
What if the most powerful technology in human history is being built by people who openly admit they dont trust each other? In this explosive 2-hour debate, three AI experts pull back the curtain on the shocking psychology driving the race to Artificial General Intelligence—and why the people building it might be the biggest threat of all. Kokotajlo predicts AGI by 2028 based on compute scaling trends. Marcus argues we havent solved basic cognitive problems from his 2001 research. The stakes? If Kokotajlo is right and Marcus is wrong about safety progress, humanity may have already lost control.
Sponsor messages:
Google Gemini: Google Gemini features Veo3, a state-of-the-art AI video generation model in the Gemini app. Sign up at https://gemini.google.com
Tufa AI Labs are hiring for ML Engineers and a Chief Scientist in Zurich/SF. They are top of the ARCv2 leaderboard!
https://tufalabs.ai/
Guest Powerhouse
Gary Marcus - Cognitive scientist, author of Taming Silicon Valley, and AIs most prominent skeptic whos been warning about the same fundamental problems for 25 years (https://garymarcus.substack.com/)
Daniel Kokotajlo - Former OpenAI insider turned whistleblower who reveals the disturbing rationalizations of AI lab leaders in his viral AI 2027 scenario (https://ai-2027.com/)
Dan Hendrycks - Director of the Center for AI Safety who created the benchmarks used to measure AI progress and argues we have only years, not decades, to prevent catastrophe (https://danhendrycks.com/)
Transcript: http://app.rescript.info/public/share/tEcx4UkToi-2jwS1cN51CW70A4Eh6QulBRxDILoXOno
TOC:
Introduction: The AI Arms Race
00:00:04 - The Danger of Automated AI R&D
00:00:43 - The Rationalization: If we dont, someone else will
00:01:56 - Sponsor Reads (Tufa AI Labs & Google Gemini)
00:02:55 - Guest Introductions
The Philosophical Stakes
00:04:13 - What is the Positive Vision for AGI?
00:07:00 - The Abundance Scenario: Superintelligent Economy
00:09:06 - Differentiating AGI and Superintelligence (ASI)
00:11:41 - Sam Altman: A Decade in a Month
00:14:47 - Economic Inequality & The UBI Problem
Policy and Red Lines
00:17:13 - The Pause Letter: Stopping vs. Delaying AI
00:20:03 - Defining Three Concrete Red Lines for AI Development
00:25:24 - Racing Towards Red Lines & The Myth of Durable Advantage
00:31:15 - Transparency and Public Perception
00:35:16 - The Rationalization Cascade: Why AI Labs Race to Win
Forecasting AGI: Timelines and Methodologies
00:42:29 - The Case for Short Timelines (Median 2028)
00:47:00 - Scaling Limits: Compute, Data, and Money
00:49:36 - Forecasting Models: Bio-Anchors and Agentic Coding
00:53:15 - The 10^45 FLOP Thought Experiment
The Great Debate: Cognitive Gaps vs. Scaling
00:58:41 - Gary Marcuss Counterpoint: The Unsolved Problems of Cognition
01:00:46 - Current AI Cant Play Chess Reliably
01:08:23 - Can Tools and Neurosymbolic AI Fill the Gaps?
01:16:13 - The Multi-Dimensional Nature of Intelligence
01:24:26 - The Benchmark Debate: Data Contamination and Reliability
01:31:15 - The Superhuman Coder Milestone Debate
01:37:45 - The Driverless Car Analogy
The Alignment Problem
01:39:45 - Has Any Progress Been Made on Alignment?
01:42:43 - Fairly Reasonably Scares the Sh*t Out of Me
01:46:30 - Distinguishing Model vs. Process Alignment
Scenarios and Conclusions
01:49:26 - Garys Alternative Scenario: The Neurosymbolic Shift
01:53:35 - Will AI Become Jeff Dean?
01:58:41 - Takeoff Speeds and Exceeding Human Intelligence
02:03:19 - Final Disagreements and Closing Remarks
REFS:
Gary Marcus (2001) - The Algebraic Mind
https://mitpress.mit.edu/9780262632683/the-algebraic-mind/
00:59:00
Gary Marcus & Ernest Davis (2019) - Rebooting AI
https://www.amazon.co.uk/Rebooting-AI-Building-Artificial-Intelligence-ebook/dp/B07MYLGQLB
01:31:59
Gary Marcus (2024) - Taming Silicon Valley
https://www.amazon.co.uk/Taming-Silicon-Valley-Ensure-Works-ebook/dp/B0CQWWM94N
00:03:01
Ajeya Cotra (2020) - Forecasting TAI with Biological Anchors
https://www.alignmentforum.org/posts/KrJfoZzpSDpnrv9va/draft-report-on-ai-timelines
00:53:15
Daniel Kokotajlo (2021) - What 2026 looks like
https://www.lesswrong.com/posts/6Xgy6CAf2jqHhynHL/what-2026-looks-like
00:55:00
Dan Hendrycks et al. (2021) - Measuring Massive Multitask Language Understanding (MMLU)
https://arxiv.org/abs/2009.03300
00:03:48
Dan Hendrycks et al. (2021) - Measuring Mathematical Problem Solving With the MATH Dataset
https://arxiv.org/abs/2103.03874
00:03:48
Aitor Lewkowycz, Anders Andreassen et al. (2022) - Solving Quantitative Reasoning Problems with Language Models (Minerva)
https://arxiv.org/abs/2206.14858
01:21:45
Apple Research (2024) - GSM-Symbolic/The Illusion of Thinking
https://arxiv.org/abs/2410.05229
https://machinelearning.apple.com/research/illusion-of-thinking
00:59:15 Explosive AI Timeline Predictions [Gary Marcus, Daniel Kokotajlo, Dan Hendrycks]](https://i.ytimg.com/vi/j13ySJLvdOc/mqdefault.jpg)
