ARC Prize Version 2 Launch Video! [Francois Chollet, Mike Knoop] @MachineLearningStreetTalk
ARC Prize Version 2 Launch Video! [Francois Chollet, Mike Knoop]  @MachineLearningStreetTalk
Uploaded March 2025 | Updated November 2026, 4 days ago
Francois Chollet and Mike Knoop join Tim Scarfe to announce ARC-AGI 2 and the ARC Prize 2025 contest. The new benchmark has been human-calibrated with 400 participants and adversarially designed so that frontier reasoning models score in the single digits, while every task remains solvable by at least two humans. The conversation covers the story behind testing OpenAI o3 on ARC v1 (where it scored 75-85%), what those results reveal about fluid intelligence versus brute-force capability, and why Chollet considers o3 the first model with genuine -- if limited -- fluid intelligence. Technical discussion ranges from chain-of-thought reasoning as natural language program synthesis, to the exponential failure modes of current reasoning systems on spatially complex tasks, to the deeper question of whether intelligence is best understood as a binary category or a spectrum of recombination depth. Chollet argues that intelligence is fundamentally about efficiency of knowledge acquisition and adaptation, not raw capability, and that ARC-AGI 2 is designed to measure exactly this distinction.

SPONSOR MESSAGES:
***
Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich.

---
REFERENCES:
Paper:
[00:00:15] On the Measure of Intelligence
arxiv.org/abs/1911.01547
[00:12:50] OpenAI o3 Performance on ARC v1
arcprize.org/blog/oai-o3-pub-breakthrough
[00:18:30] Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
arxiv.org/abs/2201.11903
[00:26:05] ARC Prize 2024: Technical Report
arxiv.org/abs/2412.04604
[00:48:57] The Bitter Lesson
incompleteideas.net/IncIdeas/BitterLesson.html
[00:53:30] Decoding Strategies in Neural Text Generation
mdpi.com/2078-2489/12/9/355/pdf
Organization:
[00:06:45] ARC Prize Foundation
arcprize.org

---
LINKS:
Full Transcript: app.rescript.info/share/19ed0e8921636ce804d8780e6b03f67d
Download PDF transcript: app.rescript.info/api/public/sessions/302bc045019e056a/pdf
ARC Prize Version 2 Launch Video! [Francois Chollet, Mike Knoop]The most natural voice agent was a French voice speaking EnglishThe Annals Published It. It Wasnt True. - Kevin BuzzardSomething Finally Was at Stake (Tom Froese)You dont fine-tune your way to AGI - Heres why. [Eiso Kant]What Building an AI Scientist Actually Requires Beyond Intelligence — Edward HughesCan Latent Program Networks Solve Abstract Reasoning? [Clement Bonnet]Mutually Assured AI Malfunction [Dan Hendrycks]Your Brain Is a Prediction Machine, Not a Processor — Karl FristonChatGPT is overconfident (Anil Ananthaswamy)People wanted faster horses - Jeff CluneThe Mathematical Foundations of Intelligence [Professor Yi Ma]
Machine Learning Street Talk |

ARC Prize Version 2 Launch Video! [Francois Chollet, Mike Knoop]

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER