What Building an AI Scientist Actually Requires Beyond Intelligence — Edward Hughes @MachineLearningStreetTalk
What Building an AI Scientist Actually Requires Beyond Intelligence — Edward Hughes  @MachineLearningStreetTalk
Uploaded September 2026 | Updated November 2026, 4 days ago
Can a machine learn the judgement that separates a plausible-looking result from a faithful experiment? Edward Hughes, Chief Scientist and co-founder of Inherent, joins Tim Scarfe to argue that creativity is not optimisation, and that the missing capability in AI is choosing which questions are worth asking.

SPONSOR:
---
Cyber Fund built the Monastery to help founders ship products that were impossible a year ago.
Apply now: https://cyber.fund
---

Edward makes the case that Move 37 was innovative rather than creative, and that the field, not the individual, decides what counts as a discovery. That reframing runs through Csikszentmihalyi, Deutsch and exaptation into open-endedness, where deceptive goals and imperfect world models turn out to be the point rather than the problem. The second half turns to the paper: Replica, a task space built by redacting figures from real papers, and Faraday, a 27-billion-parameter model trained to steer a frontier coding agent that then beats the frontier on held-out replications.

---
TIMESTAMPS:
00:00:00 Cold open: Move 37, Faraday and collective intelligence
00:01:08 Sponsor: CyberFund
00:01:46 Inherent's $50M raise and the road from string theory
00:09:14 Three timescales of learning: weights, context, culture
00:13:47 Move 37 was innovative, not creative: the field decides
00:20:39 Creativity as satisficing: the urinal and evolution
00:25:06 Exaptation and the Tristan chord: creativity in context
00:30:56 Coherence for whom? Deutsch's hard-to-vary explanations
00:35:53 Why copying is creative: Deutsch and the constraint engineer
00:42:27 Societies of agents and the strong Moravec paradox
00:45:51 Evaluate in hindsight: from Lean proofs to climate change
00:51:56 Picbreeder, local goals and why discovery needs deception
00:57:21 Spaghetti proofs, translation layers and superhuman Go
01:00:37 Does nature compress? Naturalness and real patterns
01:07:36 Why replicate? Replica's redacted figures and Faraday
01:12:31 Faraday beats Codex, Claude and GLM 5.2 on held-out tasks
01:15:31 Replication to innovation: how the Transformer happened
01:18:26 Deep replication: what Faraday learns from Voyager and GNoME
01:23:37 Can the AI scientist cheat? Goodharting the judge
01:29:09 Inside Replica: scale-down, 8xB300 runs, per-task rubrics
01:34:11 The RL crisis: getting GRPO to work with per-turn credit
01:39:43 Weights vs harnesses: AlphaEvolve, DGM and EvoTune
01:45:45 The recursive company: agents cross a phase transition
01:50:35 Collective intelligence and the electric dynamo
01:55:46 What replaces OKRs? Incumbents and the burden of knowledge

---
REFERENCES:

MLST creativity article:
archive.mlst.ai/read/why-creativity-cannot-be-interpolated

organization:
[00:01:47] Inherent
inherentlabs.ai
other:
[00:20:51] Marcel Duchamp, Fountain (1917)
tate.org.uk/art/artworks/duchamp-fountain-t07573
[00:28:25] The Tristan chord
imslp.org/wiki/Tristan_und_Isolde%2C_WWV_90_(Wagner%2C_Richard)
arxiv.org/abs/2301.07608
[00:05:41] Genie
arxiv.org/abs/2402.15391
[00:06:05] The AI Scientist
arxiv.org/abs/2408.06292
[00:12:13] Training AI Scientists to Replicate Research (Replica and Faraday)
arxiv.org/abs/2608.13331
[00:25:42] Open-Endedness is Essential for ASI
arxiv.org/abs/2406.04268
[00:50:30] MAP-Elites
arxiv.org/abs/1504.04909
[00:50:59] OMNI: Open-endedness via Models of human Notions of Interestingness
arxiv.org/abs/2306.01711
[00:51:01] OMNI-EPIC
arxiv.org/abs/2405.15568
[00:53:42] Picbreeder
pubmed.ncbi.nlm.nih.gov/20964537
[01:17:26] Neural Machine Translation by Jointly Learning to Align and Translate
arxiv.org/abs/1409.0473
[01:20:44] Voyager
arxiv.org/abs/2305.16291
[01:21:57] GNoME
nature.com/articles/s41586-023-06735-9
[01:34:11] DeepSeekMath
arxiv.org/abs/2402.03300
[01:41:29] AlphaEvolve
arxiv.org/abs/2506.13131
[01:41:53] Darwin Gödel Machine
arxiv.org/abs/2505.22954
[01:41:56] Hyperagents
arxiv.org/abs/2603.19461
[01:42:33] EvoTune
arxiv.org/abs/2504.05108
[01:44:46] Evolutionary Principles in Self-Referential Learning
https://people.idsia.ch/~juergen/diploma.html
[01:59:33] Are Ideas Getting Harder to Find?
nber.org/papers/w23782
book:
[00:16:04] Creativity: Flow and the Psychology of Discovery and Invention
search.worldcat.org/title/254487436
[00:26:22] Why Greatness Cannot Be Planned
link.springer.com/book/10.1007/978-3-319-15524-1
[00:33:03] The Beginning of Infinity
penguinrandomhouse.com/books/293575/the-beginning-of-infinity-by-david-deutsch
[01:55:47] Laws of Knowledge
penguin.co.nz/books/the-infinite-alphabet-9780241655672

---
RESCRIPT:
app.rescript.info/session/670296ba913761d0?share=6281911cac9bdbff637f10819d4d1e5c
What Building an AI Scientist Actually Requires Beyond Intelligence — Edward HughesCan Latent Program Networks Solve Abstract Reasoning? [Clement Bonnet]Mutually Assured AI Malfunction [Dan Hendrycks]Your Brain Is a Prediction Machine, Not a Processor — Karl FristonChatGPT is overconfident (Anil Ananthaswamy)People wanted faster horses - Jeff CluneThe Mathematical Foundations of Intelligence [Professor Yi Ma]Why Humans Are Still Powering AI [Sponsored] - Phelim BradleyWhen We Asked GPT-4 to Solve a Problem, It Chose Blackmail — Sara Saab & Enzo BlindowOptimize GPU performance for AI - Prof. Gennady PekhimenkoYour Brain Doesnt Command Your Body. It Predicts It. [Max Bennett]Impostor Intelligence?
Machine Learning Street Talk |

What Building an AI Scientist Actually Requires Beyond Intelligence — Edward Hughes

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER