The scientist who coined retrieval augmented generation @MachineLearningStreetTalk
The scientist who coined retrieval augmented generation  @MachineLearningStreetTalk
Uploaded September 2024 | Updated November 2026, 4 days ago
MLST is sponsored by Brave:
The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmented generation. Try it now - get 2,000 free queries monthly at brave.com/api.

Disclaimer: This is the sixth video from our Cohere partnership. We were not told what to say in the interview. Filmed in London in June 2024.

Dr. Patrick Lewis, who coined the term Retrieval Augmented Generation and now works at Cohere, provides an insider account of how RAG came to be and where it is heading. Starting from his early work on the LAMA probe and the KILT benchmark, Lewis traces the intellectual path from open-domain question answering to the creation of RAG as a framework for grounding language model outputs in retrieved evidence.

The conversation covers the full technical landscape: evaluation challenges across faithfulness, fluency, and perceived utility; the tension between data quality and scale; dense versus sparse retrieval methods and when each excels; and the practical difficulties of deploying RAG in enterprise settings where link structure and training data are scarce. Lewis offers candid perspectives on prompt engineering, Cohere's approach to building retrieval-aware models through post-training, and why current evaluation benchmarks have been outpaced by model quality.

The discussion takes a historical turn through the evolution of language representations, from Mikolov's Word2Vec -- which famously rediscovered the periodic table from chemistry literature -- through GloVe and BERT to modern generative models. Tim Scarfe and Lewis close with reflections on the parallels between AI sequential processing and human cognition, and what chain-of-thought reasoning reveals about both.

---
LINKS:
Full Transcript: app.rescript.info/share/6096fae7a5833eb4864f70e7d75ab2cd
Download PDF transcript: app.rescript.info/api/public/sessions/b2fbba8073504800/pdf
The scientist who coined retrieval augmented generationThe Gap Between Humans and Machines Is   [Dr. Max Bartolo]AI Coding Is a Slot Machine: Loss Disguised as a Win | Jeremy HowardManhattan Project for AI Safety [Connor Leahy]He Co-Invented the Transformer. Now: Continuous Thought Machines [Llion Jones / Luke Darlow]SchmidhuberedDo AI chatbots behave WEIRDLY?When AI Discovers the Next Transformer — Robert LangeTransformers Need Glasses! [Federico Barbero]AI is stuck in Plato’s Cave (Maxwell Ramstead)29.4% ARC-AGI-2 🤯 (TOP SCORE!) - Jeremy BermanMIGHT THE ROBOTS TAKE OVER? [Prof. Yoshua Bengio]
Machine Learning Street Talk |

The scientist who coined retrieval augmented generation

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER