World Models & General Intuition: Khoslas largest bet since LLMs & OpenAI @LatentSpacePod
World Models & General Intuition: Khoslas largest bet since LLMs & OpenAI  @LatentSpacePod
Uploaded December 2025 | Updated November 2026, 2 days ago
From building Medal into a 12M-user game clipping platform with 3.8B highlight moments to turning down a reported $500M offer from OpenAI (theinformation.com/articles/openai-offered-pay-500-million-startup-videogame-data) and raising a $134M seed from Khosla (techcrunch.com/2025/10/16/general-intuition-lands-134m-seed-to-teach-agents-spatial-reasoning-using-video-game-clips/) to spin out General Intuition, Pim is betting that world models trained on peak human gameplay are the next frontier after LLMs.
We sat down with Pim to dig into why game highlights are “episodic memory for simulation” (and how Medal’s privacy-first action labels became a world-model goldmine medal.tv/blog/posts/enabling-state-of-the-art-security-and-protections-on-medals-new-apm-and-controller-overlay-features), what it takes to build fully vision-based agents that just see frames and output actions in real time, how General Intuition transfers from games to real-world video and then into robotics, why world models and LLMs are complementary rather than rivals, what founders with proprietary datasets should know before selling or licensing to labs, and his bet that spatial-temporal foundation models will power 80% of future atoms-to-atoms interactions in both simulation and the real world.
We discuss:

* How Medal’s 3.8B action-labeled highlight clips became a privacy-preserving goldmine for world models
* Building fully vision-based agents that only see frames and output actions yet play like (and sometimes better than) humans
* Transferring from arcade-style games to realistic games to real-world video using the same perception–action recipe
* Why world models need actions, memory, and partial observability (smoke, occlusion, camera shake) vs. “just” pretty video generation
* Distilling giant policies into tiny real-time models that still navigate, hide, and peek corners like real players
* Pim’s path from RuneScape private servers, Tourette’s, and reverse engineering to leading a frontier world-model lab
* How data-rich founders should think about valuing their datasets, negotiating with big labs, and deciding when to go independent
* GI’s first customers: replacing brittle behavior trees in games, engines, and controller-based robots with a “frames in, actions out” API
* Using Medal clips as “episodic memory of simulation” to move from imitation learning to RL via world models and negative events
* The 2030 vision: spatial–temporal foundation models that power the majority of atoms-to-atoms interactions in simulation and the real world

—
Pim

* X: https://x.com/PimDeWitte
* LinkedIn: linkedin.com/in/pimdw

Where to find Latent Space

* X: https://x.com/latentspacepod
* Substack: https://www.latent.space/

00:00:00 Introduction and Medal's Gaming Data Advantage
00:02:08 Exclusive Demo: Vision-Based Gaming Agents
00:06:17 Action Prediction and Real-World Video Transfer
00:08:41 World Models: Interactive Video Generation
00:13:42 From Runescape to AI: Pim's Founder Journey
00:16:45 The Research Foundations: Diamond, Genie, and SEMA
00:33:03 Vinod Khosla's Largest Seed Bet Since OpenAI
00:35:04 Data Moats and Why GI Stayed Independent
00:38:42 Self-Teaching AI Fundamentals: The Francois Fleuret Course
00:40:28 Defining World Models vs Video Generation
00:41:52 Why Simulation Complexity Favors World Models
00:43:30 World Labs, Yann LeCun, and the Spatial Intelligence Race
00:50:08 Business Model: APIs, Agents, and Game Developer Partnerships
00:58:57 From Imitation Learning to RL: Making Clips Playable
01:00:15 Open Research, Academic Partnerships, and Hiring
01:02:09 2030 Vision: 80 Percent of Atoms-to-Atoms AI Interactions
World Models & General Intuition: Khoslas largest bet since LLMs & OpenAINotion’s Sarah Sachs & Simon Last on Custom Agents, Evals, and the Future of Work🔬Bio-security is an AI Arms Race - Eric Nguyen (CEO, Radical Numerics)What is JevRobots That Predict the Future: Dyna-2 & Physical AI — Jason Ma, Dyna RoboticsBrex’s AI Hail Mary — With CTO James Reggio (acquired for $5B by Capital One!)Finding Needles in Data Haystacks⚡️ Competing with ChatGPT and Sierra, building a $10M ARR company — Yasser Elsaid, Founder, Chatbase⚡️ Reverse Engineering OpenAIs Training Data — Pratyush Maini, DatologyExtreme Harness Engineering: 1M LOC, 1B toks/day, 0% human code or review — Ryan Lopopolo, OpenAI[State of MechInterp] SAEs in Production, Circuit Tracing, AI4Science, Pragmatic Interp — GoodfireThe Recommendation Engine for Commodity Traders — Anekha Sokhal, Moshi
Latent Space |

World Models & General Intuition: Khosla's largest bet since LLMs & OpenAI

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER