Uploaded July 2026 | Updated September 2026, 3 weeks ago
Feed it 67 videos from the 2022 World Cup and ask for the near misses, the shots that almost scored but did not, each with a reason, and it returns them. Ask it to track Messi across the entire corpus and describe the camera framing, and it finds the moment he slaloms past a sliding defender. James Le's point is that this is unusual because video has no memory. Almost every video AI system answers each query from scratch, and a bigger context window does not fix it, because the real problem is that there is no durable representation to retrieve into.
His fix is to stop treating video as a bag of frames and start treating it as a spatial temporal volume, then build a memory layer over it: a context graph of time bounded moments, the entities and where they appear, the relationships between them, and corpus level themes. At TwelveLabs that is an embedding encoder, a context store, and a video language model exposed as an API. The design rules are blunt: ingest once and reason many times, store primitives not answers, ground every claim to a timestamp, and let intent decide what to remember, because brand safety and sports highlights need different things from the same footage.
Speaker info:
- https://x.com/le_james94
- linkedin.com/in/khanhnamle94
- jameskle.com
Timestamps:
0:00 - Video has no memory
0:50 - Video is a spatial temporal volume, not a bag of frames
2:06 - Three problems: wrong context, wrong memory, weak reasoning
3:36 - Five properties that make video memory hard
4:53 - The TwelveLabs stack: Marengo, the context store, and Pegasus
5:56 - Search versus memory
7:48 - The context graph
9:04 - Five design principles for a video memory layer
10:45 - From a static model to a video worker
12:51 - Demo: sports and tracking Messi across the World Cup
15:37 - Demos: traffic security and ad placement
Feed it 67 videos from the 2022 World Cup and ask for the near misses, the shots that almost scored but did not, each with a reason, and it returns them. Ask it to track Messi across the entire corpus and describe the camera framing, and it finds the moment he slaloms past a sliding defender. James Le's point is that this is unusual because video has no memory. Almost every video AI system answers each query from scratch, and a bigger context window does not fix it, because the real problem is that there is no durable representation to retrieve into.
His fix is to stop treating video as a bag of frames and start treating it as a spatial temporal volume, then build a memory layer over it: a context graph of time bounded moments, the entities and where they appear, the relationships between them, and corpus level themes. At TwelveLabs that is an embedding encoder, a context store, and a video language model exposed as an API. The design rules are blunt: ingest once and reason many times, store primitives not answers, ground every claim to a timestamp, and let intent decide what to remember, because brand safety and sports highlights need different things from the same footage.
Speaker info:
- https://x.com/le_james94
- linkedin.com/in/khanhnamle94
- jameskle.com
Timestamps:
0:00 - Video has no memory
0:50 - Video is a spatial temporal volume, not a bag of frames
2:06 - Three problems: wrong context, wrong memory, weak reasoning
3:36 - Five properties that make video memory hard
4:53 - The TwelveLabs stack: Marengo, the context store, and Pegasus
5:56 - Search versus memory
7:48 - The context graph
9:04 - Five design principles for a video memory layer
10:45 - From a static model to a video worker
12:51 - Demo: sports and tracking Messi across the World Cup
15:37 - Demos: traffic security and ad placement
