Uploaded August 2023 | Updated September 2026, 2 weeks ago

---

Check out the latest (and most visual) video on this topic! The Celestial Mechanics of Attention Mechanisms: youtube.com/watch?v=RFdb2rKAqFw

This is the second of a series of 3 videos where we demystify Transformer models and explain them with visuals and friendly examples.

Video 1: The attention mechanism in high level youtu.be/OxCpWwDCDFQ
Video 2: The attention mechanism with math (this one)
Video 3: Transformer models youtube.com/watch?v=qaWMOYf4ri8

If you like this material, check out LLM University from Cohere!
https://llm.university

00:00 Introduction
01:18 Recap: Embeddings and Context
04:46 Similarity
11:09 Attention
20:46 The Keys and Queries Matrices
25:02 The Values Matrix
28:41 Self and Multi-head attention
33:54: Conclusion