Uploaded September 2024 | Updated November 2026, 3 hours ago
Attention for neural networks explained conceptually, visually, and mathematically in 15 minutes!
If you're familiar with the value weights matrix projecting into a lower dimensional space, I've created an addendum here showing the equivalence of both formulations:
drive.google.com/file/d/12XNiH2yB3jHwPvXbIHOThz2yGfkWc-cN/view?usp=sharing
0:00 The Concept
3:53 The Visual
5:41 The Math
Attention for neural networks explained conceptually, visually, and mathematically in 15 minutes!
If you're familiar with the value weights matrix projecting into a lower dimensional space, I've created an addendum here showing the equivalence of both formulations:
drive.google.com/file/d/12XNiH2yB3jHwPvXbIHOThz2yGfkWc-cN/view?usp=sharing
0:00 The Concept
3:53 The Visual
5:41 The Math










