Uploaded November 2024 | Updated November 2026, 3 days ago
Transformer models can generate language really well, but how do they do it?
A very important step of the pipeline is the Positional Encoding step, which injects the order of the words into the LLM, in order for it to tell the difference between sentences with the same words, but in different order.
This is part of a series of videos on Large Language Models:
Video 1: The attention mechanism youtube.com/watch?v=OxCpWwDCDFQ
Video 2: The math behind the attention mechanism youtube.com/watch?v=UPtG_38Oq8o
Video 3: Transformer models youtube.com/watch?v=qaWMOYf4ri8
Grokking Machine Learning Book:
manning.com/books/grokking-machine-learning
40% discount promo code: serranoyt
00:00 Introduction
03:01 How much to add to each word?
03:29 Some trigonometry
06:20 The formulas
Transformer models can generate language really well, but how do they do it?
A very important step of the pipeline is the Positional Encoding step, which injects the order of the words into the LLM, in order for it to tell the difference between sentences with the same words, but in different order.
This is part of a series of videos on Large Language Models:
Video 1: The attention mechanism youtube.com/watch?v=OxCpWwDCDFQ
Video 2: The math behind the attention mechanism youtube.com/watch?v=UPtG_38Oq8o
Video 3: Transformer models youtube.com/watch?v=qaWMOYf4ri8
Grokking Machine Learning Book:
manning.com/books/grokking-machine-learning
40% discount promo code: serranoyt
00:00 Introduction
03:01 How much to add to each word?
03:29 Some trigonometry
06:20 The formulas










