Uploaded October 2026 | Updated November 2026, 4 days ago
The answer is complete, and the model keeps writing anyway. The reason is usually a missing special token. Here are the six that matter, in one reel.
Special tokens are not words. They are signals for the model: where to start, where to stop, and where the empty space is. They have their own IDs in the vocabulary, and the tokenizer adds them, not the user.
- BOS (beginning of sequence): marks where the text starts. Llama 3 calls it begin of text. In training, many documents are joined back to back, so BOS says a new document starts here
- EOS (end of sequence): the moment the model writes EOS, generation stops. GPT-2 calls it end of text
- CLS and SEP (from BERT): CLS goes at the start of every input, and for classification the final vector at that position is used. SEP separates two parts, like a question and a passage
- UNK (unknown): in BERT's WordPiece, a piece that cannot be built from the vocabulary becomes UNK and its meaning is lost. Byte-level tokenizers do not need UNK, because every text is bytes and every byte is in the vocabulary
- PAD: a GPU runs many sentences together and they all need the same length, so short ones get PAD tokens at the end just to fill the space
- The catch: fine-tune on your data and forget EOS at the end of every example, and the model never learns to stop. The answer is complete and it keeps writing until it hits the max tokens limit. Before fine-tuning, tokenize one example and check that the start and end special tokens are really there
- Illustrative on screen: the token counts, the example sentences and replies, the ordinary token IDs and the hex bytes of the rare symbol. The BERT IDs shown for PAD (0), UNK (100), CLS (101) and SEP (102) are from bert-base-uncased
Why doesn't the model read the PAD? That is the next LLM Interview reel: why do we need a PAD token and an attention mask? Has a model of yours ever just not stopped? Comment and tell me. Follow for the LLM Concepts series.
kgptalkie.com
#shorts #LLM #SpecialTokens #Tokenizer #EOS #BOS #BERT #PAD #GenerativeAI #MachineLearning #AIExplained #LLMConcepts #KGPTalkie
The answer is complete, and the model keeps writing anyway. The reason is usually a missing special token. Here are the six that matter, in one reel.
Special tokens are not words. They are signals for the model: where to start, where to stop, and where the empty space is. They have their own IDs in the vocabulary, and the tokenizer adds them, not the user.
- BOS (beginning of sequence): marks where the text starts. Llama 3 calls it begin of text. In training, many documents are joined back to back, so BOS says a new document starts here
- EOS (end of sequence): the moment the model writes EOS, generation stops. GPT-2 calls it end of text
- CLS and SEP (from BERT): CLS goes at the start of every input, and for classification the final vector at that position is used. SEP separates two parts, like a question and a passage
- UNK (unknown): in BERT's WordPiece, a piece that cannot be built from the vocabulary becomes UNK and its meaning is lost. Byte-level tokenizers do not need UNK, because every text is bytes and every byte is in the vocabulary
- PAD: a GPU runs many sentences together and they all need the same length, so short ones get PAD tokens at the end just to fill the space
- The catch: fine-tune on your data and forget EOS at the end of every example, and the model never learns to stop. The answer is complete and it keeps writing until it hits the max tokens limit. Before fine-tuning, tokenize one example and check that the start and end special tokens are really there
- Illustrative on screen: the token counts, the example sentences and replies, the ordinary token IDs and the hex bytes of the rare symbol. The BERT IDs shown for PAD (0), UNK (100), CLS (101) and SEP (102) are from bert-base-uncased
Why doesn't the model read the PAD? That is the next LLM Interview reel: why do we need a PAD token and an attention mask? Has a model of yours ever just not stopped? Comment and tell me. Follow for the LLM Concepts series.
kgptalkie.com
#shorts #LLM #SpecialTokens #Tokenizer #EOS #BOS #BERT #PAD #GenerativeAI #MachineLearning #AIExplained #LLMConcepts #KGPTalkie










