Special Tokens in LLM: Why Your Model Never Stops @KGPTalkie
Special Tokens in LLM: Why Your Model Never Stops  @KGPTalkie
Uploaded October 2026 | Updated November 2026, 4 days ago
The answer is complete, and the model keeps writing anyway. The reason is usually a missing special token. Here are the six that matter, in one reel.

Special tokens are not words. They are signals for the model: where to start, where to stop, and where the empty space is. They have their own IDs in the vocabulary, and the tokenizer adds them, not the user.

- BOS (beginning of sequence): marks where the text starts. Llama 3 calls it begin of text. In training, many documents are joined back to back, so BOS says a new document starts here
- EOS (end of sequence): the moment the model writes EOS, generation stops. GPT-2 calls it end of text
- CLS and SEP (from BERT): CLS goes at the start of every input, and for classification the final vector at that position is used. SEP separates two parts, like a question and a passage
- UNK (unknown): in BERT's WordPiece, a piece that cannot be built from the vocabulary becomes UNK and its meaning is lost. Byte-level tokenizers do not need UNK, because every text is bytes and every byte is in the vocabulary
- PAD: a GPU runs many sentences together and they all need the same length, so short ones get PAD tokens at the end just to fill the space
- The catch: fine-tune on your data and forget EOS at the end of every example, and the model never learns to stop. The answer is complete and it keeps writing until it hits the max tokens limit. Before fine-tuning, tokenize one example and check that the start and end special tokens are really there
- Illustrative on screen: the token counts, the example sentences and replies, the ordinary token IDs and the hex bytes of the rare symbol. The BERT IDs shown for PAD (0), UNK (100), CLS (101) and SEP (102) are from bert-base-uncased

Why doesn't the model read the PAD? That is the next LLM Interview reel: why do we need a PAD token and an attention mask? Has a model of yours ever just not stopped? Comment and tell me. Follow for the LLM Concepts series.

kgptalkie.com

#shorts #LLM #SpecialTokens #Tokenizer #EOS #BOS #BERT #PAD #GenerativeAI #MachineLearning #AIExplained #LLMConcepts #KGPTalkie
Special Tokens in LLM: Why Your Model Never StopsLangChain Tutorial for Beginners: Messages, Roles and LangSmith SetupBuild Finance RAG in Less than 10 Minutes with OpenAI Agent Builder10 Python Crash Course for ML - Set | ML for Data ScienceMCP Tutorial 1 - Control Web Browser with Playwright MCP Server and AIHermes vs OpenClaw: Which AI Agent Is Better?Ornith 9B vs 35B: The KV Cache Puzzle #shortsConnect Hermes Agent to Discord: Control Your AI Agent From Your Phone (2026)Hermes Agent Lead Generation Automation - Unlimited Leads/Day07 Gen AI Interview Preparation: What is Tokenization in LLM  #ai #genai #generativeai #interviewBookMyShow System Design: 4 Friends, 4 Seats, 1 StrangerGemma 4 😮 Qwen Killer? I Don’t Think So...
Laxmi Kant | KGP Talkie |

Special Tokens in LLM: Why Your Model Never Stops

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER