Uploaded September 2026 | Updated November 2026, 50 minutes ago
DeepSeek V4.1 Flash architecture explained with drawings: how DeepSeek Sparse Attention (DSA) and Compressed Sparse Attention 2 (CSA2) let a 1M token model read at most 640 entries per layer for each new token, with a global KV cache of just 890 bytes per token.
I walk through the attention stack of DeepSeek V4.1 Flash, released on 10 September 2026, one picture at a time. We start with how query, key and value attention is calculated, then see why reading the prompt is limited by arithmetic while writing the answer is limited by moving the KV cache through GPU memory. From there you will see how the lightning indexer scores the whole context cheaply and passes the top 512 positions plus a 128 token sliding window to full attention, and how CSA2 shares one cache and one pick across layers with its Full, Reindex and Reuse modes. The last part does the math at 1M tokens: attention itself gets about 1,240x cheaper, and the whole block about 158x cheaper once the indexer is counted too. If you know what a transformer is, this video gives you the complete picture, and the written version with every diagram is linked below.
📺 Full playlist: youtube.com/playlist?list=PLc2rvfiptPSReropGbvDFpB6dneNBwqhD
⏱ Chapters:
0:00 Intro: What's New in DeepSeek V4.1 Flash
2:23 DSA and CSA2 at a Glance
3:24 The Problem with Full Attention at 1M Tokens
5:11 How DeepSeek Picks 512 + 128 Tokens
8:40 How Attention Scores Are Calculated
10:33 Why Most Attention Scores Are Near Zero
14:06 Prefill vs Decode: Where the Cost Goes
17:57 Why Decoding Is Memory Bound: HBM, SRAM and Bandwidth
21:18 890 Bytes per Token: The KV Cache Win
23:14 Shrinking the KV Cache: Pooling and 4 Cache Layers
25:13 Lightning Indexer: Solving the Chicken-and-Egg Problem
31:03 DSA Inside One Layer
32:55 CSA2: Compressed Sparse Attention 2
37:06 Layer Map: 20 Encoder + 20 Decoder Layers
38:41 Full, Reindex and Reuse Modes
40:31 The Math: 1,240x Cheaper Attention at 1M Tokens
45:31 How the Lightning Indexer Was Trained
46:31 Does Sparse Attention Change Accuracy?
47:50 Final Thoughts on DeepSeek V4.1 Flash
🔗 Resources:
Written tutorial with every diagram and number: kgptalkie.com/tutorials/llm-benchmarking/deepseek-sparse-attention-explained
DeepSeek V4.1 Flash model card and technical report: huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash
📺 Watch next: I Ran DeepSeek V4 Flash vs Qwen 3.8 Flash Next Locally on CPU
youtube.com/watch?v=sjFQtwIoHas
🎓 Go deeper with my Udemy course: Fine Tuning LLM with Hugging Face Transformers for NLP
kgptalkie.com/fine-tuning-llm
If this made sparse attention click for you, hit like, and tell me in the comments which part you want me to break down next. Subscribe and turn on the bell for more LLM architecture videos.
#DeepSeek #SparseAttention #KVCache #LLM
DeepSeek V4.1 Flash architecture explained with drawings: how DeepSeek Sparse Attention (DSA) and Compressed Sparse Attention 2 (CSA2) let a 1M token model read at most 640 entries per layer for each new token, with a global KV cache of just 890 bytes per token.
I walk through the attention stack of DeepSeek V4.1 Flash, released on 10 September 2026, one picture at a time. We start with how query, key and value attention is calculated, then see why reading the prompt is limited by arithmetic while writing the answer is limited by moving the KV cache through GPU memory. From there you will see how the lightning indexer scores the whole context cheaply and passes the top 512 positions plus a 128 token sliding window to full attention, and how CSA2 shares one cache and one pick across layers with its Full, Reindex and Reuse modes. The last part does the math at 1M tokens: attention itself gets about 1,240x cheaper, and the whole block about 158x cheaper once the indexer is counted too. If you know what a transformer is, this video gives you the complete picture, and the written version with every diagram is linked below.
📺 Full playlist: youtube.com/playlist?list=PLc2rvfiptPSReropGbvDFpB6dneNBwqhD
⏱ Chapters:
0:00 Intro: What's New in DeepSeek V4.1 Flash
2:23 DSA and CSA2 at a Glance
3:24 The Problem with Full Attention at 1M Tokens
5:11 How DeepSeek Picks 512 + 128 Tokens
8:40 How Attention Scores Are Calculated
10:33 Why Most Attention Scores Are Near Zero
14:06 Prefill vs Decode: Where the Cost Goes
17:57 Why Decoding Is Memory Bound: HBM, SRAM and Bandwidth
21:18 890 Bytes per Token: The KV Cache Win
23:14 Shrinking the KV Cache: Pooling and 4 Cache Layers
25:13 Lightning Indexer: Solving the Chicken-and-Egg Problem
31:03 DSA Inside One Layer
32:55 CSA2: Compressed Sparse Attention 2
37:06 Layer Map: 20 Encoder + 20 Decoder Layers
38:41 Full, Reindex and Reuse Modes
40:31 The Math: 1,240x Cheaper Attention at 1M Tokens
45:31 How the Lightning Indexer Was Trained
46:31 Does Sparse Attention Change Accuracy?
47:50 Final Thoughts on DeepSeek V4.1 Flash
🔗 Resources:
Written tutorial with every diagram and number: kgptalkie.com/tutorials/llm-benchmarking/deepseek-sparse-attention-explained
DeepSeek V4.1 Flash model card and technical report: huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash
📺 Watch next: I Ran DeepSeek V4 Flash vs Qwen 3.8 Flash Next Locally on CPU
youtube.com/watch?v=sjFQtwIoHas
🎓 Go deeper with my Udemy course: Fine Tuning LLM with Hugging Face Transformers for NLP
kgptalkie.com/fine-tuning-llm
If this made sparse attention click for you, hit like, and tell me in the comments which part you want me to break down next. Subscribe and turn on the bell for more LLM architecture videos.
#DeepSeek #SparseAttention #KVCache #LLM








![OpenRouter Free AI Coding Models: Free AI API Key Setup (curl + Python, No Card)
Get a free OpenRouter API key and call free AI models from your terminal and from Python, with no credit card and no graphics card. Day 10 of the Free AI Setup series: one free AI tool a day, set up step by step.
Watch the whole series in order: https://www.youtube.com/playlist?list=PLfUBzZXRNC-k
Previous, day 9, AnythingLLM: https://youtu.be/n2Y_qXUhrzM
Next, day 11, Kilo Code: https://youtu.be/RVsMCTfD7hQ
What you need: a browser and a Google or GitHub account. No card for the free models, and no GPU, because the models run on OpenRouters providers. PowerShell for the first call, Python for the second.
In this video:
- Find the free models: the Models page price filter and the public models list (every free model ID ends in :free)
- Three good free models: NVIDIA Nemotron 3 Super (120B), Nemotron 3 Ultra (550B, 1M context), Gemma 4 31B, plus the Free Models Router (openrouter/free)
- Sign up and create an API key with a credit limit
- Store the key in a Windows environment variable, OPENROUTER_API_KEY
- First call with curl.exe in PowerShell (and why plain curl fails in Windows PowerShell)
- A real 429 rate-limited upstream error on a busy free model, and the fix
- Python with the OpenAI SDK pointed at OpenRouter (two changed lines)
- The free limits: 20 requests a minute, 50 a day, or 1,000 a day after buying $10 of credits once
- Check your own limits with the /api/v1/key endpoint
- Use the key in other apps (base URL, key, model), and keep it safe
Chapters:
0:00 Intro: a 120B model answers for free
0:25 Day 10 of 30 and what OpenRouter is
0:56 What you need
1:20 Step 1: Find the free models
2:13 Step 2: Choose a model
2:55 Step 3: Sign up
3:11 Step 4: Create an API key with a credit limit
3:52 Step 5: Store the key in an environment variable
4:43 Step 6: First call with curl.exe
5:41 A real 429 error and the fix
6:34 Step 7: Python with the OpenAI SDK
7:56 Step 8: The Free Models Router
8:34 Step 9: The free limits
9:04 Check your own limits with /api/v1/key
9:44 Step 10: Use the key in other apps
10:25 Keep your key safe
11:11 ChatGPT Plus vs OpenRouter free
11:41 Next: Kilo Code
Code from the video
1) body.json (save it in the folder where you run the command)
{
model: nvidia/nemotron-3-super-120b-a12b:free,
messages: [
{ role: user, content: Explain what an API key is to a beginner in two short sentences. }
]
}
2) First call in PowerShell (type curl.exe, not curl: in Windows PowerShell plain curl is a different command)
curl.exe https://openrouter.ai/api/v1/chat/completions -H Content-Type: application/json -H Authorization: Bearer $env:OPENROUTER_API_KEY -d @body.json
3) Python: run pip install openai, save this as first_call.py, then run python first_call.py
import os
from openai import OpenAI
client = OpenAI(
base_url=https://openrouter.ai/api/v1,
api_key=os.environ[OPENROUTER_API_KEY],
)
reply = client.chat.completions.create(
model=nvidia/nemotron-3-ultra-550b-a55b:free,
messages=[{role: user, content: Give me three short tips to keep an API key safe.}],
)
print(reply.choices[0].message.content)
print(reply.model)
To let OpenRouter pick a free model for you, change the model to openrouter/free.
4) Check your limits
curl.exe https://openrouter.ai/api/v1/key -H Authorization: Bearer $env:OPENROUTER_API_KEY
Set OPENROUTER_API_KEY in the Windows environment variables first, then open a new terminal. Free models change over time: if a model gives an error, pick any model whose ID ends in :free from openrouter.ai/models.
Links
OpenRouter: https://openrouter.ai
Free models list: https://openrouter.ai/models?max_price=0
Quickstart: https://openrouter.ai/docs/quickstart
Free limits: https://openrouter.ai/docs/api/reference/limits
API keys and safety: https://openrouter.ai/docs/api/reference/authentication
Tomorrow, day 11: Kilo Code.
New free AI tool every day. Subscribe so you dont miss one: https://www.youtube.com/@KGPTalkie?sub_confirmation=1 OpenRouter Free AI Coding Models: Free AI API Key Setup (curl + Python, No Card)](https://i.ytimg.com/vi/l5oBv_Mhf3k/mqdefault.jpg)

