Uploaded March 2026 | Updated November 2026, 3 hours ago
LLM Architecture Gallery: sebastianraschka.com/llm-architecture-gallery
In this video, I take you on a visual tour of modern LLM architectures and walk through the key ideas behind models like DeepSeek, Qwen3-Next, Kimi, Sarvam, Ling 2.5, and Nemotron.
We look at what actually changed in recent LLM design, including grouped-query attention (GQA), sliding-window attention, multi-head latent attention (MLA), DeepSeek sparse attention, and hybrid linear attention.
The goal of the gallery is to make it easier to compare architectures side by side, connect the diagrams back to papers, configs, and code, and get a clearer intuition for how today’s language models differ under the hood.
Poster: redbubble.com/i/poster/LLM-Architecture-Gallery-by-Ahead-of-AI/179274487/flk2
Articles / newsletter: magazine.sebastianraschka.com/p/visual-attention-variants
00:00 Intro
00:55 Why I built the gallery
01:16 Overview of the LLM Architecture Gallery
04:17 Comparing models side by side
05:41 Benchmarks and the Artificial Intelligence Index
07:03 GPT2 XL as the baseline architecture
10:22 Grouped-Query Attention (GQA)
14:51 Sliding-Window Attention
18:40 Multi-Head Latent Attention (MLA)
25:31 Sarvam 30B vs 105B
27:41 DeepSeek Sparse Attention
30:24 Hybrid Attention and Qwen3-Next
33:20 Kimi Linear, Ling 2.5, and Nemotron
36:39 Poster, future updates, and wrap-up
LLM Architecture Gallery: sebastianraschka.com/llm-architecture-gallery
In this video, I take you on a visual tour of modern LLM architectures and walk through the key ideas behind models like DeepSeek, Qwen3-Next, Kimi, Sarvam, Ling 2.5, and Nemotron.
We look at what actually changed in recent LLM design, including grouped-query attention (GQA), sliding-window attention, multi-head latent attention (MLA), DeepSeek sparse attention, and hybrid linear attention.
The goal of the gallery is to make it easier to compare architectures side by side, connect the diagrams back to papers, configs, and code, and get a clearer intuition for how today’s language models differ under the hood.
Poster: redbubble.com/i/poster/LLM-Architecture-Gallery-by-Ahead-of-AI/179274487/flk2
Articles / newsletter: magazine.sebastianraschka.com/p/visual-attention-variants
00:00 Intro
00:55 Why I built the gallery
01:16 Overview of the LLM Architecture Gallery
04:17 Comparing models side by side
05:41 Benchmarks and the Artificial Intelligence Index
07:03 GPT2 XL as the baseline architecture
10:22 Grouped-Query Attention (GQA)
14:51 Sliding-Window Attention
18:40 Multi-Head Latent Attention (MLA)
25:31 Sarvam 30B vs 105B
27:41 DeepSeek Sparse Attention
30:24 Hybrid Attention and Qwen3-Next
33:20 Kimi Linear, Ling 2.5, and Nemotron
36:39 Poster, future updates, and wrap-up










