A Visual Tour of Modern LLM Architectures @SebastianRaschka
A Visual Tour of Modern LLM Architectures  @SebastianRaschka
Uploaded March 2026 | Updated November 2026, 3 hours ago
LLM Architecture Gallery: sebastianraschka.com/llm-architecture-gallery

In this video, I take you on a visual tour of modern LLM architectures and walk through the key ideas behind models like DeepSeek, Qwen3-Next, Kimi, Sarvam, Ling 2.5, and Nemotron.

We look at what actually changed in recent LLM design, including grouped-query attention (GQA), sliding-window attention, multi-head latent attention (MLA), DeepSeek sparse attention, and hybrid linear attention.

The goal of the gallery is to make it easier to compare architectures side by side, connect the diagrams back to papers, configs, and code, and get a clearer intuition for how today’s language models differ under the hood.

Poster: redbubble.com/i/poster/LLM-Architecture-Gallery-by-Ahead-of-AI/179274487/flk2
Articles / newsletter: magazine.sebastianraschka.com/p/visual-attention-variants

00:00 Intro
00:55 Why I built the gallery
01:16 Overview of the LLM Architecture Gallery
04:17 Comparing models side by side
05:41 Benchmarks and the Artificial Intelligence Index
07:03 GPT2 XL as the baseline architecture
10:22 Grouped-Query Attention (GQA)
14:51 Sliding-Window Attention
18:40 Multi-Head Latent Attention (MLA)
25:31 Sarvam 30B vs 105B
27:41 DeepSeek Sparse Attention
30:24 Hybrid Attention and Qwen3-Next
33:20 Kimi Linear, Ling 2.5, and Nemotron
36:39 Poster, future updates, and wrap-up
A Visual Tour of Modern LLM ArchitecturesL19.0 RNNs & Transformers for Sequence-to-Sequence Modeling   Lecture OverviewConditional Ordinal Regression for Neural Networks (CORN) With Examples in PyTorchDo Looped Transformers Hide Chain of Thought?L15.3 Different Types of Sequence Modeling TasksL17.7 VAE Latent Space Arithmetic in PyTorch   Making People Smile (Code Example)L16.5 Other Types of AutoencodersL14.6.2 Transfer Learning in PyTorch   Code ExampleL10.5.2 Dropout Co-Adaptation InterpretationL5.8 Adaline Code ExampleL8.2 Logistic Regression Loss FunctionL17.1 Variational Autoencoder Overview
Sebastian Raschka |

A Visual Tour of Modern LLM Architectures

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER