MoE Token Routing Explained: How Mixture of Experts Works (with Code) @HuggingFace
MoE Token Routing Explained: How Mixture of Experts Works (with Code)  @HuggingFace
Uploaded January 2026 | Updated November 2026, 6 days ago
This video dives deep into Token Routing, the core algorithm of Mixture of Experts (MoE) models.

Slides: huggingface.co/ariG23498/moe-routing-algorithm/blob/main/MoE_routing_algorithm.pdf
Colab Notebook: huggingface.co/ariG23498/moe-routing-algorithm/blob/main/routing_mechanism.ipynb

Chapter Timestamp:
Introduction: 00:00
Laying the Foundation for Mixture of Experts (MoE): 00:09
Focus on Token Routing: 00:50
What is a Mixture of Experts Layer?: 02:36
Problem Statement and Configurations: 04:48
Compute Router Logits: 08:31
Sparsity and Selecting Top K Experts: 10:54
Normalizing Logits to Router Probabilities: 12:43
Slot Selection: 14:39
Dropping Oversubscribed Tokens: 16:51
Updated Normalized Token Weights: 20:36
Updated Slot Selection and Token Slots: 21:34
Final Weight Matrix Construction: 24:35
Conclusion: 32:41


Correction:
As @denisflavius5365 rightly suggested the router matrix in the slides should be of the shape 3x4 and not 4x4.
MoE Token Routing Explained: How Mixture of Experts Works (with Code)Workshop: Training Agents at the frontier of AI IndustryLets talk Enterprise HubAssemble and Calibrate Moss v1: LeRobot Tutorial #6 by Jess MossNew Model: Inkling by Thinking Machine on Hugging FaceLocal Agents are the FutureHow to Build Your Own AI Agent (Like PewDiePie)Steering LLM Behavior Without Fine-TuningWhat can you build during 48h Robotics Hackathon?Science Behind the Open Cyber Frontier with Mistral Large 4Assemble and Calibrate SO-100: LeRobot Tutorial #7 by Jess MossMachine Learning Experts - Margaret Mitchell
Hugging Face |

MoE Token Routing Explained: How Mixture of Experts Works (with Code)

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER