Uploaded October 2026 | Updated November 2026, 5 hours ago
In this final video, we look into model distillation as a more efficient alternative to reinforcement learning for developing small reasoning models.
- Reasoning playlist: youtube.com/watch?v=Kh9mqTzjuEQ&list=PLfrHAQCRxtUs
- Reasoning Book: amzn.to/4aAKiFY
- Reasoning GitHub repo: github.com/rasbt/reasoning-from-scratch
- LLMs from Scratch book: amzn.to/4fqvn0D
- LLMs from Scratch repo: github.com/rasbt/LLMs-from-scratch
- LLMs from Scratch playlist: youtube.com/watch?v=yAcWnfsZhzo&list=PLTKMiZHVd_2IIEsoJrWACkIxLRdfMlw11&index=1
00:00 Introduction and series recap
01:07 Why distill a reasoning model?
02:51 Soft vs. hard distillation
11:06 On-policy vs. off-policy distillation
18:09 Preparing data for reasoning distillation
22:35 Loading the DeepSeek R1 MATH dataset
28:04 Inspecting reasoning traces and final answers
31:27 Tokenization and chat templates
35:58 Pre-tokenizing the training dataset
39:01 Sequence lengths and the 2,048-token limit
42:38 Creating training and validation splits
45:32 Loading the Qwen3-0.6B base model
46:24 Computing answer-token log probabilities
54:10 Cross-entropy loss for distillation
57:25 Training and validation loss helpers
1:02:03 Implementing the distillation training loop
1:07:26 Running the standalone training script
1:09:41 Plotting training and validation losses
1:11:31 Evaluating checkpoints on MATH-500
1:14:09 Comparing teachers and distillation results
1:16:34 Future directions for reasoning models
1:19:41 Bonus exercises and next steps
In this final video, we look into model distillation as a more efficient alternative to reinforcement learning for developing small reasoning models.
- Reasoning playlist: youtube.com/watch?v=Kh9mqTzjuEQ&list=PLfrHAQCRxtUs
- Reasoning Book: amzn.to/4aAKiFY
- Reasoning GitHub repo: github.com/rasbt/reasoning-from-scratch
- LLMs from Scratch book: amzn.to/4fqvn0D
- LLMs from Scratch repo: github.com/rasbt/LLMs-from-scratch
- LLMs from Scratch playlist: youtube.com/watch?v=yAcWnfsZhzo&list=PLTKMiZHVd_2IIEsoJrWACkIxLRdfMlw11&index=1
00:00 Introduction and series recap
01:07 Why distill a reasoning model?
02:51 Soft vs. hard distillation
11:06 On-policy vs. off-policy distillation
18:09 Preparing data for reasoning distillation
22:35 Loading the DeepSeek R1 MATH dataset
28:04 Inspecting reasoning traces and final answers
31:27 Tokenization and chat templates
35:58 Pre-tokenizing the training dataset
39:01 Sequence lengths and the 2,048-token limit
42:38 Creating training and validation splits
45:32 Loading the Qwen3-0.6B base model
46:24 Computing answer-token log probabilities
54:10 Cross-entropy loss for distillation
57:25 Training and validation loss helpers
1:02:03 Implementing the distillation training loop
1:07:26 Running the standalone training script
1:09:41 Plotting training and validation losses
1:11:31 Evaluating checkpoints on MATH-500
1:14:09 Comparing teachers and distillation results
1:16:34 Future directions for reasoning models
1:19:41 Bonus exercises and next steps










