Uploaded October 2026 | Updated November 2026, 3 days ago
Can a 27 billion parameter model teach a 4 billion model to be as good as itself? Last time, RL got Qwen 4B from 52% to 58.4% and then hit a wall. This time I used Qwen 27B as a teacher and tried all three kinds of distillation on two DGX Sparks: training on the teacher's answers, training on its probabilities, and on-policy distillation where the 27B grades every token the 4B writes.
Sponsored by Dex, the talent agent for senior engineers. Free for engineers: meetdex.link/promptengineering-1
Links
- Previous video (RL, the sandbox, the 58% wall): youtu.be/Pl_2TelxjdY
- Code: github.com/PromtEngineer/smoldataenvs-grpo-spark
- SmolDataEnvs (the tasks): huggingface.co/datasets/FineEnvs/SmolDataEnvs
- Thinking Machines, On-Policy Distillation: thinkingmachines.ai/blog/on-policy-distillation
- Qwen3 Technical Report (strong-to-weak distillation): arxiv.org/abs/2505.09388
- Hinton, Vinyals, Dean (2015), Distilling the Knowledge in a Neural Network: arxiv.org/abs/1503.02531
- On-policy distillation papers: MiniLLM arxiv.org/abs/2306.08543 · GKD arxiv.org/abs/2306.13649
Chapters
00:00 - Introduction & The Distillation Idea
02:21 - Benchmark Setup & Small Data Environments
03:35 - Why Reinforcement Learning Hit a Wall
04:40 - The 3 Ways to Distill LLMs
08:04 - Hardware Setup (Dual DGX Spark)
08:37 - Method 1: Training on Teacher Answers
10:00 - Sponsor Break
11:00 - Method 2: Distilling Teacher Probabilities
13:30 - Method 3: Live On-Policy Distillation
16:42 - Comparing All 6 Experiment Results
17:27 - The 58% Ceiling & What the Small Model Actually Knows
19:07 - The 4-Tries Headroom Test
20:24 - Hardware & Practical Implementation Tips
21:21 - Recommendations & Which Method to Use
22:19 - Next Steps: Testing 2B and 9B Models
#distillation #qwen #llm #reinforcementlearning #localai #dgxspark
Can a 27 billion parameter model teach a 4 billion model to be as good as itself? Last time, RL got Qwen 4B from 52% to 58.4% and then hit a wall. This time I used Qwen 27B as a teacher and tried all three kinds of distillation on two DGX Sparks: training on the teacher's answers, training on its probabilities, and on-policy distillation where the 27B grades every token the 4B writes.
Sponsored by Dex, the talent agent for senior engineers. Free for engineers: meetdex.link/promptengineering-1
Links
- Previous video (RL, the sandbox, the 58% wall): youtu.be/Pl_2TelxjdY
- Code: github.com/PromtEngineer/smoldataenvs-grpo-spark
- SmolDataEnvs (the tasks): huggingface.co/datasets/FineEnvs/SmolDataEnvs
- Thinking Machines, On-Policy Distillation: thinkingmachines.ai/blog/on-policy-distillation
- Qwen3 Technical Report (strong-to-weak distillation): arxiv.org/abs/2505.09388
- Hinton, Vinyals, Dean (2015), Distilling the Knowledge in a Neural Network: arxiv.org/abs/1503.02531
- On-policy distillation papers: MiniLLM arxiv.org/abs/2306.08543 · GKD arxiv.org/abs/2306.13649
Chapters
00:00 - Introduction & The Distillation Idea
02:21 - Benchmark Setup & Small Data Environments
03:35 - Why Reinforcement Learning Hit a Wall
04:40 - The 3 Ways to Distill LLMs
08:04 - Hardware Setup (Dual DGX Spark)
08:37 - Method 1: Training on Teacher Answers
10:00 - Sponsor Break
11:00 - Method 2: Distilling Teacher Probabilities
13:30 - Method 3: Live On-Policy Distillation
16:42 - Comparing All 6 Experiment Results
17:27 - The 58% Ceiling & What the Small Model Actually Knows
19:07 - The 4-Tries Headroom Test
20:24 - Hardware & Practical Implementation Tips
21:21 - Recommendations & Which Method to Use
22:19 - Next Steps: Testing 2B and 9B Models
#distillation #qwen #llm #reinforcementlearning #localai #dgxspark










