GRPO - Group Relative Policy Optimization  - How DeepSeek trains reasoning models @SerranoAcademy
GRPO - Group Relative Policy Optimization  - How DeepSeek trains reasoning models  @SerranoAcademy
Uploaded May 2025 | Updated November 2026, 1 day ago
GRPO is what DeepSeek used to train its amazing reasoning model. The biggest innovation comes from using reinforcement learning to get the model to improve itself, as opposed to self supervised learning. Learn all about it in this friendly video!

Other videos in RL for LLms:

Deep Reinforcement Learning: youtube.com/watch?v=SgC6AZss478
Reinforcement Learning with Human Feedback (RLHF): youtube.com/watch?v=Z_JUqJBpVOk
Proximal Policy Optimization (PPO): youtube.com/watch?v=TjHH_--7l8g
Direct Preference Optimization (DPO): youtube.com/watch?v=k2pD3k1485A
Group Relative Policy Optimization (GRPO): youtu.be/XeUB4h1OO1g

Get the Grokking Machine Learning book!
manning.com/books/grokking-machine-learning
Discount code (40%): serranoyt
(Use the discount code on checkout)

00:00 Introduction
00:26 Answering with context
01:40 DeepSeek vs ChatGPT
05:30 The GRPO score
07:05 Averaging over answers and steps
07:38 Quality (Advantage)
10:30 Probability of responses
15:36 Clipping the response
18:21 Not changing the model too much
GRPO - Group Relative Policy Optimization  - How DeepSeek trains reasoning modelsReinforcement Learning with Human Feedback (RLHF) - How to train and fine-tune Transformer ModelsHow does Netflix recommend movies? Matrix FactorizationWhy do we divide by n-1 to estimate the variance? A visual tour through Bessel correctionMachine Learning: Testing and Error MetricsEigenvectors and Generalized EigenspacesRead papers with Luis - The Illusion (of the illusion) of thinkingWhen is a sequence periodic? The Discrete Fourier Transform will tell usLive chat with Luis Serrano!RAG is not an Agent????Can quantum computers break the speed of information?Book by Luis Serrano - Grokking Machine Learning (40% off promo code)
Luis Serrano Academy |

GRPO - Group Relative Policy Optimization - How DeepSeek trains reasoning models

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER