Uploaded May 2021 | Updated September 2026, 4 hours ago
This is a tutorial and explanation for how to code Proximal Policy Optimization PPO. I cover how to code the core training loop of the algorithm and apply Proximal Policy Optimization to the OpenAI gym Cart Pole environment.
PPO is a reinforcement learning training method. It falls into the category of policy gradient methods, which is where a predictor is trained on a gradient derived directly from a reward function. PPO is sample efficient and very stable which makes it great from RL control problems like robotics and also many other tasks.
RL theory series: youtube.com/playlist?list=PL_49VD9KwQ_OML1Knh-Yb7FUFkhTLS0jL
^ Watch the series above if you were confused
PPO paper: arxiv.org/abs/1707.06347
Generalized Advantage Estimation (GAE) paper: arxiv.org/abs/1506.02438
This is a tutorial and explanation for how to code Proximal Policy Optimization PPO. I cover how to code the core training loop of the algorithm and apply Proximal Policy Optimization to the OpenAI gym Cart Pole environment.
PPO is a reinforcement learning training method. It falls into the category of policy gradient methods, which is where a predictor is trained on a gradient derived directly from a reward function. PPO is sample efficient and very stable which makes it great from RL control problems like robotics and also many other tasks.
RL theory series: youtube.com/playlist?list=PL_49VD9KwQ_OML1Knh-Yb7FUFkhTLS0jL
^ Watch the series above if you were confused
PPO paper: arxiv.org/abs/1707.06347
Generalized Advantage Estimation (GAE) paper: arxiv.org/abs/1506.02438
