Uploaded June 2022 | Updated September 2026, 2 hours ago

---

OpenAI's new paper, "Video PreTraining(VPT): Learning to Act by Watching Unlabeled Online Videos" trains an agent to play Minecraft using a mixture of imitation learning (in the form of behavior cloning) and reinforcement learning (RL). The results are very impressive, training a 500 million parameter model that can obtain diamonds and occasionally craft diamond tools. I cover the paper in this video, talking about their approach of using and fine-tuning a foundational model with semi-supervised learning.

Outline
0:00 - Intro & Abstract
2:10 - Paper Goal
3:55 - Methods
5:30 - Initial Results
8:06 - Fine-tuning w/ BC
9:15 - Reinforcement Learning
12:02 - Sample Video
13:14 - Method Efficiency
15:30 - Text Conditioning
16:33 - Summary
18:00 - Criticisms & Thoughts

Sources
Paper: openai.com/blog/vpt
Blog post: arxiv.org/abs/2206.11795
Source code & models: github.com/openai/Video-Pre-Training
Agent videos:
youtube.com/playlist?list=PLNAOIb_agjf3U3rSvG_BCWqJ869NdBhcP
youtube.com/playlist?list=PLNAOIb_agjf2yDSs4AqcoyPv4z_eWUiKm
youtube.com/playlist?list=PLNAOIb_agjf3e_UKweM5pQUSfTw8r-Wfc