Uploaded May 2022 | Updated September 2026, 20 hours ago
DeepMind has released "A Generalist Agent", a paper that introduces their new multi-modal model Gato. But is Gato truly a generalist agent? It is a transformer based model with the goal of generalizing over new tasks. It is trained fully autoregressively with supervised learning (no reinforcement learning) on a total of 603 different tasks. The tasks include robotics, Atari, DM Lab, Procgen, and a lot more. It also includes text and image tasks. This video is a paper review / explanation where I also give my thoughts on the paper.
Outline:
0:00 - Intro & Gato Overview
1:20 - Paper Motivation
2:56 - Model Explanation
6:13 - Data & Tasks
7:45 - Initial Results
9:38 - Zero Shot Transfer
12:58 - Generalization Ablation
17:05 - Fine-tuning
19:38 - My Thoughts
Gato blog post: deepmind.com/publications/a-generalist-agent
Gato paper: arxiv.org/pdf/2205.06175.pdf
DeepMind has released "A Generalist Agent", a paper that introduces their new multi-modal model Gato. But is Gato truly a generalist agent? It is a transformer based model with the goal of generalizing over new tasks. It is trained fully autoregressively with supervised learning (no reinforcement learning) on a total of 603 different tasks. The tasks include robotics, Atari, DM Lab, Procgen, and a lot more. It also includes text and image tasks. This video is a paper review / explanation where I also give my thoughts on the paper.
Outline:
0:00 - Intro & Gato Overview
1:20 - Paper Motivation
2:56 - Model Explanation
6:13 - Data & Tasks
7:45 - Initial Results
9:38 - Zero Shot Transfer
12:58 - Generalization Ablation
17:05 - Fine-tuning
19:38 - My Thoughts
Gato blog post: deepmind.com/publications/a-generalist-agent
Gato paper: arxiv.org/pdf/2205.06175.pdf
