Uploaded February 2025 | Updated September 2026, 2 weeks ago
In this talk, Danny Driess, a research scientist at Physical Intelligence (Pi), explores the path to creating generalist #robot models—robot policies capable of solving any task in any environment.
Danny introduces PaLM-E, one of the first large embodied #ai models, which demonstrated how general vision-language knowledge can transfer to robotics for high-level reasoning. He then explains how these insights contributed to the development of RT-2, Pi0, and Pi0-FAST. Notably, with Pi0, the team showcased one of the first generalist policies capable of fully autonomous, long-horizon dexterous tasks, such as unloading a dryer and folding laundry.
Bio: Danny Driess is a research scientist at Physical Intelligence (Pi). Prior to that, he was a senior research scientist at @googledeepmind, working in the intersection between robotics and #gemini.
Timestamps:
0:00 Introduction
1:14 LLMs, VLMs and Robots
3:26 Google DeemMind's PaLM-E: An embodied multimodal language model
5:45 How do you build a multimodal model for robotics, embodied reasoning, and planning? And what happens when you scale it up?
6:08 PaLM-E architecture: injecting multimodal information, robot environments, training data
9:00 PaLM-E conclusion
10:20 Google DeemMind's RT-2: Vision-Language-Action Models
11:42 RT-2 conclusion
12:18 Dexterity and action chunking
12:53 π0: A Vision-Language-Action Flow Model for General Robot Control
13:30 π0 model
15:58 π0 robots: training, performance comparison, finetuning
18:44 Why not RT-2 style VLA?
19:35 FAST: Efficient Action Tokenization for Vision-Language-Action Models
22:00 FAST tokenizer compression, comparison to other tokenization schemes, π0-FAST on DROID
#artificialintelligence #artificialgeneralintelligence #ai #robot #robotics #deeplearning #generalistai #machinelearning #roboticengineering #autonomousrobot #autonomousmobilerobots #educationalvideos #science #technology #techtalk #techtalks
Social Links:
Newsletter: buzzrobot.substack.com
X: https://x.com/sopharicks
Slack: join.slack.com/t/buzzrobot/shared_invite/zt-2s067rv7n-guPIMGe62rbp9ncxdnOUfQ
In this talk, Danny Driess, a research scientist at Physical Intelligence (Pi), explores the path to creating generalist #robot models—robot policies capable of solving any task in any environment.
Danny introduces PaLM-E, one of the first large embodied #ai models, which demonstrated how general vision-language knowledge can transfer to robotics for high-level reasoning. He then explains how these insights contributed to the development of RT-2, Pi0, and Pi0-FAST. Notably, with Pi0, the team showcased one of the first generalist policies capable of fully autonomous, long-horizon dexterous tasks, such as unloading a dryer and folding laundry.
Bio: Danny Driess is a research scientist at Physical Intelligence (Pi). Prior to that, he was a senior research scientist at @googledeepmind, working in the intersection between robotics and #gemini.
Timestamps:
0:00 Introduction
1:14 LLMs, VLMs and Robots
3:26 Google DeemMind's PaLM-E: An embodied multimodal language model
5:45 How do you build a multimodal model for robotics, embodied reasoning, and planning? And what happens when you scale it up?
6:08 PaLM-E architecture: injecting multimodal information, robot environments, training data
9:00 PaLM-E conclusion
10:20 Google DeemMind's RT-2: Vision-Language-Action Models
11:42 RT-2 conclusion
12:18 Dexterity and action chunking
12:53 π0: A Vision-Language-Action Flow Model for General Robot Control
13:30 π0 model
15:58 π0 robots: training, performance comparison, finetuning
18:44 Why not RT-2 style VLA?
19:35 FAST: Efficient Action Tokenization for Vision-Language-Action Models
22:00 FAST tokenizer compression, comparison to other tokenization schemes, π0-FAST on DROID
#artificialintelligence #artificialgeneralintelligence #ai #robot #robotics #deeplearning #generalistai #machinelearning #roboticengineering #autonomousrobot #autonomousmobilerobots #educationalvideos #science #technology #techtalk #techtalks
Social Links:
Newsletter: buzzrobot.substack.com
X: https://x.com/sopharicks
Slack: join.slack.com/t/buzzrobot/shared_invite/zt-2s067rv7n-guPIMGe62rbp9ncxdnOUfQ
