Uploaded April 2022 | Updated September 2026, 3 days ago
Despite numerous successes in deep robotic learning over the past decade, the generalization and versatility of robots across environments and tasks has remained a major challenge. This is because much of reinforcement and imitation learning research trains agents from scratch in a single or a few environments, training special-purpose policies from special-purpose datasets. In contrast, the rest of machine learning has drawn considerable success from repeatedly reusing broad datasets and recycling pre-trained models for a variety of purposes. Replicating this success in robotics is no easy feat, since robot data doesn’t simply exist in vast quantities on the internet. In this talk, I will discuss how our embodied learning algorithms need to reduce, reuse, and recycle — reducing the need for special-purpose online data collection, reusing existing data, and recycling pre-trained models with various downstream tasks. Towards this goal, I will present research that studies zero-shot robot generalization to new tasks and language commands, using a diverse dataset containing 100 distinct tasks. I will also discuss how we might develop recyclable pre-trained models for robot learning using large-scale datasets, including language-annotated videos of humans. In all cases, the evaluation will emphasize generalization, including to new objects, new scenes, and new tasks. I'll conclude by discussing some important open questions and future directions.
Chelsea Finn's Bio: https://ai.stanford.edu/~cbfinn/
Despite numerous successes in deep robotic learning over the past decade, the generalization and versatility of robots across environments and tasks has remained a major challenge. This is because much of reinforcement and imitation learning research trains agents from scratch in a single or a few environments, training special-purpose policies from special-purpose datasets. In contrast, the rest of machine learning has drawn considerable success from repeatedly reusing broad datasets and recycling pre-trained models for a variety of purposes. Replicating this success in robotics is no easy feat, since robot data doesn’t simply exist in vast quantities on the internet. In this talk, I will discuss how our embodied learning algorithms need to reduce, reuse, and recycle — reducing the need for special-purpose online data collection, reusing existing data, and recycling pre-trained models with various downstream tasks. Towards this goal, I will present research that studies zero-shot robot generalization to new tasks and language commands, using a diverse dataset containing 100 distinct tasks. I will also discuss how we might develop recyclable pre-trained models for robot learning using large-scale datasets, including language-annotated videos of humans. In all cases, the evaluation will emphasize generalization, including to new objects, new scenes, and new tasks. I'll conclude by discussing some important open questions and future directions.
Chelsea Finn's Bio: https://ai.stanford.edu/~cbfinn/
