Uploaded May 2025 | Updated September 2026, 2 hours ago
Future Leaders in Robotics and AI Seminar Series: Watch, Predict, Act: Robot Learning Meets Web Videos
Online Seminar
Homanga Bharadhwaj
PhD Student
Carnegie Mellon University
Developing robots that can help us in our daily activities has been a guiding aspiration in AI for decades. Yet, general-purpose robots that work out of the box, are able to perform tasks without manual interventions, and are safe to interact with remains an elusive goal. Since collecting robot interaction data in diverse scenarios is difficult due to operational constraints, a key challenge in robotics is being able to perform new tasks in novel scenes without requiring robot data collection in every scenario. Going beyond the standard of end-to-end imitation learning, in this talk I will describe an alternate paradigm towards this goal: combining robot-specific data with predictive planning from diverse web videos such as YouTube clips of humans doing daily chores.
By learning to predict motion and contextual cues from naturally diverse web data for robotic policy learning, I will demonstrate how this recipe enables training common goal/language-conditioned policies capable of multiple in-the-wild manipulation tasks in unseen offices and kitchens, across different robotic embodiments. I will conclude by laying out my vision for the coming years, encompassing the entire spectrum of learning structure from diverse web datasets, developing robot learning algorithms that can scalably use such structured predictions for broad generalization, and deploying robotic systems in-the-wild by being adaptable to deployment-time constraints and compliant with human preferences.
For more information, please visit:
https://robotics.umd.edu/FutureLeaders
Future Leaders in Robotics and AI Seminar Series: Watch, Predict, Act: Robot Learning Meets Web Videos
Online Seminar
Homanga Bharadhwaj
PhD Student
Carnegie Mellon University
Developing robots that can help us in our daily activities has been a guiding aspiration in AI for decades. Yet, general-purpose robots that work out of the box, are able to perform tasks without manual interventions, and are safe to interact with remains an elusive goal. Since collecting robot interaction data in diverse scenarios is difficult due to operational constraints, a key challenge in robotics is being able to perform new tasks in novel scenes without requiring robot data collection in every scenario. Going beyond the standard of end-to-end imitation learning, in this talk I will describe an alternate paradigm towards this goal: combining robot-specific data with predictive planning from diverse web videos such as YouTube clips of humans doing daily chores.
By learning to predict motion and contextual cues from naturally diverse web data for robotic policy learning, I will demonstrate how this recipe enables training common goal/language-conditioned policies capable of multiple in-the-wild manipulation tasks in unseen offices and kitchens, across different robotic embodiments. I will conclude by laying out my vision for the coming years, encompassing the entire spectrum of learning structure from diverse web datasets, developing robot learning algorithms that can scalably use such structured predictions for broad generalization, and deploying robotic systems in-the-wild by being adaptable to deployment-time constraints and compliant with human preferences.
For more information, please visit:
https://robotics.umd.edu/FutureLeaders
