Uploaded February 2024 | Updated September 2026, 2 weeks ago
Title: Inverse RL in real life: Lessons learned in Google Maps
Speaker: Matt Barnes (Google Research)
Date: Friday, February 16, 2024
Abstract: Inverse RL provides a strong framework for imitating humans' planning behavior, yet no approach has successfully addressed planetary-scale problems with hundreds of millions of states and demonstration trajectories. In this talk, I'll share how we scaled IRL algorithms (e.g. dominant eigenvector inspired initialization conditions) and the challenges we faced along the way. I'll also share our key observation that there exists a trade-off among classic IRL methods between the use of cheap, deterministic planners and expensive yet robust stochastic policies. This insight is leveraged in Receding Horizon Inverse Planning (RHIP), a new generalized method that enables interpolating between classic IRL algorithms and provides fine-grained control over performance trade-offs. The talk culminates in a policy that achieves a 16-24% improvement in Google Maps route quality, and to the best of our knowledge, represents the largest published benchmark of IRL algorithms in a real-world setting to date.
Bio: Matt Barnes is a senior engineer at Google Research in Seattle. Matt obtained his PhD in Robotics at Carnegie Mellon and was a postdoctoral scholar at the University of Washington with Sidd Srinivasa. His interests focus on designing large-scale systems at the intersection of learning and planning, and exploring the unique research challenges associated with bringing these systems into real-world use by billions of users.
This video is closed captioned.
Title: Inverse RL in real life: Lessons learned in Google Maps
Speaker: Matt Barnes (Google Research)
Date: Friday, February 16, 2024
Abstract: Inverse RL provides a strong framework for imitating humans' planning behavior, yet no approach has successfully addressed planetary-scale problems with hundreds of millions of states and demonstration trajectories. In this talk, I'll share how we scaled IRL algorithms (e.g. dominant eigenvector inspired initialization conditions) and the challenges we faced along the way. I'll also share our key observation that there exists a trade-off among classic IRL methods between the use of cheap, deterministic planners and expensive yet robust stochastic policies. This insight is leveraged in Receding Horizon Inverse Planning (RHIP), a new generalized method that enables interpolating between classic IRL algorithms and provides fine-grained control over performance trade-offs. The talk culminates in a policy that achieves a 16-24% improvement in Google Maps route quality, and to the best of our knowledge, represents the largest published benchmark of IRL algorithms in a real-world setting to date.
Bio: Matt Barnes is a senior engineer at Google Research in Seattle. Matt obtained his PhD in Robotics at Carnegie Mellon and was a postdoctoral scholar at the University of Washington with Sidd Srinivasa. His interests focus on designing large-scale systems at the intersection of learning and planning, and exploring the unique research challenges associated with bringing these systems into real-world use by billions of users.
This video is closed captioned.
![[Audio Descriptions] ArticuTool: A Modular Active End-Effector for Robot Assisted Feeding](https://i.ytimg.com/vi/rSqVBN42ZG0/mqdefault.jpg "[Audio Descriptions] ArticuTool: A Modular Active End-Effector for Robot Assisted Feeding
For a version without audio descriptions, visit https://youtu.be/0QLWAM63kqw")
