Uploaded October 2021 | Updated September 2026, 2 hours ago
Maryland Robotics Center Research Symposium 2021: Toward Robust Manipulation in Complex Environments
Dieter Fox
Senior Director of Robotics Research at Nvidia
Professor at Univ. of Washington
Recent advances in deep learning and GPU-based computing have enabled significant progress in several areas of robotics, including navigation, visual recognition, and object manipulation. This progress has turned applications such as autonomous driving and delivery tasks in warehouses, hospitals, or hotels into realistic application scenarios. However, robust manipulation in complex settings is still an open research problem. Various efforts at NVIDIA robotics research investigate how deep learning along with physics-based and photo-realistic simulation can be used to train manipulators in virtual environments and then deploy them in the real world. Our work shows promising results on different pieces of the manipulation puzzle, including manipulator control, touch sensing, object pose detection, and object pick and place. In this talk, I will present some of these advances. I will describe a robot manipulator that can open and close cabinet doors and drawers, detect and pickup objects, and move these objects to desired locations. Our baseline system is designed to be applicable in a wide variety of environments, only relying on 3D articulated models of the kitchen and the relevant objects. I will discuss lessons learned so far, and various research directions toward enabling more robust and general manipulation systems that do not rely on existing models.
For more information on the Maryland Robotics Center Research Symposium 2021 see:
https://robotics.umd.edu/symposium2021
Maryland Robotics Center Research Symposium 2021: Toward Robust Manipulation in Complex Environments
Dieter Fox
Senior Director of Robotics Research at Nvidia
Professor at Univ. of Washington
Recent advances in deep learning and GPU-based computing have enabled significant progress in several areas of robotics, including navigation, visual recognition, and object manipulation. This progress has turned applications such as autonomous driving and delivery tasks in warehouses, hospitals, or hotels into realistic application scenarios. However, robust manipulation in complex settings is still an open research problem. Various efforts at NVIDIA robotics research investigate how deep learning along with physics-based and photo-realistic simulation can be used to train manipulators in virtual environments and then deploy them in the real world. Our work shows promising results on different pieces of the manipulation puzzle, including manipulator control, touch sensing, object pose detection, and object pick and place. In this talk, I will present some of these advances. I will describe a robot manipulator that can open and close cabinet doors and drawers, detect and pickup objects, and move these objects to desired locations. Our baseline system is designed to be applicable in a wide variety of environments, only relying on 3D articulated models of the kitchen and the relevant objects. I will discuss lessons learned so far, and various research directions toward enabling more robust and general manipulation systems that do not rely on existing models.
For more information on the Maryland Robotics Center Research Symposium 2021 see:
https://robotics.umd.edu/symposium2021
