Uploaded October 2021 | Updated September 2026, 2 weeks ago
Shubham Tulsiani
Assistant Professor
Robotics Institute,
Carnegie Mellon University
October 22, 2021
https://www.ri.cmu.edu/event/ri-seminar-shubham-tulsiani-cmu-2021-10-22/
Towards Reconstructing Any Object in 3D
Abstract: The world we live in is incredibly diverse, comprising of over 10k natural and man-made object categories. While the computer vision community has made impressive progress in classifying images from such diverse categories, the state-of-the-art 3D prediction systems are still limited to merely tens of object classes. A key reason for this stark difference is the relative difficulty of acquiring supervision — while it is easy to annotate a semantic label for an image, obtaining ground-truth 3D for learning at scale is infeasible. But do we really need such ground-truth 3D for learning? In this talk, I will present a learning-based approach that can train from unstructured image collections, using only segmentation outputs from off-the-shelf recognition systems as supervisory signal, thus allowing us to scale to an order of magnitude more classes than existing works. I will also show how continually (instead of independently) learning 3D inference for new classes can further improve performance. Finally, while single-view image collections allow us to learn coarse category-level 3D inference, I will also show how sparse multi-view collections can allow us to infer fine instance-level 3D shapes for generic objects using as few as 8 images.
Bio: Shubham Tulsiani is an incoming Assistant Professor in the CMU School of Computer Science. Prior to this, he was research scientist at Facebook AI Research (FAIR). He received a PhD. in Computer Science from UC Berkeley in 2018. He is interested in building perception systems that can infer the spatial and physical structure of the world they observe.
Shubham Tulsiani
Assistant Professor
Robotics Institute,
Carnegie Mellon University
October 22, 2021
https://www.ri.cmu.edu/event/ri-seminar-shubham-tulsiani-cmu-2021-10-22/
Towards Reconstructing Any Object in 3D
Abstract: The world we live in is incredibly diverse, comprising of over 10k natural and man-made object categories. While the computer vision community has made impressive progress in classifying images from such diverse categories, the state-of-the-art 3D prediction systems are still limited to merely tens of object classes. A key reason for this stark difference is the relative difficulty of acquiring supervision — while it is easy to annotate a semantic label for an image, obtaining ground-truth 3D for learning at scale is infeasible. But do we really need such ground-truth 3D for learning? In this talk, I will present a learning-based approach that can train from unstructured image collections, using only segmentation outputs from off-the-shelf recognition systems as supervisory signal, thus allowing us to scale to an order of magnitude more classes than existing works. I will also show how continually (instead of independently) learning 3D inference for new classes can further improve performance. Finally, while single-view image collections allow us to learn coarse category-level 3D inference, I will also show how sparse multi-view collections can allow us to infer fine instance-level 3D shapes for generic objects using as few as 8 images.
Bio: Shubham Tulsiani is an incoming Assistant Professor in the CMU School of Computer Science. Prior to this, he was research scientist at Facebook AI Research (FAIR). He received a PhD. in Computer Science from UC Berkeley in 2018. He is interested in building perception systems that can infer the spatial and physical structure of the world they observe.
