Learning Depth with Very Sparse Supervision (IROS 2020 Pitch Video) @ailabRPG
Learning Depth with Very Sparse Supervision (IROS 2020 Pitch Video)  @ailabRPG
Uploaded August 2020 | Updated November 2026, 1 hour ago
Motivated by the astonishing capabilities of natural intelligent agents and inspired by theories from psychology, this paper explores the idea that perception gets coupled to 3D properties of the world via interaction with the environment. Existing works for depth estimation require either massive
amounts of annotated training data or some form of hard-coded geometrical constraint. This paper explores a new approach to learning depth perception requiring neither of those. Specifically, we propose a novel global-local network architecture that can be trained with the data observed by a robot exploring an environment: images and extremely sparse depth measurements, down to even a single pixel per image. From a pair of consecutive images, the proposed network outputs a latent representation of the camera’s and scene’s parameters, and a dense depth map. Experiments on several datasets show that, when ground truth is available even for just one of the image pixels, the proposed network can learn monocular dense depth estimation up to 22.5% more accurately than state-of-the-art approaches. We believe that this work, in addition to its scientific interest, lays the foundations to learn depth with extremely sparse supervision, which can be valuable to all robotic systems acting under severe bandwidth or sensing constraints.

Reference:
A.Loquercio, A. Dosovitskiy, D. Scaramuzza
Robotics and Automation Letters and IROS, 2020
PDF: http://rpg.ifi.uzh.ch/docs/IROS20_Loquercio.pdf

More info on our research on Robot Learning: http://rpg.ifi.uzh.ch/research_learning.html

Affiliations:
A. Loquercio and D. Scaramuzza are with the Robotics and Perception Group, Dep. of Informatics, University of Zurich, and Dep. of Neuroinformatics, University of Zurich and ETH Zurich, Switzerland
http://rpg.ifi.uzh.ch/

Alexey Dosovitskiy is with Google Research, Berlin.
Learning Depth with Very Sparse Supervision (IROS 2020 Pitch Video)Demonstrating Agile Flight from Pixels without State Estimation (RSS 2024)ICRA 2026 Keynote Tutorial - Learning Agile Vision-based Quadrotor FlightParabolic Flight Experiments | How Gravity affects Self-Motion estimation in Drone PilotsFisher Information Field: an Efficient and Differentiable Map for Perception-aware PlanningPowerline Tracking with Event Cameras (IROS 2021 Video Pitch)10-Year Anniversary of the UZH Robotics and Perception GroupEnd-to-End Learning of Representations for Asynchronous Event-Based Data (ICCV19)Time Lens++: Event-based Frame Interpolation with Parametric Flow and Multi-scale Fusion (CVPR 2022)Event-Based Angular Velocity Regression with Spiking Networks (ICRA20 Video Pitch)5th Year Anniversary of the Robotics and Perception GroupE-RAFT: Dense Optical Flow from Event Cameras (3DV 2021)
UZH Robotics and Perception Group |

Learning Depth with Very Sparse Supervision (IROS 2020 Pitch Video)

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER