Uploaded October 2024 | Updated November 2026, 1 hour ago
Visual Odometry (VO) is crucial for autonomous robotic navigation, especially in GPS-denied environments like planetary terrains. To improve robustness, recent model-based VO systems have begun combining standard and event-based cameras. While event cameras excel in low-light and high-speed motion, standard cameras provide dense and easier-to-track features. However, the field of image- and event-based VO still predominantly relies on model-based methods and is yet to fully integrate recent image-only advancements leveraging end-to-end learning-based architectures. Seamlessly integrating the two modalities remains challenging due to their different nature, one asynchronous, the other not, limiting the potential for a more effective image- and event-based VO. We introduce RAMP-VO, the first end-to-end learned image- and event-based VO system. It leverages novel Recurrent, Asynchronous, and Massively Parallel (RAMP) encoders capable of fusing asynchronous events with image data, providing 8× faster inference and 33% more accurate predictions than existing solutions. Despite being trained only in simulation, RAMP-VO outperforms previous methods on the newly introduced Apollo and Malapert datasets, and on existing benchmarks, where it improves image-based (ORB-SLAM, DSO, DROID-SLAM, DPVO) and event-based methods (DEVO, EDS) by up to 60% and 30%, paving the way for robust and asynchronous VO.
Reference:
Roberto Pellerito, Marco Cannici, Daniel Gehrig, Joris Belhadj, Olivier Dubois-Matra, Massimo Casasco, Davide Scaramuzza
Deep Visual Odometry with Events and Frames.
IEEE/RSJ International Conference on Intelligent Robots (IROS), 2024.
PDF: https://rpg.ifi.uzh.ch/docs/IROS24_Pellerito.pdf
Code and datasets: github.com/uzh-rpg/rampvo
Our research page on event-based vision: http://rpg.ifi.uzh.ch/research_dvs.html
For event-camera datasets, see here:
1. https://dsec.ifi.uzh.ch/
2. http://rpg.ifi.uzh.ch/davis_data.html
3. github.com/uzh-rpg/event-based_vision_resources#datasets
For an event camera simulator: http://rpg.ifi.uzh.ch/esim
For a survey paper on event cameras, see here: https://rpg.ifi.uzh.ch/docs/EventVisionSurvey.pdf
Other resources on event cameras (publications, software, drivers, where to buy, etc.): github.com/uzh-rpg/event-based_vision_resources
Affiliation:
Roberto Pellerito, Marco Cannici, Daniel Gehrig, and Davide Scaramuzza are with the Robotics and Perception Group, University of Zurich, Switzerland https://rpg.ifi.uzh.ch/
Joris Belhadj, Olivier Dubois-Matra, and Massimo Casasco are with European Space Agency
Visual Odometry (VO) is crucial for autonomous robotic navigation, especially in GPS-denied environments like planetary terrains. To improve robustness, recent model-based VO systems have begun combining standard and event-based cameras. While event cameras excel in low-light and high-speed motion, standard cameras provide dense and easier-to-track features. However, the field of image- and event-based VO still predominantly relies on model-based methods and is yet to fully integrate recent image-only advancements leveraging end-to-end learning-based architectures. Seamlessly integrating the two modalities remains challenging due to their different nature, one asynchronous, the other not, limiting the potential for a more effective image- and event-based VO. We introduce RAMP-VO, the first end-to-end learned image- and event-based VO system. It leverages novel Recurrent, Asynchronous, and Massively Parallel (RAMP) encoders capable of fusing asynchronous events with image data, providing 8× faster inference and 33% more accurate predictions than existing solutions. Despite being trained only in simulation, RAMP-VO outperforms previous methods on the newly introduced Apollo and Malapert datasets, and on existing benchmarks, where it improves image-based (ORB-SLAM, DSO, DROID-SLAM, DPVO) and event-based methods (DEVO, EDS) by up to 60% and 30%, paving the way for robust and asynchronous VO.
Reference:
Roberto Pellerito, Marco Cannici, Daniel Gehrig, Joris Belhadj, Olivier Dubois-Matra, Massimo Casasco, Davide Scaramuzza
Deep Visual Odometry with Events and Frames.
IEEE/RSJ International Conference on Intelligent Robots (IROS), 2024.
PDF: https://rpg.ifi.uzh.ch/docs/IROS24_Pellerito.pdf
Code and datasets: github.com/uzh-rpg/rampvo
Our research page on event-based vision: http://rpg.ifi.uzh.ch/research_dvs.html
For event-camera datasets, see here:
1. https://dsec.ifi.uzh.ch/
2. http://rpg.ifi.uzh.ch/davis_data.html
3. github.com/uzh-rpg/event-based_vision_resources#datasets
For an event camera simulator: http://rpg.ifi.uzh.ch/esim
For a survey paper on event cameras, see here: https://rpg.ifi.uzh.ch/docs/EventVisionSurvey.pdf
Other resources on event cameras (publications, software, drivers, where to buy, etc.): github.com/uzh-rpg/event-based_vision_resources
Affiliation:
Roberto Pellerito, Marco Cannici, Daniel Gehrig, and Davide Scaramuzza are with the Robotics and Perception Group, University of Zurich, Switzerland https://rpg.ifi.uzh.ch/
Joris Belhadj, Olivier Dubois-Matra, and Massimo Casasco are with European Space Agency










