Reinforcement Learning Meets Visual Odometry  (ECCV 2024) @ailabRPG
Reinforcement Learning Meets Visual Odometry  (ECCV 2024)  @ailabRPG
Uploaded October 2024 | Updated November 2026, 42 minutes ago
Visual Odometry (VO) is essential to downstream mobile robotics and augmented/virtual reality tasks. Despite recent advances, existing VO methods still rely on heuristic design choices that require several weeks of hyperparameter tuning by human experts, hindering generalizability and robustness. We address these challenges by reframing VO as a sequential decision-making task and applying Reinforcement Learning (RL) to adapt the VO process dynamically. Our approach introduces a neural network, operating as an agent within the VO pipeline, to make decisions such as keyframe and grid-size selection based on real-time conditions. Our method minimizes reliance on heuristic choices using a reward function based on pose error, runtime, and other metrics to guide the system. Our RL framework treats the VO system and the image sequence as an environment, with the agent receiving observations from keypoints, map statistics, and prior poses. Experimental results using classical VO methods and public benchmarks demonstrate improvements in accuracy and robustness, validating the generalizability of our RL-enhanced VO approach to different scenarios. We believe this paradigm shift advances VO technology by eliminating the need for time-intensive parameter tuning of heuristics.

Reference:
Nico Messikommer*, Giovanni Cioffi*, Mathias Gehrig , and Davide Scaramuzza
"Reinforcement Learning Meets Visual Odometry" European Conference on Computer Vision. (ECCV), 2024
PDF: https://rpg.ifi.uzh.ch/docs/ECCV24_Messikommer.pdf
Code: github.com/uzh-rpg/rl_vo

For more info on our research on:
Visual and Inertial Odometry and SLAM: http://rpg.ifi.uzh.ch/research_vo.html
Machine Learning: https://rpg.ifi.uzh.ch/research_learning.html

Affiliation: N. Messikommer, G. Cioffi, M. Gehrig, and D. Scaramuzza are with the Robotics and Perception Group, Dept. of Informatics, University of Zurich, Switzerland
http://rpg.ifi.uzh.ch/
Reinforcement Learning Meets Visual Odometry  (ECCV 2024)Low-Latency Event-Based Object Detection with Spatially-Sparse Linear Attention (ECCV 2026)Deep Drone Acrobatics (RSS 2020 Video Pitch)User-Conditioned Neural Control Policies for Mobile Robotics (ICRA 2023)Voxel Map for Visual SLAM (ICRA20 Video Pitch)Active Autonomous Aerial Exploration for Ground Robot Path PlanningExploration Without Global Consistency Using Local Volume ConsolidationFast Trajectory Optimization for Agile Quadrotor Maneuvers with a Cable-Suspended PayloadEvent-based Shape from Polarization (CVPR 2023)Rapid, Dynamic Obstacle Avoidance with an Event-based CameraMPCC++: Model Predictive Contouring Control for Time-Optimal Flight with Safety Constraints (RSS24)VIMO: Simultaneous Visual Inertial Model-based Odometry and Force Estimation (RSS 2019)
UZH Robotics and Perception Group |

Reinforcement Learning Meets Visual Odometry (ECCV 2024)

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER