Data-driven Feature Tracking for Event Cameras (CVPR 2023, Award Candidate; T-PAMI25) @ailabRPG
Data-driven Feature Tracking for Event Cameras (CVPR 2023, Award Candidate; T-PAMI25)  @ailabRPG
Uploaded April 2023 | Updated November 2026, 1 hour ago
Because of their high temporal resolution, increased resilience to motion blur, and very sparse output, event cameras have been shown to be ideal for low-latency and low-bandwidth feature tracking, even in challenging scenarios. Existing feature tracking methods for event cameras are either handcrafted or derived from first principles but require extensive parameter tuning, are sensitive to noise, and do not generalize to different scenarios due to unmodeled effects. To tackle these deficiencies, we introduce the first data-driven feature tracker for event cameras, which leverages low-latency events to track features detected in a grayscale frame. We achieve robust performance via a novel frame attention module, which shares information across feature tracks. By directly transferring zero-shot from synthetic to real data, our data-driven tracker outperforms existing approaches in relative feature age by up to 120% while also achieving the lowest latency. This performance gap is further increased to 130% by adapting our tracker to real data with a novel self-supervision strategy.

Reference:
Nico Messikommer*, Carter Fang*, Mathias Gehrig, Davide Scaramuzza
Data-driven Feature Tracking for Event Cameras
IEEE Conference of Computer Vision and Pattern Recognition (CVPR), 2023, Vancouver , CA
IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 2025
PDF: https://rpg.ifi.uzh.ch/docs/Arxiv24_Messikommer.pdf
Code: github.com/uzh-rpg/deep_ev_tracker

Our research page on event-based vision: http://rpg.ifi.uzh.ch/research_dvs.html

For event-camera datasets, see here:
1. https://dsec.ifi.uzh.ch/
2. http://rpg.ifi.uzh.ch/davis_data.html
3. github.com/uzh-rpg/event-based_vision_resources#datasets

For an event camera simulator: http://rpg.ifi.uzh.ch/esim

For a survey paper on event cameras, see here:
https://rpg.ifi.uzh.ch/docs/EventVisionSurvey.pdf

Other resources on event cameras (publications, software, drivers, where to buy, etc.):
github.com/uzh-rpg/event-based_vision_resources

Affiliation:
N. Messikommer, C. Fang, M. Gehrig, and D. Scaramuzza are with the Robotics and Perception Group, Dept. of Informatics, University of Zurich, and Dept. of Neuroinformatics, University of Zurich and ETH Zurich, Switzerland http://rpg.ifi.uzh.ch/
Data-driven Feature Tracking for Event Cameras (CVPR 2023, Award Candidate; T-PAMI25)Low Latency Automotive Vision with Event Cameras (Nature, 2024)Autonomous Overtaking in Gran Turismo Sport Using Curriculum Reinforcement Learning (ICRA 2021)High Speed and High Dynamic Range Video with an Event CameraLearning on the Fly: Rapid Policy Adaptation via Differentiable Simulation (RA-L 2026)3D Reconstruction Experiments from a Train using an Event CameraChampion-level Drone Racing using Deep Reinforcement Learning (Nature, 2023)Autonomous Drones - Droni autonomi - SuperQuark RAI1 - Main Italian TV - July 13, 2022Agilicious: Open-Source and Open-Hardware Agile Quadrotor for Vision-Based Flight (Science Robotics)Bridging the Gap between Events and Frames through Unsupervised Domain AdaptationMulti-Bracket High Dynamic Range Imaging with Event Cameras (CVPR 2022)Event-based Asynchronous Sparse Convolutional Networks
UZH Robotics and Perception Group |

Data-driven Feature Tracking for Event Cameras (CVPR 2023, Award Candidate; T-PAMI'25)

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER