Bridging the Gap between Events and Frames through Unsupervised Domain Adaptation @ailabRPG
Bridging the Gap between Events and Frames through Unsupervised Domain Adaptation  @ailabRPG
Uploaded September 2021 | Updated November 2026, 1 hour ago
Event cameras are novel sensors with outstanding properties such as high temporal resolution and high dynamic range. Despite these characteristics, event-based vision has been held back by the shortage of labeled datasets due to the novelty of event cameras. To overcome this drawback, we propose a task transfer method that allows models to be trained directly with labeled images and unlabeled event data. Compared to previous approaches, (i) our method transfers from single images to events instead of high frame rate videos, and (ii) does not rely on paired sensor data. To achieve this, we leverage the generative event model to split event features into content and motion features. This feature split enables to efficiently match the latent space for events and images, which is crucial for a successful task transfer. Thus, our approach unlocks the vast amount of existing image datasets for the training of event-based neural networks. Our task transfer method consistently outperforms methods applicable in the Unsupervised Domain Adaptation setting for object detection by 0.26 mAP (increase by 93%) and classification by 2.7% accuracy.

Reference: Nico Messikommer, Daniel Gehrig, Mathias Gehrig, and Davide Scaramuzza.
PDF: https://rpg.ifi.uzh.ch/docs/RAL22_Messikommer.pdf
Code: github.com/uzh-rpg/rpg_ev-transfer
Robotics and Automation Letters (RAL), 2022

For more info on our research on event cameras: http://rpg.ifi.uzh.ch/research_dvs.html
For more info on our research on machine learning: http://rpg.ifi.uzh.ch/research_learning.html
For event-camera datasets, see here: http://rpg.ifi.uzh.ch/davis_data.html and here: github.com/uzh-rpg/event-based_vision_resources#datasets
For an event camera simulator: http://rpg.ifi.uzh.ch/esim/index.html
For a survey paper on event cameras, see here: http://rpg.ifi.uzh.ch/docs/EventVisionSurvey.pdf
Other resources on event cameras (publications, software, drivers, where to buy, etc.): github.com/uzh-rpg/event-based_vision_resources

Affiliations: The authors are with the Robotics and Perception Group, Dep. of Informatics, University of Zurich, and Dep. of Neuroinformatics, University of Zurich and ETH Zurich, Switzerland http://rpg.ifi.uzh.ch/
Bridging the Gap between Events and Frames through Unsupervised Domain AdaptationMulti-Bracket High Dynamic Range Imaging with Event Cameras (CVPR 2022)Event-based Asynchronous Sparse Convolutional NetworksDrift-free Visual SLAM using Digital Twins (RAL 2025)E-RAFT: Dense Optical Flow from Event Cameras (3DV 2021)Motion-aware Event Suppression for Event Cameras (RSS 2026)Tightly-coupled Fusion of Global Positional Measurements in Optimization-based VIO (IROS 2020)Ultimate SLAM? Combining Events, Images, and IMU for Visual SLAM in HDR and High-Speed ScenariosData-Driven Methods for Event Cameras (Ph.D. defense of Mathias Gehrig)The Foldable Drone: A Morphing Quadrotor that can Squeeze and FlyHuman-Piloted Drone Racing: Visual Processing and Control (ICRA 2021 Video Pitch)Student-Informed Teacher Training (ICLR 2025)
UZH Robotics and Perception Group |

Bridging the Gap between Events and Frames through Unsupervised Domain Adaptation

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER