Low Latency Automotive Vision with Event Cameras (Nature, 2024) @ailabRPG
Low Latency Automotive Vision with Event Cameras (Nature, 2024)  @ailabRPG
Uploaded May 2024 | Updated November 2026, 1 hour ago
The computer vision algorithms used in today’s advanced driver assistance systems rely on image-based RGB cameras, leading to a critical bandwidth-latency trade-off for delivering safe driving experiences. To address this, event cameras have emerged as alternative vision sensors. Event cameras measure changes in intensity asynchronously, offering high temporal resolution and sparsity, drastically reducing bandwidth and latency requirements. Despite these advantages, event camera-based algorithms are either highly efficient but lag behind image-based ones in terms of accuracy or sacrifice the sparsity and efficiency of events to achieve comparable results. To overcome this, we propose a novel hybrid event- and frame-based object detector that preserves the advantages of each modality and thus does not suffer from this tradeoff. Our method exploits the high temporal resolution and sparsity of events and the rich but low temporal resolution information in standard images to generate efficient, high-rate object detections, reducing perceptual and computational latency. We show that the use of a 20 Hz RGB camera plus an event camera can achieve the same latency as a 5,000 Hz camera with the bandwidth of a 45 Hz camera without compromising accuracy. Our approach paves the way for efficient and robust perception in edge-case scenarios by uncovering the potential of event cameras.

Daniel Gehrig, Davide Scaramuzza
Low Latency Automotive Vision with Event Cameras.
Nature, May 29th, 2024
PDF Open Access: nature.com/articles/s41586-024-07409-w
Code: github.com/uzh-rpg/dagr
Dataset: https://dsec.ifi.uzh.ch/dsec-detection/
Dataset Helper Tools: github.com/uzh-rpg/dsec-det

For an event camera simulator: http://rpg.ifi.uzh.ch/esim

For a survey paper on event cameras, see here: https://rpg.ifi.uzh.ch/docs/EventVisionSurvey.pdf

Other resources on event cameras (publications, software, drivers, where to buy, etc.): github.com/uzh-rpg/event-based_vision_resources

Affiliation:
D. Gehrig and D. Scaramuzza are with the Robotics and Perception Group, University of Zurich, Switzerland http://rpg.ifi.uzh.ch/
Low Latency Automotive Vision with Event Cameras (Nature, 2024)Autonomous Overtaking in Gran Turismo Sport Using Curriculum Reinforcement Learning (ICRA 2021)High Speed and High Dynamic Range Video with an Event CameraLearning on the Fly: Rapid Policy Adaptation via Differentiable Simulation (RA-L 2026)3D Reconstruction Experiments from a Train using an Event CameraChampion-level Drone Racing using Deep Reinforcement Learning (Nature, 2023)Autonomous Drones - Droni autonomi - SuperQuark RAI1 - Main Italian TV - July 13, 2022Agilicious: Open-Source and Open-Hardware Agile Quadrotor for Vision-Based Flight (Science Robotics)Bridging the Gap between Events and Frames through Unsupervised Domain AdaptationMulti-Bracket High Dynamic Range Imaging with Event Cameras (CVPR 2022)Event-based Asynchronous Sparse Convolutional NetworksDrift-free Visual SLAM using Digital Twins (RAL 2025)
UZH Robotics and Perception Group |

Low Latency Automotive Vision with Event Cameras (Nature, 2024)

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER