Time Lens++: Event-based Frame Interpolation with Parametric Flow and Multi-scale Fusion (CVPR 2022) @ailabRPG
Time Lens++: Event-based Frame Interpolation with Parametric Flow and Multi-scale Fusion (CVPR 2022)  @ailabRPG
Uploaded June 2022 | Updated November 2026, 1 hour ago
Recently, video frame interpolation using a combination of frame- and event-based cameras has surpassed traditional image-based methods both in terms of performance and memory efficiency. However, current methods still suffer from (i) brittle image-level fusion of complementary interpolation results, that fails in the presence of artifacts in the fused image, (ii) potentially temporally inconsistent and inefficient motion estimation procedures, that run for every inserted frame and (iii) low contrast regions that do not trigger events, and thus cause events-only motion estimation to generate artifacts. Moreover, previous methods were only tested on datasets consisting of planar and far-away scenes, which do not capture the full complexity of the real world. In this work, we address the above problems by introducing multi-scale feature-level fusion and computing one-shot non-linear inter-frame motion—which can be efficiently sampled for image warping—from events and images. We also collect the first large-scale events and frames dataset consisting of more than 100 challenging scenes with depth variations, captured with a new experimental setup based on a beamsplitter. We show that our method improves the reconstruction quality by up to 0.2 dB in terms of PSNR and up to 15% in LPIPS score.

Reference:
Stepan Tulyakov, Alfredo Bochicchio, Daniel Gehrig, Stamatios Georgoulis, Yuanyou Li, Davide Scaramuzza
Time Lens++: Event-based Frame Interpolation with Parametric Non-linear Flow and Multi-scale Fusion
IEEE Conference of Computer Vision and Pattern Recognition (CVPR), 2022, New Orleans, USA
PDF: http://rpg.ifi.uzh.ch/docs/CVPR22_Tulyakov.pdf
Code and datasets: https://rpg.ifi.uzh.ch/TimeLens.html

Our research page on event-based vision: http://rpg.ifi.uzh.ch/research_dvs.html

For event-camera datasets, see here: http://rpg.ifi.uzh.ch/davis_data.html
and here: github.com/uzh-rpg/event-based_vision_resources#datasets

For an event camera simulator: http://rpg.ifi.uzh.ch/esim

For a survey paper on event cameras, see here:
https://rpg.ifi.uzh.ch/docs/EventVisionSurvey.pdf

Other resources on event cameras (publications, software, drivers, where to buy, etc.):
github.com/uzh-rpg/event-based_vision_resources

Affiliation:
D. Gehrig, and D. Scaramuzza are with the Robotics and Perception Group, Dept. of Informatics, University of Zurich, and Dept. of Neuroinformatics, University of Zurich and ETH Zurich, Switzerland http://rpg.ifi.uzh.ch/ and S. Tulyakov, A. Bochicchio, S. Georgoulis and Y. Li are with Huawei Zurich Research Center.
Time Lens++: Event-based Frame Interpolation with Parametric Flow and Multi-scale Fusion (CVPR 2022)Event-Based Angular Velocity Regression with Spiking Networks (ICRA20 Video Pitch)5th Year Anniversary of the Robotics and Perception GroupE-RAFT: Dense Optical Flow from Event Cameras (3DV 2021)TimeLens: Event-based Video Frame Interpolation (CVPR 2021)Event-Aided Sharp Radiance Field Reconstruction for Fast-Flying Drones (T-RO 2026)Data-driven Feature Tracking for Event Cameras (CVPR 2023, Award Candidate; T-PAMI25)Low Latency Automotive Vision with Event Cameras (Nature, 2024)Autonomous Overtaking in Gran Turismo Sport Using Curriculum Reinforcement Learning (ICRA 2021)High Speed and High Dynamic Range Video with an Event CameraLearning on the Fly: Rapid Policy Adaptation via Differentiable Simulation (RA-L 2026)3D Reconstruction Experiments from a Train using an Event Camera
UZH Robotics and Perception Group |

Time Lens++: Event-based Frame Interpolation with Parametric Flow and Multi-scale Fusion (CVPR 2022)

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER