Deep Drone Racing: From Simulation to Reality with Domain Randomization @ailabRPG
Deep Drone Racing: From Simulation to Reality with Domain Randomization  @ailabRPG
Uploaded May 2019 | Updated November 2026, 40 minutes ago
Dynamically changing environments, unreliable state estimation, and operation under severe resource constraints are fundamental challenges for robotics, which still limit the deployment of small autonomous drones. We address these challenges in the context of autonomous, vision-based drone racing in dynamic environments. A racing drone must traverse a track with possibly moving gates at high speed. We enable this functionality by combining the performance of a state-of-the-art path-planning and control system with the perceptual awareness of a convolutional neural network (CNN). The CNN directly maps raw images to a desired waypoint and speed. Given the CNN output, the planner generates a short minimum-jerk trajectory segment that is tracked by a model-based controller to actuate the drone towards the waypoint. The resulting modular system has several desirable features: (i) it can run fully on-board, (ii) it does not require globally consistent state estimation, and (iii) it is both platform and domain independent. We extensively test the precision and robustness of our system, both in simulation and on a physical platform. In both domains, our method significantly outperforms the prior state of the art. In order to understand the limits of our approach, we additionally compare against professional human drone pilots with different skill levels.

Reference:
A. Loquercio, E. Kaufmann, R. Ranftl, A. Dosovitskiy, V. Koltun, D. Scaramuzza
Deep Drone Racing: From Simulation to Reality with Domain Randomization
IEEE Transactions on Robotics, 2019
PDF: http://rpg.ifi.uzh.ch/docs/TRO19_Loquercio.pdf
Code: http://rpg.ifi.uzh.ch/research_drone_racing.html

Our research page on drone racing:
http://rpg.ifi.uzh.ch/research_drone_racing.html

Our research page on deep learning:
http://rpg.ifi.uzh.ch/research_learning.html

Our research page on autonomous, vision-controlled drones:
http://rpg.ifi.uzh.ch/research_mav.html
http://rpg.ifi.uzh.ch/aggressive_flight.html

Affiliations:
A. Loquercio, E. Kaufmann, and D. Scaramuzza are with the Robotics and Perception Group, Dep. of Informatics, University of Zurich, and Dep. of Neuroinformatics, University of Zurich and ETH Zurich, Switzerland
http://rpg.ifi.uzh.ch/
R. Ranftl, A. Dosovitskiy and V. Koltun are with Intel Labs
vladlen.info
Deep Drone Racing: From Simulation to Reality with Domain RandomizationLearning Minimum-Time Flight in Cluttered Environments (RAL 2022)Video to Events: Recycling Video Datasets for Event Cameras (CVPR20 1-min Video Pitch)HDVIO2.0: Wind and Disturbance Estimation with Hybrid Dynamics VIO (T-RO 2025)Learning Quadruped Locomotion Using Differentiable Simulation (CORL 2024, Oral)Recurrent Vision Transformers for Object Detection with Event Cameras (CVPR 2023)Swiss Drone Days - Zurich, June 11-12, 2022 - Teaser #1Approximate Imitation Learning for Event-based Flight in Cluttered Environments (CoRL 2026)A Benchmark Comparison of Monocular Visual-Inertial Odometry Algorithms for Flying RobotsESIM: an Open Event Camera SimulatorLearning Agile Quadrotor Flight in the Real World (RSS 2026)Learning Visuomotor Flight Through Dynamic Narrow Gap via Differentiable Simulation (CoRL 2026)
UZH Robotics and Perception Group |

Deep Drone Racing: From Simulation to Reality with Domain Randomization

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER