Uploaded June 2026 | Updated November 2026, 1 hour ago
Recent advances in learning-based control have enabled quadrotors to perform agile maneuvers that were previously difficult to achieve with conventional model-based pipelines alone. However, transferring policies trained in simulation to the real world remains challenging due to perception noise, dynamic mismatches, latency, and the limited availability of real-world flight data. This tutorial provides an overview of learning agile vision-based quadrotor flight, covering the full pipeline from simulation-based training to real-world adaptation. We discuss how to combine visual perception, state estimation, differentiable simulation, reinforcement learning, and model-predictive control to enable robust, agile flight using onboard sensing. Particular emphasis is placed on closing the sim-to-real gap through domain randomization, residual learning, system identification, perception-aware training, and real-world fine-tuning. The tutorial draws on recent results in autonomous drone racing, visual navigation, and event-camera-based flight, highlighting both the opportunities and open challenges in deploying learned agile flight systems outside the laboratory.
All the source code in this tutorial can be found here:
https://rpg.ifi.uzh.ch/software_datasets.html and github.com/uzh-rpg
Research page: https://rpg.ifi.uzh.ch/aggressive_flight.html
Speakers: Davide Scaramuzza, Rudolf Reiter, Ismail Geles
00:00:00 Introduction
00:02:00 Fly better than human pilots?
00:05:46 From traditional control to Machine Learning
00:08:45 The AI Drone - SWIFT
00:12:24 Autonomous Drone Racing
00:23:11 Takeaways
00:27:29 Simulation
00:31:10 Improve VIO through RL
00:36:24 Introduction of MPC vs. RL
00:39:18 Combine MPC and RL
00:45:11 Actor-Critic MPC
00:52:56 Learning Agile Flight in Practice
00:59:05 Sim-to-Real Gap
01:04:01 Vision-based Flight
01:12:03 Differentiable Simulation
01:14:28 Multi-Agent Racing
Affiliation:
Ismail Geles, Rudolf Reiter, and Davide Scaramuzza are with the Robotics and Perception Group, University of Zurich, Switzerland (https://rpg.ifi.uzh.ch)
Recent advances in learning-based control have enabled quadrotors to perform agile maneuvers that were previously difficult to achieve with conventional model-based pipelines alone. However, transferring policies trained in simulation to the real world remains challenging due to perception noise, dynamic mismatches, latency, and the limited availability of real-world flight data. This tutorial provides an overview of learning agile vision-based quadrotor flight, covering the full pipeline from simulation-based training to real-world adaptation. We discuss how to combine visual perception, state estimation, differentiable simulation, reinforcement learning, and model-predictive control to enable robust, agile flight using onboard sensing. Particular emphasis is placed on closing the sim-to-real gap through domain randomization, residual learning, system identification, perception-aware training, and real-world fine-tuning. The tutorial draws on recent results in autonomous drone racing, visual navigation, and event-camera-based flight, highlighting both the opportunities and open challenges in deploying learned agile flight systems outside the laboratory.
All the source code in this tutorial can be found here:
https://rpg.ifi.uzh.ch/software_datasets.html and github.com/uzh-rpg
Research page: https://rpg.ifi.uzh.ch/aggressive_flight.html
Speakers: Davide Scaramuzza, Rudolf Reiter, Ismail Geles
00:00:00 Introduction
00:02:00 Fly better than human pilots?
00:05:46 From traditional control to Machine Learning
00:08:45 The AI Drone - SWIFT
00:12:24 Autonomous Drone Racing
00:23:11 Takeaways
00:27:29 Simulation
00:31:10 Improve VIO through RL
00:36:24 Introduction of MPC vs. RL
00:39:18 Combine MPC and RL
00:45:11 Actor-Critic MPC
00:52:56 Learning Agile Flight in Practice
00:59:05 Sim-to-Real Gap
01:04:01 Vision-based Flight
01:12:03 Differentiable Simulation
01:14:28 Multi-Agent Racing
Affiliation:
Ismail Geles, Rudolf Reiter, and Davide Scaramuzza are with the Robotics and Perception Group, University of Zurich, Switzerland (https://rpg.ifi.uzh.ch)










