Uploaded January 2026 | Updated November 2026, 1 hour ago
Learning control policies in simulation enables rapid, safe, and cost-effective development of advanced robotic capabilities. However, transferring these policies to the real world remains difficult due to the sim-to-real gap, where unmodeled dynamics and environmental disturbances can degrade policy performance. Existing approaches, such as domain randomization and Real2Sim2Real pipelines, can improve policy robustness, but either struggle under out-of-distribution conditions or require costly offline retraining. In this work, we approach these problems from a different perspective. Instead of relying on diverse training conditions before deployment, we focus on rapidly adapting the learned policy in the real world in an online fashion. To achieve this, we propose a novel online adaptive learning framework that unifies residual dynamics learning with real-time policy adaptation inside a differentiable simulation. Starting from a simple dynamics model, our framework refines the model continuously with real-world data to capture unmodeled effects and disturbances such as payload changes and wind. The refined dynamics model is embedded in a differentiable simulation framework, enabling gradient backpropagation through the dynamics and thus rapid, sample-efficient policy updates beyond the reach of classical RL methods like PPO. All components of our system are designed for rapid adaptation, enabling the policy to adjust to unseen disturbances within 5 seconds of training. We validate the approach on agile quadrotor control under various disturbances in both simulation and the real world. Our framework reduces hovering error by up to 81% compared to L1-MPC and 55% compared to DATT, while also demonstrating robustness in vision-based control without explicit state estimation.
Reference:
J. Pan*, J. Xing*, R. Reiter, Y. Zhai, E. Aljalbout, and D. Scaramuzza,
"Learning on the Fly: Rapid Policy Adaptation via Differentiable Simulation."
IEEE Robotics and Automation Letters RA-L
PDF: https://rpg.ifi.uzh.ch/docs/RAL26_Pan.pdf
Project page: https://rpg.ifi.uzh.ch/lotf/
Code: github.com/uzh-rpg/learning_on_the_fly
More info on our research in Drone Racing:
https://rpg.ifi.uzh.ch/research_drone_racing.html
More info on our research in Agile Drone Flight:
https://rpg.ifi.uzh.ch/aggressive_flight.html
More info on our research on Machine Learning:
https://rpg.ifi.uzh.ch/research_learning.html
Affiliations:
J. Pan, J. Xing, R. Reiter, Y. Zhai, E. Aljalbout, and D. Scaramuzza are with the Robotics and Perception Group, Dep. of Informatics, University of Zurich, https://rpg.ifi.uzh.ch/
Learning control policies in simulation enables rapid, safe, and cost-effective development of advanced robotic capabilities. However, transferring these policies to the real world remains difficult due to the sim-to-real gap, where unmodeled dynamics and environmental disturbances can degrade policy performance. Existing approaches, such as domain randomization and Real2Sim2Real pipelines, can improve policy robustness, but either struggle under out-of-distribution conditions or require costly offline retraining. In this work, we approach these problems from a different perspective. Instead of relying on diverse training conditions before deployment, we focus on rapidly adapting the learned policy in the real world in an online fashion. To achieve this, we propose a novel online adaptive learning framework that unifies residual dynamics learning with real-time policy adaptation inside a differentiable simulation. Starting from a simple dynamics model, our framework refines the model continuously with real-world data to capture unmodeled effects and disturbances such as payload changes and wind. The refined dynamics model is embedded in a differentiable simulation framework, enabling gradient backpropagation through the dynamics and thus rapid, sample-efficient policy updates beyond the reach of classical RL methods like PPO. All components of our system are designed for rapid adaptation, enabling the policy to adjust to unseen disturbances within 5 seconds of training. We validate the approach on agile quadrotor control under various disturbances in both simulation and the real world. Our framework reduces hovering error by up to 81% compared to L1-MPC and 55% compared to DATT, while also demonstrating robustness in vision-based control without explicit state estimation.
Reference:
J. Pan*, J. Xing*, R. Reiter, Y. Zhai, E. Aljalbout, and D. Scaramuzza,
"Learning on the Fly: Rapid Policy Adaptation via Differentiable Simulation."
IEEE Robotics and Automation Letters RA-L
PDF: https://rpg.ifi.uzh.ch/docs/RAL26_Pan.pdf
Project page: https://rpg.ifi.uzh.ch/lotf/
Code: github.com/uzh-rpg/learning_on_the_fly
More info on our research in Drone Racing:
https://rpg.ifi.uzh.ch/research_drone_racing.html
More info on our research in Agile Drone Flight:
https://rpg.ifi.uzh.ch/aggressive_flight.html
More info on our research on Machine Learning:
https://rpg.ifi.uzh.ch/research_learning.html
Affiliations:
J. Pan, J. Xing, R. Reiter, Y. Zhai, E. Aljalbout, and D. Scaramuzza are with the Robotics and Perception Group, Dep. of Informatics, University of Zurich, https://rpg.ifi.uzh.ch/



![Agilicious: Open-Source and Open-Hardware Agile Quadrotor for Vision-Based Flight (Science Robotics)
Agilicious is a co-designed hardware and software framework tailored to autonomous, agile quadrotor flight, which has been developed and used since 2016 at the Robotics and Perception Group of the University of Zurich. Agilicious is described in our Science Robotics 2022 paper, linked below. It is completely open-source and open hardware and supports both model-based and neural-network-based controllers! Also, it provides high thrust-to-weight and torque-to-inertia ratios for agility, onboard vision sensors, GPU-accelerated compute hardware for real-time perception and neural-network inference, a real-time flight controller, and a versatile software stack. In contrast to existing frameworks, Agilicious offers a unique combination of flexible software and high-performance hardware. Agilicious has been used in over 30 scientific papers at our lab, including trajectory tracking for drone racing scenarios at up to 5g and 70km/h [SciRob21_Foehn], vision-based acrobatic flight [RSS20_Kaufmann), obstacle avoidance in both structured and unstructured environments using solely onboard perception [SciRob21_Loquercio], and hardware-in-the-loop simulation in virtual-reality environments! Thanks to its versatility, we believe that Agilicious supports the next generation of scientific and industrial quadrotor research. Agilicious allows you to seamlessly develop, test, reproduce, and benchmark control algorithms such as Non-linear Model Predictive Control (MPC), differential-flatness-based control (DFBC), and INDI (Incremental Non-linear Dynamic Inversion) [TRO 2022_Sun]. Agilicious also includes our state-of-the-art, high-fidelity simulator, which uses Blade Element Momentum (BEM) theory for the propeller model and provides accurate estimates of the quadcopter’s aerodynamic characteristics across the flight envelope [RSS 2021_Bauersfeld].
Agilicious webpage: https://agilicious.dev/
References:
Agilicious main reference (all open-access PDFs and videos can be found on the Agilicious page):
Foehn, Kaufmann, Romero, Penicka, Sun, Bauersfeld, Laengle, Cioffi, Song, Loquercio, Scaramuzza, “Agilicious: Open-Source and Open-Hardware Agile Quadrotor for Vision-Based Flight”, AAAS Science Robotics, 2022.
PDF: https://rpg.ifi.uzh.ch/docs/ScienceRobotics22_Foehn.pdf
Agilicious implements the algorithms described in the following papers:
[SciRob21_Foehn] Philipp Foehn, Angel Romero, Davide Scaramuzza, “Time-Optimal Planning for Quadrotor Waypoint Flight”, Science Robotics, 2021,
PDF: https://rpg.ifi.uzh.ch/docs/ScienceRobotics21_Foehn.pdf
[RSS20_Kaufmann] Elia Kaufmann, Antonio Loquercio, Rene Ranftl, Mark Müller, Vladlen Koltun, Davide Scaramuzza, “Deep Drone Acrobatics”, RSS: Robotics, Science, and Systems, 2020,
PDF: https://rpg.ifi.uzh.ch/docs/RSS20_Kaufmann.pdf
[SciRob21_Loquercio] Antonio Loquercio, Elia Kaufmann, Rene Ranftl, Mark Müller, Vladlen Koltun, Davide Scaramuzza, “Learning High-Speed Flight in the Wild”, Science Robotics, 2021,
PDF: https://rpg.ifi.uzh.ch/docs/Loquercio21_Science.pdf
[TRO 2022_Sun] Sihao Sun, Angel Romero, Philipp Foehn, Elia Kaufmann, Davide Scaramuzza, “A Comparative Study of Nonlinear MPC and Differential-Flatness-based Control for Quadrotor Agile Flight”, IEEE Transactions on Robotics, 2022,
PDF: https://rpg.ifi.uzh.ch/docs/Arxiv21_MPC_Sun.pdf
[RSS 2021_Bauersfeld] Leonard Bauersfeld, Elia Kaufmann, Philipp Foehn, Sihao Sun, Davide Scaramuzza, “NeuroBEM: Hybrid Aerodynamic Quadrotor Model”, RSS: Robotics, Science, and Systems, 2021,
PDF: https://rpg.ifi.uzh.ch/docs/RSS21_Bauersfeld.pdf
Matthias Faessler, Antonio Franchi, Davide Scaramuzza, “Differential Flatness of Quadrotor Dynamics Subject to Rotor Drag for Accurate Tracking of High-Speed Trajectories”, IEEE Robotics and Automation Letters, 2018.
PDF: https://rpg.ifi.uzh.ch/docs/RAL18_Faessler.pdf
Our research in Agile Drone Flight:
http://rpg.ifi.uzh.ch/aggressive_flight.html
Affiliations:
P. Foehn, E. Kaufmann, A. Romero, R. Penicka, S. Sun, L. Bauersfeld, T. Laengle, Y. Song, A. Loquercio, D. Scaramuzza are with the Robotics and Perception Group, University of Zurich, Switzerland
http://rpg.ifi.uzh.ch/
Music Credits: scottholmesmusic.com under Free Creative Commons License Agilicious: Open-Source and Open-Hardware Agile Quadrotor for Vision-Based Flight (Science Robotics)](https://i.ytimg.com/vi/fNYxPLyJ5YY/mqdefault.jpg)






