Dream to Fly: Model-Based Reinforcement Learning for Vision-Based Drone Flight (ICRA 2026) @ailabRPG
Dream to Fly: Model-Based Reinforcement Learning for Vision-Based Drone Flight (ICRA 2026)  @ailabRPG
Uploaded April 2026 | Updated November 2026, 17 hours ago
Autonomous drone racing has risen as a challenging robotic benchmark for testing the limits of learning, perception, planning, and control. Expert human pilots are able to agilely fly a drone through a race track by mapping the real-time feed from a single onboard camera directly to control commands. Recent works in autonomous drone racing attempting direct pixel-to-commands control policies (without explicit state estimation) have relied on either intermediate representations that simplify the observation space or performed extensive bootstrapping using Imitation Learning (IL). This work introduces an approach that learns policies from scratch, allowing a quadrotor to autonomously navigate a race track by directly mapping raw onboard camera pixels to control commands, just as human pilots do. By leveraging model-based reinforcement learning (RL) - specifically DreamerV3 - we train visuomotor policies capable of agile flight through a race track using only raw pixel observations. While model-free RL methods such as PPO struggle to learn under these conditions, DreamerV3 efficiently acquires complex visuomotor behaviors. Moreover, because our policies learn directly from pixel inputs, the perception-aware reward term employed in previous RL approaches to guide the training process is no longer needed. Our experiments demonstrate in both simulation and real-world flight how the proposed approach can be deployed on agile quadrotors. This approach advances the frontier of vision-based autonomous flight and shows that model-based RL is a promising direction for real-world robotics.

Reference:
A. Romero, A. Shenai, I. Geles, E. Aljalbout, D. Scaramuzza,
"Dream to Fly: Model-Based Reinforcement Learning for Vision-Based Drone Flight",
IEEE International Conference on Robotics and Automation, 2026
PDF: https://rpg.ifi.uzh.ch/docs/ICRA26_Romero.pdf

For more info about our research on:
Agile Drone Flight: https://rpg.ifi.uzh.ch/aggressive_flight.html
Drone Racing: https://rpg.ifi.uzh.ch/research_drone_racing.html
Machine Learning: https://rpg.ifi.uzh.ch/research_learning.html

Affiliation:
Angel Romero, Ashwin Shenai, Ismail Geles, Elie Aljalbout, and Davide Scaramuzza are with the Robotics and Perception Group, University of Zurich, Switzerland: https://rpg.ifi.uzh.ch/
Dream to Fly: Model-Based Reinforcement Learning for Vision-Based Drone Flight (ICRA 2026)Feature Detection and Tracking with the Dynamic and Active pixel Vision Sensor DAVISUZH Deep Learning Symposium: Learning Agile FilghtAEGNN: Asynchronous Event-based Graph Neural Networks (CVPR 2022)Dronet: Learning to Fly by DrivingRedesigning SLAM for Arbitrary Multi-Camera SystemsReinforcement Learning Meets Visual Odometry  (ECCV 2024)Low-Latency Event-Based Object Detection with Spatially-Sparse Linear Attention (ECCV 2026)Deep Drone Acrobatics (RSS 2020 Video Pitch)User-Conditioned Neural Control Policies for Mobile Robotics (ICRA 2023)Voxel Map for Visual SLAM (ICRA20 Video Pitch)Active Autonomous Aerial Exploration for Ground Robot Path Planning
UZH Robotics and Perception Group |

Dream to Fly: Model-Based Reinforcement Learning for Vision-Based Drone Flight (ICRA 2026)

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER