Human-Piloted Drone Racing: Visual Processing and Control (ICRA 2021 Video Pitch) @ailabRPG
Human-Piloted Drone Racing: Visual Processing and Control (ICRA 2021 Video Pitch)  @ailabRPG
Uploaded June 2021 | Updated November 2026, 1 hour ago
Humans race drones faster than algorithms, despite being limited to a fixed camera angle, body rate control, and response latencies in the order of hundreds of milliseconds. A better understanding of the ability of human pilots of selecting appropriate motor commands from highly dynamic visual information may provide key insights for solving current challenges in vision-based autonomous navigation. This work investigates the relationship between human eye movements, control behavior, and flight performance in a drone racing task. We collected a multimodal dataset from 21 experienced drone pilots using a highly realistic drone racing simulator, also used to recruit professional pilots. Our results show task-specific improvements in drone racing performance over time. In particular, we found that eye gaze tracks future waypoints (i.e., gates), with first fixations occurring on average 1.5 seconds and 16 meters before reaching the gate. Moreover, human pilots consistently looked at the inside of the future flight path for lateral (i.e., left and right turns) and vertical maneuvers (i.e., ascending and descending). Finally, we found a strong correlation between pilots’ eye movements and the commanded direction of quadrotor flight, with an average visual-motor response latency of 220 ms. These results highlight the importance of coordinated eye movements in human-piloted drone racing. We make our dataset publicly available.


Reference:
C. Pfeiffer, D. Scaramuzza
Human-Piloted Drone Racing: Visual Processing and Control
IEEE Robotics and Automation Letters (RA-L), 2021
PDF: http://rpg.ifi.uzh.ch/docs/RAL21_Pfeiffer.pdf

Slides:
http://rpg.ifi.uzh.ch/docs/ICRA21_Pfeiffer.pptx

Open-Source Dataset:
osf.io/gvdse


For more information about our research, visit these pages:
1. Publications: http://rpg.ifi.uzh.ch/publications.html
2. Drone Racing: http://rpg.ifi.uzh.ch/research_drone_racing.html
3. Vision-Based Navigation: http://rpg.ifi.uzh.ch/research_mav.html

Affiliations:
C. Pfeiffer and D. Scaramuzza are with the Robotics and Perception Group, Dep. of Informatics, University of Zurich, and Dep. of Neuroinformatics, University of Zurich and ETH Zurich, Switzerland.
http://rpg.ifi.uzh.ch/
Human-Piloted Drone Racing: Visual Processing and Control (ICRA 2021 Video Pitch)Student-Informed Teacher Training (ICLR 2025)Swiss Drone Days - Zurich, June 11-12, 2022 - Teaser #2Structure-Invariant Range-Visual-Inertial Odometry for the Mars Science Helicopter (with NASA-JPL)AutoTune: Controller Tuning for High-Speed FlightLearning High-Speed Flight in the Wild (Science Robotics, 2021)Flightmare: A Flexible Quadrotor SimulatorModel Predictive Contouring Control for Time-Optimal Quadrotor Flight (TRO 2022)Actor-Critic Model Predictive Control (ICRA 2024)RPG TV Documentary: Drones for Search and Rescue - 3Sat Nano - 20.03.2020Deep Visual Odometry with Events and Frames (IROS 2024)Efficient, Data-Driven Perception with Event Cameras (Ph.D. Defense of Daniel Gehrig)
UZH Robotics and Perception Group |

Human-Piloted Drone Racing: Visual Processing and Control (ICRA 2021 Video Pitch)

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER