Monocular Event-Based Vision for Obstacle Avoidance with a Quadrotor @KumarLabPenn
Monocular Event-Based Vision for Obstacle Avoidance with a Quadrotor  @KumarLabPenn
Uploaded November 2024 | Updated November 2026, 5 hours ago
We present the first static-obstacle avoidance method for quadrotors using just an onboard, monocular event camera. Quadrotors are capable of fast and agile flight in cluttered environments when piloted manually, but vision-based autonomous flight in unknown environments is difficult in part due to the sensor limitations of traditional onboard cameras. Event cameras, however, promise nearly zero motion blur and high dynamic range, but produce a large volume of events under significant ego-motion and further lack a continuous-time sensor model in simulation, making direct sim-to-real transfer not possible.
By leveraging depth prediction as a pretext task in our learning framework, we can pre-train a reactive obstacle avoidance events-to-control policy with approximated, simulated events and then fine-tune the perception component with limited events-and-depth real-world data to achieve obstacle avoidance in indoor and outdoor settings.
We demonstrate this across two quadrotor-event camera platforms in multiple settings and find, contrary to traditional vision-based works, that low speeds (1m/s) make the task harder and more prone to collisions, while high speeds (5m/s) result in better event-based depth estimation and avoidance. We also find that success rates in outdoor scenes can be significantly higher than in certain indoor scenes.
Project page: anishbhattacharya.com/research/evfly
In collaboration between GRASP, Penn and Robotics and Perception Group, UZH.
Reference: Bhattacharya, Cannici, Rao, Tao, Kumar, Matni, Scaramuzza (2024). Monocular Event-Based Vision for Obstacle Avoidance with a Quadrotor. Conference on Robot Learning (CoRL) 2024.
Monocular Event-Based Vision for Obstacle Avoidance with a QuadrotorDecentralization of Multiagent Policies by Learning What to CommunicateThe Open Vision Computer : An Integrated Sensing and Compute System for Mobile RobotsVision-based Perimeter Defense via Multiview Pose EstimationCoordinate-Free Dynamics and Differential Flatness of a Class of 6DOF Aerial ManipulatorsCPS: Frontier: Collaborative Research: bioCPS for Engineering Living CellsLearning Local Urban Wind Flow Fields From Range SensinghenrikSemi-Dense Visual-Inertial Odometry and Mapping for Quadrotors with SWAP ConstraintsDORA: Divergence-Oriented Data-Relay Algorithm for Partially Connected Robot TeamsAir-Ground Collaboration with SPOMP: Semantic Panoramic Online Mapping and PlanningMultiple Scarab Ground Robots Manipulating Objects
Vijay Kumar |

Monocular Event-Based Vision for Obstacle Avoidance with a Quadrotor

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER