Actor-Critic Model Predictive Control (ICRA 2024) @ailabRPG
Actor-Critic Model Predictive Control (ICRA 2024)  @ailabRPG
Uploaded April 2024 | Updated November 2026, 1 hour ago
An open research question in robotics is how to combine the benefits of model-free reinforcement learning (RL) - known for its strong task performance and flexibility in optimizing general reward formulations - with the robustness and online replanning capabilities of model predictive control (MPC). This paper provides an answer by introducing a new framework called Actor-Critic Model Predictive Control. The key idea is to embed a differentiable MPC within an actor-critic RL framework. The proposed approach leverages the short-term predictive optimization capabilities of MPC with the exploratory and end-to-end training properties of RL. The resulting policy effectively manages both short-term decisions through the MPC-based actor and long-term prediction via the critic network, unifying the benefits of both model-based control and end-to-end learning. We validate our method in both simulation and the real world with a quadcopter platform across various high-level tasks. We show that the proposed architecture can achieve real-time control performance, learn complex behaviors via trial and error, and retain the predictive properties of the MPC to better handle out of distribution behaviour.

Reference:
A. Romero, Y. Song, D. Scaramuzza,
"Actor-Critic Model Predictive Control",
IEEE International Conference on Robotics and Automation, 2024
PDF: https://rpg.ifi.uzh.ch/docs/ICRA24_Romero.pdf

For more info about our research on:
Agile Drone Flight: http://rpg.ifi.uzh.ch/aggressive_flight.html
Drone Racing: http://rpg.ifi.uzh.ch/research_drone_racing.html
Machine Learning: http://rpg.ifi.uzh.ch/research_learning.html

Affiliations:
A. Romero, Y. Song, and D. Scaramuzza are with the Robotics and Perception Group, Dep. of Informatics, University of Zurich, and Dep. of Neuroinformatics, University of Zurich and ETH Zurich, Switzerland
http://rpg.ifi.uzh.ch/

Music Credits: scottholmesmusic.com under Free Creative Commons License
Actor-Critic Model Predictive Control (ICRA 2024)RPG TV Documentary: Drones for Search and Rescue - 3Sat Nano - 20.03.2020Deep Visual Odometry with Events and Frames (IROS 2024)Efficient, Data-Driven Perception with Event Cameras (Ph.D. Defense of Daniel Gehrig)Dream to Fly: Model-Based Reinforcement Learning for Vision-Based Drone Flight (ICRA 2026)Feature Detection and Tracking with the Dynamic and Active pixel Vision Sensor DAVISUZH Deep Learning Symposium: Learning Agile FilghtAEGNN: Asynchronous Event-based Graph Neural Networks (CVPR 2022)Dronet: Learning to Fly by DrivingRedesigning SLAM for Arbitrary Multi-Camera SystemsReinforcement Learning Meets Visual Odometry  (ECCV 2024)Low-Latency Event-Based Object Detection with Spatially-Sparse Linear Attention (ECCV 2026)
UZH Robotics and Perception Group |

Actor-Critic Model Predictive Control (ICRA 2024)

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER