Reinforcement Learning for Robotics Part 3: Deploy AI Agent to Real Robot (Sim-to-Real) | DigiKey @digikey
Reinforcement Learning for Robotics Part 3: Deploy AI Agent to Real Robot (Sim-to-Real) | DigiKey  @digikey
Uploaded August 2026 | Updated November 2026, 12 hours ago
In this series, we explore using reinforcement learning algorithms (RL) to have a robot learn to balance on its own. We will deploy the trained agent to a real ESP32-based robot platform, tackle the sim-to-real gap using post-processing and domain randomization, and eventually build a full remote-controlled balance bot. The same techniques used here are what researchers use to train state-of-the-art walking robots. So, while we are starting with a simpler project, know that these skills will transfer to modern robotics.

Click here to register for Shawn’s webinar: event.on24.com/wcc/r/5384134/958EFFE494BF1C6A2AD94F9B093DC01B?partnerref=vid

To follow along with the series, you will need the following hardware:

M5Stack BALA2 Fire robot kit: digikey.com/short/7h2fb5hd

All project files, including the FreeCAD model, MuJoCo XML descriptor, and Docker environment, are available in the following GitHub repository:

github.com/ShawnHymel/reinforcement-learning-for-robotics

In the previous episode, we trained our balance bot agent entirely in simulation. Now, it's time to see how it holds up in the real world. In this episode, we deploy the trained PPO actor network to the M5Stack BALA 2 Fire robot and quickly run into the sim-to-real gap: the differences between simulation and reality that the agent never learned to handle.

You can read a full write-up of everything we cover in this video here:

digikey.com/en/maker/tutorials/2026/reinforcement-learning-for-robotics-part-3-deploy-ai-agent-to-a-robot

We start by converting the trained actor network from ONNX to a plain C header file, which gets compiled directly into the Arduino firmware. We are not accelerating the simple dense network (i.e. no TensorFlow Lite, NPU, etc.), just multiply-accumulates and a tanh activation running in 2–3ms on the ESP32. From there, we walk through the Arduino firmware that reads the IMU and wheel encoders, builds the observation vector, runs inference, and drives the motors. We keep all this within a 5ms control loop.

With the initial firmware uploaded, the robot balances briefly but eventually drifts and tips. We cover the main sources of sim-to-real error (sensor noise, motor inaccuracies, modeling errors, and calibration offsets) and then add a set of firmware fixes, including a pitch offset, motor boost, action deadband, and low-pass filter. While these fixes help the robot to balance more reliably, they require manual tuning and undermine the whole point of training a robust agent. The real solution is domain randomization, which we'll tackle in the next episode.

Coursera Deep Learning specialization: coursera.org/specializations/deep-learning
Coursera Reinforcement Learning specialization: coursera.org/specializations/reinforcement-learning
Introduction to Reinforcement Learning textbook by Sutton and Barto: incompleteideas.net/book/the-book-2nd.html
Reinforcement Learning math blog post series: shawnhymel.com/3316/what-is-reinforcement-learning
Introduction to Reinforcement Learning video: youtube.com/watch?v=3av8vozEczU

Learn more:
Maker.io - digikey.com/en/maker
DigiKey’s Blog – TheCircuit digikey.com/en/blog
Connect with DigiKey on Facebook facebook.com/digikey.electronics
And follow us on X: https://x.com/digikey

0:00 Intro
0:45 Strategy to deploy the actor network
1:30 ONNX actor to C code script
5:35 Calibrate IMU
9:34 Deploy agent to robot with Arduino
17:45 Post-processing firmware fixes
22:25 Conclusion
Reinforcement Learning for Robotics Part 3: Deploy AI Agent to Real Robot (Sim-to-Real) | DigiKeySeeing Double [Maker Update] | Maker.ioIntro to ROS Part 4: Nodes, Topics, Publishers, and Subscribers with C++  | DigiKey2025’s Notable Projects #Electronics #Diy #2025 #makerupdateIn My Lane, Flourishing [Maker Update] | Maker.ioRugged Connectors, 48V Systems & Data Centers on Wheels | DigiKeyHeadfirst [Maker Update] | Maker.ioPlaying by the Rules [Maker Update] | Maker.ioPortable Worlds [Maker Update] | Maker.ioBenchtop Power Supplies - Electronics with Becky Stern | DigiKeyPerfect Pitch Printer [Maker Update] | Maker.ioMoving Data at AI Scale - Factory Tomorrow S6E2 | DigiKey
DigiKey |

Reinforcement Learning for Robotics Part 3: Deploy AI Agent to Real Robot (Sim-to-Real) | DigiKey

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER