Uploaded August 2026 | Updated November 2026, 12 hours ago
In this series, we explore using reinforcement learning algorithms (RL) to have a robot learn to balance on its own. We will deploy the trained agent to a real ESP32-based robot platform, tackle the sim-to-real gap using post-processing and domain randomization, and eventually build a full remote-controlled balance bot. The same techniques used here are what researchers use to train state-of-the-art walking robots. So, while we are starting with a simpler project, know that these skills will transfer to modern robotics.
Click here to register for Shawn’s webinar on this topic: event.on24.com/wcc/r/5384134/958EFFE494BF1C6A2AD94F9B093DC01B?partnerref=vid
To follow along with the series, you will need the following hardware:
M5Stack BALA2 Fire robot kit: digikey.com/short/7h2fb5hd
All project files, including the FreeCAD model, MuJoCo XML descriptor, and Docker environment, are available in the following GitHub repository:
github.com/ShawnHymel/reinforcement-learning-for-robotics
In the previous episode, we deployed our trained agent to the real robot and had to patch the sim-to-real gap with firmware fixes like pitch offsets, motor boosts, and low-pass filters. It worked, but it required manual tuning and undermined the whole point of training a robust agent. This episode takes the more principled approach: domain randomization.
You can read a full write-up of everything we cover in this video here:
digikey.com/en/maker/tutorials/2026/reinforcement-learning-for-robotics-part-4-domain-randomization
Domain randomization (DR) is the most popular technique for bridging the sim-to-real gap in robotics. Rather than trying to perfectly model the robot, we deliberately introduce randomness into the simulation during training (e.g. noisy sensor readings, variable motor torque, random pushes, friction variation, etc.) so the agent learns a policy that generalizes to the real world without firmware workarounds.
We walk through the updated Gymnasium environment, which adds a new DomainRandomConfig dataclass to control all the randomization parameters. From there, we extend our curriculum learning approach to 7 training phases, slowly introducing domain randomization after the agent has mastered the basic balancing task. We cover what to randomize, why, and how to layer it in gradually without destabilizing training. Once training is complete, we deploy the new actor network to the robot (this time without all the firmware fixes) and see a noticeably more robust result.
Coursera Deep Learning specialization: coursera.org/specializations/deep-learning
Coursera Reinforcement Learning specialization: coursera.org/specializations/reinforcement-learning
Introduction to Reinforcement Learning textbook by Sutton and Barto: incompleteideas.net/book/the-book-2nd.html
Reinforcement Learning math blog post series: shawnhymel.com/3316/what-is-reinforcement-learning
Introduction to Reinforcement Learning video: youtube.com/watch?v=3av8vozEczU
Learn more:
Maker.io - digikey.com/en/maker
Digi-Key’s Blog – TheCircuit digikey.com/en/blog
Connect with Digi-Key on Facebook facebook.com/digikey.electronics
And follow us on X https://x.com/digikey
0:00 Intro
0:54 Domain randomization overview
3:00 Adding domain randomization to the gymnasium environment
9:34 Retrain agent with domain randomization enabled
19:52 Deploy agent to robot using Arduino
21:52 Conclusion
In this series, we explore using reinforcement learning algorithms (RL) to have a robot learn to balance on its own. We will deploy the trained agent to a real ESP32-based robot platform, tackle the sim-to-real gap using post-processing and domain randomization, and eventually build a full remote-controlled balance bot. The same techniques used here are what researchers use to train state-of-the-art walking robots. So, while we are starting with a simpler project, know that these skills will transfer to modern robotics.
Click here to register for Shawn’s webinar on this topic: event.on24.com/wcc/r/5384134/958EFFE494BF1C6A2AD94F9B093DC01B?partnerref=vid
To follow along with the series, you will need the following hardware:
M5Stack BALA2 Fire robot kit: digikey.com/short/7h2fb5hd
All project files, including the FreeCAD model, MuJoCo XML descriptor, and Docker environment, are available in the following GitHub repository:
github.com/ShawnHymel/reinforcement-learning-for-robotics
In the previous episode, we deployed our trained agent to the real robot and had to patch the sim-to-real gap with firmware fixes like pitch offsets, motor boosts, and low-pass filters. It worked, but it required manual tuning and undermined the whole point of training a robust agent. This episode takes the more principled approach: domain randomization.
You can read a full write-up of everything we cover in this video here:
digikey.com/en/maker/tutorials/2026/reinforcement-learning-for-robotics-part-4-domain-randomization
Domain randomization (DR) is the most popular technique for bridging the sim-to-real gap in robotics. Rather than trying to perfectly model the robot, we deliberately introduce randomness into the simulation during training (e.g. noisy sensor readings, variable motor torque, random pushes, friction variation, etc.) so the agent learns a policy that generalizes to the real world without firmware workarounds.
We walk through the updated Gymnasium environment, which adds a new DomainRandomConfig dataclass to control all the randomization parameters. From there, we extend our curriculum learning approach to 7 training phases, slowly introducing domain randomization after the agent has mastered the basic balancing task. We cover what to randomize, why, and how to layer it in gradually without destabilizing training. Once training is complete, we deploy the new actor network to the robot (this time without all the firmware fixes) and see a noticeably more robust result.
Coursera Deep Learning specialization: coursera.org/specializations/deep-learning
Coursera Reinforcement Learning specialization: coursera.org/specializations/reinforcement-learning
Introduction to Reinforcement Learning textbook by Sutton and Barto: incompleteideas.net/book/the-book-2nd.html
Reinforcement Learning math blog post series: shawnhymel.com/3316/what-is-reinforcement-learning
Introduction to Reinforcement Learning video: youtube.com/watch?v=3av8vozEczU
Learn more:
Maker.io - digikey.com/en/maker
Digi-Key’s Blog – TheCircuit digikey.com/en/blog
Connect with Digi-Key on Facebook facebook.com/digikey.electronics
And follow us on X https://x.com/digikey
0:00 Intro
0:54 Domain randomization overview
3:00 Adding domain randomization to the gymnasium environment
9:34 Retrain agent with domain randomization enabled
19:52 Deploy agent to robot using Arduino
21:52 Conclusion

![Plowing Ahead [Maker Update] | Maker.io
This week on Maker Update: remote control snow patrol, cleaning up after Wall-E, fighting plastic with plastic, and a boogie-down Bottango bot.
+Show Notes [Maker Update Ep.474]+
=Project of the Week=
1/7 PB 600 Snowcat with 12-way Blade by Willem Smith (4x3D) 0:26
https://youmagine.com/designs/ea2e008d-95f2-42de-b343-d7fddfc5ef6f#top
=More Projects=
Can I Turn M-O from Wall-E into a Real Vacuum? by Kiaras Workshop 1:58
https://www.youtube.com/watch?v=ZdyHG45VR3Y
Can a 3D Printed Combat Robot Actually Do Damage? by Makers Muse 2:56
https://www.youtube.com/watch?v=pm5FW0dlryY
Droid-Style RC Animatronics Using Bottango and ESP32 by Donald Bell 3:54
https://www.instructables.com/Droid-Style-RC-Animatronics-Using-Bottango-and-ESP/
=Tips & Tools=
Simple Screw Counter by mitxela 4:37
https://mitxela.com/projects/screwcounter
Dont Waste Time on Wrong ESP32 Pins by Vman Builds 5:05
https://www.youtube.com/watch?v=E4LxQvyeiWw
Arcade Fightstick by John Park 6:08
https://learn.adafruit.com/arcade-fightstick
=DigiKey Spotlight=
Electronics Teardowns and Recycling - Electronics with Becky Stern 6:50
https://www.youtube.com/watch?v=agOJkBQqup8
#makerupdate
Maker Update is a weekly video series covering news, projects, tips, and events that interest the maker and DIY community.
Find more online:
Blog: http://makerprojectlab.com
Instagram: https://www.instagram.com/makerprojectlab/ Plowing Ahead [Maker Update] | Maker.io](https://i.ytimg.com/vi/lUVlauwsN8I/mqdefault.jpg)
![Maker Faire Bay Area 2026 [Maker Update] | Maker.io
This week on Maker Update: a garden of inflatable delights, a tiling wall of light, combat garden gnomes and telling stories with light. We’re at Maker Faire Bay Area 2026.
+Show Notes [Maker Update Ep.503]+
=Projects from MFBA 26=
Thorsson & Associates Workshop 0:37
https://tandaworkshop.com/
Story Lamp by Story Lamp Collective 3:05
https://myteach.blog/2019/07/31/%e3%80%90innovative-teaching%e3%80%91light-up-wansheng-community-story-lamp/
Rob the Robot by Rudy Aramayo 4:45
https://www.orbitusrobotics.com/
Inflatable Costumes and Decorations by Devising Delight 7:04
https://devisingdelight.com/
Penrose Tile Wall by Chuck Sommerville 9:47
https://www.chucksommerville.com/
Arduino at Maker Faire Bay Area 11:47
https://www.arduino.cc/
https://www.youtube.com/channel/UC-8E9fWJmFdEfK46u2S6nFg
https://www.instagram.com/supermoderno.tv/
Shop for Arduino on DigiKey:
https://www.digikey.com/en/supplier-centers/arduino
#makerupdate
Maker Update is a weekly video series covering news, projects, tips, and events that interest the maker and DIY community.
Find more online:
Blog: http://makerprojectlab.com
Instagram: https://www.instagram.com/makerprojectlab/ Maker Faire Bay Area 2026 [Maker Update] | Maker.io](https://i.ytimg.com/vi/lhKduoM-UmE/mqdefault.jpg)


![Made to Fade [Maker Update] | Maker.io
This week on Maker Update: flying faders, data dispensers, tiny trucks, walkie talkies, and winking neon.
+Show Notes [Maker Update Ep.480]+
=Project of the Week=
V2 of the Motorized Fader Per-App Volume Controller! by Computer Hardware Tips and Tutorials 0:27
https://www.instructables.com/V2-of-the-Motorized-Fader-Per-App-Volume-Controlle/
=More Projects=
Star Trek Data Dispenser by Ruiz Brothers 2:32
https://learn.adafruit.com/star-trek-data-dispenser
Truck-Kun v1 by Victor Frost 3:14
https://www.heyvictorfrost.com/workshop/Truck-Kun
ESP-NOW Walkie Talkies by Liz Clark 3:43
https://learn.adafruit.com/esp-now-walkie-talkies
Neon Animated Eye by David Hamp-Gonsalves 6:32
https://davidhampgonsalves.com/neon-animated-eye/
=Tips & Tools=
Make Wireless LEDs for 1/8th the Cost by Myth Made 4:58
https://www.youtube.com/watch?v=W5IFY6Dsom0
Tips for Using AI to Write Embedded Firmware by Shawn Hymel 5:22
https://shawnhymel.com/3285/tips-for-using-ai-to-write-embedded-firmware/
=DigiKey Spotlight=
Sound Generation and Musical Microcontrollers - Electronics with Becky Stern 5:48
https://www.youtube.com/watch?v qmU0hE89Fk
#makerupdate
Maker Update is a weekly video series covering news, projects, tips, and events that interest the maker and DIY community.
Find more online:
Blog: http://makerprojectlab.com
Instagram: https://www.instagram.com/makerprojectlab/ Made to Fade [Maker Update] | Maker.io](https://i.ytimg.com/vi/lziDdwIAUts/mqdefault.jpg)


![Intro to Embedded Rust Part 12: Async Programming with Embassy | DigiKey
In this tutorial, we cover async/await programming for embedded systems with Embassy, a modern Rust framework that enables concurrent multitasking on microcontrollers without a traditional RTOS.
The written guide for this episode can be found here: https://www.digikey.com/en/maker/tutorials/2026/intro-to-embedded-rust-part-12-asynchronous-programming-with-embassy
The GitHub repository containing the Docker image and example code for this series can be found here: https://github.com/ShawnHymel/introduction-to-embedded-rust
Embassy provides cooperative task scheduling using Rusts native async/await syntax, allowing you to write code that handles multiple operations simultaneously, like monitoring buttons, blinking LEDs, and managing USB. We demonstrate how to replace bare-metal peripheral access with Embassys higher-level abstractions, creating a button-controlled LED blinker with USB logging that showcases async GPIO, timers, task spawning, and inter-task communication.
Embassy is a collection of modular crates including platform-specific HALs (embassy-rp for Raspberry Pi, embassy-stm32, embassy-nrf), the executor for running async tasks, timer abstractions, synchronization primitives, and protocol stacks for USB and networking. We demonstrate configuring Embassy dependencies in Cargo.toml, using the bind_interrupts! macro to connect hardware interrupts to handlers automatically, and marking functions with #[embassy_executor::task] to transform them into spawnable concurrent tasks.
We also dive into futures (objects representing asynchronous operations that may not have completed yet) and look at how the executor manages them, pausing and resuming tasks as needed while automatically putting the processor to sleep when all tasks are waiting, providing efficient power management without manual wfi() calls.
The tutorial showcases Embassys async-first API design with real hardware interaction. Youll see how to use async GPIO methods like pin.wait_for_low().await that yield execution until events occur, eliminating manual interrupt service routines and polling loops. We demonstrate Timer::after_millis().await for non-blocking delays, Signal for inter-task communication (replacing the Mutex‹RefCell‹Option‹T››› pattern from bare-metal interrupt handling), and embassy_usb_logger for USB serial debugging output.
We also show how to initialize the runtime and use a Spawner for launching tasks, using join_array to run multiple async operations concurrently, and structuring programs with both spawned tasks (independent lifecycle) and async functions within the main task. Youll see how Embassy abstracts interrupt handling, peripheral configuration, and resource sharing, making complex embedded applications more maintainable while preserving Rusts zero-cost abstractions and memory safety.
Related Link:
https://www.digikey.com/en/maker/tutorials/2026/intro-to-embedded-rust-part-12-asynchronous-programming-with-embassy
Links:
Getting Started with STM32 and Nucleo: https://www.youtube.com/watch?v=hyZS2p1tW-g&list=PLEBQazB0HUyRYuzfi4clXsKUSgorErmBv
Introduction to Real-Time Operating Systems (RTOS): https://www.youtube.com/watch?v=F321087yYy4&list=PLEBQazB0HUyQ4hAPU1cJED6t3DU0h34bz
Raspberry Pi Pico 2: https://www.digikey.com/en/products/detail/raspberry-pi/SC1631/24627142
Learn more:
Maker.io - https://www.digikey.com/en/maker
DigiKey’s Blog – TheCircuit https://www.digikey.com/en/blog
Connect with DigiKey on Facebook https://www.facebook.com/digikey.electronics/
And follow us on X: https://x.com/digikey
Product:
https://www.digikey.com/en/products/detail/raspberry-pi/SC1631/24627142
https://www.digikey.com/en/products/detail/raspberry-pi/SC1632/26241102
https://www.digikey.com/en/products/detail/raspberry-pi/SC1633/25862735
https://www.digikey.com/en/products/detail/raspberry-pi/SC1634/26241087
https://www.digikey.com/en/products/detail/sparkfun-electronics/13314/6797646
https://www.digikey.com/en/products/detail/lite-on-inc/LTL-4224/217584
https://www.digikey.com/en/products/detail/stackpole-electronics-inc/CF14JT220R/1830334
https://www.digikey.com/en/products/detail/te-connectivity-alcoswitch-switches/1825910-6/1632536
https://www.digikey.com/en/products/detail/dfrobot/FIT0096/7597069
https://www.digikey.com/en/products/detail/adafruit-industries-llc/1957/6827090
https://www.digikey.com/en/products/detail/cvilux-usa/DH-20M50055/13175849
Chapters:
0:00 Intro
1:05 Overview of Embassy
2:22 Configuring Project and Setting Dependencies
8:16 Create Button Demo
22:56 Build and Flash Button Demo
26:49 Add Blinking LED Task
36:20 Flash and Build Button and LED Demo
37:21 Going Further with Rust
38:13 Conclusion Intro to Embedded Rust Part 12: Async Programming with Embassy | DigiKey](https://i.ytimg.com/vi/m9FT1qSm-KM/mqdefault.jpg)

