Uploaded July 2023 | Updated September 2026, 2 weeks ago
This is a quick overview of the new paper from Meta / Facebook's Reality Labs Research group in Seoul.
QuestEnvSim: Environment-Aware Simulated Motion Tracking from Sparse Sensors
sunny-codes.github.io/projects/questenvsim.html
This is Full-Body Motion Tracking in AR/VR Using Reinforcement Learning
In this video, I'm reviewing the act of replicating a user's pose in augmented reality/virtual reality applications. Unlike most existing methods that focus on foot-floor contact alone, they take an interesting approach. They incorporate environment interaction into the mix. Imagine sitting on a couch or leaning on a desk and having your movements accurately tracked in the virtual world. But they're not tracked exactly, it's actually kinda faking it.
With the power of Reinforcement Learning, combined with physics simulation and environment observations, they show the potential to generate incredibly lifelike full-body poses even in highly constrained environments. Their physics simulation automatically enforces the necessary conditions for realistic poses. Say goodbye to artifacts like penetration and contact sliding that often plague other motion tracking techniques.
Join me as I explore the features that contribute to this method's exceptional performance. This includes sitting on chairs, lounging on a couch, stepping over boxes, rocking a chair, and even spinning around in an office chair.
Prepare to be amazed. Maybe. I don't know. Just watch the video and let me know what you think.
And for more info on my own solution, check out Glycon3d.com
This is a quick overview of the new paper from Meta / Facebook's Reality Labs Research group in Seoul.
QuestEnvSim: Environment-Aware Simulated Motion Tracking from Sparse Sensors
sunny-codes.github.io/projects/questenvsim.html
This is Full-Body Motion Tracking in AR/VR Using Reinforcement Learning
In this video, I'm reviewing the act of replicating a user's pose in augmented reality/virtual reality applications. Unlike most existing methods that focus on foot-floor contact alone, they take an interesting approach. They incorporate environment interaction into the mix. Imagine sitting on a couch or leaning on a desk and having your movements accurately tracked in the virtual world. But they're not tracked exactly, it's actually kinda faking it.
With the power of Reinforcement Learning, combined with physics simulation and environment observations, they show the potential to generate incredibly lifelike full-body poses even in highly constrained environments. Their physics simulation automatically enforces the necessary conditions for realistic poses. Say goodbye to artifacts like penetration and contact sliding that often plague other motion tracking techniques.
Join me as I explore the features that contribute to this method's exceptional performance. This includes sitting on chairs, lounging on a couch, stepping over boxes, rocking a chair, and even spinning around in an office chair.
Prepare to be amazed. Maybe. I don't know. Just watch the video and let me know what you think.
And for more info on my own solution, check out Glycon3d.com
