Uploaded October 2025 | Updated September 2026, 3 hours ago
PoolXR fuses real-world pool with virtual guidance in Mixed Reality, using a custom YOLO ball-tracking dataset and physics-based shot prediction - all on-device on Meta Quest via Unity Inference.
Aman, the creator of PoolXR, will walk you through the design, building, refinement, and optimization process, demonstrating how MR and computer vision can create a truly immersive experience.
Takeaways:
• Train a custom YOLO model with 450+ labeled images to track cue and game balls
• Overcome on-device inference challenges, including multiple objects tracking, lighting, and real-time performance
• Physics-based shot predictions visualized in Mixed Reality to guide the user
• Learn why YOLO was chosen over OpenCV for tracking on Meta Quest
Why this topic matters:
On-device inference on wearables remains a unique challenge, especially for real-time object tracking, where varying lighting, angles, and multiple moving objects can cause issues. Apps like PoolXR tackle these problems, showing how to train a custom YOLO model, run it locally on Meta Quest, and optimize real-time performance for multiple moving objects, enabling practical, real-world mixed reality use cases.
Speaker:
Aman Bohra
Aman Bohra is an XR developer with over half a decade of experience creating immersive AR/VR experiences that blend technology with storytelling. He currently works as a Senior XR Developer at DJXLabs, where he builds impactful, interactive realities and explores the future of spatial computing. Aman actively shares his journey and insights with the XR community, inspiring the next generation of creators.
About PoolXR:
PoolXR is a mixed reality-based project that uses a custom-trained object detection model (for cue and game balls) along with the new Meta Passthrough Camera API. It brings the classic game of pool to life by infusing it with virtual information and guidance, combining real-world interaction with virtual elements. amanbohra.com/projects/pool-xr
About SensAI Hackademy:
SensAI Hackademy gives you early access to devkits, wearables, and expert-led courses. Join our community and team up with top XR, AI, CV, and mobile developers in year-round online and in-person hackathons and events. We bring together leading engineers, researchers, and creators to collaborate, learn, and push the boundaries of wearable technologies—accelerating the future of immersive, intelligent experiences. sensaihackademy.com
After kicking off the SensAI Hack 2025 Global Tour in Cologne✅, we’re now bringing the innovation energy to SensAI Barcelona, right before Unite 2025, and then heading to SensAI San Francisco, the heart of tech innovation.
🚀 Apply Now (sensaihack.com/) to join the upcoming hackathons and be part of the next wave of breakthroughs!
Timestamp:
0:00:00 - Welcome
0:00:39 - SensAI Hackademy Introduction
0:06:06 - SensAI Hackathons
0:12:32 - PoolXR Introduction
0:22:05 - Why to choose PoolXR
0:36:49 - Data Prep
0:48:06 - Model Training
1:01:12 - Unity Integration
1:19:45 - PCA
1:20:47 - Mapping
1:25:11 - Ball World Locking
1:26:07 - Shot Prediction
1:29:14 - Current Limitations
1:31:10 - Q&A
PoolXR fuses real-world pool with virtual guidance in Mixed Reality, using a custom YOLO ball-tracking dataset and physics-based shot prediction - all on-device on Meta Quest via Unity Inference.
Aman, the creator of PoolXR, will walk you through the design, building, refinement, and optimization process, demonstrating how MR and computer vision can create a truly immersive experience.
Takeaways:
• Train a custom YOLO model with 450+ labeled images to track cue and game balls
• Overcome on-device inference challenges, including multiple objects tracking, lighting, and real-time performance
• Physics-based shot predictions visualized in Mixed Reality to guide the user
• Learn why YOLO was chosen over OpenCV for tracking on Meta Quest
Why this topic matters:
On-device inference on wearables remains a unique challenge, especially for real-time object tracking, where varying lighting, angles, and multiple moving objects can cause issues. Apps like PoolXR tackle these problems, showing how to train a custom YOLO model, run it locally on Meta Quest, and optimize real-time performance for multiple moving objects, enabling practical, real-world mixed reality use cases.
Speaker:
Aman Bohra
Aman Bohra is an XR developer with over half a decade of experience creating immersive AR/VR experiences that blend technology with storytelling. He currently works as a Senior XR Developer at DJXLabs, where he builds impactful, interactive realities and explores the future of spatial computing. Aman actively shares his journey and insights with the XR community, inspiring the next generation of creators.
About PoolXR:
PoolXR is a mixed reality-based project that uses a custom-trained object detection model (for cue and game balls) along with the new Meta Passthrough Camera API. It brings the classic game of pool to life by infusing it with virtual information and guidance, combining real-world interaction with virtual elements. amanbohra.com/projects/pool-xr
About SensAI Hackademy:
SensAI Hackademy gives you early access to devkits, wearables, and expert-led courses. Join our community and team up with top XR, AI, CV, and mobile developers in year-round online and in-person hackathons and events. We bring together leading engineers, researchers, and creators to collaborate, learn, and push the boundaries of wearable technologies—accelerating the future of immersive, intelligent experiences. sensaihackademy.com
After kicking off the SensAI Hack 2025 Global Tour in Cologne✅, we’re now bringing the innovation energy to SensAI Barcelona, right before Unite 2025, and then heading to SensAI San Francisco, the heart of tech innovation.
🚀 Apply Now (sensaihack.com/) to join the upcoming hackathons and be part of the next wave of breakthroughs!
Timestamp:
0:00:00 - Welcome
0:00:39 - SensAI Hackademy Introduction
0:06:06 - SensAI Hackathons
0:12:32 - PoolXR Introduction
0:22:05 - Why to choose PoolXR
0:36:49 - Data Prep
0:48:06 - Model Training
1:01:12 - Unity Integration
1:19:45 - PCA
1:20:47 - Mapping
1:25:11 - Ball World Locking
1:26:07 - Shot Prediction
1:29:14 - Current Limitations
1:31:10 - Q&A
