Uploaded April 2025 | Updated September 2026, 2 hours ago
Project: figlab.com/research/2025/patterntrack
Code: github.com/FIGLAB/PatternTrack
As augmented reality devices (e.g., smartphones and headsets) proliferate in the market, multi-user AR scenarios are set to become more common. Co-located users will want to share coherent and synchronized AR experiences, but this is surprisingly cumbersome with current methods. In response, we developed PatternTrack, a novel tracking approach that repurposes the structured infrared light patterns emitted by VCSEL-driven depth sensors, like those found in the Apple Vision Pro, iPhone, iPad, and Meta Quest 3. Our approach is infrastructure-free, requires no pre-registration, works on featureless surfaces, and provides the real-time 3D position and orientation of other users’ devices. In our evaluation — tested on six different surfaces and with inter-device distances of up to 260 cm — we found a mean 3D positional tracking error of 11.02 cm and a mean angular error of 6.81°.
Daehwa Kim, Robert Xiao, and Chris Harrison. 2025. PatternTrack: Multi-Device Tracking Using Infrared, Structured-Light Projections from Built-in LiDAR (CHI '25). Association for Computing Machinery, New York, NY, USA. DOI:doi.org/10.1145/3706598.3713388
Project: figlab.com/research/2025/patterntrack
Code: github.com/FIGLAB/PatternTrack
As augmented reality devices (e.g., smartphones and headsets) proliferate in the market, multi-user AR scenarios are set to become more common. Co-located users will want to share coherent and synchronized AR experiences, but this is surprisingly cumbersome with current methods. In response, we developed PatternTrack, a novel tracking approach that repurposes the structured infrared light patterns emitted by VCSEL-driven depth sensors, like those found in the Apple Vision Pro, iPhone, iPad, and Meta Quest 3. Our approach is infrastructure-free, requires no pre-registration, works on featureless surfaces, and provides the real-time 3D position and orientation of other users’ devices. In our evaluation — tested on six different surfaces and with inter-device distances of up to 260 cm — we found a mean 3D positional tracking error of 11.02 cm and a mean angular error of 6.81°.
Daehwa Kim, Robert Xiao, and Chris Harrison. 2025. PatternTrack: Multi-Device Tracking Using Infrared, Structured-Light Projections from Built-in LiDAR (CHI '25). Association for Computing Machinery, New York, NY, USA. DOI:doi.org/10.1145/3706598.3713388
