Uploaded March 2025 | Updated September 2026, 2 hours ago
Presentation from ACM UIST 2024.
Paper PDF: figlab.com/research/2024/egotouch
In augmented and virtual reality (AR/VR) experiences, a user’s arms and hands can provide a convenient and tactile surface for touch input. Prior work has shown on-body input to have significant speed, accuracy, and ergonomic benefits over in-air interfaces, which are common today. In this work, we demonstrate high accuracy, bare hands (i.e., no special instrumentation of the user) skin input using just an RGB camera, like those already integrated into all modern XR headsets. Published at ACM UIST 2024.
Vimal Mollyn and Chris Harrison. 2024. EgoTouch: On-Body Touch Input Using AR/VR Headset Cameras. In Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology (UIST '24). Association for Computing Machinery, New York, NY, USA, Article 69, 1–11.
Presentation from ACM UIST 2024.
Paper PDF: figlab.com/research/2024/egotouch
In augmented and virtual reality (AR/VR) experiences, a user’s arms and hands can provide a convenient and tactile surface for touch input. Prior work has shown on-body input to have significant speed, accuracy, and ergonomic benefits over in-air interfaces, which are common today. In this work, we demonstrate high accuracy, bare hands (i.e., no special instrumentation of the user) skin input using just an RGB camera, like those already integrated into all modern XR headsets. Published at ACM UIST 2024.
Vimal Mollyn and Chris Harrison. 2024. EgoTouch: On-Body Touch Input Using AR/VR Headset Cameras. In Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology (UIST '24). Association for Computing Machinery, New York, NY, USA, Article 69, 1–11.
![TouchTools [CMU/Qeexo]](https://i.ytimg.com/vi/N8s8NJf34fM/mqdefault.jpg "TouchTools [CMU/Qeexo]
Gestures on todays touch devices are simplistic, solely relying on the number of fingers (1-finger pan, 2-finger zoom, 3-finger swipe, etc). We propose instead that gesture design be inspired by the manipulation of real world artifacts specifically tools. The result is a set of rich gestures for touch interaction that leverages user familiarity and fluency with real world objects.")
