Uploaded August 2024 | Updated September 2026, 6 hours ago
This is the video from the bachelor thesis from Bernhard Messerli at the Bern University of Applied Science in Switzerland.
In the project, an Android app was developed that allows the user to virtually test a shoe model before paying for it. This technology belongs to the field of augmented reality in computer graphics. Unity is the heart of the application in which this extension is presented.
The challenge is that the camera is monocular, so no depth information is available. The constantly changing position of the captured foot must be captured in real time and transferred to the virtual shoe model to scale it accordingly.
Implementing the occlusion, in which the hidden parts of the foot should not be rendered, is particularly challenging.
The implementation of this project relies on artificial intelligence. A YOLOv8 model is trained using modern deep-learning algorithms. This requires a user-defined dataset of labeled foot images. Using these labeled key points, the virtualized shoe model can now be placed as precisely as possible on the foot in the video frame of a mobile device.
The finished Shoe AR app aims to help customers try on shoes from the industry partner virtually on their own feet. In addition, the app should show how shoes could harmonize with certain clothing items, for example.
This is the video from the bachelor thesis from Bernhard Messerli at the Bern University of Applied Science in Switzerland.
In the project, an Android app was developed that allows the user to virtually test a shoe model before paying for it. This technology belongs to the field of augmented reality in computer graphics. Unity is the heart of the application in which this extension is presented.
The challenge is that the camera is monocular, so no depth information is available. The constantly changing position of the captured foot must be captured in real time and transferred to the virtual shoe model to scale it accordingly.
Implementing the occlusion, in which the hidden parts of the foot should not be rendered, is particularly challenging.
The implementation of this project relies on artificial intelligence. A YOLOv8 model is trained using modern deep-learning algorithms. This requires a user-defined dataset of labeled foot images. Using these labeled key points, the virtualized shoe model can now be placed as precisely as possible on the foot in the video frame of a mobile device.
The finished Shoe AR app aims to help customers try on shoes from the industry partner virtually on their own feet. In addition, the app should show how shoes could harmonize with certain clothing items, for example.
