Uploaded March 2020 | Updated September 2026, 3 hours ago
Hololens for Machine Maintenance for Gilgen Logistic Machines:
At the core of this pilot project was the linking of several relatively new and IT technologies:
Data exchange via a network server: so that the partners involved, the Gilgen supporter, and the operator can get in touch, they must be connected to the Internet. For this we need two servers:
Dark Rift server: This is used for initial contact and for the continuous flow of data for the 3D synchronization of the Hololens.
Node-DSS: This is a simple signaling server for contacting the video communication in order to link the two clients of the WebRTC connection (the Hololens and the supporter's browser app).
Peer-To-Peer WebRTC: WebRTC stands for Web Real-Time Communication and is an open standard that defines protocols and programming interfaces for real-time communication of computer-to-computer connections. This allows web browsers and other clients, like our Hololens, to exchange video and audio streams directly without going through a server.
Augmented Reality with the Microsoft Hololens:
Pose Estimation: The Hololens glasses are AR glasses to show the wearer 3D objects in the glasses. This happens stereoscopically and in real-time (i.e. at least 30 times per second) and can show the wearer of an object in the room that is actually not there. The most important aspect of AR is that the glasses can determine the position and orientation of the wearer very quickly and precisely. With this pose estimation, the computer can generate two perspectively correct images and fade them into the special glasses.
Marker detection: in order to precisely identify the Gilgen machine with the holo lens, we use a QR code that we recognize from the video stream. In addition, we use this marker to specify the initial position of the machine.
3D model visualization in the browser: The 3D objects that we show in the Hololens are also displayed in the supporter's browser. The most important point in this project was that the supporter could select and highlight a problematic part in his 3D representation of the machine. This selection is now transmitted to the Hololens via the network and thus also displayed to the operator in 3D in the glasses. The supervisor can give the operator precise instructions as if he were standing next to the operator.
This project is based on the bachelor thesis of Joël Frutiger from fall 2019.
Hololens for Machine Maintenance for Gilgen Logistic Machines:
At the core of this pilot project was the linking of several relatively new and IT technologies:
Data exchange via a network server: so that the partners involved, the Gilgen supporter, and the operator can get in touch, they must be connected to the Internet. For this we need two servers:
Dark Rift server: This is used for initial contact and for the continuous flow of data for the 3D synchronization of the Hololens.
Node-DSS: This is a simple signaling server for contacting the video communication in order to link the two clients of the WebRTC connection (the Hololens and the supporter's browser app).
Peer-To-Peer WebRTC: WebRTC stands for Web Real-Time Communication and is an open standard that defines protocols and programming interfaces for real-time communication of computer-to-computer connections. This allows web browsers and other clients, like our Hololens, to exchange video and audio streams directly without going through a server.
Augmented Reality with the Microsoft Hololens:
Pose Estimation: The Hololens glasses are AR glasses to show the wearer 3D objects in the glasses. This happens stereoscopically and in real-time (i.e. at least 30 times per second) and can show the wearer of an object in the room that is actually not there. The most important aspect of AR is that the glasses can determine the position and orientation of the wearer very quickly and precisely. With this pose estimation, the computer can generate two perspectively correct images and fade them into the special glasses.
Marker detection: in order to precisely identify the Gilgen machine with the holo lens, we use a QR code that we recognize from the video stream. In addition, we use this marker to specify the initial position of the machine.
3D model visualization in the browser: The 3D objects that we show in the Hololens are also displayed in the supporter's browser. The most important point in this project was that the supporter could select and highlight a problematic part in his 3D representation of the machine. This selection is now transmitted to the Hololens via the network and thus also displayed to the operator in 3D in the glasses. The supervisor can give the operator precise instructions as if he were standing next to the operator.
This project is based on the bachelor thesis of Joël Frutiger from fall 2019.
