Drone Navigation without GPS HuCE - cpvrLab 2020-09-18 | The Global Positioning System (GPS) is a crucial component in consumer drones. GPS aided navigation is of straightforward usage. This project investigates the possibilities of drone navigation in case of GPS absence. In particular, our goal is to fly a drone from A to B without using GPS signals but with a camera attached.The pose estimation of a camera given its image is a well known and difficult computer vision problem. In general, the problem results in the estimation of 6 parameters (degrees of freedom (DoF)) using a trade-off between optimality and performance. The 6 parameters are the position in space: x-,y-,z-coordinate, and an arbitrary orientation.The Swiss Federal Office of Topography, swisstopo, provides to us maps in two styles: an orthographic projection of aerial photographs (referred to as ortho map) and its stripped-down abstractions, well known as the 1:25’000 maps (referred as the abstract map). Our approach to the pose estimation problem is to match the camera image on such a swisstopo map. To improve the matching we apply a learned transformation to the camera image and the map. For more details on the image, transformation learning see the video youtu.be/5JEFe2_L4So
JoysteerVR: A Driving Simulator for Steer-by-Wire Systems HuCE - cpvrLab 2025-06-22 | This is a video about Marko Bozic's bachelor's thesis at the Bern University of Applied Sciences in Switzerland.Driving a vehicle is an impossible task for most people with severe physiological impairments. They are therefore heavily dependent on carers and relatives to remain mobile. The company Bozzio AG in Biel has set itself the task of giving these people the opportunity to drive independently. To this end, they have developed a ‘drive-by-wire’ system, the “Joysteer” human-machine interface (HMI), which enables a vehicle to be controlled using control elements that can only be operated with the fingers. This system controls the mechanics via electric motors and can therefore be installed in almost all vehicles and customized to the customer's needs. However, operating this system is not easy to learn and requires a well-practised feel for the controllers. Learning this is frightening and expensive for customers.The aim of this bachelor's thesis is to develop a driving simulator for Bozzio AG, which will serve as a training tool for customers. Learner drivers can use the simulator to gain and deepen their initial experience in a safe environment. In addition to the real driving lessons, they can receive furthertraining lessons on the simulator. This gives customers a risk-free and cost-effective option to optimally prepare for their driving test.The driving simulator should run natively in virtual reality on the Oculus Quest 3 and be controllable with the Joysteer HMI. It was developed in the Godot Game Engine for this purpose. In the simulator, trainees can drive through various scenarios to practise common manoeuvres, such as parking or cornering on fast roads. The entire system is designed to be portable, such that only the VR headset and HMIs are required to use it. Before the project was launched in September 2024, a prototype had already been developed in Unity. This prototype was then ported to the Godot engine and further developed. The driving behaviour of the virtual vehicle was adapted as closely as possible to the real-world counterpart using regression analysis, a machine learning technique, to achieve a realistic driving experience.
Multilabel Rib Segmentation UsingDomain-Adversarial Training HuCE - cpvrLab 2025-06-22 | This is a video about Tim Schär's bachelor's thesis at the Bern University of Applied Sciences in Switzerland. Adolescent idiopathic scoliosis patients require frequent imaging during growth periods; however, current clinical approaches present significant limitations. While CT scans provide excellent bone contrast for segmentation, they expose patients to substantial ionizing radiation. Biplanar EOS X-ray imaging offers reduced radiation exposure while capturing anatomically relevant standing-position deformities. However, automated segmentation of overlapping rib structures from these 2D projections remains challenging. The severe scarcity of labeled EOS datasets further hinders the development of machine learning based segmentation models. This thesis develops a comprehensive automated pipeline that transforms biplanar EOS images into patient-specific 3D rib cage models through domain adversarial training and geometric reconstruction. To address data limitations, synthetic training datasets are generated from over 750 CT volumes using physics-based Digitally Reconstructed Radiography with multilabel segmentation preservation. A custom domain adversarial neural network architecture employs a twelve-channel output to maintain individual rib continuity while reducing the domain gap between synthetic and clinical images through gradient reversal mechanisms. The final component reconstructs volumetricmodels by establishing a mathematical correspondence between orthogonal projections, achieved through centerline extraction, and parametric representation.The segmentation network achieved Dice scores of 0.95 for coronal and 0.93 for sagittal views on synthetic data, with domain adversarial training producing measurable improvements in generalization to clinical EOS images. The 3D reconstruction component successfully captures patient-specific anatomical characteristics, including the overall shape of the rib cage, individual rib curvature patterns, the spatial relationships between rib pairs, and patient-specific spinal deformations with associated rib adaptations. This work establishes a foundation for extracting comprehensive anatomical information from limited projection data while providing methodological frameworks for medical imaging challenges involving synthetic data generation and cross-domain adaptation.
Reinforcement Learning for Autonomous Driving using Visual Lane Detection HuCE - cpvrLab 2025-06-22 | This is a video about Ammar Hammad's bachelor's thesis at the Bern University of Applied Sciences in Switzerland. It explores and demonstrates the usability of a neural network for traffic lane detection using color camera input in autonomous vehicles within a simulated environment created in the Unity game engine. Several deep learning-based approaches for lane detection are examined and evaluated for their applicability in Unity. To improve prediction performance and ensure full Unity compatibility, a neural network was implemented from scratch, using the architecture of BezierLaneNet (Z. Feng et al., 2022) as a reference. A Bézier-based road generation system was developed to facilitate both the generation of training data and the deployment of simulations. After iterative testing and adjustments, the custom network achieved performance comparable to BezierLaneNet, as demonstrated by the conducted evaluations. To enable downstream use of the network output, additional processing steps were developed, including the projection of predicted Bézier control points from image to world coordinates and collision detection based on tracked lane geometry. Finally, to showcase and assess the practical applicability of the network, a reinforcement learning environment was built using the Bézier-based road system, in which the trained network is deployed to provide perceptual input to an autonomous driving agent. The resulting system demonstrates an end-to-end pipeline comprising the Bézier-based road generation system, a customized BezierLaneNet, post-processing modules, and a reinforcement learning component for control.The underlying music was also composed by Ammar Hammad. See alsosoundcloud.com/user-365397860/spring-outside
AetherVR: Play VR without VR Headset HuCE - cpvrLab 2024-11-01 | This is the video of the final project of our apprentice Marino von Wattenwyl.To facilitate VR development in the cpvrLab, he developed a VR simulator called AetherVR. With AetherVR, VR applications can be started directly on the PC and don't have to be loaded onto a VR headset first. The system uses the webcam to track the user's head and hands to control the virtual headset and virtual controllers. AetherVR is implemented as an OpenXR runtime. OpenXR is a standard API for accessing VR devices that is usually implemented by manufacturers of VR hardware so that applications run on their devices. Marino has implemented this API to trick VR applications into thinking they are running on a VR headset. The system uses OpenCV to connect to the webcam and MediaPipe to track the user.
Shoe AR App HuCE - cpvrLab 2024-08-02 | This is the video from the bachelor thesis from Bernhard Messerli at the Bern University of Applied Science in Switzerland.In the project, an Android app was developed that allows the user to virtually test a shoe model before paying for it. This technology belongs to the field of augmented reality in computer graphics. Unity is the heart of the application in which this extension is presented.The challenge is that the camera is monocular, so no depth information is available. The constantly changing position of the captured foot must be captured in real time and transferred to the virtual shoe model to scale it accordingly.Implementing the occlusion, in which the hidden parts of the foot should not be rendered, is particularly challenging.The implementation of this project relies on artificial intelligence. A YOLOv8 model is trained using modern deep-learning algorithms. This requires a user-defined dataset of labeled foot images. Using these labeled key points, the virtualized shoe model can now be placed as precisely as possible on the foot in the video frame of a mobile device.The finished Shoe AR app aims to help customers try on shoes from the industry partner virtually on their own feet. In addition, the app should show how shoes could harmonize with certain clothing items, for example.
GlaciAR : Augmenting Melting Glaciers HuCE - cpvrLab 2024-06-17 | This is the project video of Aaron Grand's bachelor thesis at the Computer Science Department of the Bern University of Applied Sciences in the spring semester of 2024.The aim of the project is to interactively display and visualize the predicted changes in selected Swiss glaciers by the year 2100. The augmented reality (AR) process enables the glacier condition on-site to be precisely superimposed over the running video. In this way, users can experience the effects of climate change directly on-site. The application is aimed at a broad audience, especially tourists, and is intended to raise awareness of the consequences of glacier melting and the need for climate protection measures. GlaciAR uses location-dependent data and enables users to experience the current and future conditions of the glaciers directly on-site through an immersive perspective. The technical implementation is carried out using Unity and AR Foundation, using various methods such as coroutines, permission handling, exception handling and the use of Unity Addressables for efficient data management.
StreamingATLOD: Streaming-Assisted Terrain Level of Detail HuCE - cpvrLab 2024-06-17 | This is the project video of Amar Tabakovic's bachelor thesis at the Computer Science Department of the Bern University of Applied Sciences in the spring semester of 2024.Terrains are an important part of various practical computer graphics applications, such as video games, flight simulators, and geographical information systems. Since terrains are expensive to render, special rendering optimizations, such as level of detail (LOD). Besides the rendering performance, another important aspect of terrain rendering is the management of terrain data. Terrain datasets that are too large to fit entirely in memory must be streamed in from the disk or over the network and streamed out of memory.This thesis describes the implementation of StreamingATLOD, a large-scale terrain rendering system with LOD capable of rendering the Earth by streaming in height data and satellite imagery from web APIs based on the XYZ tiling scheme. StreamingATLOD is currently built around the data and APIs from the company MapTiler, which serves height data tiles and satellite imagery tiles. Both layers come projected in the Web Mercator projection and are used up to zoom level 14, corresponding to a precision of 9.5 meters per pixel.The implemented terrain LOD algorithm is mainly based on Chunked LOD. The terrain is organized as a quadtree, where each quadtree node represents a section of the terrain at the resolution of the node’s depth. Each node gets rendered with heightmap displacement in the vertex shader followed by a projection onto the globe. To hide cracks between adjacent nodes, skirts are rendered. The Web Mercator projection cuts off at a latitude of 85.05◦, resulting in holes at the North and South Pole. These holes get covered with a circular pole mesh. Terrain sections that are not visible are culled with view-frustum culling and horizon culling.For more information see: github.com/AmarTabakovic/bachelor-thesis
Together in Aventicum-VR HuCE - cpvrLab 2024-06-05 | This is the project video of Dan Simonin's bachelor thesis at the Computer Science Department of the Bern University of Applied Sciences in the spring semester of 2024.As virtual reality (VR) evolves, the Computer Perception and Virtual Reality Laboratory at the BFH wants to upgrade its existing Unity VR solution by adding networking capabilities. In addition to full-body avatars and grouped teleportation, multiple players in the same physical space must see their friends' avatars in their actual physical locations to enable real-time interactions.Use casesTwo distinct use cases were considered during the development process to guide feature choices to be as generic as possible:1) This use case involves sending headsets to a school via a postal system, allowing children to virtually visit with friends the Cigognier temple of Aventicum, once the capital of Roman Switzerland, before exploring the existing ruins in person.2) The second one is a professional application where architects can present the current state of a project to their clients.FeaturesStandaloneEven though using Android headsets like the Meta Quest comes with the downside of limited performance, which needs to be managed carefully, this makes the entire project versatile. Removing the requirement for a computer reduces the project's size, keeping it simple and future-proof. Additionally, it makes the headsets easier to send through the post.CalibrationHeadsets need to be calibrated so players can see each other's avatars in the same physical locations, enabling physical interactions between them. This calibration is done using two physical markers each player selects using the headset's passthrough feature, which displays the camera feed to perform augmented reality.Grouped TeleportationIf teleportation is not managed over the network, the calibration will be disrupted if a player teleports. To prevent this, teleportation is managed so that only one player can teleport at a time, and the ray displaying the teleportation destination is synchronized over the players.AvatarsVirtual reality relies heavily on a sense of presence, making it crucial for players to notice minimal discrepancies from reality. Since this project emphasizes physical interaction, the visual representation must be highly accurate to enhance the sense of presence. Using inverse kinematics to compute bone positions based on the headset and hand positions, a customized full-body avatar is displayed for each player.
VR for Architects HuCE - cpvrLab 2024-03-10 | This is an introductory video about how architects can visualize their designs in virtual reality (VR) as easily as possible. It was created for the course "Digital Modelling II" at the Bern University of Applied Sciences architecture department in Switzerland. The method is a two-step process:- First, we build a viewer app for mobile VR headsets that support the OpenXR standard. We only must do this once. This step takes about 10 min.- Second, we build the architectural scene and upload it to a server without a connection to a headset. This scene and any other available can be downloaded and explored in the headset. This build and upload process is much faster and allows also, that the uploaded scenes can be shared with other users. With this VR viewer, one can visit and link 3D spaces like one can view and link websites with a web browser. This step takes about 20 min. for the example project shown in this tutorial.TIMESTAMPS00:00 Introduction01:38 Open Project03:25 Build for Viewer06:43 Test the Viewer08:56 Import Model12:00 Create a new Scene13:26 Place Model15:41 Set Teleporting16:27 Build for Upload18:07 Test the Upload21:11 Bake Lighting24:11 Test Baked Scene25:49 Outlook
VR Together HuCE - cpvrLab 2024-01-18 | This is the video from the semester project of Dan Simonin at the Bern University of Applied Sciences. It demonstrates a multiuser VR experience where 4 users in the same room can experience the same virtual space correctly aligned. All headsets communicate over wifi and sync each other with their headset and hand poses.
Virtual Avatar Platform HuCE - cpvrLab 2023-12-20 | The Avatar Behavior Platform (ABP) was developed as part of an interdepartmental project between BFH-TI (Technology & Informatics) and BFH-S (Social Sciences). The project was funded as part of the strategic topic of Human Digital Transformation of the BFH.With the Avatar Behavior Platform, 3D applications can be developed where the user can be confronted with one or more virtual avatars. With the ABP, an avatar can be equipped with a simple artificial behavior. This includes, for example, following the user, avoiding, keeping distance, and maintaining eye contact. The avatar can react to certain events with predefined audio clips with the user. For example, the user points to a virtual object or moves to a specific location and the avatar begins to speak. These applications are developed with the 3D engine Unity and can be used on the PC, in the browser, or in the VR glasses. The movement space can be freely defined with 3D models. Any 3D character models with the corresponding animations are suitable as avatars.
Quest-3 Performance Test with an animated Roman crane HuCE - cpvrLab 2023-12-20 | In our recent enhancement of the AventivumVR scene, previously optimized with light-baking for performance, we introduced a crane featuring dynamic lighting, along with shape key and transform animations. This incorporation of dynamic lighting on select objects led to a near doubling of the draw calls and triangles. The Quest-3's performance threshold hovers around 250 draw calls and approximately 1.5 million triangles per frame. A notable aspect of using dynamic lighting, particularly evident at the start of the clip on the stone, is the significantly improved quality of normal mapping. Interestingly, adding animations does not majorly impact performance; instead, it's the increase in draw calls due to dynamic lighting that's more demanding. Achieving optimal performance hinges on efficient culling of non-visible objects and minimizing the use of dynamic lighting. Our next phase involves integrating skinned animations for the worker characters.
Visual Inertial Odometry HuCE - cpvrLab 2023-06-24 | This is the video of Fredy Maeder's bachelor thesis in spring 2023 at the computer science department of the Bern University of Applied Sciences in Switzerland.A system based on Visual Inertial Odometry has been developed for GPS-free navigation of a drone. For this purpose, the Structure from Motion technology was adopted from the preliminary study and additionally combined with inertial navigation. The implementation reconstructs a path that was completed with a smartphone. The data collection is done with an Android application, which was developed especially for this application.OpenCV was used to implement a Structure from Motion, which computes a relative 3D path from multiple 2D camera images. This path is scale-less. The problem of the missing scale was tried to be solved by inertial navigation. Accelerations are measured continuously during the movement. By a double integration of these data, a covered distance can be inferred.Finally, both signals were merged and converted to a path in geocoordinates. This could be compared with additionally measured GPS data. The drift of the measured acceleration data falsifies the data as expected. Better acceleration sensors could bring an improvement here. Nevertheless, plausible calculations could be performed.
Product Placement in 3D Environments HuCE - cpvrLab 2023-06-22 | This is the video of Bryan Tabinas and Ratheeban Rajakumar's bachelor thesis at the computer science department of the Bern University of Applied Sciences in Switzerland in spring 2023.The goal of the thesis was to validate the idea of product placements in 3D environments and explore the market potential for a start-up.Interviews with potential customers and desk research into the digital advertisement economy, lead to the findings of their needs and understanding of said industry standards. Using these insights important features for the platform could be conceptualized.Parallelly a platform was developed as a prototype to showcase the potential of the idea. The platform was able to handle real-time bidding requests and track impressions. Using the platform, a demonstration of how billboard placements could look has been realized within an own developed Unity game and with a modification for an established 3D simulation game.
Paragliding AR HuCE - cpvrLab 2023-06-22 | This is the video for Janick Stucki's bachelor thesis in the spring semester of 2023 at the computer science department of the Bern University of Applied Sciences.Paragliding is difficult, but for some pilots, it is not enough, and competitions are organized. The aim is to fly a predefined route as quickly as possible. GPS trackers (time and GPS position) and 360-degree cameras are used for the evaluation.The aim of the Paragliding-AR project is to combine 360-degree videos and GPS tracker data. The tracker data can be visualized in an application and augmented into the corresponding videos. Various tools were used to implement the application:- Python for converting the GPS data into a local east-north-up system- FFmpeg and ExifTool for reading out metadata of the 360-degree videos.- Unity and C# as the basis for the application- OpenCV for Unity for stabilization testsIn order for the image of the 360-degree videos and the flight tracks to match, the skybox where the video shows must be aligned using the flight direction from the GPS data. In order to obtain even better results, further approaches were investigated with OpenCV.
Utility Designer for Unity HuCE - cpvrLab 2023-06-15 | Utility Designer is a generic tool for the Unity Asset Store that speeds up the process of creating intelligent and dynamic AIs by combining the concept of utility AI and behavior trees.In a utility AI system, the NPC can have different user-defined states, all of which are scored based on the character's needs and environmental factors.The scores change over time and the highest scoring state determines the NPC's action.If a state does not meet all of its preconditions, it will be marked red and will not receive any score.A behavior tree in the execution tab is used to define these actions. We are currently looking at the evaluation tab, and by switching to the execution tab we can now see the custom-defined behavior of that state.A behavior tree is very flexible and allows us to easily create any kind of behavior that's needed to execute our states.Utility Designer has a powerful API that allows the creation of custom nodes for the behavior tree. Various callbacks and properties simplify the process of creating custom behaviors for your specific needs.
The Cigognier Sanctuary in mobile VR HuCE - cpvrLab 2023-05-11 | The Cigognier Sanctuary was a temple complex in the Roman city of Aventicum (today Avenches) in Switzerland. For more information please aventicum.org/fr/site-monuments/sanctuaire-du-cigognier.The project aims to find you how detailed we can populate an architectural scene with light baking for mobile VR headsets such as the Meta Quest and the Pico headsets.
Optimal Racing Car Agent for Unity HuCE - cpvrLab 2023-02-06 | This is the project video of Sascha Ledermann's bachelor thesis in autumn 2022 at the Bern University of Applied Sciences in Switzerland.This bachelor thesis shows that an agent can be trained in Unity with MLAgent to run a time trial faster than any human player. Even if the agent drives a route for the first time. For this purpose, an environment is implemented that makes it possible to dynamically create racetracks, drive them and then display statistics and visualizations of the race completed.In order to be able to optimally train the agent, data is made available in the environment, which the agent can use to record its environment. Care is taken to ensure that the information received does not give him any advantage over a human player.A reward system is also created for the agent, which enables him to achieve an optimal solution during training. To round off the entire training process, the hyperparameters are evaluated, allowing the agent to perform fast, stable, and efficient training.
LIDAR Processing - Implementing a 3D Scanner HuCE - cpvrLab 2023-01-23 | This is the project video from the bachelor thesis of Marc Hutzli at the Bern University of Applied Sciences during the autumn semester of 2022.
What a neural network learns during training. Video 3 HuCE - cpvrLab 2022-09-15 | This video shows what a shallow neural network learns in its weights to 100 neurons during 30 epochs of training. Each square shows the 28 by 28 weights from the MNIST input images.
What a neural network learns during training. Video 5 HuCE - cpvrLab 2022-09-15 | This video shows what a shallow neural network learns in its weights to 100 neurons during 30 epochs of training. Each square shows the 28 by 28 weights from the MNIST input images.
What a neural network learns during training. Video 4 HuCE - cpvrLab 2022-09-15 | This video shows what a shallow neural network learns in its weights to 100 neurons during 30 epochs of training. Each square shows the 28 by 28 weights from the MNIST input images.
What a neural network learns during training. Video 1. HuCE - cpvrLab 2022-09-15 | This video shows what a shallow neural network learns in its weights to 100 neurons during 30 epochs of training. Each square shows the 28 by 28 weights from the MNIST input images.
What a neural network learns during training. Video 2 HuCE - cpvrLab 2022-09-15 | This video shows what a shallow neural network learns in its weights to 100 neurons during 30 epochs of training. Each square shows the 28 by 28 weights from the MNIST input images.
Stratoon Highlight Cut (Stratospheric Balloon Flight Project) HuCE - cpvrLab 2022-07-20 | Bachelor Thesis Project Stratoon of the students Oliver Gafner, Frederik Heck, and Yannik Stuker (Summer 2020). See also https://stratoon.ch
Particle System in OpenGL HuCE - cpvrLab 2022-06-30 | This is the video from the bachelor thesis of Marc Affolter at the Berne University of Applied Sciences in Switzerland. The particle system is GPU-based. I.e., the particles are updated and drawn on the GPU. To remain on GPU, a technology called transform feedback was used.The particle system was integrated into the SLProject framework (github.com/cpvrlab/SLProject).
Mobile App DYO Watch Configurator HuCE - cpvrLab 2022-06-24 | The project was a feasibility study for an augmented reality watch configurator app with a mechanical eta-6497 watch movement. The project was realized with Unity3D and OpenCV.
White Blood Cell Classification HuCE - cpvrLab 2022-06-20 | This is the video from the bachelor thesis from Patrick Oktay at the Berne University of Applied Sciences in Switzerland. The goal was to count and classify the three types of white blood cells in a video stream.
Robot programming by demonstration HuCE - cpvrLab 2022-03-04 | Robot programming is often laborious for tasks with open trajectories such as gluing or soldering since it is difficult and time-consuming to mathematically describe the trajectory. To simplify the programming, the 6D-tracked pen can be used to teach the trajectory by demonstration, that is, by directly doing the task on the workspace.
Non Euclidean Geometry Rendering HuCE - cpvrLab 2022-02-03 | Dies ist das Video der Bachelorarbeit von Kevin Riesen im Herbstsemester 2021 an der Berner Fachhochschule in der Schweiz.Die Applikation verbindet einen Mixed-Reality Ansatz mit verschiedenen nicht euklidischen Spielelementen um damit ein einzigartiges Erlebnis zu kreieren. Hierfür wird ein Würfel mittels AruCo Markern getrackt und in die virtuelle Welt übertragen. Die nicht euklidischen Spielelemente reizen dabei mit Hilfe des Stencil-Buffers die Möglichkeiten der Game Engine Unity aufs Äusserste.Das Video wurde in Zusammenarbeit mit Stiuvou GmbH produziert.
Near Real-time Weather Rendering System HuCE - cpvrLab 2022-02-02 | This is the video from the bachelor thesis from Matthias Thomann in Spring 2021 at the Berne University of Applied Sciences. Clouds contribute a substantial part to the overall ambiance in games, but an implementation of such effects often proves to be more challenging than anticipated. To get as close as possible to real clouds, this project engages in researching and developing a near real-time weather rendering system. This means that real weather forecasts from meteoblue are usedto visualize past, current, and forecast weather at any given time of day. The environment is created with elevation model data from ArcGIS. Live photographs from Roundshot cameras can be viewed side-by-side with the rendered output for comparison.The bachelor thesis dives into the science of clouds and illustrates the ten distinct classifications and how each of those could be represented in a weather simulation. In order to achieve high fidelity, the implementation relies on concepts like Voronoi noise generation and raymarching, which means generating a random 3D cloud pattern and rendering it volumetrically.At last, the goal of the project is to create a fully-featured, near-real-time weather rendering system in Unity. It is able to render procedural and volumetric cloudscapes, for any given date and time. An intuitive user interface allows the user to control the weather simulation manually or let it run automatically based on meteoblue weather reports.The achieved solution shows great results and fulfills almost all specified requirements. It is only missing the cirrus clouds but features shadow casting and a rain particle system, which were both not originally planned. For future work, the weather rendering system could be incorporated in a game or further improved to achieve even higher visual realism.
Beamed Interactive Game HuCE - cpvrLab 2022-01-27 | Das ist das Projektvideo der Bachelor Thesis von Luca Berger und Christian Nussbaum im Herbstsemester 2021 an der Berner Fachhochschule. Beamed Interactive Game ist eine Spielesammlung, welche sich über das Werfen von Bällen steuern lässt. So wird an der Aufprallstelle des Balles eine Interaktion im jeweiligen Spiel ausgelöst. Architekturmässig wurde die Applikation in zwei Teilsysteme aufgeteilt: Den Detektionsmechanismus (C++), welcher für die Ball- und Treffererkennung zuständig ist, sowie die Spielesammlung selbst (Unity), welche die einzelnen Spiele enthält.
Swiss Sailing VR HuCE - cpvrLab 2022-01-26 | This is the video from the bachelor thesis from Damian Schüpbach from the Berne University of Applied Sciences in Switzerland. His project is virtual reality visualization of a regatta of the 49er boat class. The user, who operates the corresponding VR headset, can sit on any boat in the regatta and follow the competition from their perspective. So that the regatta can be reproduced as realistically as possible, GPS data of the sailing boats, which were recorded at the time of the competition, are used. However, since the position data is not recorded continuously, but only about every 1-3 seconds, the Catmull-Rom algorithm is used to interpolate between the points in order to create a refined path. The product is implemented in the game engine Unity.
Together in VR: The Bachelor Thesis from Diego Omini. HuCE - cpvrLab 2022-01-26 | This is the video from the bachelor thesis from Diego Omini at the Berne University of Applied Sciences in the winter semester of 2021. Virtual reality experiences are mostly limited to a single person playing either alone or with online multiplayer. In this project, the student developed an application in Unity that allows multiple people that are in the same physical space to be also correctly positioned in the same virtual world. The project shows also how to teleport a group of users so that they can explore the virtual environment together. In this example video, they explore the theatre and temple of Augusta Raurica, the biggest Roman city in Switzerland.
Billiard-AI (Bachelor Thesis Fall 2021) HuCE - cpvrLab 2022-01-20 | Billiard-AI is the bachelor thesis of Luca Ritz and Lukas Seglias in Computer Science at the Berne University of Applied Sciences in Switzerland. Billiard-AI is a system in which a camera above the pool table continuously takes pictures and the position and color of all balls are recognized in real-time. The player is shown the ball positions and tips for billiard shots directly on the table via a projector. Possible shots are found by analyzing the game situation and their difficulty is evaluated using various criteria. Both direct impacts and impacts over the rails are taken into account and the required speed of the cue ball is calculated. The simplest and most promising ones are suggested. A physics simulation calculates the game situation after the shot. The path of each ball involved is shown by lines on the table and the process is visualized by an animation of the projected balls. This makes it clear where which balls collide, are pocketed, or come to a standstill. The player can align the billiard cue (stick) along the displayed lines and follow the animated speed so that the shot is successful.
Libellley 360° HuCE - cpvrLab 2021-11-18 | Libell'ley 360° is a 360° VR video shot with the Insta 360 One camera mounted under a drone. You can view this video also in a VR headset.
Erleb-AR Teaser Bern HuCE - cpvrLab 2021-11-17 | Mit der Erleb-AR App können Sie die Römerstadt Aventicum und Augusta Raurica erkunden und den Christoffelturm an seinem ursprünglichen Standort bestaunen. Vieles vom reichen Schweizer Kulturerbe ist heute nur noch teilweise in Form von Ruinen oder archäologischen Funden vorhanden oder ganz verschwunden. Neue Technologien ermöglichen es, Kamerabilder mit Modellen von historischen Gebäuden, Objekten und Personen zu verschmelzen. Mit der so angereicherten Realität ("Augmented Reality - AR") ist man nicht mehr allein auf die eigene Vorstellungskraft angewiesen, sondern wird am Ort des Geschehens in die Vergangenheit zurückversetzt. Die Erleb-AR App wurde am Institut für Human Centered Engineering HuCE am Labor für Computerwahrnehmung und virtuelle Realität (cpvrLab) der Berner Fachhochschule mit grosszügiger finanzieller Unterstützung des Bundesamtes für Kultur BAK entwickelt.
GoalieVR - Ice Hockey Goalie Training in VR HuCE - cpvrLab 2021-07-12 | Video of the bachelor thesis of Sylvain Barthe and Sandro Scherrer of the Bern University of Applied Science.
Hololens-2 Render Comparison HuCE - cpvrLab 2021-06-18 | The left part was captured through the left waveguide display with a Samsung-20FE. The right part was captured with the center camera Hololens-2.
Erleb-AR: Explore Swiss Cultural Heritage with Outdoor AR HuCE - cpvrLab 2021-05-04 | With the Erleb-AR app you can explore the Roman city of Aventicum and Augusta Raurica and marvel at the Christoffel tower at its original site. Much of the rich Swiss cultural heritage is only partially available today, in the form of ruins or archaeological findings, or it has completely disappeared. New emerging technologies make it possible to merge camera images with models of historical buildings, objects and people. With reality enriched in this way ("Augmented Reality - AR") one no longer has to rely solely on their own imagination, but one is transported back to the past at the scene of the event. The Erleb-AR app was developed at the Institute for Human Centered Engineering HuCE at the laboratory for computer perception and virtual reality (cpvrLab) of the Bern University of Applied Sciences with generous financial support from the Federal Office of Culture FOC.
Project Erleb-AR Models. HuCE - cpvrLab 2021-04-29 | These are the models shown in the outdoor augmented reality app called Erleb-AR (https://web.erleb-ar.bfh.science). This screen capture was taken from the desktop development app. The blueish background is the video from the webcam covered with a blueish scotch tape.
Multi-modal Human-Machine Interaction for a Sensor Packaging Task (True Cobotics Project) HuCE - cpvrLab 2021-03-15 | Thanks to our multi-modal interface, a user can program a UR3 robot using the natural language or intuitive gestures for a sensor packaging task. Want to try yourself ? https://urfree-robot.web.appThis work is part of the True Cobotics Project, where an architecture has been developed to allow intuitive interactions between the user and the machine. The same architecture can be used for tasks as diverse as : packaging sensors, bringing drinks in an elderly home or emptying a dishwasher. The project was funded by Innosuisse and included 3 research partners: the Bern University of Applied Sciences (robotics), HE-Arc (AI) and the Swiss Smart Factory (AR) and 3 industrial partners: F&P robotics, nViso and mimacom.
Rolex4K in Meshroom HuCE - cpvrLab 2020-12-08 | A short tutorial on how to create a 3D mesh from a video using the free software ffmpeg (ffmpeg.org) and Meshroom.(https://alicevision.org/).
Automatic Video Object Removal HuCE - cpvrLab 2020-10-23 | This is the project video of the Master Thesis of Daniel Cameron from October 2020. In this project, a prototype video editing system based on “inpainting” is demonstrated. Inpainting is an image editing method for the replacement of masked regions of an image with a suitable background. The resulting video is thus free from the selected objects and more readily adaptable for further simulation. The system utilizes a three-step approach to simulation: (1) detection, (2) mask grouping, and (3) inpainting. The detection step involves the identification of objects within the video-based upon a given object class definition and the production of pixel-level masks. Next, the object masks are grouped and tracked through the frame sequence to determine persistence and allow correction of classified results. Finally, the grouped masks are used to target specific objects instances in the video for inpainting removal.The end result of this project is a video editing platform in the context of locomotive route simulation. The final video output demonstrates the system’s ability to automatically remove moving pedestrians in a video sequence, which commonly occurs in most street tram simulations. This work also addresses the limitations of the system, in particular, the inability to remove quasi-stationary objects. The overall outcome of the project is a video editing system with automation capabilities equal to or surpassing most commercial inpainting software.
Image Transformation Learning for Drone Navigation without GPS HuCE - cpvrLab 2020-09-18 | In this video, we explain the image transformation learning, that we use for transforming drone video images into an abstract, map style images that can be used for drone pose estimation without GPS (as can be seen in an extra video: youtu.be/ilIBzMu8QDY)The Swiss Federal Office of Topography, swisstopo, provides to us maps in two styles: an orthographic projection of aerial photographs (referred to as ortho map) and its stripped-down abstractions, well known as the 1:25’000 maps (referred as the abstract map). Our approach to the pose estimation problem is to match the camera image on such a swisstopo map.
AR for Video-based Train Simulation HuCE - cpvrLab 2020-07-09 | Concept video for Augmented Reality in Video-based Train Simulation
