Uploaded May 2024 | Updated November 2026, 20 minutes ago
This is the accompanying video of our 2024 IEEE International Conference on Robotics and Automation (ICRA) paper "Watching the Air Rise: Learning-Based Single-Frame Schlieren Detection".
Abstract:
Detecting air flows caused by phenomena such as heat convection is valuable in multiple scenarios, including leak identification and locating thermal updrafts for extending UAV flight duration. Unfortunately, the heat signature of these flows is often too subtle to be seen by a thermal camera. While convection also leads to fluctuations in air density and hence causes so-called schlieren – intensity and color variations in images – existing techniques such as Background-oriented schlieren (BOS) allow detecting them only against a known background and from a static camera, making these approaches unsuitable for moving vehicles. In this work we demonstrate the feasibility of visualizing air movement by predicting the corresponding schlieren-induced optical flow from a single greyscale image captured by a moving camera against an unfamiliar background. We first record and label a set of optical flows in an indoor setup using standard BOS techniques. We then train a convolutional neural network (CNN) by applying the previously collected optical flow distortions to a dataset containing a mixture of real and synthetically generated images to predict the two-dimensional optical flow from a single image. Finally, we evaluate our approach on the task of extracting the optical flow caused by schlieren from both a static and moving camera on previously unseen flow patterns and background images.
This is the accompanying video of our 2024 IEEE International Conference on Robotics and Automation (ICRA) paper "Watching the Air Rise: Learning-Based Single-Frame Schlieren Detection".
Abstract:
Detecting air flows caused by phenomena such as heat convection is valuable in multiple scenarios, including leak identification and locating thermal updrafts for extending UAV flight duration. Unfortunately, the heat signature of these flows is often too subtle to be seen by a thermal camera. While convection also leads to fluctuations in air density and hence causes so-called schlieren – intensity and color variations in images – existing techniques such as Background-oriented schlieren (BOS) allow detecting them only against a known background and from a static camera, making these approaches unsuitable for moving vehicles. In this work we demonstrate the feasibility of visualizing air movement by predicting the corresponding schlieren-induced optical flow from a single greyscale image captured by a moving camera against an unfamiliar background. We first record and label a set of optical flows in an indoor setup using standard BOS techniques. We then train a convolutional neural network (CNN) by applying the previously collected optical flow distortions to a dataset containing a mixture of real and synthetically generated images to predict the two-dimensional optical flow from a single image. Finally, we evaluate our approach on the task of extracting the optical flow caused by schlieren from both a static and moving camera on previously unseen flow patterns and background images.
![Aerial Manipulation with an Omnidirectional Flying Platform
This video showcases our current research to address the challenges of aerial manipulation with omnidirectional flying robots at ETH Zurichs Autonomous systems lab. This topic connects several different topics of active research at our institute, including design and control of omnidirectional aerial manipulators, planning frameworks, and grasp detection.
For further details on the content of this video, links to the related research papers are listed below.
Paper References
[1] Allenspach, M., Bodie, K., Brunner, M., Rinsoz, L., Taylor, Z., Kamel, M., Siegwart, R. and Nieto, J., 2020. Design and optimal control of a tiltrotor micro-aerial vehicle for efficient omnidirectional flight., The International Journal of Robotics Research, 39(10-11), pp.1305-1325.
link: https://arxiv.org/pdf/2003.09512.pdf
[2] Zhang, W., Brunner, M., Ott, L., Kamel, M., Siegwart, R. and Nieto, J., 2020. Learning dynamics for improving control of overactuated flying systems., IEEE Robotics and Automation Letters, 5(4), pp.5283-5290.
link: https://arxiv.org/pdf/2006.13153.pdf
[3] Zhang, W., Ott, L., Tognon, M., Siegwart, R. and Nieto, J., 2021. Active Model Learning using Informative Trajectories for Improved Closed-Loop Control on Real Robots. arXiv preprint arXiv:2101.08100.
link: https://arxiv.org/pdf/2101.08100.pdf
[4] Bodie, K., Brunner, M., Pantic, M., Walser, S., Pfändler, P., Angst, U., Siegwart, R. and Nieto, J., 2019. An omnidirectional aerial manipulation platform for contact-based inspection., Robotics: Science and System XV, 15.
link: https://arxiv.org/pdf/1905.03502.pdf
[5] Bodie, K., Tognon, M. and Siegwart, R. Dynamic End Effector Tracking with an Omnidirectional Parallel Aerial Manipulator. (under review, link to follow)
[6] Breyer, M., Chung, J.J., Ott, L., Siegwart, R. and Nieto, J., 2021. Volumetric Grasping Network: Real-time 6 DOF Grasp Detection in Clutter., arXiv preprint arXiv:2101.01132.
link: https://arxiv.org/pdf/2101.01132.pdf
[7] Pantic, M., Ott, L., Cadena, C., Siegwart, R. and Nieto, J., 2021. Mesh Manifold based Riemannian Motion Planning for Omnidirectional Micro Aerial Vehicles., IEEE Robotics and Automation Letters, 6(3), pp.4790-4797.
link: https://arxiv.org/pdf/2102.10313.pdf
[8] Allenspach, M., Lawrance, N. , Tognon, M. and Siegwart, R. 6 DOF Teleoperation of an Omnidirectional Aerial Vehicle. (under review, link to follow) Aerial Manipulation with an Omnidirectional Flying Platform](https://i.ytimg.com/vi/njlyQnRwNLs/mqdefault.jpg)




![Visual-inertial teach and repeat powered by Google Tango
Visual-Inertial Teach and Repeat for Aerial Inspection
M Fehr, T Schneider, M Dymczyk, J Sturm, R Siegwart
Arxiv: https://arxiv.org/abs/1803.09650
Submitted to:
Workshop on Aerial Robotic Inspection and Maintenance
International Conference on Robotics and Automation 2018
Related Publications:
Real-time visual-inertial mapping, re-localization and planning onboard MAVs in unknown environments
M Burri, H Oleynikova, MW Achtelik, R Siegwart
A robust and modular multi-sensor fusion approach applied to mav navigation
S Lynen, MW Achtelik, S Weiss, M Chli, R Siegwart
Linear vs Nonlinear MPC for Trajectory Tracking Applied to Rotary Wing Micro Aerial Vehicles
M Kamel, M Burri, R Siegwart
Related Videos:
https://www.youtube.com/watch?v=tuUMwcTJx8s
https://www.youtube.com/watch?v=Ot76i9p2ZZo
https://www.youtube.com/watch?v=lrGSwAPzMOQ
Contact:
marius.fehr[at]mavt.ethz.ch
thomas.schneider[at]mavt.ethz.ch Visual-inertial teach and repeat powered by Google Tango](https://i.ytimg.com/vi/pDlQXsOrgI4/mqdefault.jpg)




