Visual Saliency-aware Receding Horizon Autonomous Exploration with Application to Aerial Robotics Kostas Alexis 2026-09-27 | Visual Saliency-aware Receding Horizon Autonomous Exploration with Application to Aerial Robotics
Degradation Resilient LiDAR-Radar-Inertial Odometry Kostas Alexis 2024-01-29 | Enabling autonomous robots to operate robustly in challenging environments is necessary in a future with increased autonomy. For many autonomous systems, estimation and odometry remains a single point of failure, from which it can often be difficult, if not impossible, to recover. As such robust odometry solutions are of key importance. In this work a method for tightly-coupled LiDAR-Radar-Inertial fusion for odometry is proposed, enabling the mitigation of the effects of LiDAR degeneracy by leveraging a complementary perception modality while preserving the accuracy of LiDAR in well-conditioned environments. The proposed approach combines modalities in a factor graph-based windowed smoother with sensor information-specific factor formulations which enable, in the case of degeneracy, partial information to be conveyed to the graph along the non-degenerate axes. The proposed method is evaluated in real-world tests on a flying robot experiencing degraded conditions including geometric self-similarity as well as obscurant occlusion. For the benefit of the community we release the datasets presented: github.com/ntnu-arl/lidar_degeneracy_datasets
ARL | Open-Source Contributions 2023 Kostas Alexis 2023-12-30 | Celebrating the end of the year with a quick outline of our open-source contributions for the year!Repositories:* github.com/ntnu-arl/aerial_gym_simulator* github.com/ntnu-arl/ORACLE* github.com/ntnu-arl/sevae* github.com/ntnu-arl/trace_paw* github.com/ntnu-arl/vectornav* github.com/ntnu-arl/underwater-datasets* github.com/ntnu-arl/lidar_degeneracy_datasets
ARL | Open-Source Contributions 2023 Kostas Alexis 2023-12-29 | Celebrating the end of the year with a quick outline of our open-source contributions for the year!Slight video correction (updating one experiment): youtu.be/azX70eZ8QLw This video will stay unlisted for a while with a link to the new one.
An Online Self-calibrating Refractive Camera Model with Application to Underwater Odometry Kostas Alexis 2023-11-03 | This work presents a camera model for refractive media such as water and its application in underwater visual-inertial odometry. The model is self-calibrating in real-time and is free of known correspondences or calibration targets. It is separable as a distortion model (dependent on refractive index n and radial pixel coordinate) and a virtual pinhole model (as a function of n). We derive the self-calibration formulation leveraging epipolar constraints to estimate the refractive index and subsequently correct for distortion. Through experimental studies using an underwater robot integrating cameras and inertial sensing, the model is validated regarding the accurate estimation of the refractive index and its benefits for robust odometry estimation in an extended envelope of conditions. Lastly, we show the transition between media and the estimation of the varying refractive index online, thus allowing computer vision tasks across refractive media.
Model Predictive Attitude Control of a Jumping-and-Flying Quadruped for Planetary Exploration Kostas Alexis 2023-10-06 | Exploration of new planetary environments necessitates the development of novel concepts of locomotion capable of overcoming the potential challenges present on their surface or even in large-scale subsurface voids such as lava tubes. Planets such as Mars present terrain geometries - both on the surface and underground - that may involve steep and tall obstacles or wide but sharp ditches. Walking robots have recently attracted significant attention owing to their unique ability to conquer perilous terrain as compared to wheeled or tracked systems. However, walking alone may not suffice against such challenging terrain profiles. Simultaneously, autonomous navigation over the largely unknown worlds of other planets can often benefit from high vantage points to acquire informative sensor data and thus better plan future steps. Responding to this need, this work concentrates on the core technologies necessary for developing an autonomous jumping quadruped tailored to having the ability to jump for multiple meters high and thus being able to efficiently overcome major obstacles while also being able to benefit from "bird's eye views" to plan its path ahead. The focus is especially on planets with reduced gravity such as Mars. Within this broader scope, this work introduces the design of a quadruped capable of high jumping and thus short-period flight and specifically contributes the design, implementation, and simulation-based testing of a hierarchical controller for a legged robot that facilitates control of its attitude during its flight phase. The proposed flight controller for quadruped systems utilizes model predictive control principles to manipulate leg actuators and generate appropriate torques to achieve the desired attitude setpoint mid-flight during a jump. The controller is implemented in simulation and tested on a high-fidelity quadruped model considering both the kinematics and dynamics of the platform, as well as the dynamics of the motors.
Design and Experimental Verification of a Jumping Legged Robot for Martian Lava Tube Exploration Kostas Alexis 2023-10-05 | Unlisted video for our work on "Design and Experimental Verification of a Jumping Legged Robot for Martian Lava Tube Exploration"Summary: The potential of Martian lava tubes for resource extraction and habitat sheltering highlights the need for robots capable to undertake the grueling task of their exploration. Driven by this motivation, in this work we introduce a legged robot system optimized for jumping in the low gravity of Mars, designed with leg configurations adaptable to both bipedal and quadrupedal systems. This design utilizes torque-controlled actuators coupled with springs for high-power jumping, robust locomotion, and an energy-efficient resting pose. Key design features include a 5-bar mechanism as leg concept, combined with springs connected by a high-strength cord. The selected 5-bar link lengths and spring stiffness were optimized for maximizing the jump height in Martian gravity and realized as a robot leg. Two such legs combined with a compact body allowed jump testing of a bipedal prototype. The robot is 0.472 m tall and weighs 7.9 kg. Jump testing with significant safety margins resulted in a measured jump height of 1.141 m in Earth's gravity, while a total of $4$ jumping experiments are presented. Simulations utilizing the full motor torque and kinematic limits of the design resulted in a maximum possible jump height of 1.52 m in Earth's gravity and 3.63 m in Mars' gravity, highlighting the versatility of jumping as a form of locomotion and overcoming obstacles in lower gravity.
ORACLE Library of Deep Learning-based Safe Navigation Methods: Indicative Results Kostas Alexis 2023-09-28 | We *open-source* the ORACLE library of methods on deep learned collision-free navigation of aerial robots that assume a) no access to a map of the environment or an estimate of the robot’s position, and presents robust sim2real transfer. ORACLE enables safe uncertainty-aware flight, while its visually-attentive variant (A-ORACLE) combines that capacity with implicit information sampling, and seVAE-ORACLE alters the architecture to offer modularization and partial training on both synthetic and real data (if available). For navigation, the method(s) only consider the current depth image (e.g., from an RGB-D or a passive stereo camera) and a partial state estimate (linear velocities, roll/pitch, angular rate around the z-axis), alongside a bearing vector to the desired direction. The two core navigation methods, ORACLE and seVAE-ORACLE focus either on a) uncertainty-awareness by considering the robot’s partial state estimate uncertainty and epistemic uncertainty over the neural net through a deep ensemble, or b) modularized encoding of the high-dimensional depth data through a semantically-enhanced variational autoencoder – allowing to retain information over hard-to-perceive obstacles in aggressively compressing latent spaces - before the collision prediction step. A-ORACLE uses also a mask over the depth image that highlights visually-attentive regions in order to select the action primitive that not only allows the robot to navigate the environment safely but also to attend (using its camera) to salient regions either from a bottom-up or a top-down perspective (depending how the mask is derived).Open-Source code access: github.com/ntnu-arl/ORACLE Detailed wiki: github.com/ntnu-arl/ORACLE/wiki
ORACLE & A-ORACLE - How it Works Kostas Alexis 2023-09-26 | Functionality overview presentation for the ORACLE and Attentive ORACLE methods. Open-source at: github.com/ntnu-arl/ORACLE
Reinforcement Learning for Collision-free Flight Exploiting Deep Collision Encoding Kostas Alexis 2023-09-22 | This work contributes a novel deep navigation policy that enables collision-free flight of aerial robots based on a modular approach exploiting deep collision encoding and reinforcement learning. The proposed solution builds upon a deep collision encoder that is trained on both simulated and real depth images using supervised learning such that it compresses the high-dimensional depth data to a low-dimensional latent space encoding collision information while accounting for the robot size. This compressed encoding is combined with an estimate of the robot's odometry and the desired target location to train a deep reinforcement learning navigation policy that offers low-latency computation and robust sim2real performance. A set of simulation and experimental studies in diverse environments are conducted and demonstrate the efficiency of the emerged behavior and its resilience in real-life deployments.
Autonomous Under Canopy Navigation and Mapping in Dense Forests (DigiForest @ Evo, Finland) Kostas Alexis 2023-05-15 | In this video we present results from the recent field-testing campaign of the DigiForest project at Evo, Finland. The DigiForest project started in September 2022 and runs up to February 2026. It brings together diverse partners working on aerial robots, walking robots, autonomous lightweight harvesters, as well as forestry decision makers and commercial companies with the goal to create a full data pipeline for digitized forestry. During this field campaign, our lab tested two distinct autonomy stacks onboard flying robots performing under-canopy navigation, exploration and mapping. First, we tested our traditional autonomy pipeline involving onboard SLAM, volumetric mapping and path planning on such maps. Our focus was to test performance against different tree distributions. Second, we tested two variations of a new learning-based navigation approach that calculates collision-free actions without assuming access to any form of consistent map or even position estimates. Instead, an appropriately trained neural network uses a partial estimate of the robot state (including roll and pitch angles, yaw rate and linear velocities), as well as the associated covariance and the immediate depth image from an RGBD sensor to derive an admissible motion primitive in velocity space that flies towards a desired direction in a collision-free manner. These results are preliminary and one of the future goals will be to enable safe fast flight in perceptually-degraded dense forests involving complex distributions of thin hard-to-perceive branches.
Autonomous Exploration of Ballast Water Tank with Navigation through Manholes Kostas Alexis 2023-05-08 | This video presents results on autonomous exploration of multiple ballast water tank compartments inside an FPSO vessel. The autonomous mission necessitated the traversal of several manholes with dimensions as narrow as 0.7 by 0.5 meters. To execute the mission, the robot performs simultaneous localization and mapping, represents the world using voxels thus allowing it to conduct volumetric calculations, plans exploratory paths within each compartment, and detects and localizes the manholes within them. Detection of manholes takes place on point cloud data and is necessary since due to their constrained size, autonomous traversal across them merely by means of sampling paths in the volumetric map is unlikely. The robot subsequently utilizes such detections to automatically traverse across compartments. Overall, in this mission, the aerial robot explored six ballast tank compartments and returned back to its original take-off location.
Autonomous Exploration and Visual Inspection of Ballast Water Tank in an FPSO Kostas Alexis 2023-04-21 | This video presents results on autonomous exploration and visual inspection of a ballast tank inside an FPSO vessel. Specifically, RMF – a collision tolerant aerial robot implementing multi-modal SLAM and path planning functionality – is deployed inside the ballasts of the vessel and performs the autonomous inspection of 3 tank compartments without any prior knowledge of the environment other than a rough estimate of the geometric midpoint of each compartment. Such information is readily available and does not require access to hard-to-access CAD models of ships. The mission takes place in less than 4 minutes and ensures both the geometric mapping of those compartments and their visual inspection with certain resolution guarantees. In further detail, the process starts with the robot in the first compartment where it performs autonomous exploration to map it geometrically with its LiDAR. It then calculates a path to inspect all the mapped surfaces visually subject to a minimum resolution of observation. Upon completion of this process in one compartment, the method proceeds to the next compartment and the whole procedure is iteratively repeated. After geometrically exploring and visually inspecting a given number of compartments, the system autonomously returns to the deployment point. This result is part of our effort to ensure semantics-aware autonomous missions that ensure first the exploration and then the detailed visual inspection of structures of interest inside ballast tanks and other key areas of maritime vessels or broadly industry facilities.
Semantically-enhanced Deep Collision Prediction for Autonomous Navigation using Aerial Robots Kostas Alexis 2023-03-05 | This work contributes a novel and modularized learning-based method for aerial robots navigating cluttered environments containing hard-to-perceive thin obstacles without assuming access to a map or the full pose estimation of the robot. The proposed solution builds upon a semantically-enhanced Variational Autoencoder that is trained with both real-world and simulated depth images to compress the input data, while preserving semantically-labeled thin obstacles and handling invalid pixels in the depth sensor's output. This compressed representation, in addition to the robot's partial state involving its linear/angular velocities and its attitude are then utilized to train an uncertainty-aware 3D Collision Prediction Network in simulation to predict collision scores for candidate action sequences in a predefined motion primitives library. A set of simulation and experimental studies in cluttered environments with various sizes and types of obstacles, including multiple hard-to-perceive thin objects, were conducted to evaluate the performance of the proposed method and compare against an end-to-end trained baseline. The results demonstrate the benefits of the proposed semantically-enhanced deep collision prediction for learning-based autonomous navigation.Functionality overview video: youtu.be/yoO5MqSPfKw
Semantically-enhanced Deep Collision Prediction for Autonomous Navigation - How it Works Kostas Alexis 2023-03-05 | This work contributes a novel and modularized learning-based method for aerial robots navigating cluttered environments containing hard-to-perceive thin obstacles without assuming access to a map or the full pose estimate of the robot.Video with experimental results: youtu.be/Ni4VywUQCPw
Manhole Detection and Traversal for Exploration of Ballast Water Tanks using Micro Aerial Vehicles Kostas Alexis 2023-02-10 | This work presents a method for the autonomous exploration of multiple compartments of a Ballast Water Tank inside a vessel using Micro Aerial Vehicles. Navigation across the compartments of ballast tanks often requires the robot to pass through narrow cross-section ``manholes'' (e.g., 0.8 x 0.6m. Hence, this work presents an algorithm to explicitly detect and localize such manholes using 3D LiDAR data and a strategy to reliably navigate through them to enable autonomous exploration of multiple compartments of the tank. Two ablation studies are presented analyzing the effective 3D space with respect to the manhole in which reliable detection takes place. Furthermore, the method is evaluated onboard a collision-tolerant aerial robot in two autonomous exploration experiments in relevant mock-up scenarios.
Semantics-aware Exploration and Inspection Path Planning Kostas Alexis 2022-09-23 | This work contributes a novel strategy for semantics-aware autonomous exploration and inspection path planning. Attuned to the fact that environments that need to be explored often involve a sparse set of semantic entities of particular interest, the proposed method offers volumetric exploration combined with two new planning behaviors that together ensure that a complete mesh model is reconstructed for each semantic, while its surfaces are observed at appropriate resolution and through suitable viewing angles. Evaluated in extensive simulation studies and experimental results using a flying robot, the planner delivers efficient combined exploration and high-fidelity inspection planning that is focused on the semantics of interest. Comparisons against relevant methods of the state-of-the-art are further presented.
Factor Graph-based Tightly-coupled LiDAR-Inertial SLAM Kostas Alexis 2022-09-23 | This work presents a method for tightly-coupled LiDAR-inertial SLAM utilizing factor graphs as the underlying representation. The method extracts LOAM-style features and performs a scan to scan registration step, while also incorporating preintegrated IMU constraints within its optimization. A scan to submap registration step follows, which instead of using a monolithic map, utilizes a dynamic submap built using spatial keyframes over a sliding window, thus enabling past registration errors (within the keyframe window) to be corrected given new observations. Incremental fixed lag smoothing is performed over the maintained factor graphs in the individual scan to scan and scan to submap optimization steps to ensure bounded computational complexity. The method is evaluated on datasets collected with an aerial robot inside subterranean environments and an industrial facility.
Radar-Inertial ICP-based Pose Graph SLAM Kostas Alexis 2022-09-23 | A plethora of localization solutions for different sensor modalities exists, yet robust sensing in degraded perception conditions remains an open challenge. For example, the perception sensors most typically used in mobile robotics, like visible-light cameras and LiDARs, can face significant problems when operating in environments with dense obscurants (e.g., fog, smoke or dust). However, millimeter wave radars offer an avenue to penetrate such conditions and overcome these limitations. Motivated by the above, in this work we propose a new radar-inertial SLAM method that utilizes doppler velocity-based radar-inertial odometry combined with aggregate radar point cloud registration on both short- and long-term (loop closure) associations. Experimental evaluations are conducted on both publicly available and self-collected datasets and allow to demonstrate the performance of the method.
IROS2022 Talk:MIMOSA: A Multi-Modal SLAM Framework for Resilient Autonomy against Sensor Degradation Kostas Alexis 2022-09-23 | IROS 2022 Talk for our work on "MIMOSA: A Multi-Modal SLAM Framework for Resilient Autonomy against Sensor Degradation".Abstract: This paper presents a framework for Multi-Modal SLAM (MIMOSA) that utilizes a nonlinear factor graph as the underlying representation to provide loosely-coupled fusion of any number of sensing modalities. Tailored to the goal of enabling resilient robotic autonomy in GPS-denied and perceptually-degraded environments, MIMOSA currently contains modules for pointcloud registration, fusion of multiple odometry estimates relying on visible-light and thermal vision, as well as inertial measurement propagation. A flexible backend utilizes the estimates from various modalities as relative transformation factors. The method is designed to be robust to degeneracy through the maintenance and tracking of modality-specific health metrics, while also being inherently tolerant to sensor failure. We detail this framework alongside our implementation for handling high-rate asynchronous sensor measurements and evaluate its performance on data from autonomous subterranean robotic exploration missions using legged and aerial robots.
IROS2022 Talk: Risk-Aware Motion Planning for Collision-Tolerant Aerial Robots s.t. Uncertainty Kostas Alexis 2022-09-22 | Talk for our paper on "Risk-Aware Motion Planning for Collision-Tolerant Aerial Robots Subject to Localization Uncertainty"Abstract: This paper contributes a novel strategy towards risk-aware motion planning for collision-tolerant aerial robots subject to localization uncertainty. Attuned to the fact that micro aerial vehicles are often tasked to navigate within GPS-denied, possibly unknown, confined and obstacle-filled environments the proposed method exploits collision-tolerance at the robot design level to mitigate the risks of collisions especially as their likelihood increases with growing uncertainty. Accounting for the maximum kinetic energy with which an impact is considered safe, alongside the robot dynamics, the planner builds a set of admissible uncertainty-aware and collision-inclusive paths over a horizon involving multiple motion steps. The first step of the best path is executed by the robot, while the procedure is then repeated in a receding horizon manner. Evaluated in extensive simulation studies and experimental results with a collision-tolerant flying robot, the planner successfully considers the interplay between uncertainty and the likelihood of a collision, balances the risks of possible impacts and enables to navigate safely within highly cluttered environments.
Autonomous Exploration and Inspection inside Industrial Tank Environments Kostas Alexis 2022-07-27 | In this experiment, we demonstrate a combined exploration and inspection mission using the RMF-Owl collision tolerant aerial robot inside the Nutec RelyOn facilities in Trondheim, Norway. The robot is tasked to autonomously explore and inspect the surfaces of the environment - within a height boundary - with its onboard camera sensor given no prior knowledge of the map. To accomplish its task, the system utilizes a modified version of our open-sourced Graph-Based Exploration Path Planner. The planner utilizes voxblox as the volumetric mapping framework for planning and periodically annotates the mapped voxels lying inside the camera frustum, while a depth sensor is used for volumetric exploration. The planner then calculates a path that maximizes the umount of unseen surface that will be seen by the camera. The work is part of our ongoing efforts for optimized exploration and inspection path planning inside vessel ballast and cargo tanks, as well as industrial process and storage tanks.
Team CERBERUS Wins the DARPA Subterranean Challenge Kostas Alexis 2022-06-06 | This article presents the CERBERUS robotic system-of-systems, which won the DARPA Subterranean Challenge Final Event in 2021. The Subterranean Challenge was organized by DARPA with the vision to facilitate the novel technologies necessary to reliably explore diverse underground environments despite the grueling challenges they present for robotic autonomy. Due to their geometric complexity, degraded perceptual conditions combined with lack of GPS support, austere navigation conditions, and denied communications, subterranean settings render autonomous operations particularly demanding. In response to this challenge, we developed the CERBERUS system which exploits the synergy of legged and flying robots, coupled with robust control especially for overcoming perilous terrain, multi-modal and multi-robot perception for localization and mapping in conditions of sensor degradation, and resilient autonomy through unified exploration path planning and local motion planning that reflects robot-specific limitations. Based on its ability to explore diverse underground environments and its high-level command and control by a single human supervisor, CERBERUS demonstrated efficient exploration, reliable detection of objects of interest, and accurate mapping. In this article, we report results from both the preliminary runs and the final Prize Round of the DARPA Subterranean Challenge, and discuss highlights and challenges faced, alongside lessons learned for the benefit of the community.Team Website: subt-cerberus.org
ICRA2022 Talk Autonomous Teamed Exploration of Subterranean Environments with Legged & Aerial Robots Kostas Alexis 2022-04-13 | ICRA 2022 Talk on our paper:Mihir Kulkarni, Mihir Rahul Dharmadhikari, Marco Tranzatto, Samuel Zimmermann, Victor Reijgwart, Paolo De Petris, Huan Nguyen, Nikhil Vijay Khedekar, Christos Papachristos, Lionel Ott, Roland Siegwart, Marco Hutter, Kostas Alexis, "Autonomous Teamed Exploration of Subterranean Environments using Legged and Aerial Robots", IEEE International Conference on Robotics and Automation (ICRA), May 23-27, 2022, Philadelphia (PA), USAAbstract: This paper presents a novel strategy for autonomous teamed exploration of subterranean environments using legged and aerial robots. Tailored to the fact that subterranean settings, such as cave networks and underground mines, often involve complex, large-scale and multi-branched topologies, while wireless communication within them can be particularly challenging, this work is structured around the synergy of an onboard exploration path planner that allows for resilient long-term autonomy, and a multi-robot coordination framework. The onboard path planner is unified across legged and flying robots and enables navigation in environments with steep slopes, and diverse geometries. When a communication link is available, each robot of the team shares submaps to a centralized location where a multi-robot coordination framework identifies global frontiers of the exploration space to inform each system about where it should re-position to best continue its mission. The strategy is verified through a field deployment inside an underground mine in Switzerland using a legged and a flying robot collectively exploring for $45\textrm{ min}$, as well as a longer simulation study with three systems.GBPlanner 2 Open-Source release: github.com/ntnu-arl/gbplanner_ros
ICRA2022 Talk: ORACLE - Motion Primitives-based Navigation Planning using Deep Collision Prediction Kostas Alexis 2022-04-13 | ICRA 2022 Talk on our paper:Huan Nguyen, Sondre Holm Fyhn, Paolo De Petris, Kostas Alexis, "Motion Primitives-based Navigation Planning using Deep Collision Prediction", IEEE International Conference on Robotics and Automation (ICRA), May 23-27, 2022, Philadelphia (PA), USAAbstract: This paper contributes a method to design a novel navigation planner exploiting a learning-based collision prediction network. The neural network is tasked to predict the collision cost of each action sequence in a predefined motion primitives library in the robot's velocity-steering angle space, given only the current depth image and the estimated linear and angular velocities of the robot. Furthermore, we account for the uncertainty of the robot's partial state by utilizing the Unscented Transform and the uncertainty of the neural network model by using Monte Carlo dropout. The uncertainty-aware collision cost is then combined with the goal direction given by a global planner in order to determine the best action sequence to execute in a receding horizon manner. To demonstrate the method, we develop a resilient small flying robot integrating lightweight sensing and computing resources. A set of simulation and experimental studies, including a field deployment, in both cluttered and perceptually-challenging environments is conducted to evaluate the quality of the prediction network and the performance of the proposed planner.
MIMOSA: A Multi-Modal SLAM Framework for Resilient Autonomy against Sensor Degradation Kostas Alexis 2022-03-08 | This work presents a framework for Multi-Modal SLAM (MIMOSA) that utilizes a nonlinear factor graph as the underlying representation to provide loosely-coupled fusion of any number of sensing modalities Tailored to the goal of enabling resilient robotic autonomy in GPS-denied and visually degraded environments, MIMOSA currently contains modules for pointcloud registration and fusion of odometry estimates relying on visible-light and thermal vision. A flexible backend utilizes the estimates from various modalities as relative transformation factors. The method is designed to be robust to degeneracy through the maintenance and tracking of modality-specific health metrics, while also being inherently tolerant to sensor failure.
Risk-aware Motion Planning for Collision-tolerant Aerial Robots subject to Uncertainty Kostas Alexis 2022-03-07 | This work contributes a novel strategy towards risk-aware motion planning for collision-tolerant aerial robots subject to localization uncertainty. Attuned to the fact that micro aerial vehicles are often tasked to navigate within GPS-denied, possibly unknown, confined and obstacle-filled environments the proposed method exploits collision-tolerance at the robot design level to mitigate the risks of collisions especially as their likelihood increases with growing uncertainty. Accounting for the maximum kinetic energy with which an impact is considered safe, alongside the robot dynamics, the planner builds a set of admissible uncertainty-aware and collision-inclusive paths over a horizon involving multiple motion steps. The first step of the best path is executed by the robot, while the procedure is then repeated in a receding horizon manner. Evaluated in extensive simulation studies and experimental results with a collision-tolerant flying robot, the planner successfully considers the interplay between uncertainty and the likelihood of a collision, balances the risks of possible impacts and enables to navigate safely within highly cluttered environments.
Team CERBERUS: DARPA Subterranean Challenge Technical Approach and Lessons Learned Kostas Alexis 2021-12-07 | Team CERBERUS' DARPA Subterranean Challenge Technical Approach and Lessons Learned. We outline our team's approach with respect to the robotic systems legged and flying mobility concepts, methods for resilient multi-modal and multi-robot localization and mapping, autonomy and especially exploration path planning, artifact detection and localization on the map, as well as communications and networking. Finally, we outline our competitive runs and results especially during the Prize Run of the DARPA Subterranean Challenge finals. Presenters:* Kostas Alexis* Marco Tranzatto* Shehryar Khattak* Mihir Dharmadhikari* Mihir Kulkarni* Samuel ZimmermannTeam CERBERUS info page: http://www.subt-cerberus.orgNote: There is a typo error in 26min: We mention 40,007 labels total because we also accounted for labels for the cellphone artifact. Visual detection was however not reliable thus the associated row ("Cellphone - 3556 labels") is not shown in the table.
Mihir Kulkarni: COHORT: CoOperation for HeterOgeneous Robot Teams Kostas Alexis 2021-12-03 | This is the master thesis presentation of Mihir Kulkarni outlining his contributions in multi-robot exploration, alongside developing simulation tools and contributing into the artifact detection pipeline and one of the robots of Team CERBERUS winning in the DARPA Subterranean Challenge.
Mihir Dharmadhikari: Autonomous Informative Path Planning for Aerial and Legged Robots Kostas Alexis 2021-11-30 | Master thesis presentation by Mihir Dharmadhikari, member of the Autonomous Robots Lab, on "Autonomous Informative Path Planning for Aerial and Legged Robots". The work has been core to our success in the DARPA Subterranean Challenge. The methods presented build on top of the extensive research conducted by Dr. Tung Dang, graduate of our laboratory.
Autonomous Distributed Radiation Field Characterization and Informative Planning: Experiment #3 Kostas Alexis 2021-10-24 | This work presents a strategy for field estimation and informative path planning towards autonomous mapping and radiological characterization of distributed gamma radiation fields within confined GPS-denied environments using aerial robots. First, an online distributed radiation field estimation and spectroscopic analysis framework is presented which combines sequentially acquired measurements to estimate both the field intensity and gradient and propagate the belief over the initially unknown map, while simultaneously classifying each map region with respect to its dominating isotope. As such a process depends on the quality of the acquired measurements and given the limited endurance of small flying robots, we further contribute an informative path planner responsible for iteratively guiding the robot towards the next-best radiation measurement location such that high estimation confidence is achieved in short time. A bifurcated global and local planning architecture is proposed enabling the robot to guide itself towards the radiologically most interesting areas quickly and acquire sufficient measurements within those. We further develop a tailor-made collision-tolerant micro flying robot that is equipped with a lightweight scintillator and silicon photomultiplier combination, alongside GPS-denied localization and mapping capabilities. A set of experimental studies are presented involving the autonomous characterization of distributed radiation fields containing live uranium ore and radium sources within GPS-denied industrial settings.Experiment #1: youtu.be/aXcC6SpczHQ Experiment #2: youtu.be/Jr0xnczLy3UExperiment #3: youtu.be/C1dFA2fanU4
Autonomous Distributed Radiation Field Characterization and Informative Planning: Experiment #2 Kostas Alexis 2021-10-24 | This work presents a strategy for field estimation and informative path planning towards autonomous mapping and radiological characterization of distributed gamma radiation fields within confined GPS-denied environments using aerial robots. First, an online distributed radiation field estimation and spectroscopic analysis framework is presented which combines sequentially acquired measurements to estimate both the field intensity and gradient and propagate the belief over the initially unknown map, while simultaneously classifying each map region with respect to its dominating isotope. As such a process depends on the quality of the acquired measurements and given the limited endurance of small flying robots, we further contribute an informative path planner responsible for iteratively guiding the robot towards the next-best radiation measurement location such that high estimation confidence is achieved in short time. A bifurcated global and local planning architecture is proposed enabling the robot to guide itself towards the radiologically most interesting areas quickly and acquire sufficient measurements within those. We further develop a tailor-made collision-tolerant micro flying robot that is equipped with a lightweight scintillator and silicon photomultiplier combination, alongside GPS-denied localization and mapping capabilities. A set of experimental studies are presented involving the autonomous characterization of distributed radiation fields containing live uranium ore and radium sources within GPS-denied industrial settings.Experiment #1: youtu.be/aXcC6SpczHQ Experiment #2: youtu.be/Jr0xnczLy3UExperiment #3: youtu.be/C1dFA2fanU4
Autonomous Distributed Radiation Field Characterization and Informative Planning: Experiment #1 Kostas Alexis 2021-10-24 | This work presents a strategy for field estimation and informative path planning towards autonomous mapping and radiological characterization of distributed gamma radiation fields within confined GPS-denied environments using aerial robots. First, an online distributed radiation field estimation and spectroscopic analysis framework is presented which combines sequentially acquired measurements to estimate both the field intensity and gradient and propagate the belief over the initially unknown map, while simultaneously classifying each map region with respect to its dominating isotope. As such a process depends on the quality of the acquired measurements and given the limited endurance of small flying robots, we further contribute an informative path planner responsible for iteratively guiding the robot towards the next-best radiation measurement location such that high estimation confidence is achieved in short time. A bifurcated global and local planning architecture is proposed enabling the robot to guide itself towards the radiologically most interesting areas quickly and acquire sufficient measurements within those. We further develop a tailor-made collision-tolerant micro flying robot that is equipped with a lightweight scintillator and silicon photomultiplier combination, alongside GPS-denied localization and mapping capabilities. A set of experimental studies are presented involving the autonomous characterization of distributed radiation fields containing live uranium ore and radium sources within GPS-denied industrial settings.Experiment #1: youtu.be/aXcC6SpczHQ Experiment #2: youtu.be/Jr0xnczLy3UExperiment #3: youtu.be/C1dFA2fanU4
Motion Primitives-based Navigation Planning using Deep Collision Prediction Kostas Alexis 2021-09-29 | This work contributes a method to design a novel navigation planner exploiting a learning-based collision prediction network. The neural network is tasked to predict the collision cost of each action sequence in a predefined motion primitives library in the robot's velocity-steering angle space, given only the current depth image and the estimated linear and angular velocities of the robot. Furthermore, we account for the uncertainty of the robot's partial state by utilizing the Unscented Transform and the uncertainty of the neural network model by using Monte Carlo dropout. The uncertainty-aware collision cost is then combined with the goal direction given by a global planner in order to determine the best action sequence to execute in a receding horizon manner. To demonstrate the method, we develop a resilient small flying robot integrating lightweight sensing and computing resources. A set of simulation and experimental studies, including a field deployment, in both cluttered and perceptually-challenging environments is conducted to evaluate the quality of the prediction network and the performance of the proposed planner.
Autonomous Teamed Exploration of Subterranean Environments using Legged and Aerial Robots Kostas Alexis 2021-09-12 | This paper presents a novel strategy for autonomous teamed exploration of subterranean environments using legged and aerial robots. Tailored to the fact that subterranean settings, such as cave networks and underground mines, often involve complex, large-scale and multi-branched topologies, while wireless communication within them can be particularly challenging, this work is structured around the synergy of an onboard exploration path planner that allows for resilient long-term autonomy, and a multi-robot coordination framework. The onboard path planner is unified across legged and flying robots and enables navigation in environments with steep slopes, and diverse geometries. When a communication link is available, each robot of the team shares submaps to a centralized location where a multi-robot coordination framework identifies global frontiers of the exploration space to inform each system about where it should re-position to best continue its mission. The strategy is verified through a field deployment inside an underground mine in Switzerland using a legged and a flying robot collectively exploring for more than 45min, as well as a longer simulation study with three systems. Open-source:* github.com/ntnu-arl/gbplanner_rosDetailed videos for each mission: * ANYmal C mission: youtu.be/puyRbPtPyXw* RMF-Obelix mission: youtu.be/F9udtbZFV-Q* COHORT/GBPlanner2 simulation: youtu.be/JYiujM2PVz8Videos explaining the methods used:* GBPlanner2: youtu.be/bTqFp1aODqU* COHORT: youtu.be/wGpbPmkLN40* COHORT/GBPlanner2 synergy: youtu.be/gbcof8KBTZEWebsite with up-to-date results and soon open-source code: autonomousrobotslab.com/exploration-planning.htmlThis work is the output of the collaboration within the CERBERUS team of the DARPA Subterranean Challenge and especially across the Autonomous Robots Lab (autonomousrobotslab.com), the Robotic Systems Lab (https://rsl.ethz.ch/), and the Autonomous Systems Lab (https://asl.ethz.ch/).
COHORT-GBPlanner2 Interfacing for Teamed Exploration - How It Works Kostas Alexis 2021-09-10 | In this video, we provide a functionality overview of how our Graph-Based exploration path Planner 2.0 (GBPlanner2) method and our CoOperation for HeterOgeneous Robot Teams (COHORT) multi-robot coordination system work together to enable teamed explorationRelevant results: * Legged robots: youtu.be/puyRbPtPyXw* Flying robots: youtu.be/F9udtbZFV-Q* youtu.be/JYiujM2PVz8
COHORT: CoOperation for HeterOgeneous Robot Teams - How it Works Kostas Alexis 2021-09-10 | In this video, we provide a functionality overview of our new CoOperation for HeterOgeneous Robot Teams (COHORT) method. Relevant result: youtu.be/JYiujM2PVz8
GBPlanner2: Graph-Based exploration path Planner 2.0 - How it Works Kostas Alexis 2021-09-10 | In this video, we provide a functionality overview of our new Graph-Based exploration path Planner 2.0 (GBPlanner2) method. Videos of experimental results with the method can be found at:* Legged robots: youtu.be/puyRbPtPyXw* Flying robots: youtu.be/F9udtbZFV-QThe previous version is already open-source: github.com/ntnu-arl/gbplanner_ros
Autonomous Teamed Exploration of Subterranean Environments - Simulation study Kostas Alexis 2021-09-10 | This video is part of our work on "Autonomous Teamed Exploration of Subterranean Environments using Legged and Aerial Robots"In this work we present a novel strategy for autonomous teamed exploration of subterranean environments using legged and aerial robots. Tailored to the fact that underground environments, such as cave networks and underground mines, often involve complex, large-scale and multi-branched topologies, while establishing a persistent communications network may not be possible, this work is structured around the synergy of an onboard exploration path planner that allows for resilient long-term autonomy, and a centralized multi-robot coordination framework. The onboard path planner is unified across legged and flying robots and enables navigation in environments with steep slopes, and diverse geometries. When a communication link is available, each robot of the team shares submaps to a centralized location where a multi-robot coordination framework identifies global frontiers of the exploration space to best inform each system where it should re-position to best continue its mission. The proposed strategy is verified through a field deployment inside an underground mine in Switzerland using a legged and a flying robot collectively exploring for more than $45\textrm{min}$, as well as a longer simulation study involving three systems.
Autonomous subterranean exploration with the RMF aerial robot inside the Hagerbach underground mine Kostas Alexis 2021-09-10 | This video is part of our work on "Autonomous Teamed Exploration of Subterranean Environments using Legged and Aerial Robots"In this work we present a novel strategy for autonomous teamed exploration of subterranean environments using legged and aerial robots. Tailored to the fact that underground environments, such as cave networks and underground mines, often involve complex, large-scale and multi-branched topologies, while establishing a persistent communications network may not be possible, this work is structured around the synergy of an onboard exploration path planner that allows for resilient long-term autonomy, and a centralized multi-robot coordination framework. The onboard path planner is unified across legged and flying robots and enables navigation in environments with steep slopes, and diverse geometries. When a communication link is available, each robot of the team shares submaps to a centralized location where a multi-robot coordination framework identifies global frontiers of the exploration space to best inform each system where it should re-position to best continue its mission. The proposed strategy is verified through a field deployment inside an underground mine in Switzerland using a legged and a flying robot collectively exploring for more than $45\textrm{min}$, as well as a longer simulation study involving three systems.
ICRA2021: Autonomous Distributed 3D Radiation Field Estimation for Nuclear Env. Characterization Kostas Alexis 2021-05-24 | Frank Mascarich, Paolo De Petris, Dinh Huan Nguyen, Nikhil Vijay Khedekar, Kostas Alexis, "Autonomous Distributed 3D Radiation Field Estimation for Nuclear Environment Characterization", IEEE International Conference on Robotics and Automation (ICRA), May 30-June 5, 2021, Xi'an China
ICRA2021: Hypergame-Based Adaptive Behavior Path Planning fr Combined Exploration & Visual Search Kostas Alexis 2021-05-23 | Mihir Rahul Dharmadhikari, Harshal Deshpande, Tung Dang, Kostas Alexis, "Hypergame-based Adaptive Behavior Path Planning for Combined Exploration and Visual Search", IEEE International Conference on Robotics and Automation (ICRA), May 30-June 5, 2021, Xi'an China
ICRA2021: Resilient Collision-Tolerant Navigation in Confined Environments Kostas Alexis 2021-05-23 | Paolo De Petris, Dinh Huan Nguyen, Mihir Kulkarni, Frank Mascarich, Kostas Alexis, "Resilient Collision-tolerant Navigation in Confined Environments", IEEE International Conference on Robotics and Automation (ICRA), May 30-June 5, 2021, Xi'an China
ICRA2021: Forceful Aerial Manipulation Based on an Aerial Robotic Chain:Hybrid Modeling & Control Kostas Alexis 2021-05-23 | Dinh Huan Nguyen, Kostas Alexis, "Forceful Aerial Manipulation based on an Aerial Robotic Chain: Hybrid Modeling and Control", IEEE Robotics and Automation Letters with IEEE ICRA 2021 presentation
Field Deployment inside the Løkken Mine Kostas Alexis 2021-03-22 | In this video we present results from a field deployment inside the Løkken Mine underground pyrite mine in Norway. The Løkken mine was operative from 1654 to 1987 and contains narrow but long corridors, alongside vast rooms and challenging vertical stopes. In this field study we evaluated selected autonomous exploration and visual search capabilities of a subset of the aerial robots of Team CERBERUS towards the goal of complete subterranean autonomy.SLAM Solution: CompSLAMS. Khattak, C. Papachristos, K. Alexis, "Keyframe-based Thermal-Inertial Odometry", Journal of Field Robotics, 2019, 1–28, doi.org/10.1002/rob.21932S. Khattak, D. H. Nguyen, F. Mascarich, T. Dang, and K. Alexis, "Complementary Multi–Modal Sensor Fusion for Resilient Robot Pose Estimation in Subterranean Environments", International Conference on Unmanned Aircraft Systems (ICUAS), Athens, Greece, 2020Open-Source Git Repo: TBAExploration Solution: GBPlanner T. Dang, M. Tranzatto, S. Khattak, F. Mascarich, K. Alexis, M. Hutter, "Graph-based Subterranean Exploration Path Planning using Aerial and Legged Robots", Journal of Field Robotics, November, 2020, doi.org/10.1002/rob.21993T. Dang, F. Mascarich, S. Khattak, C. Papachristos, K. Alexis, "Graph-based Path Planning for Autonomous Robotic Exploration in Subterranean Environments", IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019, Macau, ChinaOpen-Source Git Repo: github.com/unr-arl/gbplanner_ros The field tests were conducted in the Løkken Mine with the help and support of the Orkla Industrimuseum
CERBERUS: Deployment at the DARPA Subterranean Challenge Urban Circuit Kostas Alexis 2021-01-15 | This paper reports the technological progress and performance of team “CERBERUS” after participating in the Tunnel and Urban Circuits of the DARPA Subterranean Challenge. The SubT Challenge is an international robotics competition organized by the Defense Advanced Research Projects Agency to inspire advances in resilient robotic autonomy in subterranean settings. Teams compete to explore, map, and search a completely unknown environment including tunnels, mines, caves, and urban subterranean infrastructure.Each team receives a 60 minute time slot to find various objects of interest, called artifacts, in an underground setting. For each artifact found and correctly localized, the team earns a point.Finally, and critically, only a single human supervisor may communicate with the robots once they are deployed. Motivated by and responding to these challenges, the CERBERUS collaborative walking and flying robotic system-of-systems envisions the autonomous exploration of such subterranean environments through the synergistic operation of diverse systems with unique navigation capabilities. The principal idea behind CERBERUS' vision is that the combination of legged and aerial systems offers unique advantages tailored to the needs of complex underground settings. Walking robots present potential benefits for overcoming complex terrain, both in terms of the size of obstacles that can be negotiated for a certain overall robot size, and the robust traversal of dynamic terrain. Aerial robots, especially when equipped with collision-tolerance capabilities, offer the advantage of seamless navigation that is not bound to the extreme terrain challenges often present in environments such as caves. CERBERUS focuses on the co-deployment of such legged and aerial robots further equipped with resilient multi-modal and multi-robot localization and mapping, artifact detection and universal exploration path planning capabilities. A robot-deployable communications network solution is being realized which allows the CERBERUS robotic system to deploy its own communications network underground. The results presented in this paper outline the experience of the Tunnel and Urban Circuit, the first relating to the exploration of the underground NIOSH Mining Program’s Safety Research and Experimental Mines and the second, an unfinished nuclear facility at the Satsop Business Park in Elma, Washington. Team CERBERUS is an international partnership involving the following members:* Autonomous Robots Lab, University of Nevada, Reno & Norwegian University of Science and Technology* Robotic Systems Lab, ETH Zurich* Autonomous Systems Lab, ETH Zurich* HiPeR Lab, University of California, Berkeley* Dynamic Robot Systems Group, University of Oxford * Flyability * Sierra Nevada Corporation
Model Predictive Control for Micro Aerial Vehicles: A Survey Kostas Alexis 2021-01-03 | This paper presents a review of the design and application of model predictive control strategies for Micro Aerial Vehicles and specifically multirotor configurations such as quadrotors. The diverse set of works in the domain is organized based on the control law being optimized over linear or nonlinear dynamics, the integration of state and input constraints, possible fault-tolerant design, if reinforcement learning methods have been utilized and if the controller refers to free-flight or other tasks such as physical interaction or load transportation. A selected set of comparison results are also presented and serve to provide insight for the selection between linear and nonlinear schemes, the tuning of the prediction horizon, the importance of disturbance observer-based offset-free tracking and the intrinsic robustness of such methods to parameter uncertainty. Furthermore, an overview of recent research trends on the combined application of modern deep reinforcement learning techniques and model predictive control for multirotor vehicles is presented. Finally, this review concludes with explicit discussion regarding selected open-source software packages that deliver off-the-shelf model predictive control functionality applicable to a wide variety of Micro Aerial Vehicle configurations.Link to Survey Paper: arxiv.org/abs/2011.11104Selected – but indicative – open-source MPC releases for MAVs* github.com/ethz-asl/mav_control_rw* github.com/uzh-rpg/rpg_mpc* github.com/DentOpt/denmpc* github.com/klaxalk/multirotor-control-boardSelected – but indicative – open-source MPC software packages with broader scope* github.com/ethz-adrl/control-toolbox* cvxgen.com/docs/index.html* http://acado.sourceforge.net/doc/html/d4/d26/example_013.html* yalmip.github.io* mpt3.org* do-mpc.com/en/latest
Resource-aware Online Parameter Adaptation for Computationally-constrained Visual-Inertial Odometry Kostas Alexis 2020-11-06 | In this work, a computational resources-aware parameter adaptation method for visual-inertial navigation systems is proposed with the goal of enabling the improved deployment of such algorithms on computationally constrained systems. Such a capacity can prove critical when employed on ultra-lightweight systems or alongside mission critical computationally expensive processes. To achieve this objective, the algorithm proposes selected changes in the vision front-end and optimization back-end of visual-inertial odometry algorithms, both prior to execution and in real time based on an online profiling of available resources. The method also utilizes information from the motion dynamics experienced by the system to manipulate parameters online. The general policy is demonstrated on three established algorithms, namely S-MSCKF, VINS-Mono and OKVIS and has been verified experimentally on the EuRoC dataset. The proposed approach achieved comparable performance at a fraction of the original computational cost.
Autonomous Distributed 3D Radiation Field Estimation for Nuclear Environment Characterization Kostas Alexis 2020-11-06 | In this work, a new method for autonomous distributed 3D nuclear radiation field mapping was proposed and experimentally verified based on a custom-designed resilient micro flyer. The algorithm uses sequentially-acquired radiation readings to reconstruct an accurate estimate of the radiation field. To achieve this goal it estimates the field mean and gradient, while it further allows automatic guidance of a robot in order to iteratively acquire informative scintillator readings. A set of experiments is presented and serves to verify the quality of the estimated map and the benefits of curiosity-driven path planning in time-constrained missions.
Hypergame-based Adaptive Behavior Path Planning for Combined Exploration and Visual Search Kostas Alexis 2020-11-05 | In this work we present an adaptive behavior path planning method for autonomous exploration and visual search of unknown environments. As volumetric exploration and visual coverage of unknown environments, with possibly different sensors, are non-identical objectives, a principled combination of the two is proposed. In particular, the method involves three distinct planning policies, namely exploration, and sparse or dense visual coverage. A hypergame formulation is proposed which allows the robot to select for the next-best planning behavior in response to the currently encountered environment challenges in terms of geometry and visual conditions, alongside a self-assessment of its performance. The proposed planner is evaluated in a collection of experimental and simulation studies in diverse environments, while comparative results against a state-of-the-art exploration method are also presented.
Resilient Collision-tolerant Navigation in Confined Environments Kostas Alexis 2020-11-03 | This work presents the design and autonomous navigation policy of the Resilient Micro Flyer, a new type of collision-tolerant robot tailored to flying through extremely confined environments and manhole-sized tubes. The robot maintains a low weight (sub-500g) and implements a combined rigid-compliant design through the integration of elastic flaps around its stiff collision-tolerant frame. These passive flaps ensure compliant collisions, contact sensing and smooth navigation in contact with the environment. Focusing on resilient autonomy, capable of running on resource-constrained hardware, we demonstrate the beneficial role of compliant collisions for the reliability of the onboard visual-inertial odometry and propose a safe navigation policy that exploits both collision-avoidance using lightweight time-of-flight sensing and adaptive control in response to collisions. The robot further realizes an explicit manhole navigation mode that exploits the direct mechanical feedback provided by the flaps and a special navigation strategy to self-align inside manholes with non-straight geometry. Comprehensive experimental studies are presented to evaluate, both individually and as a whole, how resilience is achieved based on the robot design and its navigation scheme.
