Learning Camera Performance Models for Active Multi-Camera Visual Teach and Repeat Oxford Dynamic Robot Systems Group 2021-03-29 | In dynamic and cramped industrial environments, achieving reliable Visual Teach and Repeat (VT&R) with a single-camera is challenging. In this work, we develop a robust method for non-synchronized multi-camera VT&R. Our contribution are expected Camera Performance Models (CPM) which evaluate the camera streams from the teach step to determine the most informative one for localization during the repeat step. By actively selecting the most suitable camera for localization, we are able to successfully complete missions when one of the cameras is occluded, faces into feature poor locations or if the environment has changed. Furthermore, we explore the specific challenges of achieving VT&R on a dynamic quadruped robot, ANYmal. The camera does not follow a linear path (due to the walking gait and holonomicity) such that precise path-following cannot be achieved. Our experiments feature forward and backward facing stereo cameras showing VT&R performance in cluttered indoor and outdoor scenarios. We compared the trajectories the robot executed during the repeat steps demonstrating typical tracking precision of less than 10 cm on average. With a view towards omni-directional localization, we show how the approach generalizes to four cameras in simulation.Accepted in the IEEE International Conference on Robotics and Automation (ICRA) 2021Authors: Matías Mattamala, Milad Ramezani, Marco Camurri, Maurice FallonPre-print: arxiv.org/abs/2103.14070PDF: arxiv.org/pdf/2103.14070.pdf
SiLVR: Scalable Lidar-Visual Reconstruction with Neural RadianceFields for Robotic Inspection Oxford Dynamic Robot Systems Group 2024-03-22 | Accepted to ICRA 2024Website: ori-drs.github.io/projects/silvrPre-print: arxiv.org/abs/2403.06877[Abstract]We present a neural-field-based large-scale reconstruction system that fuses lidar and vision data to generate high-quality reconstructions that are geometrically accurate and capture photo-realistic textures. This system adapts the state-of-the-art neural radiance field (NeRF) representation to also incorporate lidar data which adds strong geometric constraints on the depth and surface normals. We exploit the trajectory from a real-time lidar SLAM system to bootstrap a Structure-from-Motion (SfM) procedure to both significantly reduce the computation time and to provide metric scale which is crucial for lidar depth loss. We use submapping to scale the system to large-scale environments captured over long trajectories. We demonstrate the reconstruction system with data from a multi-camera, lidar sensor suite onboard a legged robot, hand-held while scanning building scenes for 600 metres, and onboard an aerial robot surveying a multi-storey mock disaster site-building. Authors: Yifu Tao , Yash Bhalgat, Lanke Frank Tarimo Fu, Matias Mattamala , Nived Chebrolu, Maurice Fallon
Batch Differentiable Pose Refinement for In-The-Wild Camera/LiDAR Extrinsic Calibration Oxford Dynamic Robot Systems Group 2024-03-21 | Published in Conference on Robot Learning CoRL 2023 in Atlanta, Georgia, USA.Lanke Frank Tarimo Fu and Maurice FallonPre-print: openreview.net/pdf?id=mTZcxs2O7k
Fast Traversability Estimation for Wild Visual Navigation Oxford Dynamic Robot Systems Group 2024-03-21 | Published in Robotics Science and Systems in Daegu, Korea, July 2023.Jonas Frey, Matias Mattamala, Nived Chebrolu, Cesar Cadena, Maurice Fallon, Marco HutterPre-print: arxiv.org/abs/2305.08510Project Page: sites.google.com/leggedrobotics.com/wild-visual-navigation
LiSTA: Geometric Object-Based Change Detection in Cluttered Environments Oxford Dynamic Robot Systems Group 2024-03-05 | ori-drs.github.io/LiSTA_datasetWe present LiSTA (LiDAR Spatio-Temporal Analysis), a system to detect probabilistic object-level change over time using multi-mission SLAM. Many applications require such a system, including construction, robotic navigation, long-term autonomy, and environmental monitoring. We focus on the semi-static scenario where objects are added, subtracted, or changed in position over weeks or months. Our system combines multi-mission LiDAR SLAM, volumetric differencing, object instance description, and correspondence grouping using learned descriptors to keep track of an open set of objects. Object correspondences between missions are determined by clustering the object's learned descriptors. We demonstrate our approach using datasets collected in a simulated environment and a real-world dataset captured using a LiDAR system mounted on a quadruped robot monitoring an industrial facility containing static, semi-static, and dynamic objects. Our method demonstrates superior performance in detecting changes in semi-static environments compared to existing methods.Accepted to IEEE International Conference on Robotics and Automation (ICRA) 2024 ConferenceAuthors: Joseph Rowell, Lintong Zhang, Maurice FallonPre-print: arxiv.org/abs/2403.02175PDF: arxiv.org/pdf/2403.02175.pdf
Language-EXtended Indoor SLAM (LEXIS): A Versatile System for Real-time Visual Scene Understanding Oxford Dynamic Robot Systems Group 2024-03-05 | Abstract:Versatile and adaptive semantic understanding would enable autonomous systems to comprehend and interact with their surroundings. Existing fixed-class models limit the adaptability of indoor mobile and assistive autonomous systems. In this work, we introduce LEXIS, a real-time indoor Simultaneous Localization and Mapping (SLAM) system that harnesses the open-vocabulary nature of Large Language Models (LLMs) to create a unified approach to scene understanding and place recognition. The approach first builds a topological SLAM graph of the environment (using visual-inertial odometry) and embeds Contrastive Language-Image Pretraining (CLIP) features in the graph nodes. We use this representation for flexible room classification and segmentation, serving as a basis for room-centric place recognition. This allows loop closure searches to be directed towards semantically relevant places. Our proposed system is evaluated using both public, simulated data and real-world data, covering office and home environments. It successfully categorizes rooms with varying layouts and dimensions and outperforms the state-of-the-art (SOTA). For place recognition and trajectory estimation tasks we achieve equivalent performance to the SOTA, all also utilizing the same pre-trained model. Lastly, we demonstrate the system's potential for planning. Accepted at ICRA 2024Authors: Christina Kassab, Matias Mattamala, Lintong Zhang, Maurice Fallon Pre-print: arxiv.org/abs/2309.15065
R-LGP:A Reachability-guided LGP Framework for Optimal Task and Motion Planning on Mobile Manipulator Oxford Dynamic Robot Systems Group 2024-02-28 | This paper presents an optimization-based solution to task and motion planning (TAMP) on mobile manipulators. Logic-geometric programming (LGP) has shown promising capabilities for optimally dealing with hybrid TAMP problems that involve abstract and geometric constraints. However, LGP does not scale well to high-dimensional systems (e.g. mobile manipulators) and can suffer from obstacle avoidance issues due to local minima. In this work, we extend LGP with a sampling-based reachability graph to enable solving optimal TAMP on high-DoF mobile manipulators. The proposed reachability graph can incorporate environmental information (obstacles) to provide the planner with sufficient geometric constraints. This reachability-aware heuristic efficiently prunes infeasible sequences of actions in the continuous domain, hence, it reduces replanning by securing feasibility at the final full path trajectory optimization. Our framework proves to be time-efficient in computing optimal and collision-free solutions, while outperforming the current state of the art on metrics of success rate, planning time, path length, and number of steps. We validate our framework on the physical Toyota HSR robot and report comparisons on a series of mobile manipulation tasks of increasing difficulty.Accepted to IEEE International Conference on Robotics and Automation (ICRA)Authors: Kim Tien Ly, Valeriy Semenov, Mattia Risiglione, Wolfgang Merkt, Ioannis HavoutisPre-print: arxiv.org/abs/2310.02791PDF: arxiv.org/pdf/2310.02791.pdf
Osprey: Multi-Session Autonomous Aerial Mapping with LiDAR-based SLAM and Next Best View Planning Oxford Dynamic Robot Systems Group 2023-11-06 | Aerial mapping systems are important for many surveying applications (e.g., industrial inspection or agricultural monitoring). Fully autonomous systems can significantly improve efficiency. This video presents Osprey, an autonomous aerial mapping system with state-of-the-art multi-session mapping capabilities. It enables a non-expert operator to specify a bounded target area that the aerial platform can then map autonomously, over multiple flights if necessary.Field experiments with Osprey demonstrate mapping of three sites, with a total ground coverage of 7085m2 and a maximum height of 27m. True colour maps were created from images captured by Osprey using pointcloud and NeRF reconstruction methods. These maps provide useful data for structural inspection tasks.Authors: Rowan Border, Nived Chebrolu, Yifu Tao, Jonathan Gammell, Maurice Fallon
Extrinsic Calibration of Camera to LIDAR using a Differentiable Checkerboard Model Oxford Dynamic Robot Systems Group 2023-09-28 | Multi-modal sensing often involves determining correspondences between each domain’s signals, which in turn depends on the accurate extrinsic calibration of the sensors. We present a framework for extrinsic calibration of a camera and a LIDAR using only a simple off-the-shelf checkerboard. It is designed to operate even when the LIDAR observes a significantly truncated portion of the checkerboard. In our experiments, we achieve calibration accuracy in the order of 2-4 mm and demonstrate a 30% error reduction compared to state-of-the-art approaches. We are able to achieve this improvement while using only partial LIDAR views of the checkerboard that allows for a simpler data capture process.Accepted to International Conference on Intelligent Robots and SystemsDetroit, October 2023Authors:Lanke Frank Tarimo Fu, Nived Chebrolu, Maurice FallonPDF:dropbox.com/s/arhpp59d502fuks/2023IROS_fu.pdf?dl=0
InstaLoc: One-shot Global Lidar Localisation in Indoor Environments through Instance Learning Oxford Dynamic Robot Systems Group 2023-07-06 | Localization for autonomous robots in prior maps is crucial for their functionality. This paper offers a solution to this problem for indoor environments called InstaLoc, which operates on an individual lidar scan to localize it within a prior map. We draw on inspiration from how humans navigate and position themselves by recognizing the layout of distinctive objects and structures. Mimicking the human approach, InstaLoc identifies and matches object instances in the scene with those from a prior map. As far as we know, this is the first method to use panoptic segmentation directly inferring on 3D lidar scans for indoor localization. InstaLoc operates through two networks based on spatially sparse tensors to directly infer dense 3D lidar point clouds. The first network is a panoptic segmentation network that produces object instances and their semantic classes. The second smaller network produces a descriptor for each object instance. A consensus based matching algorithm then matches the instances to the prior map and estimates a six degrees of freedom (DoF) pose for the input cloud in the prior map. The significance of InstaLoc is that it has two efficient networks. It requires only one to two hours of training on a mobile GPU and runs in real-time at 1 Hz. Our method achieves between two and four times more detections when localizing, as compared to baseline methods, and achieves higher precision on these detections.Accepted by Robotics: Science and Systems 2023Authors: Lintong Zhang, Tejaswi Digumarti, Georgi Tinchev, Maurice FallonPre-print: arxiv.org/abs/2305.09552PDF: arxiv.org/pdf/2305.09552.pdf
Deep IMU Bias Inference for Robust Visual-Inertial Odometry With Factor Graphs Oxford Dynamic Robot Systems Group 2023-05-31 | Published in IEEE Robotics and Automation Letters and Presented at ICRA 2023 in London, UK.Russell Buchanan, Varun Agrawal, Marco Camurri, Frank Dellaert, Maurice FallonPre-print: arxiv.org/abs/2211.04517
Learning Low-Frequency Motion Control for Robust and Dynamic Robot Locomotion Oxford Dynamic Robot Systems Group 2023-03-01 | Abstract:Robotic locomotion is often approached with the goal of maximizing robustness and reactivity by increasing motion control frequency. We challenge this intuitive notion by demonstrating robust and dynamic locomotion with a learned motion controller executing at as low as 8 Hz on a real ANYmal C quadruped. The robot is able to robustly and repeatably achieve a high heading velocity of 1.5 m/s, traverse uneven terrain, and resist unexpected external perturbations. We further present a comparative analysis of deep reinforcement learning (RL) based motion control policies trained and executed at frequencies ranging from 5 Hz to 200 Hz. We show that low-frequency policies are less sensitive to actuation latencies and variations in system dynamics. This is to the extent that a successful sim-to-real transfer can be performed even without any dynamics randomization or actuation modeling. We support this claim through a set of rigorous empirical evaluations. Moreover, to assist reproducibility, we provide the training and deployment code along with an extended analysis on the project website. Authors: Siddhant Gangapurwala, Luigi Campanaro and Ioannis HavoutisAccepted to IEEE International Conference on Robotics and Automation (ICRA) 2023 Project Website: ori-drs.github.io/lfmcCode:Training (RaiSim - C++): github.com/ori-drs/lfmc_gymDeployment (RaiSim - C++): github.com/ori-drs/lfmc_cvalDeployment (PyBullet/RaiSim - Python): github.com/ori-drs/lfmc_pyvalPre-print: arxiv.org/abs/2209.14887Dog Clip: pexels.com/video/video-of-dog-playing-with-a-ball-4651948
Inspection of Energy Facilities with Legged Robots Oxford Dynamic Robot Systems Group 2022-12-12 | Full Report:ori.ox.ac.uk/news/oxford-quadruped-robot-research-tested-on-a-live-chevron-facilityIn Spring 2022 our group demoed ANYmal and Spot carrying out automated inspection at Chevron's Blending Plant in Ghent, Belgium.This demonstration was coordinated by the EPSRC/ISCF ORCA Hub and the Net Zero Technology Centre. It marked the culmination of the 4-year ORCA Hub - which developed robotic systems for inspection of assets in the energy sector.This trial marked the first demonstration of quadruped robots on an Chevron site in Europe. Many thanks to all the partners for helping to enable this trial - in particular David Wavell from ORCA Hub.Spot team lead: Maurice FallonANYmal team lead: Ioannis HavoutisMusic from BenSound.com - "Once Again"
Mapping Oxfords Sheldonian Theatre Oxford Dynamic Robot Systems Group 2022-12-12 | This video overviews a project in Spring 2022 where researchers from Oxford Robotics Institute collaborated with Hilti to produce a highly detailed 3D model of the Sheldonian Theatre.Further Details:ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9968057hilti-challenge.comori.ox.ac.uk/labs/drshttp://dynamic.robots.ox.ac.uk/maps/sheldonian.htmlAudio from:bensound.com
Automated Inspection of Construction with Legged Robots Oxford Dynamic Robot Systems Group 2022-10-19 | During Summer 2022 our group demoed ANYmal and Spot carrying out in the context of construction progress monitoring at Costain's Gatwick Airport Train Station site. This was the final demo of the MEMMO Horizon Europe Project.ANYmal crossed terrain obstacles and steps while Spot autonomously collected images and lidar maps.Many thanks to Costain for hosting us on site.
Factor Graph Fusion of GNSS Sensing with IMU and Lidar for Robot Localization w/o a Base Station Oxford Dynamic Robot Systems Group 2022-10-05 | Jonas Beuchert, Marco Camurri, and Maurice Fallon.“Factor Graph Fusion of Raw GNSS Sensing with IMU and Lidar for Precise Robot Localization without a Base Station”.In: arXiv preprint arXiv:2209.14649 (2022). URL: arxiv.org/abs/2209.14649
Learning and Deploying Robust Locomotion Policies with Minimal Dynamics Randomization Oxford Dynamic Robot Systems Group 2022-09-29 | Training deep reinforcement learning (DRL) locomotion policies often requires massive amounts of data to converge to the desired behavior. In this regard, simulators provide a cheap and abundant source. For successful sim-to-real transfer, exhaustively engineered approaches such as system identification, dynamics randomization, and domain adaptation are generally employed. As an alternative, we investigate a simple strategy of random force injection (RFI) to perturb system dynamics during training. We show that the application of random forces enables us to emulate dynamics randomization. This allows us to obtain locomotion policies that are robust to variations in system dynamics. We further extend RFI, referred to as extended random force injection (ERFI), by introducing an episodic actuation offset. We demonstrate that ERFI provides additional robustness for variations in system mass offering on average a 61% improved performance over RFI. We also show that ERFI is sufficient to perform a successful sim-to-real transfer on two different quadrupedal platforms, ANYmal C and Unitree A1, even for perceptive locomotion over uneven terrain in outdoor environments.Authors: Luigi Campanaro, Siddhant Gangapurwala, Wolfgang Merkt, Ioannis HavoutisPre-print: arxiv.org/abs/2209.12878Website: sites.google.com/view/erfi-icra
Towards Real-Time Forest Inventory using Handheld LiDAR Oxford Dynamic Robot Systems Group 2022-08-20 | Paper appearing in Journal of Robotics and Autonomous Systems (RAS).More details:https://digiforest.eu/news/2022-08-19-realtime-forest-inventoryContributions:• An online mapping system for forest environments using a handheld LiDAR.• Segmentation, tracking and mapping of trees in the forest.• Estimation of Diameter at Breast Height (DBH) for individual trees in the forest.• Pose-graph SLAM system with loop closure integration to correct for odometry drift.• Multi-session mapping capability to merge scans in a post-processing step.• Demonstration in challenging large scale forests spanning several kilometers.Authors: Alexander Proudman, Milad Ramezani, Sundara Tejaswi Digumarti, Nived Chebrolu, Maurice Fallon
Strategies for Large Scale Elastic and Semantic LiDAR Reconstruction Oxford Dynamic Robot Systems Group 2022-07-29 | This paper presents novel strategies for spawning and fusing submaps within an elastic dense 3D reconstruction system. The proposed system uses spatial understanding of the scanned environment to control memory usage growth by fusing overlapping submaps in different ways. This allowsthe number of submaps and memory consumption to scale with the size of the environment rather than the duration of exploration. By analysing spatial overlap and semantic information, our system segments distinct spaces on-the-fly during exploration, such as rooms, stairwells, and indoor-outdoortransitions. The proposed system associates semantically labelled submaps with poses of SLAM pose graph to enable global elasticity. A probabilistic model to merge the voxel labels of the different submaps is incorporated to ensure correct semantic submap fusion when SLAM loop closures occur. Additionally, we present a new mathematical formulation of relative uncertainty between poses to improve the global consistency of the reconstruction. Performance is demonstrated using experiments exploring multi-floor multi-room indoor environments, indoor-outdoor transitions and large-scale outdoor experiments. Relative to our baseline, the presented approach demonstrates improved scalability and accuracy.Accepted in Robotics and Autonomous Systems (RAS) journal - European Conference on Mobile Robots (ECMR) 2021 special issue. Authors: Yiduo Wang, Milad Ramezani, Matias Mattamala, Sundara Tejaswi Digumarti and Maurice FallonPDF: doi.org/10.1016/j.robot.2022.104185
3D Lidar Reconstruction with Probabilistic Depth Completion for Robotic Navigation - IROS 2022 Oxford Dynamic Robot Systems Group 2022-07-27 | "3D Lidar Reconstruction with Probabilistic Depth Completionfor Robotic Navigation" - IROS 2022PDF: arxiv.org/abs/2207.12520Website: ori.ox.ac.uk/labs/drs/depth-completionAuthors: Yifu Tao, Marija Popović, Yiduo Wang, Sundara Tejaswi Digumarti, Nived Chebrolu, Maurice Fallon[Abstract]Safe motion planning in robotics requires planning into space which has been verified to be free of obstacles. However, obtaining such environment representations using lidars is challenging by virtue of the sparsity of their depth measurements. We present a learning-aided 3D lidar reconstruction framework that upsamples sparse lidar depth measurements with the aid of overlapping camera images so as to generate denser reconstructions with more definitively free space than can be achieved with the raw lidar measurements alone. We use a neural network with an encoder-decoder structure to predict dense depth images along with depth uncertainty estimates which are fused using a volumetric mapping system. We conduct experiments on real-world outdoor datasets captured using a handheld sensing device and a legged robot. Using input data from a 16-beam lidar mapping a building network, our experiments showed that the amount of estimated free space was increased by more than 40% with our approach. We also show that our approach trained on a synthetic dataset generalises well to real-world outdoor scenes without additional fine-tuning. Finally, we demonstrate how motion planning tasks can benefit from these denser reconstructions.
RLOC: Terrain-Aware Legged Locomotion using Reinforcement Learning and Optimal Control Oxford Dynamic Robot Systems Group 2022-05-10 | "RLOC: Terrain-Aware Legged Locomotion using Reinforcement Learning and Optimal Control"- Siddhant Gangapurwala, Mathieu Geisert, Romeo Orsolino, Maurice Fallon and Ioannis Havoutis. IEEE Transactions on Robotics (T-RO) 2022.Description:Overview of the proposed RLOC perceptive locomotion control architecture. Existing perceptive locomotion approaches are often developed for a target control system. When such a framework is transferred to a system with different dynamics, the control performance tends to significantly deteriorate. In contrast, RLOC introduces a training method which allows the perceptive footstep planner to be robust to considerable variations in system dynamics. In this work, this is demonstrated by performing a zero-shot transfer from the training domain (a simulation of ANYmal version B robot) to the test domain (physical ANYmal version C robot). Such a framework is especially relevant for scenarios where a quadrupedal robot is deployed for loading-unloading tasks and where the dynamics of the system are expected to continuously change. The link to the manuscript and the abstract are provided below.Article: ieeexplore.ieee.org/document/9779429Preprint: arxiv.org/abs/2012.03094DOI: 10.1109/TRO.2022.3172469Supplementary Movies:S1 - Footstep Planning: youtu.be/1Hm77CfaouES2 - Domain Adaptive Tracking: youtu.be/T-A8s-ajuwwS3 - Recovery Policy: youtu.be/-ToiFl-eBt8S4 - RLOC Framework: youtu.be/_SFOfxUgCzES5 - Stability Margin: youtu.be/vSOU-USXK64S6 - Demonstrations: youtu.be/rIr0tyqTjGwAbstract: We present a unified model-based and data-driven approach for quadrupedal planning and control to achieve dynamic locomotion over uneven terrain. We utilize on-board proprioceptive and exteroceptive feedback to map sensory information and desired base velocity commands into footstep plans using a reinforcement learning (RL) policy trained in simulation over a wide range of procedurally generated terrains. When run online, the system tracks the generated footstep plans using a model-based controller. We evaluate the robustness of our method over a wide variety of complex terrains. It exhibits behaviors which prioritize stability over aggressive locomotion. Additionally, we introduce two ancillary RL policies for corrective whole-body motion tracking and recovery control. These policies account for changes in physical parameters and external perturbations. We train and evaluate our framework on a complex quadrupedal system, ANYmal version B, and demonstrate transferability to a larger and heavier robot, ANYmal C, without requiring retraining.
Enter the 2022 Hilti SLAM Challenge - Map Oxfords Sheldonian Theatre Oxford Dynamic Robot Systems Group 2022-04-28 | The Hilti SLAM (Simultaneous Localization and Mapping) Challenge tests the state-of-the-art of robotic map building.hilti-challenge.comHilti, the Oxford Robotics Institute and the Robotics and Perception Group from University of Zürich have created a benchmark which focuses on the SLAM problem in environments with challenging features such as changing light conditions, difficult geometries, and fast movements.We provide mm-accurate groundtruth as well as full 3D scans from a survey-grade laserscanner (will be published after the challenge). Our automatic evaluation system offers fast feedback on the accuracy and gives a detailed analysis on the result.Music: bensound.com
Unsupervised Learning of Terrain Representations for Haptic Monte Carlo Localization Oxford Dynamic Robot Systems Group 2022-03-07 | Mikołaj Łysakowski*, Michal Ryszard Nowicki*, Russell Buchanan, Marco Camurri, Maurice Fallon, Krzysztof Tadeusz Walas**Institute of Robotics and Machine Intelligence, Poznan University of Technology, Poznan, PolandThis paper was accepted to IEEE ICRA 2022Haptic sensing has recently been used effectively for legged robot localization in extreme scenarios where cameras and LiDAR might fail, such as dusty mines and foggy sewers. However, existing haptic sensing mainly relies on supervised classification, with training and evaluation executed over explicit terrain classes. Defining classes is a significant limitation to real-world applications, where prior labelling and handcrafted classes are often impractical. This paper proposes a novel haptic localization system based on a fully unsupervised terrain representation learned solely from the force/torque sensors located in the quadruped robot's feet. Instead of using the detected terrain class for localization, we propose an improved autoencoder architecture to generate a sparse map of encodings on the first run and to localize against this sparse map during subsequent runs.We compare our approach to a haptic localization system based on supervised terrain classification, showing that the unsupervised method has comparable or better performance than the supervised one for the same trajectories while clearly outperforming the proprioceptive odometry estimator available on the robot.Therefore, the proposed approach is well-suited for a routine maintenance application, increasing the platform's robustness.
VILENS: Visual, Inertial, Lidar, and Leg Odometryfor All-Terrain Legged Robots Oxford Dynamic Robot Systems Group 2022-02-21 | We present VILENS (Visual Inertial Lidar Legged Navigation System), an odometry system for legged robots based on factor graphs. The key novelty is the tight fusion of four different sensor modalities to achieve reliable operation when the individual sensors would otherwise produce degenerate estimation. To minimize leg odometry drift, we extend the robot's state with a linear velocity bias term which is estimated online. This bias is observable because of the tight fusion of this preintegrated velocity factor with vision, lidar, and IMU factors. Extensive experimental validation on different ANYmal quadruped robots is presented, for a total duration of 2 h and 1.8 km traveled. The experiments involved dynamic locomotion over loose rocks, slopes, and mud which caused challenges like slippage and terrain deformation. Perceptual challenges included dark and dusty underground caverns, and open and feature-deprived areas. We show an average improvement of 62% translational and 51% rotational errors compared to a state-of-the-art loosely coupled approach. To demonstrate its robustness, VILENS was also integrated with a perceptive controller and a local path planner.Submitted to IEEE Transactions on Robotics journal]Authors: David Wisth, Marco Camurri, Maurice FallonPre-print: arxiv.org/abs/2107.07243 PDF: arxiv.org/pdf/2107.07243
An Efficient Locally Reactive Controller for Safe Navigation in Visual Teach and Repeat Missions Oxford Dynamic Robot Systems Group 2022-01-12 | To achieve successful field autonomy, mobile robots need to freely adapt to changes in their environment. Visual navigation systems such as Visual Teach and Repeat (VT&R) often assume the space around the reference trajectory is free, but if the environment is obstructed path tracking can fail or the robot could collide with a previously unseen obstacle. In this work, we present a locally reactive controller for a VT&R system that allows a robot to navigate safely despite physical changes to the environment. Our controller uses a local elevation map to compute vector representations and outputs twist commands for navigation at 10 Hz. They are combined in a Riemannian Motion Policies (RMP) controller that requires less than 2 ms to run on a CPU. We integrated our controller with a VT&R system onboard an ANYmal C robot and tested it in indoor cluttered spaces and a large-scale underground mine. We demonstrate that our locally reactive controller keeps the robot safe when physical occlusions or loss of visual tracking occur such as when walking close to walls, crossing doorways, or traversing narrow corridors.Accepted in IEEE Robotics and Automation Letters (RA-L) and International Conference on Robotics and Automation (ICRA) 2022.Authors: Matías Mattamala, Nived Chebrolu, Maurice FallonPre-print: arxiv.org/abs/2201.03938PDF: arxiv.org/pdf/2201.03938.pdf
3D Scanning of the Chernobyl Nuclear Power Plant Oxford Dynamic Robot Systems Group 2021-11-07 | This video demonstrates some of the results from a scientific deployment to Chernobyl NPP in September 2021 led by University of Bristol.Our 3D scanning technology was used to build 3D models of various sites with with radiation data (not shown) can be used to monitor the activity levels of remaining nuclear radiation.More details here:bristol.ac.uk/news/2021/september/chornobyl-2021.html
Learning Inertial Odometry for Dynamic Legged Robot State Estimation Oxford Dynamic Robot Systems Group 2021-11-04 | To be presented at CoRL 2021Authors: Russell Buchanan, Marco Camurri, Frank Dellaert, Maurice FallonPaper available (arXiv): arxiv.org/abs/2111.00789Abstract:This paper introduces a novel proprioceptive state estimator for legged robots based on a learned displacement measurement from IMU data. Recent research in pedestrian tracking has shown that motion can be inferred from inertial data using convolutional neural networks. A learned inertial displacement measurement can improve state estimation in challenging scenarios where leg odometry is unreliable, such as slipping and compressible terrains. Our work learns to estimate a displacement measurement from IMU data which is then fused with traditional leg odometry. Our approach greatly reduces the drift of proprioceptive state estimation, which is critical for legged robots deployed in vision and lidar denied environments such as foggy sewers or dusty mines. We compared results from an EKF and an incremental fixed-lag factor graph estimator using data from several real robot experiments crossing challenging terrains. Our results show a reduction of relative pose error by 37% in challenging scenarios when compared to a traditional kinematic-inertial estimator without learned measurement. We also demonstrate a 22% reduction in error when used with vision systems in visually degraded environments such as an underground mine.
Navigating by Touch: Haptic Localization via Geometric Sensing and Terrain Classification Oxford Dynamic Robot Systems Group 2021-10-12 | Russell Buchanan, Jakub Bednarek, Marco Camurri, Michał R. Nowicki, Krzysztof Walas and Maurice Fallon. Navigating by touch: haptic Monte Carlo localization via geometric sensing and terrain classification. Autonomous Robot 45, 843–857 (2021).Available: link.springer.com/article/10.1007/s10514-021-10013-wLegged robot navigation in extreme environments can hinder the use of cameras and lidar due to darkness, air obfuscation or sensor damage, whereas proprioceptive sensing will continue to work reliably. In this paper, we propose a purely proprioceptive localization algorithm which fuses information from both geometry and terrain type to localize a legged robot within a prior map. First, a terrain classifier computes the probability that a foot has stepped on a particular terrain class from sensed foot forces. Then, a Monte Carlo-based estimator fuses this terrain probability with the geometric information of the foot contact points. Results demonstrate this approach operating online and onboard an ANYmal B300 quadruped robot traversing several terrain courses with different geometries and terrain types over more than 1.2 km. The method keeps pose estimation error below 20 cm using a prior map with trained network and using sensing only from the feet, leg joints and IMU.This research has been conducted as part of the ANYbotics research community. It was part funded by the EU H2020 Project THING (Grant ID 780883) and a Royal Society University Research Fellowship (Fallon).
Robots to the Rescue - Oxford Robotics Institute prepares for the DARPA SubT Challenge Oxford Dynamic Robot Systems Group 2021-09-22 | This video demonstrates the preparations by our group for the DARPA Subterranean Challenge. Over 3 years we worked with NTNU (Norway), ETH (Zurich), University of Nevada (Reno) and other partners to develop a team of walking and flying robots for disaster response.The DARPA SubT challenge concludes with an international competition in the Louisville Mega Cavern in Kentucky, USA in Sept 2021.More information:ori.ox.ac.uk/projects/darpa-subt-challenge––––––––––––––––––––––––––––––The Reckoning by AERØHEAD soundcloud.com/aeroheadCreative Commons — Attribution 3.0 Unported — CC BY 3.0Free Download / Stream: bit.ly/the-reckoning-aeroheadMusic promoted by Audio Library youtu.be/NQk5C6Z_WOA––––––––––––––––––––––––––––––
Scalable and Elastic LiDAR Reconstruction in Complex Environments Through Spatial Analysis Oxford Dynamic Robot Systems Group 2021-09-20 | This paper presents novel strategies for spawning and fusing submaps within an elastic dense 3D reconstruction system. The proposed system uses spatial understanding of the scanned environment to control memory usage growth by fusing overlapping submaps in different ways. This allows the number of submaps and memory consumption to scale with the size of the environment rather than the duration of exploration. By analysing spatial overlap, our system segments distinct spaces, such as rooms and stairwells on the fly during exploration. Additionally, we present a new mathematical formulation of relative uncertainty between poses to improve the global consistency of the reconstruction. Performance is demonstrated using a multi-floor multi-room indoor experiment, a large-scale outdoor experiment and simulated datasets. Relative to our baseline, the presented approach demonstrates improved scalability and accuracy.Accepted in the European Conference on Mobile Robots (ECMR) 2021Authors: Yiduo Wang, Milad Ramezani, Matías Mattamala, Maurice FallonPre-print: arxiv.org/abs/2106.15446PDF: arxiv.org/pdf/2106.15446.pdf
Balancing the Budget: Feature Selection and Tracking for Multi-Camera Visual-Inertial Odometry Oxford Dynamic Robot Systems Group 2021-09-14 | Lintong Zhang, David Wisth, Marco Camurri, Maurice FallonPaper(ArXiv:): arxiv.org/abs/2109.05975 (IEEE): ieeexplore.ieee.org/document/9662207The paper has been accepted for IEEE RA-L/ICRA 2022The multi-camera IMU Lidar dataset can be found at ori-drs.github.io/newer-college-dataset/multi-camAbstractWe present a multi-camera visual-inertial odometry system based on factor graph optimization which estimates motion by using all cameras simultaneously while retaining a fixed overall feature budget. We focus on motion tracking in challenging environments, such as narrow corridors, dark spaces with aggressive motions, and abrupt lighting changes. These scenarios cause traditional monocular or stereo odometry to fail. While tracking motion with extra cameras should theoreticallyprevent failures, it leads to additional complexity and computational burden. To overcome these challenges, we introduce two novel methods to improve multi-camera feature tracking. First, instead of tracking features separately in each camera, we track features continuously as they move from one camera to another. This increases accuracy and achieves a more compact factor graph representation. Second, we select a fixed budget of tracked features across the cameras to reduce back-end optimization time. We have found that using a smaller set of informative features can maintain the same tracking accuracy. Our proposed method was extensively tested using a hardware-synchronized device consisting of an IMU and four cameras (a front stereo pair and two lateral) in scenarios including: an underground mine, large open spaces, and building interiors with narrow stairs and corridors. Compared to stereo-only state-of-the-art visual-inertial odometry methods, our approach reduces the drift rate, relative pose error, by up to 80 % in translation and 39% in rotation.
Safe Visual Navigation in an Underground Mine Oxford Dynamic Robot Systems Group 2021-08-21 | We demonstrate autonomous and safe visual navigation in a decommissioned underground mine. Our systems builds upon our Visual Teach and Repeat pipeline with an extra safety layer given by a reactive twist controller.This is part of our ongoing research on visual navigation for legged platforms in challenging environments:youtube.com/watch?v=lK_xu37BrNgyoutube.com/watch?v=iAY0lyjAnqY
Towards Autonomous Aerial Reconstruction and Monitoring Oxford Dynamic Robot Systems Group 2021-07-26 | This video demonstrates initial progress towards 3D mapping and reconstruction for aerial monitoring and inspectionAerial Robotics Overviewori.ox.ac.uk/labs/drs/aerial-roboticsVILENS - multi-sensor odometryori.ox.ac.uk/labs/drs/vilens-tightly-fused-multi-sensor-odometrySLAMori.ox.ac.uk/labs/drs/lidar-simultaneous-localization-and-mappingMusic by:bensound.com
Online Estimation of Diameter at Breast Height for Trees in Forests Using a Handheld LiDAR Oxford Dynamic Robot Systems Group 2021-07-21 | While mobile LiDAR sensors are increasingly used to scan in ecology and forestry applications, reconstruction and characterisation are typically carried out offline. Motivated by this, we present an online LiDAR system that can run on a handheld device to segment and track individual trees and identify them in a fixed coordinate system. Segments relating to each tree are accumulated over time, and a model of the trees are completed as more scans are captured from different perspectives. Using this reconstruction we then fit a cylinder model to each tree trunk by solving a least-squares optimisation over the points to estimate the trees' Diameter at Breast Height (DBH). Points that are not part of the tree are further removed using RANSAC. Experimental results demonstrate that our system can estimate DBH to within 7 cm accuracy for 90% of individual trees in a forest (Wytham Woods, Oxford UK).
Real-time Mapping of Industrial Structures with the Spot Quadruped Robot Oxford Dynamic Robot Systems Group 2021-07-20 | Robot loaned by the RACE (UKAEA) Hot Robotics Facility, funded by NNUFhotrobotics.co.uknnuf.ac.uk/homeori.ox.ac.uk/labs/drsMusic: bensound.com
Mapping of Industrial Facilities for Gamma Radiation using the Spot Robot Oxford Dynamic Robot Systems Group 2021-07-20 | Robot loaned by the RACE (UKAEA) Hot Robotics Facility, funded by NNUFhotrobotics.co.uknnuf.ac.uk/homeori.ox.ac.uk/labs/drsMusic: bensound.com
Ensuring Safe Visual Teach and Repeat Legged Navigation using Local World Representations Oxford Dynamic Robot Systems Group 2021-06-02 | This paper presents fast and informative local world representations for the safe navigation of legged robots during Visual Teach and Repeat (VT&R) missions. We show how meaningful representations can be computed from the local elevation map using simple image processing techniques, and its closed loop integration with a Riemannian Motion Policies-based twist controller. We demonstrate our approach with ANYmal C in a challenging indoor environment.Accepted to the 5th Full-Day Workshop on Legged Robots at ICRA 2021Authors: Matías Mattamala, Nived Chebrolu, Maurice FallonPDF: drive.google.com/file/d/1pYGr4q2F-uGriIovyVI859n7xHnoSCsc/view?usp=sharingPresentation: youtu.be/Dvy6L799dn4
[Presentation] Elastic and Efficient LiDAR Reconstruction for Large-Scale Exploration Tasks Oxford Dynamic Robot Systems Group 2021-06-02 | Video presentation of the article arxiv.org/abs/2010.09232"Elastic and Efficient LiDAR Reconstruction for Large-Scale Exploration Tasks"Yiduo Wang, Nils Funk, Milad Ramezani, Sotiris Papatheodorou, Marija Popovic, Marco Camurri, Stefan Leutenegger, Maurice FallonIEEE International Conference on Robotics and Automation (ICRA), 2021Abstract:We present an efficient, elastic 3D LiDAR reconstruction framework which can reconstruct up to maximum LiDAR ranges (60 m) at multiple frames per second, thus enabling robot exploration in large-scale environments. Our approach only requires a CPU. We focus on three main challenges of large-scale reconstruction: integration of long-range LiDAR scans at high frequency, the capacity to deform the reconstruction after loop closures are detected, and scalability for long-duration exploration. Our system extends upon a state-of-the-art efficient RGB-D volumetric reconstruction technique, called supereight, to support LiDAR scans and a newly developed submapping technique to allow for dynamic correction of the 3D reconstruction. We then introduce a novel pose graph clustering and submap fusion feature to make the proposed system more scalable for large environments. We evaluate the performance using two public datasets including outdoor exploration with a handheld device and a drone, and with a mobile robot exploring an underground room network. Experimental results demonstrate that our system can reconstruct at 3 Hz with 60 m sensor range and ~5 cm resolution, while state-of-the-art approaches can only reconstruct to 25 cm resolution or 20 m range at the same frequency.
[Presentation] SKD: Keypoint Detection for Point Clouds using Saliency Estimation Oxford Dynamic Robot Systems Group 2021-05-31 | We present a novel keypoint detector that uses saliency to determine the best candidates from a point cloud for tasks such as registration and reconstruction. The approach can be applied to any differentiable deep learning descriptor by using the gradients of that descriptor with respect to input 3D position to estimate an initial set of candidate keypoints. By using a neural network over the set of candidates we learn to refine the point selection until a final set of keypoints is obtained. The key intuition behind this approach is that keypoints are not extracted solely as a result of the geometry surrounding a point, but also take into account the descriptor’s response. To improve the performance of the learned keypoint descriptor we combine the saliency and the feature descriptor to allow the network to select good keypoint candidates. The approach was evaluated on two large LIDAR datasets - the Oxford RobotCar dataset and the KITTI dataset, where we obtain up to 50% improvement over the state-of-the-art in both matchability and repeatability. When performing sparse matching with the keypoints suggested by our method we achieve a higher inlier ratio and faster convergence.Accepted in the IEEE International Conference on Robotics and Automation (ICRA) 2021Authors: Georgi Tinchev, Adrián Peñate-Sánchez, Maurice FallonPDF: robots.ox.ac.uk/~mobile/drs/Papers/2021RAL_tinchev.pdf
RA-L/ICRA 2021 - Unified Landmark Tracking for Odometry [Finalist ICRA Best Student Paper] Oxford Dynamic Robot Systems Group 2021-05-28 | RA-L/ICRA 2021"Unified Multi-Modal Landmark Tracking for Tightly Coupled Lidar-Visual-Inertial Odometry"David Wisth, Marco Camurri, Sandipan Das, Maurice FallonPaper link: arxiv.org/abs/2011.06838VILENS website: ori.ox.ac.uk/labs/drs/vilens-tightly-fused-multi-sensor-odometryAbstract:We present an efficient multi-sensor odometry system for mobile platforms that jointly optimizes visual, lidar, andinertial information within a single integrated factor graph. Thisruns in real-time at full framerate using fixed lag smoothing.To perform such tight integration, a new method to extract 3Dline and planar primitives from lidar point clouds is presented.This approach overcomes the suboptimality of typical frame-to-frame tracking methods by treating the primitives as landmarksand tracking them over multiple scans. True integration oflidar features with standard visual features and IMU is madepossible using a subtle passive synchronization of lidar andcamera frames. The lightweight formulation of the 3D featuresallows for real-time execution on a single CPU. Our proposedsystem has been tested on a variety of platforms and scenarios,including underground exploration with a legged robot andoutdoor scanning with a dynamically moving handheld device,for a total duration of 96 min and 2.4 km traveled distance. Inthese test sequences, using only one exteroceptive sensor leadsto failure due to either underconstrained geometry (affectinglidar) or textureless areas caused by aggressive lighting changes(affecting vision). In these conditions, our factor graph naturallyuses the best information available from each sensor modalitywithout any hard switches.
[Presentation] Learning Camera Performance Models for Active Multi-Camera Visual Teach and Repeat Oxford Dynamic Robot Systems Group 2021-05-28 | In dynamic and cramped industrial environments, achieving reliable Visual Teach and Repeat (VT&R) with a single-camera is challenging. In this work, we develop a robust method for non-synchronized multi-camera VT&R. Our contribution are expected Camera Performance Models (CPM) which evaluate the camera streams from the teach step to determine the most informative one for localization during the repeat step. By actively selecting the most suitable camera for localization, we are able to successfully complete missions when one of the cameras is occluded, faces into feature poor locations or if the environment has changed. Furthermore, we explore the specific challenges of achieving VT&R on a dynamic quadruped robot, ANYmal. The camera does not follow a linear path (due to the walking gait and holonomicity) such that precise path-following cannot be achieved. Our experiments feature forward and backward facing stereo cameras showing VT&R performance in cluttered indoor and outdoor scenarios. We compared the trajectories the robot executed during the repeat steps demonstrating typical tracking precision of less than 10 cm on average. With a view towards omni-directional localization, we show how the approach generalizes to four cameras in simulation.Accepted in the IEEE International Conference on Robotics and Automation (ICRA) 2021Authors: Matías Mattamala, Milad Ramezani, Marco Camurri, Maurice FallonPre-print: arxiv.org/abs/2103.14070PDF: arxiv.org/pdf/2103.14070.pdf
[Presentation] Receding-Horizon Perceptive Trajectory Optimization with Learned Initialization Oxford Dynamic Robot Systems Group 2021-05-27 | Video presentation of the article arxiv.org/abs/2104.09078"Receding-Horizon Perceptive Trajectory Optimization for Dynamic Legged Locomotion with Learned Initialization"Oliwier Melon, Romeo Orsolino, David Surovik, Mathieu Geisert, Ioannis Havoutis, Maurice FallonIEEE International Conference on Robotics and Automation (ICRA), 2021Abstract:To dynamically traverse challenging terrain, legged robots need to continually perceive and reason about upcoming features, adjust the locations and timings of future footfalls and leverage momentum strategically. We present a pipeline that enables flexibly-parametrized trajectories for perceptive and dynamic quadruped locomotion to be optimized in an online, receding-horizon manner. The initial guess passed to the optimizer affects the computation needed to achieve convergence and the quality of the solution. We consider two methods for generating good guesses. The first is a heuristic initializer which provides a simple guess and requires significant optimization but is nonetheless suitable for adaptation to upcoming terrain. We demonstrate experiments using the ANYmal C quadruped, with fully onboard sensing and computation, to cross obstacles at moderate speeds using this technique. Our second approach uses latent-mode trajectory regression (LMTR) to imitate expert data—while avoiding invalid interpolations between distinct behaviors—such that minimal optimization is needed. This enables high-speed motions that make more expansive use of the robot’s capabilities. We demonstrate it on flat groundwith the real robot and provide numerical trials that progress toward deployment on terrain. These results illustrate a paradigm for advancing beyond short-horizon dynamic reactions, toward the type of intuitive and adaptive locomotion planning exhibited by animals and humans.
Multi-Floor Mapping and Exploration with the Spot Robot Oxford Dynamic Robot Systems Group 2021-05-23 | This video demonstrates the mapping system developed by our group running onthe Spot robot as it explores all floors of Oxford Robotics Institute
Real-Time Trajectory Adaptation for Quadrupedal Locomotion using Deep Reinforcement Learning Oxford Dynamic Robot Systems Group 2021-04-15 | "Real-Time Trajectory Adaptation for Quadrupedal Locomotion using Deep Reinforcement Learning," Siddhant Gangapurwala, Mathieu Geisert, Romeo Orsolino, Maurice Fallon and Ioannis Havoutis. IEEE International Conference on Robotics and Automation (ICRA), 2021Abstract:We present a control architecture for real-time adaptation and tracking of trajectories generated using a terrain-aware trajectory optimization solver. This approach enables us to circumvent the computationally exhaustive task of online trajectory optimization, and further introduces a control solution robust to systems modeled with approximated dynamics. We train a policy using deep reinforcement learning (RL) to introduce additive deviations to a reference trajectory in order to generate a feedback-based trajectory tracking system for a quadrupedal robot. We train this policy across a multitude of simulated terrains and ensure its generality by introducing training methods that avoid overfitting and convergence towards local optima. Additionally, in order to capture terrain information, we include a latent representation of the height maps in the observation space of the RL environment as a form of exteroceptive feedback. We test the performance of our trained policy by tracking the corrected set points using a model-based whole-body controller and compare it with the tracking behavior obtained without the corrective feedback in several simulation environments, and show that introducing the corrective feedback results in increase of the success rate from 72.7% to 92.4% for tracking precomputed dynamic long horizon trajectories on flat terrain and from 47.5% to 80.3% on a complex modular uneven terrain. We also show successful transfer of our training approach to the real physical system and further present cogent arguments in support of our framework.
Receding-Horizon Perceptive Trajectory Optimization with Learned Initialization Oxford Dynamic Robot Systems Group 2021-04-15 | "Receding-Horizon Perceptive Trajectory Optimization for Dynamic Legged Locomotion with Learned Initialization"Oliwier Melon, Romeo Orsolino, David Surovik, Mathieu Geisert, Ioannis Havoutis, Maurice FallonIEEE International Conference on Robotics and Automation (ICRA), 2021Abstract:To dynamically traverse challenging terrain, legged robots need to continually perceive and reason about upcoming features, adjust the locations and timings of future footfalls and leverage momentum strategically. We present a pipeline that enables flexibly-parametrized trajectories for perceptive and dynamic quadruped locomotion to be optimized in an online, receding-horizon manner. The initial guess passed to the optimizer affects the computation needed to achieve convergence and the quality of the solution. We consider two methods for generating good guesses. The first is a heuristic initializer which provides a simple guess and requires significant optimization but is nonetheless suitable for adaptation to upcoming terrain. We demonstrate experiments using the ANYmal C quadruped, with fully onboard sensing and computation, to cross obstacles at moderate speeds using this technique. Our second approach uses latent-mode trajectory regression (LMTR) to imitate expert data—while avoiding invalid interpolations between distinct behaviors—such that minimal optimization is needed. This enables high-speed motions that make more expansive use of the robot’s capabilities. We demonstrate it on flat groundwith the real robot and provide numerical trials that progress toward deployment on terrain. These results illustrate a paradigm for advancing beyond short-horizon dynamic reactions, toward the type of intuitive and adaptive locomotion planning exhibited by animals and humans.
ICRA2020 - Actively Mapping Industrial Structure with Information Gain-Based Planning on a Quadruped Oxford Dynamic Robot Systems Group 2021-04-10 | Actively Mapping Industrial Structures with Information Gain-Based Planning on a Quadruped RobotAuthors: Yiduo Wang, Milad Ramezani and Maurice FallonFor more information please visit: ori.ox.ac.uk/active-mapping
Online LiDAR-SLAM for Legged Robots with Robust Registration and Deep-Learned Loop Closure Oxford Dynamic Robot Systems Group 2021-03-25 | Online LiDAR-SLAM for Legged Robots with Robust Registration and Deep-Learned Loop ClosureAuthors: Milad Ramezani, Georgi Tinchev, Egor Iuganov and Maurice FallonFor more information please visit: ori.ox.ac.uk/lidar-slam
Elastic and Efficient LiDAR Reconstruction for Large-Scale Exploration Tasks Oxford Dynamic Robot Systems Group 2021-03-25 | We present an efficient, elastic 3D LiDAR reconstruction framework which can reconstruct up to maximum LiDAR ranges (60 m) at multiple frames per second, thus enabling robot exploration in large-scale environments. Ourapproach only requires a CPU.ori.ox.ac.uk/labs/drs
[Presentation] Reliable Trajectories for Quadrupeds using Analytical Costs & Learned Initializations Oxford Dynamic Robot Systems Group 2021-03-17 | Video presentation of the article: ieeexplore.ieee.org/abstract/document/9196562“Reliable Trajectories for Dynamic Quadrupeds using Analytical Costs and Learned Initializations” Oliwier Melon, Mathieu Geisert, David Surovik, Ioannis Havoutis, Maurice FallonIEEE International Conference on Robotics and Automation (ICRA), 2020Abstract:Dynamic traversal of uneven terrain is a major objective in the field of legged robotics. The most recent model predictive control approaches for these systems can generate robust dynamic motion of short duration; however, planning over a longer time horizon may be necessary when navigating complex terrain. A recently-developed framework, Trajectory Optimization for Walking Robots (TOWR), computes such plans but does not guarantee their reliability on real platforms, under uncertainty and perturbations. We extend TOWR with analytical costs to generate trajectories that a state-of-the-art whole-body tracking controller can successfully execute. To reduce online computation time, we implement a learning-based scheme for initialization of the nonlinear program based on offline experience. The execution of trajectories as long as 16 footsteps and 5.5 s over different terrains by a real quadruped demonstrates the effectiveness of the approach on hardware. This work builds toward an online system which can efficiently and robustly replan dynamic trajectories.
LIDAR SLAM/CML with celebrity narration Oxford Dynamic Robot Systems Group 2021-01-29 | Imitation etc etc ...referencing: youtu.be/G64dGf1fWIw
