Uploaded April 2024 | Updated September 2026, 6 hours ago
Microsoft Future Leaders in Robotics and AI Seminar Series: Towards Socially Aware Visual Navigation with Hierarchical Learning
Online Seminar
Faith Johnson
PhD Candidate
Rutgers University
Visual navigation follows the intuition that humans can navigate without detailed maps. A common approach is interactive exploration while building a topological graph with images at nodes that can be used for planning. Recent variations learn from passive videos and can navigate using complex social and semantic cues. However, a significant number of training videos are needed, large graphs are utilized, and scenes are not unseen since odometry is utilized. We introduce a new approach to visual navigation using feudal learning, which employs a hierarchical structure consisting of a worker agent, a mid-level manager, and a high-level manager. Key to the feudal learning paradigm, agents at each level see a different aspect of the task and operate at different spatial and temporal scales. Two unique modules are developed in this framework. For the high- level manager, we learn a memory proxy map in a self supervised manner to record prior observations in a learned latent space and avoid the use of graphs and odometry. For the mid-level manager, we develop a waypoint network that outputs intermediate subgoals imitating human waypoint selection during local navigation. This waypoint network is pre-trained using a new, small set of teleoperation videos that we make publicly available, with training environments different from testing environments. The resulting feudal navigation network achieves near SOTA performance, while providing a novel no-RL, no-graph, no-odometry, no-metric map approach to the image goal navigation task.
For more information on the Maryland Robotics Center see:
https://robotics.umd.edu
Microsoft Future Leaders in Robotics and AI Seminar Series: Towards Socially Aware Visual Navigation with Hierarchical Learning
Online Seminar
Faith Johnson
PhD Candidate
Rutgers University
Visual navigation follows the intuition that humans can navigate without detailed maps. A common approach is interactive exploration while building a topological graph with images at nodes that can be used for planning. Recent variations learn from passive videos and can navigate using complex social and semantic cues. However, a significant number of training videos are needed, large graphs are utilized, and scenes are not unseen since odometry is utilized. We introduce a new approach to visual navigation using feudal learning, which employs a hierarchical structure consisting of a worker agent, a mid-level manager, and a high-level manager. Key to the feudal learning paradigm, agents at each level see a different aspect of the task and operate at different spatial and temporal scales. Two unique modules are developed in this framework. For the high- level manager, we learn a memory proxy map in a self supervised manner to record prior observations in a learned latent space and avoid the use of graphs and odometry. For the mid-level manager, we develop a waypoint network that outputs intermediate subgoals imitating human waypoint selection during local navigation. This waypoint network is pre-trained using a new, small set of teleoperation videos that we make publicly available, with training environments different from testing environments. The resulting feudal navigation network achieves near SOTA performance, while providing a novel no-RL, no-graph, no-odometry, no-metric map approach to the image goal navigation task.
For more information on the Maryland Robotics Center see:
https://robotics.umd.edu
![Maryland Robotics Center Seminar, January 31, 2025: Markus P. Nemitz [REUPLOAD]](https://i.ytimg.com/vi/g1aHjIuysYI/mqdefault.jpg "Maryland Robotics Center Seminar, January 31, 2025: Markus P. Nemitz [REUPLOAD]
Maryland Robotics Center Seminar: Design. Print. Deploy. Enabling Swarm Robotics via Additive Manufacturing
Markus P. Nemitz, Ph.D.
Assistant Professor, Mechanical Engineering
Tufts University
Swarm robots offer transformative potential for applications where rapid and efficient coverage of large areas is critical. However, the high cost and fragility of advanced robots, coupled with the limited functionality of affordable alternatives, have historically hindered their large-scale deployment, confining much of swarm robotics research to simulations. While drones have successfully evolved into capable, low-cost swarm robots through commercialization, their fragility and inability to physically interact with the environment have limited their use in contact-based tasks and underwater operations. To address these challenges, our research focuses on the rapid design and fabrication of low-cost, capable, and scalable terrestrial swarm robots using additive manufacturing. In this talk, I will present strategies for creating increasingly intelligent yet affordable robots, advancing the cost-capability trade-off in robotics. I will showcase developments in 3D-printed soft quadrupeds, fluidic actuators, and controllers, exploring their current performance, future potential, and how they bridge the gap between laboratory prototypes and real-world applications. By advancing swarm engineering, we unlock new opportunities for distributed problem solving, such as explosive ordnance disposal, where expendable robots can be produced at a cost lower than the landmines they neutralize.
For more information on the Maryland Robotics Center see:
https://robotics.umd.edu")
