Structured World Models for Robots – Krishna Murthy @HopkinsEngineer
Structured World Models for Robots – Krishna Murthy  @HopkinsEngineer
Uploaded August 2024 | Updated November 2026, 1 day ago
Computer Science Seminar Series
April 12, 2024

“Structured World Models for Robots”
Krishna Murthy, Massachusetts Institute of Technology

Humans have an innate ability to construct detailed mental representations of the world from limited sensory data. These "world models" are central to natural intelligence, allowing us to perceive, reason about, and act in the physical world. Krishna Murthy's research seeks to create "computational world models"—artificial intelligence techniques that enable robots to understand and operate in the world around as effectively as humans. Despite the impressive successes of modern machine learning approaches in media such as text, images, and video—where abundant training data is readily available—these advancements have not translated to robotics. Building generally capable robotic systems presents unique challenges, including this lack of data and the need to adapt learning algorithms to a wide variety of embodiments, environments, and tasks of interest. In his talk, Murthy will present how his research contributes to the design of computational models for spatial, physical, and multimodal understanding. He will discuss differentiable computing approaches that have advanced the field of spatial perception, enabling an understanding of the structure of the 3D world, its constituent objects, and their semantic and physical properties from videos. He will also detail how his work interfaces advances in large image, language, and audio models with 3D scenes, enabling robots and computer vision systems to flexibly query these structured world models for a wide range of tasks. Finally, he will outline his vision for the future, where structured world models and modern scaling-based approaches work in tandem to create versatile robot perception and planning algorithms with the potential to meet and ultimately surpass human-level capabilities.

Krishna Murthy is a postdoctoral researcher at the Massachusetts Institute of Technology working with Antonio Torralba and Josh Tenenbaum. He previously completed his PhD at Mila and the University of Montreal, where he was advised by Liam Paull. Murthy's research focuses on building computational world models to help embodied agents perceive, reason about, and act in the physical world. He has led the organization of multiple workshops on themes spanning differentiable programming, physical reasoning, 3D vision and graphics, and ML research dissemination. His research has been recognized with graduate fellowship awards from NVIDIA and Google (2021); a Best Paper Award from the Institute of Electrical and Electronics Engineers' Robotics and Automation Letters (2019); and an induction to the Robotics: Science and Systems Pioneers cohort (2020).
Structured World Models for Robots – Krishna MurthyThe Don P. Giddens Inaugural Professorial Lecture: Mark DredzeMaryland Partnership in Energy (MDPE) Information SessionBlueprint MedTech Annual Meeting 2025: From Innovation to Impact - October 20, 20252026 WSE Pledge of the Computing Professional Induction CeremonyBuilding Certifiably Safe and Correct Large-Scale Autonomous Systems – Chuchu FanNortheast States Collaborative Webinar on Identifying Candidate Interregional Transmission ProjectsEmeritus Professorial Lecture: Aviel RubinAI-Driven Scientific Discovery – Yuanqi DuFairness in Algorithmic Services – Allison KoeneckeGenome Assembly: The Long and Short of It – Sergey KorenMeet Bailey Surtees, WSE Alum 17
Johns Hopkins Engineering |

Structured World Models for Robots – Krishna Murthy

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER