Uploaded June 2026 | Updated September 2026, 3 weeks ago
For a young Tixiao Shan growing up in China, a broken Nintendo 64 controller wasn’t the end of a game—it was the beginning of a journey to understand how electronics work. That drive has since propelled him to the forefront of research on robotic perception. At nonprofit research institute SRI International’s Scene Understanding and Navigation (SUN) Group in Princeton, N.J., he is teaching autonomous machines how to perceive and navigate our messy, unpredictable world. While we marvel at videos of next-gen robots walking or running around test rooms, Shan and his colleagues know that being able to move around a space is useful only if you understand that space.
To solve this problem, Shan tries to help robots build a picture of their surroundings by tracking their own movement with sensors in real time and then using artificial intelligence to interpret those data so they can understand what those surroundings are. For example, a robot that encounters a chair would know that the obstacle is a chair, that it is light enough to move and that it should be replaced when the robot is done. By bridging the gap between basic mapping and humanlike understanding, this technology would allow robots to take over mundane, repetitive or dangerous tasks, whether part of household maintenance or saving lives in war.
Shan’s recent endeavors, including the Graph2Nav framework, allow robots to recognize objects and understand the relations between them instead of merely seeing the world as a collection of geometric shapes and obstacles. If a user asks the robot to find a phone in a bedroom, because it understands relational concepts, the robot knows a phone is likely to be found on a desk or near a bed and plans its navigation accordingly.
Shan has licensed his technology to start-ups working on household robot assistants and sees a future where his technology drives smart domestic robots that could help seniors live more independent lives. “In the near future, we can have such robots, especially for senior people who cannot move freely or who have trouble moving things around,” Shan says. “The robot can do that for them.” —David M. Ewalt
For a young Tixiao Shan growing up in China, a broken Nintendo 64 controller wasn’t the end of a game—it was the beginning of a journey to understand how electronics work. That drive has since propelled him to the forefront of research on robotic perception. At nonprofit research institute SRI International’s Scene Understanding and Navigation (SUN) Group in Princeton, N.J., he is teaching autonomous machines how to perceive and navigate our messy, unpredictable world. While we marvel at videos of next-gen robots walking or running around test rooms, Shan and his colleagues know that being able to move around a space is useful only if you understand that space.
To solve this problem, Shan tries to help robots build a picture of their surroundings by tracking their own movement with sensors in real time and then using artificial intelligence to interpret those data so they can understand what those surroundings are. For example, a robot that encounters a chair would know that the obstacle is a chair, that it is light enough to move and that it should be replaced when the robot is done. By bridging the gap between basic mapping and humanlike understanding, this technology would allow robots to take over mundane, repetitive or dangerous tasks, whether part of household maintenance or saving lives in war.
Shan’s recent endeavors, including the Graph2Nav framework, allow robots to recognize objects and understand the relations between them instead of merely seeing the world as a collection of geometric shapes and obstacles. If a user asks the robot to find a phone in a bedroom, because it understands relational concepts, the robot knows a phone is likely to be found on a desk or near a bed and plans its navigation accordingly.
Shan has licensed his technology to start-ups working on household robot assistants and sees a future where his technology drives smart domestic robots that could help seniors live more independent lives. “In the near future, we can have such robots, especially for senior people who cannot move freely or who have trouble moving things around,” Shan says. “The robot can do that for them.” —David M. Ewalt
