Learning, Reasoning, and Planning with Neuro-Symbolic Concepts

Uploaded April 2025 | Updated September 2026, 2 weeks ago

Allen School Colloquia Series
Title: Learning, Reasoning, and Planning with Neuro-Symbolic Concepts
Speaker: Jiayuan Mao (MIT)
Date: April 14, 2025

Abstract: I aim to build complete intelligent agents that can continually learn, reason, and plan: answer queries, infer human intentions, and make long-horizon plans spanning hours to days. In this talk, I will describe a general learning and reasoning framework based on neuro-symbolic concepts. Drawing inspiration from theories and studies in cognitive science, neuro-symbolic concepts serve as compositional abstractions of the physical world, representing object properties, relations, and actions. These concepts can be combinatorially reused in flexible and novel ways. Technically, each neuro-symbolic concept is represented as a combination of symbolic programs, which define how concepts can be structurally combined (similar to the ways that words form sentences in human language), and modular neural networks, which ground concept names in sensory inputs and agent actions. I show that systems that leverage neuro-symbolic concepts demonstrate superior data efficiency, enable agents to reason and plan more quickly, and achieve strong generalization in novel situations and for novel goals. This is illustrated in visual reasoning in 2D, 3D, motion, and video data, as well as in diverse decision-making tasks spanning virtual agents and real-world robotic manipulation.

BIO: Jiayuan Mao is a Ph.D. student at MIT, advised by Professors Josh Tenenbaum and Leslie Kaelbling. Her research agenda is to build machines that can continually learn concepts (e.g., properties, relations, rules, and skills) from their experiences and apply them for reasoning and planning in the physical world. Her research topics include visual reasoning, robotic manipulation, scene and activity understanding, and language acquisition. She was named a Rising Star in EECS (2024) and in Generative AI (2024). Her research has received Best Paper Awards at CogSci 2024, SoCal NLP 2024, and the CoRL 2024 Workshop on Language and Robot Learning, as well as a Best Paper nomination at ACL 2019.• Slides: I will submit my slides one week in advance.

This video is closed captioned.

Learning, Reasoning, and Planning with Neuro-Symbolic Concepts–Jiayuan Mao (MIT)

$Pareto-efficient AI systems—Simran Arora (Stanford) Allen School Colloquia Series Title: Pareto-efficient AI systems: Expanding the quality and efficiency frontier of AI Speaker: Simran Arora (Stanford) Date: April 1, 2025 Abstract: We have made exciting progress in AI by scaling massive models on massive amounts of data center compute. However, this represents a small fraction of AI’s potential. My work expands the Pareto frontier between the AI capabilities we can achieve and the long tail of compute constraints. In this talk, we piece-by-piece build up to a language model architecture that expands the Pareto frontier between quality and throughput efficiency. The Transformer, AI’s current workhorse architecture, is memory hungry, limiting its throughput, or amount of data it can process per second. This has led to a Cambrian explosion of alternate architecture candidates proposed across prior work. Prior work paints an exciting picture: there are architectures that are asymptotically faster than the Transformer, while also matching its quality. However, I ask, if we’re using asymptotically faster building blocks, what if anything are we giving up in quality? 1. In part one, we understand the tradeoffs and show indeed, there’s no free lunch. I present my work to identify and explain the fundamental quality and efficiency tradeoffs between different classes of architectures. Methods I developed for this analysis are now ubiquitous in the development of efficient language models. 2. In part two, we measure how existing architecture candidates fare along on the tradeoff space. While many proposed architectures are asymptotically fast, they are not wall-clock fast compared to the Transformer. I present ThunderKittens, a programming library that I built to help AI researchers develop hardware-efficient AI algorithms. 3. In part three, we expand the Pareto frontier of the tradeoff space. I present the BASED architecture, which is built from simple, hardware-efficient components. In culmination, I released a suite of state-of-the-art 8B-405B parameter Transformer-free language models, per standard evaluations, all on an academic budget. Given the massive investment into AI models, this work blending AI and systems has had significant impact and adoption in research, open-source, and industry. Bio: Simran Arora is a PhD student at Stanford University advised by Chris Ré. Her research blends AI and systems towards expanding the Pareto frontier between AI capabilities and efficiency. Her machine learning research has appeared as Oral and Spotlight presentations at NeurIPS, ICML, and ICLR, including an Outstanding Paper award at NeurIPS and Best Paper award at ICML ES-FoMo. Her systems work has appeared at VLDB, SIGMOD, CIDR, and CHI, and her systems artifacts are widely used in research, open-source, and industry. In 2023, Simran created and taught the CS229s Systems for Machine Learning course at Stanford. She has also been supported by a SGF Sequoia Fellowship and the Stanford Computer Science Graduate Fellowship. This video is closed captioned.$

IFDS Workshop–Multi-frequency progressive refinement for learned inverse scattering

OpenThePaths 2025–Keynote: Community-First Autonomy

[Audio Descriptions] Faculty In Focus: Natasha Jaques

[Audio Descriptions] I Am CSE: Ather Sharif

IFDS Workshop Short Talks–Detecting Model Misuse Through LLM fingerprinting

[Audio Descriptions] Faculty In Focus: Stephanie Wang

2026 Winter Robotics Colloquium: Terry Fong (NASA)

IFDS Workshop Short Talks–Heavy-tailed noise in Linear and Kernel Bandits

CSE 481 Networks and Mobile AI Capstone–Group 4: Physical Therapy

Genesis: Artificial Intelligence, Hope, and the Human Spirit—Craig Mundie (Mundie & Associates)