Uploaded June 2024 | Updated November 2026, 4 days ago
Institute for Assured Autonomy & Computer Science Seminar Series
November 12, 2020
“Anomaly Detection Through Explanations”
Leilani Gilpin, Massachusetts Institute of Technology
Under most conditions, complex systems are imperfect. When errors occur, as they inevitably will, systems need to be able to (1) localize the error and (2) take appropriate action to mitigate the repercussions of that error. In this talk, Leilani Gilpin presents new methodologies for detecting and explaining errors in complex systems. Her novel contribution is a system-wide monitoring architecture, which is composed of introspective, overlapping committees of subsystems. Each subsystem is encapsulated in a “reasonableness” monitor, an adaptable framework that supplements local decisions with commonsense data and reasonableness rules. This framework is dynamic and introspective: It allows each subsystem to defend its decisions in different contexts—to the committees it participates in and to itself. For reconciling system-wide errors, she developed a comprehensive architecture called “Anomaly Detection through Explanations" (ADE). The ADE architecture contributes an explanation synthesizer that produces an argument tree, which in turn can be traced and queried to determine the support of a decision and to construct counterfactual explanations. She has applied this methodology to detect incorrect labels in semiautonomous vehicle data and to reconcile inconsistencies in simulated anomalous driving scenarios. Her work has opened up the new area of explanatory anomaly detection, working toward a vision in which complex systems will be articulate by design: They will be dynamic, internal explanations will be part of the design criteria, system-level explanations will be provided, and they can be challenged in an adversarial proceeding.
Leilani H. Gilpin is a research scientist at Sony AI and a collaborating researcher at the Massachusetts Institute of Technology Computer Science and Artificial Intelligence Laboratory. Her research focuses on enabling opaque autonomous systems to explain themselves for robust decision-making, system debugging, and accountability. Her current work integrates explainability into reinforcement learning. She has a PhD in computer science from MIT, an MS in computational and mathematical engineering from Stanford University, and a BS in mathematics with honors, a BS in Computer Science with highest honors, and a music minor from the University of California San Diego. She is currently co-organizing the Association for the Advancement of Artificial Intelligence's Fall Symposium on Anticipatory Thinking, where she is the lead of the autonomous vehicle challenge problem. Outside of research, Gilpin enjoys swimming, cooking, rowing, and org-mode.
Institute for Assured Autonomy & Computer Science Seminar Series
November 12, 2020
“Anomaly Detection Through Explanations”
Leilani Gilpin, Massachusetts Institute of Technology
Under most conditions, complex systems are imperfect. When errors occur, as they inevitably will, systems need to be able to (1) localize the error and (2) take appropriate action to mitigate the repercussions of that error. In this talk, Leilani Gilpin presents new methodologies for detecting and explaining errors in complex systems. Her novel contribution is a system-wide monitoring architecture, which is composed of introspective, overlapping committees of subsystems. Each subsystem is encapsulated in a “reasonableness” monitor, an adaptable framework that supplements local decisions with commonsense data and reasonableness rules. This framework is dynamic and introspective: It allows each subsystem to defend its decisions in different contexts—to the committees it participates in and to itself. For reconciling system-wide errors, she developed a comprehensive architecture called “Anomaly Detection through Explanations" (ADE). The ADE architecture contributes an explanation synthesizer that produces an argument tree, which in turn can be traced and queried to determine the support of a decision and to construct counterfactual explanations. She has applied this methodology to detect incorrect labels in semiautonomous vehicle data and to reconcile inconsistencies in simulated anomalous driving scenarios. Her work has opened up the new area of explanatory anomaly detection, working toward a vision in which complex systems will be articulate by design: They will be dynamic, internal explanations will be part of the design criteria, system-level explanations will be provided, and they can be challenged in an adversarial proceeding.
Leilani H. Gilpin is a research scientist at Sony AI and a collaborating researcher at the Massachusetts Institute of Technology Computer Science and Artificial Intelligence Laboratory. Her research focuses on enabling opaque autonomous systems to explain themselves for robust decision-making, system debugging, and accountability. Her current work integrates explainability into reinforcement learning. She has a PhD in computer science from MIT, an MS in computational and mathematical engineering from Stanford University, and a BS in mathematics with honors, a BS in Computer Science with highest honors, and a music minor from the University of California San Diego. She is currently co-organizing the Association for the Advancement of Artificial Intelligence's Fall Symposium on Anticipatory Thinking, where she is the lead of the autonomous vehicle challenge problem. Outside of research, Gilpin enjoys swimming, cooking, rowing, and org-mode.










