When Can We Trust What a Model Has Learned? Identifiability, Causality, and Reliable AI @SimonsInstitute
When Can We Trust What a Model Has Learned? Identifiability, Causality, and Reliable AI  @SimonsInstitute
Uploaded October 2026 | Updated November 2026, 3 days ago
Pradeep Ravikumar (Carnegie Mellon University)
https://simons.berkeley.edu/talks/pradeep-ravikumar-carnegie-mellon-university-2026-10-05
Trustworthy AI: From Hallucinations to Reliable Autonomy

Modern AI systems can achieve remarkable predictive and generative performance while remaining unreliable under distribution shift, novel compositions, or intervention. A fundamental reason is non-identifiability: many different internal representations and mechanisms can explain the same observed data and training objective, while implying very different behavior outside the training distribution. Thus, before asking whether a learned representation is interpretable, truthful, or causally meaningful, we should ask a more basic question: what aspects of what the model has learned are actually determined by the available evidence?

I will discuss a line of work that uses identifiability and causality as organizing principles for reliable representation learning. The central idea is that additional structure, such as sparsity, variation across environments, weak supervision, and assumptions about how causal mechanisms change, can turn otherwise underdetermined latent representations into recoverable, semantically meaningful abstractions. I will describe examples ranging from discovering latent causal variables and mechanisms to identifying interpretable concepts inside foundation models, where controlled contrasts and environmental variation can distinguish causally active concepts from merely correlated directions.

This perspective suggests a route from prediction to reliable autonomy: rather than relying solely on increasingly capable models together with post-hoc checks, we can ask what latent variables, concepts, and mechanisms are identifiable; what interventions they support; and which aspects of their behavior can therefore be expected to remain stable as the world changes.
When Can We Trust What a Model Has Learned? Identifiability, Causality, and Reliable AIGround Energy estimation of Quantum Impurity model is in BQPStructure vs. Randomness (Part 2)Graphs versus Manifolds in Eigenvalues and Distances Between SubsetsDerandomizing Space-Bounded Computation (Part 1)Probability, Information, and DiscrepancyCondensers/extractors for block sources surveyStrongly explicit sorting on HDXsQuantifying Reliability in AI: Hallucinations and UncertaintySpectral approximation of directed graphs and moreHigh-Accuracy Sampling for Diffusion Models and Log-Concave DistributionsExplicit free expanders
Simons Institute for the Theory of Computing |

When Can We Trust What a Model Has Learned? Identifiability, Causality, and Reliable AI

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER