Agreement and Alignment for Human-AI Collaboration – Aaron Roth @HopkinsEngineer
Agreement and Alignment for Human-AI Collaboration – Aaron Roth  @HopkinsEngineer
Uploaded October 2025 | Updated November 2026, 2 hours ago
Gerald M. Masson Distinguished Lecture Series
October 23, 2025

“Agreement and Alignment for Human-AI Collaboration”
Aaron Roth, University of Pennsylvania

As AI models become increasingly powerful, it is an attractive proposition to use them in important decision-making pipelines in collaboration with human decision-makers. But how should a human being and a machine learning model collaborate to reach decisions that are better than either of them could achieve on their own? If the human and the AI model were perfect Bayesians, operating in a setting with a commonly known and correctly specified prior, Aumann's classical agreement theorem would give us one answer: They could engage in conversation about the task at hand, and their conversation would be guaranteed to converge to (accuracy-improving) agreement. This classical result, however, would require making many implausible assumptions, both about the knowledge and computational power of both parties. We show how to recover similar (and more general) results using only computationally and statistically tractable assumptions, which substantially relax full Bayesian rationality. In the second part of the talk, we go on to consider a more difficult problem: that the AI model might be acting at least in part to advance the interests of its designer, rather than the interests of its user, which might be in tension. We show how market competition between different AI providers can mitigate this problem assuming only a mild "market alignment" assumption—that the user's utility function lies in the convex hull of the AI providers' utility functions—even when no single provider is well aligned. In particular, we show that in all Nash equilibria of the AI providers under this market alignment condition, the user is able to advance her own goals as well as they could have in collaboration with a perfectly aligned AI model. This talk describes the results of three papers—Tractable Agreement Protocols (2025 ACM Symposium on Theory of Computing), Collaborative Prediction: Tractable Information Aggregation via Agreement (ACM-SIAM Symposium on Discrete Algorithms), and Emergent Alignment from Competition—which are joint works with Natalie Collina, Ira Globus-Harris, Surbhi Goel, Varun Gupta, Emily Ryu, and Mirah Shi.

Aaron Roth is the Henry Salvatori Professor of Computer and Cognitive Science in the Department of Computer and Information Science at the University of Pennsylvania, with a secondary appointment in the Wharton Department of Statistics and Data Science. He is affiliated with the Warren Center for Network and Data Sciences and is the co-director of the Networked & Social Systems Engineering program. Roth is also an Amazon Scholar at Amazon Web Services. He is the recipient of the Hans Sigrist Prize, a Presidential Early Career Award for Scientists and Engineers, a Sloan Research Fellowship, an NSF CAREER Award, and research awards from Yahoo, Amazon, and Google. Roth's research focuses on the algorithmic foundations of data privacy, algorithmic fairness, game theory, learning theory, and machine learning. He has authored two books—The Algorithmic Foundations of Differential Privacy with Cynthia Dwork and The Ethical Algorithm with Michael Kearns.
Agreement and Alignment for Human-AI Collaboration – Aaron RothDoctoral Hooding Ceremony - May 19, 2025What Generative Visual Models Understand (and Dont) About the Physical World – Anand BhattadAn Overview of the NIST Trustworthy AI Program – Elham TabassiDedication of the Sardella Department Head & Installation of the Inaugural Recipient Michael KesslerExperimental Design for Data-Efficient Scientific Discovery – Quan NguyenMicro-Robotic Swarm Locomotion – Oren GalLinking Data with Decision-Making for Air Quality — Tracey Holloway (Wierman Lecture - Mar 28, 2025)New HyFAM Method Combines Additive and Formative Manufacturing to Fill the Gaps in 3D PrintingAnomaly Detection Through Explanations – Leilani GilpinDesign Day 2026 Keynote Speaker: Carol ReileyJohns Hopkins Engineering Innovation Pre-college Programs Info Session 2026
Johns Hopkins Engineering |

Agreement and Alignment for Human-AI Collaboration – Aaron Roth

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER