Uploaded October 2026 | Updated November 2026, 2 days ago
Adam Kalai ( RESI)
https://simons.berkeley.edu/talks/adam-kalai-resi-2026-10-07
Trustworthy AI: From Hallucinations to Reliable Autonomy
Large language models are typically evaluated using accuracy-based metrics that reward confident guessing while giving little or no credit for acknowledging uncertainty. Because evaluations influence which models and methods are selected, these incentives can work against techniques that reduce hallucinations at the cost of answering fewer questions. The talk discusses approaches to evaluation that better reward useful uncertainty and reflect the differing costs of mistakes, correct answers, and abstentions, along with open questions in designing evaluations whose incentives better match real-world objectives.
Based on joint work with Santosh Vempala, Ofir Nachum, and Edwin Zhang.
Adam Kalai ( RESI)
https://simons.berkeley.edu/talks/adam-kalai-resi-2026-10-07
Trustworthy AI: From Hallucinations to Reliable Autonomy
Large language models are typically evaluated using accuracy-based metrics that reward confident guessing while giving little or no credit for acknowledging uncertainty. Because evaluations influence which models and methods are selected, these incentives can work against techniques that reduce hallucinations at the cost of answering fewer questions. The talk discusses approaches to evaluation that better reward useful uncertainty and reflect the differing costs of mistakes, correct answers, and abstentions, along with open questions in designing evaluations whose incentives better match real-world objectives.
Based on joint work with Santosh Vempala, Ofir Nachum, and Edwin Zhang.










