Evaluating factuality in long-form generation @SimonsInstitute
Evaluating factuality in long-form generation  @SimonsInstitute
Uploaded October 2026 | Updated November 2026, 3 days ago
Mohit Iyyer (University of Maryland, College Park)
https://simons.berkeley.edu/talks/mohit-iyyer-university-maryland-college-park-2026-10-09
Trustworthy AI: From Hallucinations to Reliable Autonomy

In this talk, I consider the growing problem of evaluating the factuality of long-form texts generated by LLMs (e.g., articles, essays, books): as the length and complexity of long-form writing tasks increases, how do we even measure progress? First, I propose a high-level framework (applicable to both human and automatic evaluation) that decomposes a long-form text into simpler atomic units before then evaluating each unit on a specific aspect. I demonstrate the framework's effectiveness at evaluating factuality and faithfulness on tasks such as long-form question answering and book summarization, and also discuss how we can speed up this pipeline to scale up evaluations. Next, I'll discuss evaluation in the context of agentic writing pipelines, in which models can use tools and write/execute code to aid the production of long-form reports, focusing on our recent AnalystBench project. I'll conclude with some open problems: (1) reliance on faulty retrievers / search engines; (2) verifying complex, highly-compositional claims; and (3) evaluating interpretive claims rather than fully objective ones.
Evaluating factuality in long-form generationTalk by Max Hopkins (Institute for Advanced Study)Sampling Sphere Packings with Continuum Glauber DynamicsEntropic independenceAldous-type Spectral Gaps in Unitary GroupsSparse Random Graphs and Random Matrix Statistics (Part 2)Pseudorandomness, symmetry, smoothingRandom hyperbolic surfaces (Part 2)The Quest for Explicit Constructions in Coding TheoryA Look at Entropic Ricci Curvature on Weighted GraphsSign-balance of random Laplace eigenfunctionsRevisiting the xor lemma and exponential correlation bounds for polynomials
Simons Institute for the Theory of Computing |

Evaluating factuality in long-form generation

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER