Observability: Role of Evals, Benchmarks & Data in Frontier AI | Alex Ratner from Snorkel AI @scaleconference
Observability: Role of Evals, Benchmarks & Data in Frontier AI | Alex Ratner from Snorkel AI  @scaleconference
Uploaded June 2026 | Updated November 2026, 2 days ago
The excitement around agentic AI is real — backed by quantitative progress on model cards and genuine leaps in capability. But our ability to measure AI has been outpaced by our ability to develop it, and closing this evaluation gap is one of the most important problems facing the field. More enduring benchmarks are needed to advance the next vectors of capability and chart the path to reliable agents.

In this talk, Snorkel AI Co-Founder and CEO Alex Ratner will share insights from major research and benchmark collaborations on agentic coding and continual learning, along with practical tips from working with global frontier labs and leading academics. He'll focus on three dimensions where today's models most often break down, and where the next generation of benchmarks will need to deliver real signal: environment complexity (how dynamic and rich the operating world is), autonomy horizon (how far an agent can act independently), and output complexity (how sophisticated and verifiable the deliverable is).

Want to learn more about the @Scale Conference? Visit our website here: atscaleconference.com
Observability: Role of Evals, Benchmarks & Data in Frontier AI | Alex Ratner from Snorkel AIThe way we work is changing!Agentic Product Safety from Pindrop | Nicholas Holland from PindropHow can world models help advance robotics?Teaching AI to Fight Fires: Autonomous Reliability Agents at Meta Scale | Gaurav Mitra from MetaRegister Today: Join the @Scale community!Live Q&A | Systems & Reliability 2026How AI Is Reshaping Product Management | Danran Chen, Senior Product Manager of AI at ZoomLive from SCCC: Building scalable AI cloud interconnects with SRv6 | Alexey Gorovoy, NebiusClosing Keynote | Peter Hoose from MetaLive Panel: AI Generated Media Provenance at ScaleInside Microsofts Approach to Secure Production Debugging
@Scale |

Observability: Role of Evals, Benchmarks & Data in Frontier AI | Alex Ratner from Snorkel AI

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER