Uploaded May 2025 | Updated September 2026, 2 weeks ago
In this video, Moritz Hardt, Max Planck Institute for Intelligent Systems, Germany dives into the crucial role that benchmarks play in the machine learning community, a concept that has evolved since the 1980s. While benchmarks are central to research, there’s still much we don’t fully understand about them. In this talk, we explore the foundations of an emerging science of benchmarks, focusing on key insights such as annotator errors, the external validity of model rankings, and the potential of multi-task benchmarks. The findings challenge traditional views and highlight the importance of advancing our understanding of benchmarks in machine learning.
This talk took place on Friday, October 25 at the 2024 SIAM Conference on Mathematics of Data Science.
Learn more about SIAM siam.org
Join SIAM siam.org/membership/individual-membership
Attend a SIAM Conference siam.org/conferences-events
#benchmark #annotating #mathematicalmodeling #machinelearning #spellcheck #datascience #mathematics #mathematicseducation
In this video, Moritz Hardt, Max Planck Institute for Intelligent Systems, Germany dives into the crucial role that benchmarks play in the machine learning community, a concept that has evolved since the 1980s. While benchmarks are central to research, there’s still much we don’t fully understand about them. In this talk, we explore the foundations of an emerging science of benchmarks, focusing on key insights such as annotator errors, the external validity of model rankings, and the potential of multi-task benchmarks. The findings challenge traditional views and highlight the importance of advancing our understanding of benchmarks in machine learning.
This talk took place on Friday, October 25 at the 2024 SIAM Conference on Mathematics of Data Science.
Learn more about SIAM siam.org
Join SIAM siam.org/membership/individual-membership
Attend a SIAM Conference siam.org/conferences-events
#benchmark #annotating #mathematicalmodeling #machinelearning #spellcheck #datascience #mathematics #mathematicseducation
