Uploaded January 2026 | Updated September 2026, 1 week ago
Computer Science Seminar Series
January 20, 2026
“Beyond the Benchmarking Paradigm: Audits and Evaluation in the Age of Artificial Intelligence”
Inioluwa Deborah Raji, University of California, Berkeley
Despite great potential, there is a growing gap between what AI systems promise and what they deliver, with real human costs. AI auditing is the practice of independently evaluating deployed AI systems to determine how they behave, what risks they pose, and whether they meet their intended objectives. This interdisciplinary endeavor requires both a technical expansion of our current AI evaluation paradigm and a framework for ensuring that audit investigations are sufficiently material for downstream legal actions and normative debates. At the intersection of law and public policy, applied economics and computer science, we can advance AI auditing policy and practice in material ways—by anchoring notions of engineering responsibility in AI development, expanding our vocabulary of AI evaluation methods, and pushing to connect AI audit outcomes to organizational and legal consequences. Through case studies of AI use in health care and government, we demonstrate how novel evaluation methods such as incident reporting, workflow simulations, and pilot experiments can supplement standard practices like data benchmarking to more adequately inform AI governance, shaping a range of outcomes from documentation and procurement to regulatory enforcement and product safety compliance. As auditing makes its way into key policy proposals as a primary mechanism for AI accountability, we must think critically about the necessary technical and institutional infrastructure required for this form of oversight to successfully enable safe widespread AI adoption.
Inioluwa Deborah Raji is a researcher at the University of California, Berkeley who is interested in algorithmic auditing. She has worked closely with industry, civil society, and within academia to push forward various projects to operationalize ethical considerations in machine learning practice and push forward benchmarking and model evaluation norms in the field. In particular, Raji aims to study how model engineering choices (from evaluation to data choices) impact consumer protection, product liability, procurement, anti-discrimination practice, and other forms of legal and institutional accountability related to functional harms. She is on the advisory boards for the Center for Democracy and Technology AI Governance Lab, the Health AI Partnership, TeachAI, REAL ML, and the Leadership Conference on Civil and Human Rights Center for Civil Rights and Technology. For her efforts, Raji has been named to Forbes' 30 Under 30, MIT Technology Review's Innovators Under 35, and TIME's 100 Most Influential People in AI lists. She is also the recipient of the 2024 Tech For Humanity Prize and the 2024 Mozilla Rise25 award, and is a co-recipient of the Electronic Frontier Foundation Pioneer Award along with Joy Buolamwini and Timnit Gebru. Raji received her bachelor's of applied science in engineering science from the University of Toronto. She is currently completing her PhD in computer science at UC Berkeley.
Computer Science Seminar Series
January 20, 2026
“Beyond the Benchmarking Paradigm: Audits and Evaluation in the Age of Artificial Intelligence”
Inioluwa Deborah Raji, University of California, Berkeley
Despite great potential, there is a growing gap between what AI systems promise and what they deliver, with real human costs. AI auditing is the practice of independently evaluating deployed AI systems to determine how they behave, what risks they pose, and whether they meet their intended objectives. This interdisciplinary endeavor requires both a technical expansion of our current AI evaluation paradigm and a framework for ensuring that audit investigations are sufficiently material for downstream legal actions and normative debates. At the intersection of law and public policy, applied economics and computer science, we can advance AI auditing policy and practice in material ways—by anchoring notions of engineering responsibility in AI development, expanding our vocabulary of AI evaluation methods, and pushing to connect AI audit outcomes to organizational and legal consequences. Through case studies of AI use in health care and government, we demonstrate how novel evaluation methods such as incident reporting, workflow simulations, and pilot experiments can supplement standard practices like data benchmarking to more adequately inform AI governance, shaping a range of outcomes from documentation and procurement to regulatory enforcement and product safety compliance. As auditing makes its way into key policy proposals as a primary mechanism for AI accountability, we must think critically about the necessary technical and institutional infrastructure required for this form of oversight to successfully enable safe widespread AI adoption.
Inioluwa Deborah Raji is a researcher at the University of California, Berkeley who is interested in algorithmic auditing. She has worked closely with industry, civil society, and within academia to push forward various projects to operationalize ethical considerations in machine learning practice and push forward benchmarking and model evaluation norms in the field. In particular, Raji aims to study how model engineering choices (from evaluation to data choices) impact consumer protection, product liability, procurement, anti-discrimination practice, and other forms of legal and institutional accountability related to functional harms. She is on the advisory boards for the Center for Democracy and Technology AI Governance Lab, the Health AI Partnership, TeachAI, REAL ML, and the Leadership Conference on Civil and Human Rights Center for Civil Rights and Technology. For her efforts, Raji has been named to Forbes' 30 Under 30, MIT Technology Review's Innovators Under 35, and TIME's 100 Most Influential People in AI lists. She is also the recipient of the 2024 Tech For Humanity Prize and the 2024 Mozilla Rise25 award, and is a co-recipient of the Electronic Frontier Foundation Pioneer Award along with Joy Buolamwini and Timnit Gebru. Raji received her bachelor's of applied science in engineering science from the University of Toronto. She is currently completing her PhD in computer science at UC Berkeley.
