Skip to main content
Stanford UniversityAI Measurement ScienceAIMS — AI Measurement Science

Search AIMS

Find pages, publications, events, course projects, and software. Results update as you type; press Enter to open the first result.

Seminar on AI Measurement Science

A biweekly hybrid seminar on the science of measuring AI systems across machine learning, statistics, psychometrics, and policy.

Get announcements

Upcoming seminars

Serena Wang

Serena Wang

University of British Columbia

Time
1–2 p.m. PT
Format
Online (Zoom)

From metric selection to metric discovery in AI evaluation

Register

Abstract

Evaluation metrics are a cornerstone of the AI ecosystem, driving the decisions of developers, consumers, and investors alike. This talk will address two challenges in evaluation that are exacerbated in the development of AI systems. First is a computational challenge: the explosion of metrics to measure complex and hard-to-define capabilities in AI systems has led to rapidly increasing computational costs to evaluation. To mitigate this, we'll discuss metric selection, with algorithms for efficient and provably representative selection of metrics based in social choice theory. Second is an informational challenge: evaluators continue to face a fundamental information problem of not knowing whether they could be missing some important metrics entirely. Thus, moving beyond selection, we'll then discuss metric discovery through an economic model of incentives for agents to reveal unknown unknown metrics under information asymmetry.

About the speaker

Serena is an Assistant Professor in Computer Science at the University of British Columbia and a Canada CIFAR AI Chair. She was a Postdoctoral Fellow at Harvard University hosted by Ariel Procaccia, and completed her PhD at UC Berkeley in 2024 advised by Michael Jordan. Serena’s research focuses on understanding and improving the long term societal impacts of AI by rethinking algorithms and their surrounding incentives and practices. Her recent work concerns evaluation processes for AI systems and beyond, including robustness, incentives, and representation in a multi-stakeholder ecosystem. Her interdisciplinary research agenda combines ideas from machine learning, statistics, economics, and the social sciences.

Meera Desai

Meera Desai

University of Michigan

What AI Benchmarks Actually Measure: Adapting Convergent and Discriminant Validity to Interrogate Fifty-Six AI Benchmarks

Register
Hadi Khalaf

Hadi Khalaf

Harvard University

Robust AI Evaluation through Maximal Lotteries

Register

Abstract

The standard way to evaluate language models on subjective tasks is through pairwise comparisons: an annotator chooses the “better” of two model responses for a given prompt. These comparisons are then aggregated into a single ranking via the Bradley–Terry (BT) framework, forcing heterogeneous preferences into a total order and violating basic social-choice desiderata. In contrast, social choice theory provides an alternative approach called maximal lotteries, which aggregates pairwise preferences without imposing any assumptions on their structure. However, we show that maximal lotteries can be highly sensitive to heterogeneity among annotators and across prompts. We introduce robust lotteries, which optimize worst-case performance under plausible shifts in the preference data. On large-scale preference datasets, robust lotteries achieve more reliable win rate guarantees across the annotator distribution and recover a stable set of top performing models.

Past seminars

Anastasios Angelopoulos

Anastasios Angelopoulos

Arena (LMArena)

Agent Arena: Causal Evaluation of Agents in the Real World

Register

Abstract

Agent Arena is a large-scale platform for evaluating the real-world utility of agents. Available at arena.ai/agent, it has attracted millions of organic users and generated hundreds of thousands of real-world agentic trajectories for evaluation. Its methodology builds on causal inference and a large multifactor A/B test that isolates the effects of the orchestrator, harness, tools, and other agent components. This talk will describe the platform, its user base, and its evaluation framework, along with concrete stories from the intense race among leading AI labs to build the best models for agentic workflows.

About the speaker

Anastasios N. Angelopoulos is the co-founder and CEO of Arena, the leading open platform for evaluating AI models through real-world human feedback. A scientist at heart, Anastasios has spent his academic career working at the intersection of theoretical statistics and machine learning, with a focus on model evaluation, statistical inference, and rigorous reliability guarantees for black-box AI models.

He earned his Ph.D. in Computer Science from UC Berkeley working with Michael I. Jordan and Jitendra Malik, and was a Postdoctoral Scholar with Ion Stoica. His research centered on conformal prediction, prediction-powered inference, and the statistical foundations of AI reliability. Before that, he studied electrical engineering at Stanford University.

Outside of research and startups, he’s a longtime musician, and proud dog dad.

Nathan Jo

Nathan Jo

MIT

How Assumptions Change our Inferences of AI Evaluations

Register

Abstract

AI evals are ultimately inferences about a latent object (e.g., a model's capability), but oftentimes evals only report a proxy for the target of inference (e.g., accuracy on a benchmark). What justifies using accuracy as a proxy for a model's capability? Underlying this decision is a statistical model representing how data should relate to the target of inference, yet this crucial piece is often left implicit. In this talk, I will illustrate this idea using two examples. First, consider the task of assessing algorithmic monoculture (the extent to which an algorithmic ecosystem behaves similarly). A statement about excessive observed agreement is only meaningful relative to a null model of how much one would reasonably expect in the absence of true model dependence. Different, equally plausible, null models, can therefore produce sharply different conclusions about whether monoculture is present. Second, when assessing AI capability, benchmark performance becomes evidence of capability only through an assumed theory connecting question responses to latent abilities. Different theories of capability can lead to different interpretations of the same observed performance. Together, these cases suggest that AI evals are best understood as inference problems grounded in explicit, often normative, assumptions. I will discuss why this observation creates an obligation to articulate, justify, and test the models through which AI behavior is interpreted.

Mike Hardy

Mike Hardy

Stanford University

AI Cartography: Mapping the Latent Landscape of AI Benchmark Ecosystems

Register

Abstract

While aggregate leaderboard scores drive AI development, they contain substantial measurement noise whose sources and magnitudes remain unquantified, making it unclear when rankings reflect genuine capability differences versus evaluation artifacts. We introduce a framework for measuring the latent landscape in AI benchmark ecosystems. Applying Confirmatory Factor Analysis and Generalizability Theory to 4,000+ models from the Open LLM Leaderboard, we decompose sources of ranking variance, uncovering evidence of local dependence among leaderboard items, showing that contributor metadata explains more rank-relevant variance than architecture or deployment categories, and finding that the latent general-factor slope is far more stable than manifest-score scaling laws. We provide actionable diagnostics to determine how benchmark rankings can be trusted and how benchmark design can be improved.

Naveen Raman

Naveen Raman

Carnegie Mellon University

Healthcare LLM Benchmarks Are Only as Good as Their Explicit Assumptions

Register

Abstract

Benchmarks are necessary for healthcare evaluation, but are not sufficient for predicting deployment performance. Our position is that the evaluation–deployment gap arises not because of poorly designed benchmarks, but from implicit assumptions about how users interact with models that cannot be surfaced from benchmarks alone. To make this precise, we propose a classification of assumptions into two categories: task, which can be tested from conversation data alone, and outcome, which requires outcome data and behavioral studies for testing. Critically, outcome assumptions depend on human behavior, something that even well-designed benchmarks cannot directly observe. To demonstrate the operationality of this framework, we retrospectively analyze a healthcare RCT as a case study and find that the gap naturally separates into task and outcome gaps of roughly equal size. To address this, we make two contributions: first, we propose BenchmarkCards, an artifact that documents assumptions, and second, we propose staged evaluation, a procedure that systematically tests assumptions and evaluates performance.

Jenny Huang

Jenny Huang

MIT

Dropping Just a Handful of Preferences Can Change Top Large Language Model Rankings

Register

Abstract

We propose a method for evaluating the robustness of widely used LLM ranking systems — variants of a Bradley–Terry model — to dropping a worst-case very small fraction of preference data. In experiments on large-scale preference datasets, top model rankings prove remarkably sensitive: dropping just 0.003% of human preferences can change the top-ranked model on Chatbot Arena. MT-bench preferences are notably more robust than those from Chatbot Arena, likely due to MT-bench's use of expert annotators and carefully constructed prompts, and neither rankings based on crowdsourced human evaluations nor those based on LLM-as-a-judge preferences are systematically more sensitive than the other.

Yatong Chen

Yatong Chen

Max Planck Institute for Intelligent Systems

Leaderboard Incentives: Model Rankings under Strategic Post-Training

Register

Abstract

Influential benchmarks incentivize competing model developers to strategically allocate post-training resources toward improvements on the leaderboard, a phenomenon dubbed benchmaxxing or training on the test task. In this work, we initiate a principled study of the incentive structure that benchmarks induce. We model benchmarking as a Stackelberg game between a benchmark designer who chooses an evaluation protocol and multiple model developers who compete simultaneously in a subgame given by the designer's choice. First, we prove that current benchmarks induce games for which no Nash equilibrium between model developers exists. This result suggests one explanation for why current practice leads to misaligned incentives, prompting model developers to strategize in opaque ways. However, we prove that under mild conditions, a recently proposed evaluation protocol, called tune-before-test, induces a benchmark with a unique Nash equilibrium that ranks models by latent quality. This positive result demonstrates that benchmarks need not set bad incentives, even if current evaluations do.

Soham Ray

Soham Ray

Sierra

τ-Bench and Beyond: Evaluating Conversational Agents from Text to Real-Time Voice

Register

Abstract

As LLM agents move from demos into real customer-facing deployments, evaluating them rigorously becomes as hard — and as important — as building them. This talk traces the evolution of the τ evaluation suite, a family of benchmarks for task-oriented agents built around realistic tools, policies, and simulated users. τ-bench introduced the core setting — an agent completing real tasks with a simulated user under domain policy — and τ²-bench made it dual-control, with agent and user coordinating actions on a shared environment; it is now widely used to evaluate frontier models. τ³-bench extends evaluation along two axes: knowledge retrieval over large policy corpora, and live, full-duplex voice. The second half of the talk goes deep on voice, where the evaluation problem changes shape entirely. Voice breaks many assumptions text benchmarks rely on: turn-taking is fluid, speech recognition is noisy, accents and background audio degrade understanding, and latency itself becomes a correctness constraint. I'll discuss how τ-voice models these phenomena — realistic audio conditions, interruption dynamics, and interaction-quality metrics beyond task success — what frontier speech-to-speech models get right and wrong today, and open problems in evaluating agents that must listen, think, and act at the same time.

Zeyu Tang

Zeyu Tang

Stanford University

In-Situ Behavioral Eval in Agentic Systems

Register

Abstract

LLM fairness should be evaluated through in-situ behavioral pattern rather than standardized-test Q&A benchmarks. We show that the standardized-test paradigm can be structurally unreliable: surface-level prompt construction choices, although entirely orthogonal to the fairness question being tested, account for the majority of score variance, shift fairness conclusions in both the direction and the magnitude, and result in severe discordance in model rankings. We develop MAC-Fairness, a framework that embeds controlled variation factors into in-situ behavioral evaluation, examining how models' disparate-treatment behaviors shift when identity is varied as part of natural multi-agent conversation. In-situ behavioral evaluation reveals stable, model-specific, disparate-treatment behavioral signatures that could generalize across different fairness benchmarks, a form of evidence the standardized-test paradigm does not offer.

Organizers