The AIMS Seminar.

Abstract

Benchmarks are necessary for healthcare evaluation, but are not sufficient for predicting deployment performance. Our position is that the evaluation–deployment gap arises not because of poorly designed benchmarks, but from implicit assumptions about how users interact with models that cannot be surfaced from benchmarks alone. To make this precise, we propose a classification of assumptions into two categories: task, which can be tested from conversation data alone, and outcome, which requires outcome data and behavioral studies for testing. Critically, outcome assumptions depend on human behavior, something that even well-designed benchmarks cannot directly observe. To demonstrate the operationality of this framework, we retrospectively analyze a healthcare RCT as a case study and find that the gap naturally separates into task and outcome gaps of roughly equal size. To address this, we make two contributions: first, we propose BenchmarkCards, an artifact that documents assumptions, and second, we propose staged evaluation, a procedure that systematically tests assumptions and evaluates performance.

Abstract

We propose a method for evaluating the robustness of widely used LLM ranking systems — variants of a Bradley–Terry model — to dropping a worst-case very small fraction of preference data. In experiments on large-scale preference datasets, top model rankings prove remarkably sensitive: dropping just 0.003% of human preferences can change the top-ranked model on Chatbot Arena. MT-bench preferences are notably more robust than those from Chatbot Arena, likely due to MT-bench's use of expert annotators and carefully constructed prompts, and neither rankings based on crowdsourced human evaluations nor those based on LLM-as-a-judge preferences are systematically more sensitive than the other.

Abstract

Influential benchmarks incentivize competing model developers to strategically allocate post-training resources toward improvements on the leaderboard, a phenomenon dubbed benchmaxxing or training on the test task. In this work, we initiate a principled study of the incentive structure that benchmarks induce. We model benchmarking as a Stackelberg game between a benchmark designer who chooses an evaluation protocol and multiple model developers who compete simultaneously in a subgame given by the designer's choice. First, we prove that current benchmarks induce games for which no Nash equilibrium between model developers exists. This result suggests one explanation for why current practice leads to misaligned incentives, prompting model developers to strategize in opaque ways. However, we prove that under mild conditions, a recently proposed evaluation protocol, called tune-before-test, induces a benchmark with a unique Nash equilibrium that ranks models by latent quality. This positive result demonstrates that benchmarks need not set bad incentives, even if current evaluations do.

Abstract

As LLM agents move from demos into real customer-facing deployments, evaluating them rigorously becomes as hard — and as important — as building them. This talk traces the evolution of the τ evaluation suite, a family of benchmarks for task-oriented agents built around realistic tools, policies, and simulated users. τ-bench introduced the core setting — an agent completing real tasks with a simulated user under domain policy — and τ²-bench made it dual-control, with agent and user coordinating actions on a shared environment; it is now widely used to evaluate frontier models. τ³-bench extends evaluation along two axes: knowledge retrieval over large policy corpora, and live, full-duplex voice. The second half of the talk goes deep on voice, where the evaluation problem changes shape entirely. Voice breaks many assumptions text benchmarks rely on: turn-taking is fluid, speech recognition is noisy, accents and background audio degrade understanding, and latency itself becomes a correctness constraint. I'll discuss how τ-voice models these phenomena — realistic audio conditions, interruption dynamics, and interaction-quality metrics beyond task success — what frontier speech-to-speech models get right and wrong today, and open problems in evaluating agents that must listen, think, and act at the same time.

Abstract

LLM fairness should be evaluated through in-situ behavioral pattern rather than standardized-test Q&A benchmarks. We show that the standardized-test paradigm can be structurally unreliable: surface-level prompt construction choices, although entirely orthogonal to the fairness question being tested, account for the majority of score variance, shift fairness conclusions in both the direction and the magnitude, and result in severe discordance in model rankings. We develop MAC-Fairness, a framework that embeds controlled variation factors into in-situ behavioral evaluation, examining how models' disparate-treatment behaviors shift when identity is varied as part of natural multi-agent conversation. In-situ behavioral evaluation reveals stable, model-specific, disparate-treatment behavioral signatures that could generalize across different fairness benchmarks, a form of evidence the standardized-test paradigm does not offer.