AI Measurement Science
Sang T. Truong 
Sanmi Koyejo
Preface
Decisions about AI systems—whether to deploy one, which to prefer, how to govern its use—rest on measurements of quantities such as reasoning, planning, and safety. The quality of those measurements is therefore a precondition for sound decisions, yet it is rarely examined. New benchmarks appear faster than their measurement properties can be characterized; leaderboards report scores without specifying the scale on which they are defined; and a system’s average accuracy is largely uninformative about why it succeeds or where it fails. The field has accumulated a large number of evaluation results, but comparatively little theory governing how those results should be produced and interpreted.
This book develops the foundations and practical tools of measurement science for AI evaluation. Its organizing premise is that evaluation is a problem of inference - where we estimate latent properties of a system, such as ability along a given dimension, from observed responses, - rather than a procedure that ends once a dataset is collected and a metric is computed. From this premise the book builds outward. It begins with validity, the question of whether an evaluation measures the construct it claims to, and with the structure of benchmark data. It then develops the probabilistic models that make latent properties estimable — item response theory, Bradley–Terry models, and factor models — together with methods for fitting them, for decomposing measurement error, and for designing evaluations that extract more information per item. On this base it treats the harder questions: when benchmark results generalize across shifts in distribution, and how to evaluate systems that are themselves optimizing against the benchmark, which brings in causal inference and mechanism design. Throughout, each method is presented with its assumptions, the conditions under which it applies, and the ways it can fail, since characterizing the failure modes of an evaluation is as much a part of measurement as the estimate itself.
The book is intended for researchers, practitioners, and students who work on or care about AI evaluation. We assume some background in probability, statistics, and machine learning, but provide sufficient context and references for readers to follow the main ideas. The book also includes interactive code examples and datasets. The science of AI measurement is a fast-moving area with many open challenges, and we hope this book will inspire readers to further explore and advance it. We hope with the present book to help the field move from ad hoc benchmarking toward principled measurement—so that our decision about AI systems rest on solid scientific ground.
Stanford, 2026, Truong & Koyejo
Structure of this book
The chapters build in sequence — from validity and the structure of benchmark data, through the models, estimation, reliability, and design that form the methodological core, to causal generalization and the strategic dimensions that arise once benchmarks become optimization targets.
1 Validity opens the book with validity—the question of whether an evaluation measures the construct it claims to. The chapter models a construct as a latent variable embedded in a nomological network that causally produces item responses, and adopts existence-and-causation condition as the standard validity must meet. A central difficulty is that this network is not identifiable from data alone—substantively different structures can imply the same observed responses—so validity cannot be read off the data and must instead be built up as an accumulation of evidence. The chapter organizes that evidence, following the unitary view of validity, into five kinds (content, criterion, construct, external, and consequential), and catalogs the major threats at the conceptual level, deferring their operationalization to the later chapters.
2 Data surveys the AI evaluation data landscape: the response matrix abstraction, a taxonomy of 70+ benchmarks spanning knowledge, coding, agentic, multilingual, cultural, and preference evaluation, and the practical data quality issues that arise in real-world evaluation.
3 Models lays the mathematical modeling groundwork for the rest of the book. It covers probabilistic models for AI evaluation data — including Item Response Theory (Rasch, 2PL, 3PL) and Bradley-Terry models for pairwise comparisons. A central theme is the relationship between model assumptions and what they allow us to infer: sufficiency, specific objectivity, and the tradeoffs between model complexity and interpretability. The chapter also introduces prediction-powered evaluation—an amortized model that maps external features to latent parameters for cold-start prediction of unseen models and items.
4 Learning studies how to estimate latent parameters from evaluation data. It covers maximum likelihood estimation, gradient-based optimization, the EM algorithm, Bayesian inference, regularization, and model selection for the unidimensional IRT models.
5 Reliability addresses the question of signal versus noise in AI measurement. Building on the response model of 3 Models, it derives a variance decomposition—person, item, interaction, and residual—and defines reliability as the person share of that variance, the generalizability coefficient. It develops three routes for estimating the variance components (plug-in, Bayesian, and method of moments), turns generalizability theory into a design tool that sizes items and raters to a target reliability, and treats the rater facet in depth for LLM-as-a-judge and ordinal-rating protocols—including why systematic biases such as a judge’s position bias can be reliable yet escape reliability analysis. Classical test theory appears as the special case recovered when only total scores are available. It is also where the validity threats of 1 Validity become operational: the item-level diagnostics—differential item functioning, dimensionality assessment, and item-fit statistics—are each a structured reading of the same variance decomposition.
8 Predictability turns from confirming a known construct to discovering an unknown one. Where the preceding chapters assume the construct is named in advance and ask whether the items measure it, this chapter is handed a collection of items and asks the data how many distinct things they measure and which items measure which. It uses parallel analysis to recover dimensionality, then develops the multidimensional factor model—its definition, estimation, the structure matrix and item clustering for diagnosing benchmark heterogeneity, and masking experiments that test how well the learned model generalizes to unseen models and items. It closes with network models (the Gaussian graphical model and the Ising model) that drop the common-cause assumption entirely—treating item correlations as direct associations rather than the signature of a shared latent ability—and shows why latent-variable and network structures can be observationally close.
6 Efficiency develops the statistical foundations for efficient evaluation design. It introduces Fisher information for item selection, Computerized Adaptive Testing (CAT), D-optimal design for constructing maximally informative item pools, and efficient paired-comparison schedules for model arenas.
9 Intervention examines the causal foundations of AI measurement. It covers structural causal models for evaluation data, distribution shift (covariate, label, and concept shift) and when benchmark results generalize across contexts, interventional and counterfactual reasoning for diagnosing construct-irrelevant variance, and methods for robust prediction under shift including conformal inference and data attribution.
10 Ecosystem steps back from the statistics to the institutions that consume benchmark scores. It maps the stakeholders of a modern benchmark—builders, evaluators and platforms, adopters, investors, and regulators—and argues that as a benchmark’s audience expands, Goodhart’s Law turns from an academic curiosity into a structural force, analyzed through the economics of signaling and the market for lemons. It develops a mechanism-based taxonomy of gaming on both sides of the market—builders who game the metric and evaluators who game the audience—and connects each gaming channel to a design lever, framing the governance frontier (multi-principal evaluation, the role of open-source models, and how to organize trustworthy third-party evaluation). The chapter is the diagnosis that motivates the design machinery of 11 Design.
11 Design considers the strategic and economic dimensions of evaluation design. It introduces decision-making under ambiguity (Maxmin Expected Utility), Bayesian persuasion for optimal information disclosure, robust mechanism design, and the strategic considerations that arise when benchmark results influence development incentives.
12 Application applies the measurement framework to adversarial evaluation: red-teaming as structured measurement, attack success rate as an estimand (and when comparisons are valid), adversarial robustness as a multidimensional latent trait, synthetic data for evaluation at scale, and prediction-powered inference for combining human and automated judgments.
13 Conclusion synthesizes the book’s themes, distills lessons from the field, identifies six open challenges (beyond binary responses, multidimensional ability, temporal dynamics, agentic evaluation, scalable oversight, and fairness in evaluation), presents ten capstone projects for the CS321M course, and offers a practitioner’s checklist for designing rigorous AI evaluations.
How to engage with this book
Three chapters are essential for every reader—1 Validity, 2 Data, and 3 Models—which frame the measurement question, the data landscape, and the modeling toolkit that the rest of the book builds on; they belong in both reading tracks. Beyond this shared core, the book splits into two tracks. Readers focused on building and improving rigorous evaluations should follow the methodological track—4 Learning, 5 Reliability, 6 Efficiency, and 9 Intervention—which develops estimation, reliability, efficient design, and the analysis of when results generalize. Readers focused on the strategic and governance dimensions of AI evaluation should follow 10 Ecosystem, 11 Design, and 12 Application, which cover the incentives surrounding benchmarks, the mechanism design that responds to them, and adversarial testing.
Prior knowledge
The book assumes knowledge of the fundamentals of statistics, linear algebra, and machine learning. Code examples are written in Python using PyTorch, scikit-learn, and pandas, so familiarity with Python is valuable for readers who wish to engage with the interactive examples. For readers seeking to strengthen their prerequisites or deepen their understanding of the fields this book draws from, we recommend the following companion texts:
- Borsboom (2005) — A philosophically rigorous account of what it means to measure a psychological attribute. Essential reading for the validity and construct modeling discussions in 5 Reliability and 1 Validity.
- Lord and Novick (1968) — The classical statistical theory of mental testing that underlies much of modern psychometrics. For readers wanting depth in reliability, item analysis, and test construction before the measurement chapters.
- Murphy (2022) — A modern, comprehensive treatment of probabilistic modeling, Bayesian inference, and statistical learning. Excellent preparation for the latent variable models and estimation methods in 3 Models through 6 Efficiency.
Citation
To cite this book, please use the following entry: