Measuring AI requires the same scientific foundations that other fields rely on.
A Stanford community building the methods, courses, and software to close the gap.
The measurement problem
Interpretable measurements start with sound foundations.
A model scores 87% on a benchmark. Compared to what baseline? With what sampling error? Measuring which construct? Psychometrics, metrology, and educational testing have answers. AI measurement can draw on that work.
AIMS defines constructs before collecting data, reports uncertainty with every score, and designs benchmarks for durability. The textbook (used in CS321M) covers the theory. The software and competitions put it into practice.
The framework
Four disciplines, one integrated science.
Each pillar addresses a core question in measuring AI systems well.
Measurement foundations
Define what is being measured, how to represent it, and where metrics diverge from the construct.
Measurement methods
Protocols that are comparable, decision-relevant, and built to last.
Statistical discipline
Uncertainty, sampling, and distribution shift as core parts of every AI measurement.
Community infrastructure
Shared resources for teaching, competition, software, and discussion that reinforce each other.
Get involved
An open community, by design.
AIMS runs with the standards of a research lab and the energy of an open-source project. Join the discussion and help shape the work.
“When you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind: it may be the beginning of knowledge, but you have scarcely, in your thoughts, advanced to the stage of science, whatever the matter may be.”