AI measurements increasingly shape research, deployment, and policy. We develop the theory, methods, and open infrastructure to make those measurements valid, reliable, and fit for the decisions they inform.
About AIMSSelected work from the AIMS community on measuring and understanding AI systems.
Preprint · 2025
A framework for connecting evaluation evidence to the claims we make about AI capabilities.
Preprint · 2026
Real developer interactions reveal how coding agents are used, where they help, and where they fall short.
NeurIPS · 2025
An investigation of how private testing and selective disclosure can distort AI leaderboard rankings.
The Measurement Data Bank is a curated exchange of item-level AI measurement data: what was measured, under which conditions, what was observed, and how each outcome was recorded or scored. Analyze those measurements with torch_measure.
Explore data and analysisMeasurement Data Bank
Explore AI measurement science through Stanford's CS321M course and an open, evolving textbook.
This course introduces the foundations, methods, and applications of Artificial Intelligence (AI) measurement science through three connected themes: Measurement as Predictive Modeling, Measurement Validity and Reliability, and Design, Governance, and Applications.
This textbook provides a comprehensive reference on Artificial Intelligence (AI) measurement science, covering its theoretical foundations, core methodologies, and practical applications.
We bring people working on AI measurement together through talks, workshops, and shared challenges.
Invited talks and open discussion across machine learning, statistics, psychometrics, and policy.
A one-day workshop at COLM 2026 on measurement under interaction, strategic optimization, and non-stationarity.
A public challenge to predict model responses from sparse observations.
Continue the conversation between events.
A benchmark can guide progress without certifying readiness.
The competition accepts code submissions through 20 November 2026 at 23:59 AoE.
AIMS is organizing a one-day workshop on rigorous AI measurement, co-located with COLM 2026 in San Francisco. The research and competition tracks are now open.