Mathematics
LiveAoPSBench
LiveAoPSBench: a contamination-resistant, timestamped evaluation set of 5328 Olympiad-level math problems mined from the Art of Problem Solving forum. Per-(model, problem) short-answer correctness (pass@1).
5,321items
13subjects
apache-2.0license
mathematicsdomain
reasoningdomain
textmodality
item-level responses released
Saturation status: Unknown
Response matrix
Fit to width. Hover for subject & item; click a cell for details.
Correct (1)Incorrect (0)Unobserved
Scale: 1 = correct · 0 = incorrect
Subjects
- 1DeepSeek-R1-Distill-Qwen-32B0.5223
- 2DeepSeek-R1-Distill-Qwen-7B0.4795
- 3QwQ-32B-Preview0.4685
- 4DeepSeek-R1-Distill-Llama-8B0.4463
- 5Qwen2.5-Math-72B-Instruct0.3634
- 6DeepSeek-R1-Distill-Qwen-1.5B0.359
- 7phi-40.3341
- 8Qwen2.5-Math-7B-Instruct0.3035
- 9Qwen2.5-Math-1.5B-Instruct0.2858
- 10deepseek-math-7b-rl0.1301
- 11Llama-3.1-8B-Instruct0.1134
- 12deepseek-math-7b-instruct0.1027
- 13Llama-3.2-3B-Instruct0.1006