Mathematics
MathConstruct
MathConstruct: 121 math-olympiad constructive-proof problems parameterised into 455 instances, graded by deterministic checkers into binary correct/incorrect; released per-(model x instance) results for 19 (CoT) + 10 (code) LLMs.
455items
20subjects
90%observed
MITlicense
mathematicsdomain
reasoningdomain
textmodality
Response matrix
Fit to width. Hover for subject & item; click a cell for details.
Correct (1)Incorrect (0)Unobserved
Scale: 1 = correct · 0 = incorrect
Subjects
- 1gpt-5--high0.811
- 2o3-mini--high0.5956
- 3deepseek-chat0.5
- 4o3-mini0.4527
- 5o10.4462
- 6deepseek-ai/DeepSeek-R10.3253
- 7gemini-2.0-flash-thinking-exp0.2703
- 8o1-mini0.2549
- 9deepseek-reasoner0.25
- 10deepseek-ai/DeepSeek-R1-Distill-Llama-70B0.1868
- 11gemini-2.0-flash-exp0.1516
- 12Qwen/QwQ-32B-Preview0.1341
- 13claude-3-5-sonnet-latest0.1022
- 14Qwen/Qwen2.5-72B-Instruct-Turbo0.0967
- 15gpt-4o0.078
- 16meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo0.0725
- 17claude-3-5-haiku-latest0.0681
- 18meta-llama/Llama-3.3-70B-Instruct-Turbo0.0626
- 19gpt-4o-mini0.0571
- 20meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo-128K0.0022