Stanford UniversityAI Measurement ScienceAIMS — AI Measurement Science

Reasoning

Humanity's Last Exam

Humanity's Last Exam — 2500-question frontier benchmark; per-item binary correct/incorrect from supaihq judged data + deepwriter Gemini-3 data.

1,792items
19subjects
MITlicense
knowledgedomain
reasoningdomain
textmodality
item-level responses released
Saturation status: Unknown
Original sourcePaper← All benchmarks

Response matrix

Fit to width. Hover for subject & item; click a cell for details.

Humanity's Last Exam response matrix: AI models (rows) against items (columns)
Correct (1)Incorrect (0)Unobserved

Scale: 1 = correct · 0 = incorrect

Subjects

  1. 1Sup-AI-Ensemble0.5215
  2. 2Gemini-3-Pro-Preview0.4565
  3. 3GPT-5-Pro0.3953
  4. 4GPT-5.10.3823
  5. 5Qwen3-VL-Thinking0.3333
  6. 6Claude-Opus-4.50.2966
  7. 7Grok-40.2905
  8. 8DeepSeek-V3.2-Thinking0.2413
  9. 9GLM-4.60.2308
  10. 10Claude-Sonnet-4.50.1811
  11. 11Kimi-K2-Thinking-Turbo0.1755
  12. 12Qwen3-Next-80B-A3B-Thinking0.175
  13. 13Qwen3-Max0.1731
  14. 14MiniMax-M20.1667
  15. 15Gemini-2.5-Pro0.1651
  16. 16Mistral-Magistral-Medium0.1351
  17. 17Mistral-Large0.0588
  18. 18Gemini-2.5-Flash0
  19. 19DeepSeek-V3.2-Exp-Thinking0
Full data on Hugging FaceBack to the gallery