Skip to main content
AIMSAI Measurement Science

Science

MetaAgent (GPQA-Diamond)

Per-(system x GPQA-Diamond question) binary correctness for MetaAgent and single-/multi-agent prompting baselines (Direct, CoT, CoT-SC, LLM-Debate, Self-Refine, SPP) over GPT-4o and GPT-3.5-Turbo, recovered from the authors' released evaluation logs.

200items
10subjects
90%observed
sciencedomain
reasoningdomain
knowledgedomain
textmodality
Original sourcePaperBuild script← All benchmarks

Response matrix

Fit to width. Hover for subject & item; click a cell for details.

MetaAgent (GPQA-Diamond) response matrix: AI models (rows) against items (columns)
Correct (1)Incorrect (0)Unobserved

Scale: 1 = correct · 0 = incorrect

Subjects

  1. 1CoT-SC (GPT-4o)0.5537
  2. 2CoT (GPT-4o)0.5
  3. 3Direct (GPT-4o)0.478
  4. 4LLM-Debate (GPT-4o)0.4444
  5. 5Self-Refine (GPT-4o)0.4365
  6. 6SPP (GPT-4o)0.4343
  7. 7CoT (GPT-3.5-Turbo)0.2692
  8. 8MetaAgent (GPT-3.5-Turbo)0.2678
  9. 9Direct (GPT-3.5-Turbo)0.2623
  10. 10CoT-SC (GPT-3.5-Turbo)0.15
Full data on Hugging FaceBack to the gallery