Reasoning

StatQA

StatQA: statistical-analysis question answering over tabular datasets. Given a dataset and a refined question, a model selects the relevant columns and applicable statistical methods; it is correct on an item iff both its column selection and method selection exactly match the gold set. Evaluated across prompting strategies (zero-/one-shot, CoT, stats-prompt) for GPT and LLaMA models.

1,158items
10subjects
CC-BY-4.0license
mathematicsdomain
reasoningdomain
textmodality
tabularmodality
item-level responses released
Saturation status: Unknown

Response matrix

Fit to width. Hover for subject & item; click a cell for details.

StatQA response matrix: AI models (rows) against items (columns)
Correct (1)Incorrect (0)Unobserved

Scale: 1 = correct · 0 = incorrect

Subjects

  1. 1llama3_8b_sft0.7713
  2. 2llama3_8b_instruct_sft0.7592
  3. 3llama2_7b_sft0.6672
  4. 4gpt-4o0.5037
  5. 5gpt-40.4688
  6. 6gpt-3.5-turbo0.4107
  7. 7llama3_8b0.2917
  8. 8llama3_8b_instruct0.2191
  9. 9llama2_13b0.1505
  10. 10llama2_7b0.1209