Reasoning
CharXiv
CharXiv tests whether multimodal models can read real charts from scientific papers, answering both short descriptive questions and an open ended reasoning question, with a judge model grading the descriptive answers.
5,000items
37subjects
CC-BY-SA-4.0license
reasoningdomain
imagemodality
textmodality
item-level responses released
Saturation status: No
Response matrix
Rasch analysis p = σ(θ − z + c)
184,839 responses, 80/20 split over cells · 37 subjects · 5,000 items · 2 conditions
AUC train
0.869
AUC test
0.846
Loading session strips…
Correct (1)Incorrect (0)Unobserved
Scale: 1 = correct · 0 = incorrect