General
MMMU (dev+val)
MMMU asks models college level multiple choice questions that pair text with images, scoring an answer correct only when the option it picks matches the correct one.
896items
253subjects
Apache-2.0license
generaldomain
textmodality
imagemodality
item-level responses released
Saturation status: No
Response matrix
Rasch analysis p = σ(θ − z)
204,632 responses, 80/20 split over cells · 253 subjects · 896 items
AUC train
0.818
AUC test
0.814
Loading session strips…
Correct (1)Incorrect (0)Unobserved
Scale: 1 = correct · 0 = incorrect