Knowledge
MMLU
MMLU tests broad knowledge and reasoning with multiple choice exam questions spanning school and professional topics in the sciences, humanities and social sciences, scored by whether the model picks the right option.
13,937items
150subjects
MITlicense
knowledgedomain
reasoningdomain
textmodality
item-level responses released
Saturation status: No
Response matrix
The 14,042 evaluated questions map to 13,937 distinct items (identical question texts share one item). Subject is recorded in test_condition. Model panel capped to 150.
Rasch analysis p = σ(θ − z + c)
2,106,300 responses, 80/20 split over cells · 150 subjects · 13,937 items · 57 conditions
AUC train
0.790
AUC test
0.782
Loading session strips…
Correct (1)Incorrect (0)Unobserved
Scale: 1 = correct · 0 = incorrect