Knowledge
IKP
IKP asks models factual questions that range from things almost anyone knows to extreme long tail trivia, counting refusals as wrong. Because accuracy tracks model size closely, it doubles as a way to estimate the parameter count of a closed model.
1,400items
201subjects
CC-BY-4.0license
knowledgedomain
textmodality
item-level responses released
Saturation status: No
Response matrix
Rasch analysis p = σ(θ − z)
268,351 responses, 80/20 split over cells · 201 subjects · 1,400 items
AUC train
0.966
AUC test
0.964
Loading session strips…
Correct (1)Incorrect (0)Unobserved
Scale: 1 = correct · 0 = incorrect