General
JudgeTuning LMSys human agreement
Released evaluations of seven named LLM judges on 3000 LMSys paired-answer comparisons.
3,000items
7subjects
Apache-2.0 for JudgeTuning code. Original LMSys data retain their upstream terms.license
preference_evaluationdomain
textmodality
item-level responses released
Saturation status: No
Response matrix
Loading response matrix…
Correct (1)Incorrect (0)Unobserved
Scale: 1 = correct · 0 = incorrect