Law
LegalBench
LegalBench tests whether models can answer legal reasoning questions, with a response counted correct when it matches the reference answer closely enough to pass an approximate string match.
2,047items
31subjects
Apache-2.0license
lawdomain
reasoningdomain
textmodality
item-level responses released
Saturation status: No
Response matrix
Scoped to the 5 HELM-Lite tasks with full public per-instance predictions (of LegalBench's 162 total tasks).
Rasch analysis p = σ(θ − z + c)
63,457 responses, 80/20 split over cells · 31 subjects · 2,047 items · 5 conditions
AUC train
0.844
AUC test
0.816
Loading session strips…
Correct (1)Incorrect (0)Unobserved
Scale: 1 = correct · 0 = incorrect