Law
LegalBench
LegalBench legal-reasoning tasks evaluated by Stanford HELM Lite (v1.0.0). Per-(model, item) responses for 5 LegalBench tasks (abercrombie, corporate_lobbying, function_of_decision_section, international_citizenship_questions, proa) x 31 LLMs. Response is quasi_exact_match (0/1); trace holds the model's predicted text. Scoped to the 5 HELM-Lite tasks with full public per-instance predictions (of LegalBench's 162 total tasks).
2,047items
31subjects
Apache-2.0license
lawdomain
reasoningdomain
textmodality
item-level responses released
Saturation status: Unknown
Response matrix
Fit to width. Hover for subject & item; click a cell for details.
Correct (1)Incorrect (0)Unobserved
Scale: 1 = correct · 0 = incorrect
Subjects
- 1openai_gpt-4-06130.6619
- 2writer_palmyra-x-v30.6522
- 3meta_llama-2-70b0.6282
- 4mistralai_mixtral-8x7b-32kseqlen0.6229
- 5openai_gpt-4-1106-preview0.5965
- 6openai_text-davinci-0030.5916
- 7meta_llama-2-13b0.5892
- 8google_text-unicorn@0010.5862
- 901-ai_yi-34b0.5838
- 10writer_palmyra-x-v20.576
- 11openai_text-davinci-0020.574
- 12google_text-bison@0010.5677
- 13anthropic_claude-2.10.5628
- 14mistralai_mistral-7b-v0.10.5608
- 15openai_gpt-3.5-turbo-06130.554
- 16anthropic_claude-v1.30.5374
- 17anthropic_claude-2.00.5335
- 18meta_llama-2-7b0.5296
- 1901-ai_yi-6b0.5222
- 20cohere_command0.4929
- 21ai21_j2-jumbo0.491
- 22AlephAlpha_luminous-supreme0.4768
- 23anthropic_claude-instant-1.20.4734
- 24anthropic_claude-instant-v10.4734
- 25tiiuae_falcon-40b0.4475
- 26AlephAlpha_luminous-extended0.4255
- 27meta_llama-65b0.4177
- 28ai21_j2-grande0.4035
- 29cohere_command-light0.4001
- 30AlephAlpha_luminous-base0.3967
- 31tiiuae_falcon-7b0.3532