Medicine
MedHELM Medbullets
MedHELM medbullets: per-(model, item) exact_match in {0,1} (1=correct) on 308 USMLE Step 2/3-style clinical vignette multiple-choice questions. 13 models from the MedHELM v4.0.0 release.
298items
13subjects
100%observed
Apache-2.0license
medicinedomain
textmodality
Response matrix
Fit to width. Hover for subject & item; click a cell for details.
Correct (1)Incorrect (0)Unobserved
Scale: 1 = correct · 0 = incorrect
Subjects
- 1openai/o4-mini-2025-04-160.8701
- 2openai/gpt-5-2025-08-070.8701
- 3openai/gpt-5-mini-2025-08-070.8344
- 4openai/o3-mini-2025-01-310.8117
- 5google/gemini-2.5-pro-preview-05-060.8084
- 6openai/gpt-4o-2024-05-130.711
- 7deepseek-ai/deepseek-r10.6558
- 8anthropic/claude-3-7-sonnet-202502190.6494
- 9anthropic/claude-3-5-sonnet-202410220.6396
- 10google/gemini-2.0-flash-0010.6299
- 11meta/llama-3.3-70b-instruct0.6071
- 12openai/gpt-4o-mini-2024-07-180.5649
- 13google/gemini-1.5-pro-0010.4773