Skip to main content
AIMSAI Measurement Science

Medicine

MedHELM RaceBasedMed

MedHELM race_based_med: per-(model, item) exact_match in {0,1} (1=correct) on 167 detecting whether a medical answer relies on harmful race-based reasoning. 13 models from the MedHELM v4.0.0 release.

167items
13subjects
100%observed
Apache-2.0license
medicinedomain
textmodality
Original sourcePaperBuild script← All benchmarks

Response matrix

Fit to width. Hover for subject & item; click a cell for details.

MedHELM RaceBasedMed response matrix: AI models (rows) against items (columns)
Correct (1)Incorrect (0)Unobserved

Scale: 1 = correct · 0 = incorrect

Subjects

  1. 1deepseek-ai/deepseek-r10.9162
  2. 2openai/gpt-4o-2024-05-130.8982
  3. 3openai/o3-mini-2025-01-310.8743
  4. 4openai/o4-mini-2025-04-160.8743
  5. 5google/gemini-2.0-flash-0010.8563
  6. 6openai/gpt-5-mini-2025-08-070.8563
  7. 7google/gemini-2.5-pro-preview-05-060.8263
  8. 8google/gemini-1.5-pro-0010.8204
  9. 9anthropic/claude-3-5-sonnet-202410220.7725
  10. 10openai/gpt-4o-mini-2024-07-180.7665
  11. 11openai/gpt-5-2025-08-070.7545
  12. 12anthropic/claude-3-7-sonnet-202502190.6587
  13. 13meta/llama-3.3-70b-instruct0.5689
Full data on Hugging FaceBack to the gallery