Stanford UniversityAI Measurement ScienceAIMS — AI Measurement Science

Medicine

AfriMed-QA

AfriMed-QA: medical QA for African healthcare contexts; binary correctness on MCQ items.

6,910items
31subjects
CC-BY-NC-SA-4.0license
medicinedomain
multilingualdomain
textmodality
item-level responses released
Saturation status: Unknown
Original sourcePaper← All benchmarks

Response matrix

Fit to width. Hover for subject & item; click a cell for details.

AfriMed-QA response matrix: AI models (rows) against items (columns)
Correct (1)Incorrect (0)Unobserved

Scale: 1 = correct · 0 = incorrect

Subjects

  1. 1gpt-4-turbo0.8243
  2. 2gemini_ultra0.7973
  3. 3gpt-4o0.7928
  4. 4claude-3-5-sonnet-202406200.777
  5. 5llama3-405b-instruct-maas0.7627
  6. 6gpt-40.7568
  7. 7claude-3-opus-202402290.7455
  8. 8medpalm20.742
  9. 9Meta-Llama-3-70B-Instruct0.7379
  10. 10gpt-4o-mini0.7176
  11. 11medlm0.702
  12. 12gemini_pro0.684
  13. 13gpt-3.5-turbo0.683
  14. 14Phi-3-medium-128k-instruct0.6708
  15. 15Meta-Llama-3-8B-Instruct0.6677
  16. 16claude-3-haiku-202403070.6639
  17. 17claude-3-sonnet-202402290.6504
  18. 18Meta-Llama-3.1-8B-Instruct0.6189
  19. 19Phi-3-mini-4k-instruct0.6036
  20. 20Mixtral-8x7B-Instruct-v0.10.6033
  21. 21Phi-3-mini-128k-instruct0.5903
  22. 22Llama3-OpenBioLLM-70B0.5862
  23. 23JSL-MedLlama-3-8B-v2.00.5726
  24. 24gpt-3.5-turbo-11060.5629
  25. 25Meditron-7B0.5102
  26. 26Mistral-7B-Instruct-v0.30.5084
  27. 27Mistral-7B-Instruct-v0.20.4847
  28. 28Meta-Llama-3-8B0.4724
  29. 29PMC-LLaMA-7B0.4629
  30. 30Llama3-OpenBioLLM-8B0.45
  31. 31BioMistral-7B0.4402
Full data on Hugging FaceBack to the gallery