Stanford UniversityAI Measurement ScienceAIMS — AI Measurement Science

Multilingual

culturaleval (OALL Arabic)

culturaleval — OALL Arabic EXAMS + translated-MMLU per-item LM correctness.

14,578items
20subjects
MITlicense
multilingualdomain
culturaldomain
knowledgedomain
textmodality
item-level responses released
Saturation status: Unknown
Original sourcePaper← All benchmarks

Response matrix

Fit to width. Hover for subject & item; click a cell for details.

culturaleval (OALL Arabic) response matrix: AI models (rows) against items (columns)
Correct (1)Incorrect (0)Unobserved

Scale: 1 = correct · 0 = incorrect

Subjects

  1. 1meta-llama/Meta-Llama-3-70B0.5698
  2. 2meta-llama/Meta-Llama-3-70B-Instruct0.559
  3. 3CohereForAI/c4ai-command-r-v010.4883
  4. 4FreedomIntelligence/AceGPT-v1.5-13B-Chat0.4162
  5. 5meta-llama/Meta-Llama-3-8B-Instruct0.4022
  6. 6core42/jais-30b-v30.3856
  7. 7FreedomIntelligence/AceGPT-13B-chat0.3528
  8. 8FreedomIntelligence/AceGPT-7B-chat0.3359
  9. 9meta-llama/Llama-2-70b-hf0.3349
  10. 10FreedomIntelligence/AceGPT-13B0.3335
  11. 1101-ai/Yi-9B-200K0.3126
  12. 12mistralai/Mistral-7B-Instruct-v0.20.3073
  13. 13core42/jais-30b-v10.3072
  14. 1401-ai/Yi-1.5-34B-Chat0.3044
  15. 1501-ai/Yi-1.5-9B-Chat0.298
  16. 16FreedomIntelligence/AceGPT-v1.5-13B0.2923
  17. 17core42/jais-13b0.2821
  18. 18FreedomIntelligence/AceGPT-7B0.2766
  19. 19meta-llama/Llama-2-13b-hf0.2711
  20. 20meta-llama/Llama-2-7b-hf0.2562
Full data on Hugging FaceBack to the gallery