Stanford UniversityAI Measurement ScienceAIMS — AI Measurement Science

Safety

MMLU Moral Scenarios

MMLU moral_scenarios per-(model, question) accuracy (acc in {0,1}) over 895 machine-ethics MCQ items, from the Open LLM Leaderboard v1 details datasets. Model panel capped to 150.

895items
150subjects
MITlicense
safetydomain
reasoningdomain
textmodality
item-level responses released
Saturation status: Unknown
Original sourcePaper← All benchmarks

Response matrix

Fit to width. Hover for subject & item; click a cell for details.

MMLU Moral Scenarios response matrix: AI models (rows) against items (columns)
Correct (1)Incorrect (0)Unobserved

Scale: 1 = correct · 0 = incorrect

Subjects

  1. 1liuxiang886__llama2-70B-qlora-gpt40.6592
  2. 2garage-bAInd__Platypus2-70B-instruct0.6525
  3. 3MayaPH__GodziLLa2-70B0.6313
  4. 4deepnight-research__llama-2-70B-inst0.6045
  5. 5upstage__Llama-2-70b-instruct-v20.6045
  6. 6upstage__Llama-2-70b-instruct0.5933
  7. 7quantumaikr__llama-2-70b-fb16-guanaco-1k0.5911
  8. 8augtoma__qCammel-70-x0.5609
  9. 9upstage__llama-65b-instruct0.4905
  10. 10lilloukas__Platypus-30B0.4894
  11. 11lilloukas__GPlatty-30B0.4838
  12. 12upstage__llama-30b-instruct-20480.4726
  13. 13HiTZ__alpaca-lora-65b-en-pt-es-ca0.4682
  14. 14jordiclive__Llama-2-70b-oasst-1-2000.4659
  15. 15garage-bAInd__Camel-Platypus2-13B0.4581
  16. 16upstage__llama-30b-instruct0.4547
  17. 17augtoma__qCammel-130.4425
  18. 18jarradh__llama2_70b_chat_uncensored0.4134
  19. 19camel-ai__CAMEL-33B-Combined-Data0.4123
  20. 20OpenBuddyEA__openbuddy-llama-30b-v7.1-bf160.3944
  21. 21shareAI__llama2-13b-Chinese-chat0.3933
  22. 22OpenBuddy__openbuddy-llama-65b-v8-bf160.3899
  23. 23layoric__llama-2-13b-code-alpaca0.3676
  24. 24Lajonbot__Llama-2-13b-hf-instruct-pl-lora_unload0.3609
  25. 25Aeala__GPT4-x-AlpacaDente2-30b0.3564
  26. 26NousResearch__Nous-Hermes-Llama2-13b0.3553
  27. 27OptimalScale__robin-65b-v2-delta0.3441
  28. 28jlevin__guanaco-unchained-llama-2-7b0.3397
  29. 29Aeala__GPT4-x-AlpacaDente-30b0.3341
  30. 30kevinpro__Vicuna-13B-CoT0.3307
  31. 31OpenBuddy__openbuddy-llama2-13b-v8.1-fp160.3263
  32. 32lvkaokao__llama2-7b-hf-instruction-lora0.3117
  33. 33WizardLM__WizardLM-13B-V1.20.305
  34. 34quantumaikr__QuantumLM-70B-hf0.3039
  35. 35shareAI__bimoGPT-llama2-13b0.3039
  36. 36mosaicml__mpt-30b-instruct0.3028
Full data on Hugging FaceBack to the gallery