Stanford UniversityAI Measurement ScienceAIMS — AI Measurement Science

Safety

HELM RealToxicityPrompts

HELM RealToxicityPrompts: per-(model, prompt) toxic_frac binarized to {0,1} (1=any toxic continuation, Perspective API). ~42 models x 1000 RealToxicityPrompts items.

1,000items
42subjects
Apache-2.0license
safetydomain
textmodality
item-level responses released
Saturation status: Unknown
Original sourcePaper← All benchmarks

Response matrix

Fit to width. Hover for subject & item; click a cell for details.

HELM RealToxicityPrompts response matrix: AI models (rows) against items (columns)
Correct (1)Incorrect (0)Unobserved

Scale: 1 = correct · 0 = incorrect

Subjects

  1. 1openai/davinci0.159
  2. 2ai21/j2-grande0.15
  3. 3together/yalm0.14
  4. 4ai21/j2-jumbo0.138
  5. 5together/opt-66b0.138
  6. 6ai21/j1-grande-v2-beta0.136
  7. 7openai/text-davinci-0020.134
  8. 8anthropic/stanford-online-all-v4-s30.131
  9. 9microsoft/TNLGv2_7B0.13
  10. 10together/glm0.128
  11. 11openai/curie0.128
  12. 12openai/ada0.128
  13. 13openai/babbage0.127
  14. 14microsoft/TNLGv2_530B0.127
  15. 15together/opt-175b0.127
  16. 16together/gpt-j-6b0.123
  17. 17together/bloom0.121
  18. 18together/gpt-neox-20b0.119
  19. 19ai21/j1-jumbo0.119
  20. 20ai21/j2-large0.117
  21. 21ai21/j1-grande0.116
  22. 22writer/palmyra-instruct-300.115
  23. 23together/t0pp0.114
  24. 24ai21/j1-large0.111
  25. 25openai/text-babbage-0010.108
  26. 26openai/text-curie-0010.105
  27. 27together/ul20.1
  28. 28openai/text-ada-0010.082
  29. 29openai/text-davinci-0030.073
  30. 30AlephAlpha/luminous-supreme0.07
  31. 31AlephAlpha/luminous-base0.058
  32. 32AlephAlpha/luminous-extended0.057
  33. 33cohere/xlarge-202211080.037
  34. 34cohere/medium-202211080.032
  35. 35cohere/command-xlarge-beta0.031
  36. 36cohere/small-202207200.031
Full data on Hugging FaceBack to the gallery