Safety
HELM XSTest
HELM XSTest: per-(model, prompt) safety_score in {0,1} (1=safe) from HELM's safety classifier. 87 models x 450 XSTest prompts probing exaggerated safety / over-refusal. HELM-sourced.
450items
81subjects
Apache-2.0license
safetydomain
textmodality
item-level responses released
Saturation status: Unknown
Response matrix
Fit to width. Hover for subject & item; click a cell for details.
Correct (1)Incorrect (0)Unobserved
Scale: 1 = correct · 0 = incorrect
Subjects
- 1google/gemini-2.5-flash-preview-04-170.9756
- 2deepseek-ai/deepseek-r1-05280.9756
- 3qwen/qwen3-235b-a22b-instruct-2507-fp80.9756
- 4qwen/qwen3-235b-a22b-fp8-tput0.9756
- 5google/gemini-2.5-pro-preview-03-250.9733
- 6zai-org/glm-4.5-air-fp80.9733
- 7writer/palmyra-x-0040.9711
- 8qwen/qwen3-next-80b-a3b-thinking0.9689
- 9moonshotai/kimi-k2-instruct0.9689
- 10xai/grok-3-mini-beta0.9689
- 11openai/gpt-5.1-2025-11-130.9667
- 12openai/gpt-4.1-2025-04-140.9667
- 13openai/o4-mini-2025-04-160.9644
- 14google/gemini-2.5-flash-lite0.9644
- 15openai/gpt-4-turbo-2024-04-090.96
- 16openai/o3-2025-04-160.9578
- 17openai/o1-2024-12-170.9578
- 18openai/gpt-4o-2024-05-130.9578
- 19openai/gpt-5-mini-2025-08-070.9556
- 20qwen/qwen2-72b-instruct0.9511
- 21openai/gpt-4.1-mini-2025-04-140.9511
- 22openai/gpt-5-nano-2025-08-070.9511
- 23openai/gpt-4.1-nano-2025-04-140.9467
- 24google/gemini-3-pro-preview0.9444
- 25openai/gpt-5-2025-08-070.9444
- 26deepseek-ai/deepseek-v30.9444
- 27writer/palmyra-med0.94
- 28meta/llama-4-maverick-17b-128e-instruct-fp80.94
- 29openai/o1-mini-2024-09-120.94
- 30openai/gpt-4.5-preview-2025-02-270.9356
- 31anthropic/claude-sonnet-4-202505140.9356
- 32anthropic/claude-opus-4-202505140.9333
- 33anthropic/claude-3-7-sonnet-202502190.9311
- 34meta/llama-3-70b-chat0.9311
- 35ibm/granite-4.0-h-small0.9289
- 36writer/palmyra-fin0.9289