Safety
AgentDojo
AgentDojo: prompt-injection evaluation across many tool-using agents and environments.
1,081items
29subjects
MITlicense
agents_and_tool_usedomain
safetydomain
textmodality
item-level responses released
Saturation status: Unknown
Response matrix
Fit to width. Hover for subject & item; click a cell for details.
Correct (1)Incorrect (0)Unobserved
Scale: 1 = correct · 0 = incorrect
Subjects
- 1gpt-4o-2024-05-13 (spotlighting_with_delimiting)0.5101
- 2gpt-4o-2024-05-13 (repeat_user_prompt)0.5094
- 3gpt-4-0125-preview0.5051
- 4claude-3-7-sonnet-202502190.464
- 5claude-3-5-sonnet-202406200.4551
- 6gpt-4o-2024-05-130.4447
- 7gpt-4-turbo-2024-04-090.4363
- 8Meta-SecAlign-70B (repeat_user_prompt)0.4204
- 9gpt-4o-mini-2024-07-180.4161
- 10Meta-SecAlign-70B0.4059
- 11claude-3-5-sonnet-202410220.403
- 12claude-3-sonnet-20240229 (repeat_user_prompt)0.4
- 13claude-3-opus-202402290.3517
- 14gpt-4o-2024-05-13 (tool_filter)0.3517
- 15gemini-1.5-pro-0020.3473
- 16claude-3-sonnet-202402290.3198
- 17meta-llama_Llama-3.3-70B-Instruct0.3069
- 18gemini-1.5-pro-0010.3046
- 19gemini-2.0-flash-exp0.3025
- 20gemini-2.0-flash-0010.2773
- 21meta-llama_Llama-3.3-70B-Instruct (repeat_user_prompt)0.2664
- 22gemini-1.5-flash-0010.2533
- 23gpt-3.5-turbo-01250.246
- 24meta-llama_Llama-3-70b-chat-hf0.2352
- 25claude-3-haiku-202403070.2315
- 26gemini-1.5-flash-0020.2012
- 27command-r0.1874
- 28gpt-4o-2024-05-13 (transformers_pi_detector)0.17
- 29command-r-plus0.1599