Safety

AgentDojo

AgentDojo: prompt-injection evaluation across many tool-using agents and environments.

1,081items
29subjects
MITlicense
agents_and_tool_usedomain
safetydomain
textmodality
item-level responses released
Saturation status: Unknown

Response matrix

Fit to width. Hover for subject & item; click a cell for details.

AgentDojo response matrix: AI models (rows) against items (columns)
Correct (1)Incorrect (0)Unobserved

Scale: 1 = correct · 0 = incorrect

Subjects

  1. 1gpt-4o-2024-05-13 (spotlighting_with_delimiting)0.5101
  2. 2gpt-4o-2024-05-13 (repeat_user_prompt)0.5094
  3. 3gpt-4-0125-preview0.5051
  4. 4claude-3-7-sonnet-202502190.464
  5. 5claude-3-5-sonnet-202406200.4551
  6. 6gpt-4o-2024-05-130.4447
  7. 7gpt-4-turbo-2024-04-090.4363
  8. 8Meta-SecAlign-70B (repeat_user_prompt)0.4204
  9. 9gpt-4o-mini-2024-07-180.4161
  10. 10Meta-SecAlign-70B0.4059
  11. 11claude-3-5-sonnet-202410220.403
  12. 12claude-3-sonnet-20240229 (repeat_user_prompt)0.4
  13. 13claude-3-opus-202402290.3517
  14. 14gpt-4o-2024-05-13 (tool_filter)0.3517
  15. 15gemini-1.5-pro-0020.3473
  16. 16claude-3-sonnet-202402290.3198
  17. 17meta-llama_Llama-3.3-70B-Instruct0.3069
  18. 18gemini-1.5-pro-0010.3046
  19. 19gemini-2.0-flash-exp0.3025
  20. 20gemini-2.0-flash-0010.2773
  21. 21meta-llama_Llama-3.3-70B-Instruct (repeat_user_prompt)0.2664
  22. 22gemini-1.5-flash-0010.2533
  23. 23gpt-3.5-turbo-01250.246
  24. 24meta-llama_Llama-3-70b-chat-hf0.2352
  25. 25claude-3-haiku-202403070.2315
  26. 26gemini-1.5-flash-0020.2012
  27. 27command-r0.1874
  28. 28gpt-4o-2024-05-13 (transformers_pi_detector)0.17
  29. 29command-r-plus0.1599