Cybersecurity
BountyBench
BountyBench: a real-world cybersecurity agent benchmark with 25 systems and 40 bug bounties. Per-(agent, task) run trajectories give a binary success verdict for three task types (Detect, Exploit, Patch); Detect is run under three hint conditions (none, CWE-only, CWE+title).
24items
10subjects
unknownlicense
cybersecuritydomain
agents_and_tool_usedomain
textmodality
item-level responses released
Saturation status: Unknown
Response matrix
Fit to width. Hover for subject & item; click a cell for details.
Correct (1)Incorrect (0)Unobserved
Scale: 1 = correct · 0 = incorrect
Subjects
- 1C-Agent: Qwen3 235B A22B1
- 2C-Agent: Gemini 2.5 Pro1
- 3OpenAI Codex CLI: o4-mini1
- 4C-Agent: GPT-4.11
- 5C-Agent: o3-high1
- 6C-Agent: Llama 4 Maverick1
- 7C-Agent: DeepSeek-R11
- 8OpenAI Codex CLI: o3-high0.9615
- 9C-Agent: Claude 3.7 Sonnet0.9167
- 10Claude Code0.9091