Skip to main content
AIMSAI Measurement Science

Software Engineering

Terminal-Bench

Terminal-Bench: per-trial pass/fail of agent-model combos on terminal tasks.

89items
173subjects
98%observed
Apache-2.0license
software_engineeringdomain
textmodality
Original sourcePaperBuild script← All benchmarks

Response matrix

Fit to width. Hover for subject & item; click a cell for details.

Terminal-Bench response matrix: AI models (rows) against items (columns)
Correct (1)Incorrect (0)Unobserved

Scale: 1 = correct · 0 = incorrect

Subjects

  1. 1oracle | claude-opus-4-61
  2. 2codex | claude-opus-4-61
  3. 3vix | claude-opus-4-70.907
  4. 4jj-agent-harness | multiple0.8708
  5. 5nexau | gpt-5.50.8488
  6. 6Lemoncode-agent | gpt-5.3-codex0.8449
  7. 7capy-build | gpt-5.50.8315
  8. 8pilot-real | claude-opus-4-60.8295
  9. 9ob1-cli | gpt-5.40.8247
  10. 10codex | gpt-5.50.8225
  11. 11polaris | claude-opus-4-70.8221
  12. 12polaris | gemini-3.1-pro-preview0.8221
  13. 13polaris | gpt-5.50.8221
  14. 14Lemoncode-agent | gemini-3.1-pro-preview0.822
  15. 15forge | claude-opus-4-60.818
  16. 16forge | gpt-5.40.818
  17. 17judy | gemini-3.1-pro-preview0.8022
  18. 18claude-wozcode-plugin | claude-opus-4-70.8022
  19. 19Lemoncode-agent | gpt-3.5-codex0.7986
  20. 20forge | gemini-3.1-pro-preview0.786
  21. 21sage | gpt-5.3-codex0.7843
  22. 22ob1-cli | claude-opus-4-60.7742
  23. 23ob1-cli | claude-opus-4-50.7742
  24. 24ob1-cli | gpt-5.3-codex0.7742
  25. 25Factory Droid | gpt-5.3-codex0.773
  26. 26terminus-kira-env-bootstrap | claude-opus-4-60.764
  27. 27ruley | gpt-5.3-codex0.7593
  28. 28codelia | gpt-5.3-codex0.7573
  29. 29capy-build | claude-opus-4-60.7528
  30. 30simple_codex | gpt-5.3-codex0.7494
  31. 31terminus-3-3 | claude-opus-4-60.7489
  32. 32terminus-3-3 | gemini-3.1-pro-preview0.7483
  33. 33final | claude-4.6-opus0.7339
  34. 34judy | claude-opus-4.60.7191
  35. 35mux | gpt-5.3-codex0.7157
  36. 36spoox-m | gpt-5.3-codex0.7146
Full data on Hugging FaceBack to the gallery