Software Engineering
Terminal-Bench 2.1
Terminal-Bench 2.1 tests whether an agent can finish a real task inside a Linux container it drives through a shell, passing only when a test suite run against the state it leaves behind succeeds entirely.
89items
7subjects
Apache-2.0license
software_engineeringdomain
agents_and_tool_usedomain
textmodality
item-level responses released
Saturation status: No
Response matrix
Rasch analysis p = σ(θ − z + c)
5,785 responses, 80/20 split over cells · 7 subjects · 89 items · 10 conditions
AUC train
0.900
AUC test
0.883
Loading session strips…
Correct (1)Incorrect (0)Unobserved
Scale: 1 = correct · 0 = incorrect