Software Engineering
SWE-smith
SWE-smith agent trajectories: per-(model x task-instance) outcomes from SWE-agent runs over synthesized software-engineering task instances. Each cell is binary resolved (the agent patch passes the instance tests, 1) or not (0), recorded under three trajectory-format splits (tool/xml/ticks).
13,472items
3subjects
MITlicense
software_engineeringdomain
agents_and_tool_usedomain
textmodality
item-level responses released
Saturation status: Unknown
Response matrix
Fit to width. Hover for subject & item; click a cell for details.
Correct (1)Incorrect (0)Unobserved
Scale: 1 = correct · 0 = incorrect
Subjects
- 1claude-3-7-sonnet-202502190.4341
- 2claude-3-5-sonnet-202410220.4116
- 3gpt-4o-2024-08-060.3195