Software Engineering
SWE-chat
SWE-chat measures whether developers accept what a coding agent did during real sessions on their own repositories, with a judge model reading each following prompt to decide whether the developer pushed back.
31,930items
34subjects
ODC-By-1.0license
software_engineeringdomain
agents_and_tool_usedomain
textmodality
item-level responses released
Saturation status: Yes
Response matrix
Observational field data, not a controlled benchmark: every prompt was answered by exactly one system, so this is a per-developer session timeline rather than a subject x item comparison. A cell is one user prompt, blue when the developer accepted the agent's answer and red when their next message pushed back.
Loading session strips…
accepted (1)pushed back (0)Unobserved
Scale: 1 = accepted · 0 = pushed back