Agents & Tool Use
OSWorld 2.0
OSWorld 2.0 tests whether computer use agents can finish long real world tasks on a live Ubuntu desktop and its web applications, working from screenshots, with the final state of the machine checked against every task requirement.
108items
8subjects
apache-2.0license
agents_and_tool_usedomain
textmodality
imagemodality
gui_screenshotmodality
item-level responses released
Saturation status: No
Response matrix
Rasch analysis p = σ(θ − z + c)
1,835 responses, 80/20 split over cells · 8 subjects · 108 items · 21 conditions
1 of 8 subjects answered every item alike, so its θ is unbounded and sits pegged to the column edge.
AUC train
0.980
AUC test
0.937
Loading session strips…
Correct (1)Incorrect (0)Unobserved
Scale: 1 = correct · 0 = incorrect