Agents & Tool Use
REAL
REAL tests whether web agents can complete user tasks on deterministic clones of real websites, counting an attempt as successful only when every one of the task's automatic checks passes.
233items
44subjects
unknownlicense
agents_and_tool_usedomain
textmodality
gui_screenshotmodality
item-level responses released
Saturation status: No
Response matrix
Rasch analysis p = σ(θ − z + c)
4,449 responses, 80/20 split over cells · 44 subjects · 233 items · 12 conditions
AUC train
0.939
AUC test
0.899
Loading session strips…
Correct (1)Incorrect (0)Unobserved
Scale: 1 = correct · 0 = incorrect