Mathematics

Abstract-Reason

Abstract-Reason benchmark: rule-based symbolic reasoning tasks (basic/extended computation, non-decimal number bases, symbolic function inference, symbolic manipulation) with systematic symbol remapping to separate genuine abstraction from memorization.

9,089items
1subjects
MITlicense
reasoningdomain
mathematicsdomain
textmodality
item-level responses released
Saturation status: Unknown

Response matrix

Fit to width. Hover for subject & item; click a cell for details.

Abstract-Reason response matrix: AI models (rows) against items (columns)
Correct (1)Incorrect (0)Unobserved

Scale: 1 = correct · 0 = incorrect