General
GeckoNum
GeckoNum: numerical reasoning in text-to-image models. 1,386 numeric prompts are rendered by 7 T2I models (5 seeds); humans annotate each generated image on three tasks (object counting, relational description choice, and yes/no DSG questions). Each row grades whether one model's generated image matched the numeric prompt.
2,027items
7subjects
CC-BY-4.0license
reasoningdomain
imagemodality
textmodality
item-level responses released
Saturation status: Unknown
Response matrix
Fit to width. Hover for subject & item; click a cell for details.
Correct (1)Incorrect (0)Unobserved
Scale: 1 = correct · 0 = incorrect
Subjects
- 1dalle_30.4317
- 2muse_b0.3601
- 3imagen_c0.3382
- 4muse_a0.3299
- 5imagen_d0.2817
- 6imagen_a0.2683
- 7imagen_b0.266