Stanford UniversityAI Measurement ScienceAIMS — AI Measurement Science

General

Visual Riddles

Visual Riddles: a commonsense and world-knowledge challenge for large vision and language models. Each item is an AI-generated image with an open-ended question requiring visual reasoning plus world knowledge. Per-(model, item) responses are human-rated correct/incorrect on the open-ended task.

386items
6subjects
CC-BY-4.0license
reasoningdomain
knowledgedomain
imagemodality
textmodality
item-level responses released
Saturation status: Unknown
Original sourcePaper← All benchmarks

Response matrix

Fit to width. Hover for subject & item; click a cell for details.

This condition combination wasn’t evaluated — try a different attack, category, or judge.
Correct (1)Incorrect (0)Unobserved

Scale: 1 = correct · 0 = incorrect

Subjects

  1. 1gemini-1.5-pro0.3744
  2. 2GPT40.3425
  3. 3gemini-pro-vision0.305
  4. 4llava-v1.6-34b0.15
  5. 5llava-v1.5-7b0.135
  6. 6InstructBlip0.1275
Full data on Hugging FaceBack to the gallery