Multimodal
Build AI2D_TEST response matrix from VLMEval/OpenVLMRecords
Response matrix
Each row is an AI model and each column an item, ordered so the strongest models and easiest items gather toward one corner. 254 subjects × 3,088 items, 98% of cells evaluated. The heatmap shows a representative 762 of 3,088 items — evenly sampled across difficulty — so each cell stays square and legible.
Scale: 1 = correct · 0 = incorrect
Sample items
A spread of items across the difficulty range, each shown with a few subjects' actual answers.
Carmine made a food web diagram. If there were a sudden decrease in the amount of Algae, which would be most affected? A: the Sun B: the trees C: small marine animals D: the wolves (A) D (B) B (C) A
How subjects answered
D
D. the wolves
C
Using only the information in the diagram, choose the answer which describes humans? A: Carnivores B: Producer C: Omnivores D: Herbivores
How subjects answered
A. Carnivores
A
Based on the given image and considering that fish eat small marine animals (which presumably include shrimp), and humans catch those fish to consume them for food, we can conclude that: The answer is C. Humans are omnivores because they eat both plants and animals. Option A (carnivores) would be incorrect as it implies that humans primarily feed on other meat-eating organisms. Option B (producer) refers to an organism capable of producing its own energy through photosynthesis or chemosynthesis; this does not describe human beings who rely on consuming other living things for their nutrition …
Which leaf has an asymmetrical base? A: Truncate B: Narrowly cuneate C: Attenuate D: Oblique
How subjects answered
D
D
A. Truncate
What feeds on nematodes? A: plants B: birds C: arthropods D: fungi
How subjects answered
C
C
B
Which bird is adapted for clinging to and climbing trees? A: Eagle B: Sparrow C: Woodpecker D: Blackbird
How subjects answered
C
The bird adapted for clinging to and climbing trees is the woodpecker. Woodpeckers have zygodactyl feet, which means they have two toes facing forward and two facing backward, allowing them to cling to tree bark effectively. Therefore, the correct answer is: C. Woodpecker
The correct answer is A, theeagle.
What is the function of Mitochondrian? A: Exhaling Oxygen. B: Drinking Water. C: Conversion of food to usable energy. D: Protection of cell membrane.
How subjects answered
C
C
C
Subjects
254 subjects, ranked by mean response (accuracy) across this benchmark's items.
+ 218 more subjects evaluated.