Science

AI2D_TEST

AI2D_TEST tests whether vision language models can answer multiple choice questions about science diagrams, scored by whether the answer letter they give matches the correct choice.

3,088items

254subjects

CC-BY-SA-4.0license

sciencedomain

knowledgedomain

textmodality

imagemodality

item-level responses released

Saturation status: Yes

Full data on Hugging Face Back to the gallery

Paper and credit

2016

A Diagram Is Worth A Dozen Images (opens in a new tab)

Original source

Authors

Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo +2 moreShow all

01Aniruddha Kembhavi
02Mike Salvato
03Eric Kolve
04Minjoon Seo
05Hannaneh Hajishirzi
06Ali Farhadi

Citation

BibTeX citationShow

BibTeX

1@misc{kembhavi2016diagramworthdozenimages,
2      title={A Diagram Is Worth A Dozen Images},
3      author={Aniruddha Kembhavi and Mike Salvato and Eric Kolve and Minjoon Seo and Hannaneh Hajishirzi and Ali Farhadi},
4      year={2016},
5      eprint={1603.07396},
6      archivePrefix={arXiv},
7      primaryClass={cs.CV},
8      url={https://arxiv.org/abs/1603.07396},
9}

Response matrix

Rasch analysis p = σ(θ − z)

770,916 responses, 80/20 split over cells · 254 subjects · 3,088 items

AUC train

0.901

AUC test

0.896

Loading session strips…

Correct (1)Incorrect (0)Unobserved

Scale: 1 = correct · 0 = incorrect