Mathematics

MMMU_DEV_VAL

College-level multimodal questions across 30 subjects, with per-question model predictions released by OpenVLMRecords.

1,050items

253subjects

MMMU dataset card: Apache-2.0. OpenVLMRecords and VLMEvalKit: Apache-2.0, as declared upstream.license

reasoningdomain

sciencedomain

mathematicsdomain

textmodality

imagemodality

item-level responses released

Full data on Hugging Face Back to the gallery

Paper and credit

2024

MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI (opens in a new tab)

Original source

Authors

Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng +18 moreShow all

01Xiang Yue
02Yuansheng Ni
03Kai Zhang
04Tianyu Zheng
05Ruoqi Liu
06Ge Zhang
07Samuel Stevens
08Dongfu Jiang
09Weiming Ren
10Yuxuan Sun
11Cong Wei
12Botao Yu
13Ruibin Yuan
14Renliang Sun
15Ming Yin
16Boyuan Zheng
17Zhenzhu Yang
18Yibo Liu
19Wenhao Huang
20Huan Sun
21Yu Su
22Wenhu Chen

Citation

BibTeX citationShow

BibTeX

1@misc{yue2024mmmumassivemultidisciplinemultimodal,
2      title={MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI},
3      author={Xiang Yue and Yuansheng Ni and Kai Zhang and Tianyu Zheng and Ruoqi Liu and Ge Zhang and Samuel Stevens and Dongfu Jiang and Weiming Ren and Yuxuan Sun and Cong Wei and Botao Yu and Ruibin Yuan and Renliang Sun and Ming Yin and Boyuan Zheng and Zhenzhu Yang and Yibo Liu and Wenhao Huang and Huan Sun and Yu Su and Wenhu Chen},
4      year={2024},
5      eprint={2311.16502},
6      archivePrefix={arXiv},
7      primaryClass={cs.CL},
8      url={https://arxiv.org/abs/2311.16502},
9}

Response matrix

Loading response matrix…

Grade unavailable

Scale: Individual grades unavailable; prompts and recorded answers remain browsable