General

DORIS-MAE

DORIS-MAE Anno-GPT annotation matrix. DORIS-MAE (NeurIPS 2023 D&B) is a scientific-document retrieval benchmark of complex, multi-level aspect-based queries over arXiv CS abstracts. Its Anno-GPT framework uses chatgpt-3.5-turbo-0301 as an expert-level relevance annotator. The public Zenodo dataset releases, for all 165,144 (aspect, candidate-abstract) pairs, ChatGPT's 3-level relevance label (0=unrelated, 1=partial, 2=fully answers the aspect) plus its explanation text. This is a single-subject per-item matrix (subject=ChatGPT-3.5; item=one (aspect, document) pair; response=0/1/2 label). For the 250 Test_set pairs, three human annotators' majority-vote label is attached as gold. The 17 retrieval models benchmarked in the paper are reported only as aggregate metrics (no per-item outputs released), so they are excluded.