Skip to main content
AIMSAI Measurement Science

Science & Engineering

CARE

CARE: a benchmark suite for the Classification And Retrieval of Enzymes. Task 1 predicts the Enzyme Commission (EC) number of a protein from its sequence; Task 2 retrieves the EC number of a chemical reaction. Methods produce a ranked list of predicted EC numbers per item; scored here as top-1 accuracy against the gold EC.

2,019items
11subjects
56%observed
MITlicense
biologydomain
chemistrydomain
sciencedomain
textmodality
othermodality
Original sourcePaperBuild script← All benchmarks

Response matrix

Every model, scored item by item.

Each row is an AI model and each column an item, ordered so the strongest models and easiest items gather toward one corner. 11 subjects × 2,019 items, 56% of cells evaluated.

Fit to width. Hover for subject & item; click a cell for details.

CARE response matrix: AI models (rows) against items (columns)
Correct (1)Incorrect (0)Unobserved

Scale: 1 = correct · 0 = incorrect

Sample items

What the questions look like — and how subjects answer.

A spread of items across the difficulty range, each shown with a few subjects' actual answers.

Item 10% solve rateanswer: 3.4.13.23

L-Glu-L-Trp + H2O = L-glutamate + L-tryptophan

How subjects answered

  • CLIPZyme incorrect

    3.4.19.13 > 2.3.2.2 > 3.5.1.2 > 3.4.13.18 > 3.4.19.5 > 3.4.11.10 > 3.4.11.1 > 4.3.2.10 > 3.5.1.18 > 3.4.11.23

  • CREEP incorrect

    3.4.17.21 > 3.4.11.20 > 3.4.13.21 > 3.4.13.20 > 3.5.1.13 > 3.4.11.2 > 3.4.13.23 > 3.4.13.18 > 3.4.13.19 > 3.4.19.1

  • CREEP_text incorrect

    3.4.13.20 > 3.4.13.21 > 3.4.13.19 > 3.4.13.18 > 3.4.13.23 > 3.4.17.21 > 3.4.13.4 > 3.4.11.20 > 3.4.11.25 > 3.4.11.22

  • ChatGPT_text incorrect

    3.4.13.18

  • Similarity incorrect

    3.4.13.19 > 3.4.11.17 > 3.4.13.21 > 3.4.11.23 > 3.4.11.1 > 6.3.2.48 > 3.4.13.18 > 3.4.17.4 > 6.3.2.49 > 3.4.11.22

  • random incorrect

    1.2.1.46 > 1.1.1.356 > 3.4.24.7 > 1.13.11.54 > 4.2.1.129 > 1.13.11.58 > 1.3.1.112 > 1.14.14.95 > 1.2.1.94 > 1.3.1.53

Item 20% solve rateanswer: 3.3.2.9

ethylene oxide + H2O = ethane-1,2-diol

How subjects answered

  • CLIPZyme incorrect

    3.5.1.101 > 4.3.2.8 > 2.5.1.80 > 2.5.1.110 > 3.4.11.5 > 2.5.1.34 > 3.5.3.3 > 2.5.1.122 > 3.4.13.23 > 3.5.1.26

  • CREEP incorrect

    3.2.1.207 > 3.2.1.84 > 3.2.1.177 > 3.2.1.20 > 3.2.1.91 > 3.2.1.78 > 3.2.1.8 > 3.2.1.74 > 3.2.1.4 > 3.2.1.3

  • CREEP_text incorrect

    5.3.99.7 > 3.4.23.19 > 2.4.1.219 > 3.2.1.125 > 3.4.23.32 > 3.2.1.186 > 3.2.1.161 > 3.2.1.204 > 3.2.1.72 > 3.2.1.206

  • ChatGPT_text incorrect

    3.3.2.8

  • Similarity incorrect

    4.2.1.143 > 3.4.22.32 > 3.4.21.95 > 3.4.21.38 > 5.3.99.7 > 3.4.22.14 > 3.4.21.64 > 4.2.1.28 > 1.2.98.1 > 4.2.1.30

  • random incorrect

    1.1.1.303 > 1.13.11.11 > 2.3.2.3 > 2.5.1.103 > 3.4.21.99 > 2.5.1.22 > 3.6.1.43 > 2.5.1.19 > 3.2.1.209 > 2.5.1.143

Item 30% solve rateanswer: 2.1.1.321

S-adenosyl-L-methionine + GGFGGRGGFG-NH2 = S-adenosyl-L-homocysteine + GGFGG-(Nomega-methyl-)RGGFG-NH2

How subjects answered

  • CLIPZyme incorrect

    1.1.98.7 > 5.2.1.8 > 5.3.4.1 > 3.4.22.70 > 3.4.21.89 > 2.1.1.360 > 1.8.98.7 > 3.5.1.28 > 3.4.24.64 > 3.4.24.65

  • CREEP incorrect

    1.1.99.38 > 2.3.2.13 > 4.3.1.14 > 2.3.2.6 > 3.4.19.14 > 3.4.22.70 > 2.3.1.257 > 2.1.4.3 > 1.3.1.114 > 3.4.19.13

  • CREEP_text incorrect

    2.3.1.256 > 2.3.1.257 > 2.3.2.13 > 2.3.2.15 > 3.4.19.14 > 2.3.1.108 > 2.3.1.33 > 2.1.1.318 > 2.3.2.2 > 2.7.11.23

  • ChatGPT_text incorrect

    2.1.1.1

  • Similarity incorrect

    1.1.98.7 > 1.8.98.7 > 1.1.99.38 > 1.21.98.1 > 3.5.4.28 > 4.3.2.1 > 3.5.3.24 > 3.5.3.22 > 2.1.4.3 > 3.5.3.7

  • random incorrect

    1.1.1.401 > 4.4.1.26 > 4.6.1.2 > 1.14.13.32 > 2.8.1.6 > 2.1.1.327 > 1.3.1.53 > 1.5.1.5 > 1.2.1.91 > 1.11.1.22

Item 40% solve rateanswer: 6.6.1.2

MMSKIDLDISNLPDTTISVREVFGIDTDLRVPAYSKGDAYVPDLDPDYLFDRETTLAILAGFAHNRRVMVSGYHGTGKSTHIEQVAARLNWPCVRVNLDSHVSRIDLVGKDAIVVKDGLQVTEFKDGILPWAYQHNVALVFDEYDAGRPDVMFVIQRVLESSGRLTLLDQSRVIRPHPAFRLFATANTVGLGDTTGLYHGTQQINQAQMDRWSIVTTLNYLPHDKEVDIVAAKVKGFTADKGRETVSKMVRVADLTRAAFINGDLSTVMSPRTVITWAENAHIFGDIAFAFRVTFLNKCDELERALVAEHYQRAFGIELKECAANIVLEATA

How subjects answered

  • CLEAN incorrect

    3.6.4.12 > 6.6.1.1 > 5.6.2.4 > 2.7.11.31 > 2.7.1.78 > 7.6.2.4 > 7.4.2.8 > 6.1.1.9 > 2.7.11.26 > 7.6.2.5

  • ChatGPT incorrect

    1.1.1.237

  • Foldseek incorrect

    3.6.4.12

  • Pika incorrect

    3.6.4.13

  • random incorrect

    1.14.14.39 > 5.1.3.25 > 2.6.1.40 > 3.5.4.29 > 1.3.1.95 > 1.4.1.18 > 2.3.1.129 > 3.5.3.15 > 2.6.1.106 > 3.4.24.83

Item 510% solve rateanswer: 1.2.7.6

D-glyceraldehyde-3-phosphate + H2O + oxidized methyl viologen = 3-phospho-D-glycerate + reduced methyl viologen + H+

How subjects answered

  • CREEP_text correct

    1.2.1.9 > 1.2.1.89 > 3.2.2.17 > 1.1.99.14 > 1.1.1.29 > 1.17.1.5 > 1.1.1.402 > 1.2.5.1 > 1.17.1.9 > 1.1.1.110

  • Similarity correct

    1.7.2.6 > 1.3.99.33 > 1.2.1.90 > 1.7.2.2 > 1.2.1.89 > 1.2.1.59 > 1.2.1.9 > 1.2.1.46 > 1.7.7.1 > 1.2.1.13

  • CLIPZyme incorrect

    4.1.1.48 > 4.3.2.10 > 2.2.1.10 > 4.1.2.4 > 4.1.2.56 > 4.1.1.23 > 4.1.2.40 > 3.4.19.9 > 4.1.2.8 > 5.3.1.24

  • ChatGPT incorrect

    1.1.1.48

  • ChatGPT_text incorrect

    1.2.1.9

  • random incorrect

    4.2.1.12 > 1.1.1.362 > 2.3.1.259 > 5.5.1.13 > 2.2.1.3 > 1.14.11.46 > 1.14.19.5 > 4.3.1.15 > 2.4.1.359 > 1.1.1.261

Item 617% solve rateanswer: 6.3.1.11

MHDIDDFLKKNRVTEIEAIIPDMAGIARGKIIPRSKFESGESMRLPQAVMIQTVTGDYPEDGTLTGVTDPDMVCVPDASTIRMIPWAVDPTAQVIHDCVHFDGTPVAISPRRVLRRVLELYKAKGWKPVIAPELEFYLVDMNKDPDLPLQPPIGRTGRPETGRQAYSIEAVNEFDPLFEDIYEYCEVQELEVDTLIHEVGAAQMEINFMHGDPLKLADSVFLFKRTVREAALRHKMYATFMAKPMEGEPGSAMHMHQSLVDEETGHNLFTGPDGKPTSLFTSYIAGLQKYTPALMPIFAPYINSYRRLSRFMAAPINVAWGYDNRTVGFRIPHSGPAARRIENRIPGVDCNPYLAIAATLAAGYLGMTQKLEATEPLLSDGYELPYQLPRNLEEGLTLMGACEPIAEVLGEKFVKAYLALKETEYEAFFRVISSWERRHLLLHV

How subjects answered

  • BLAST correct

    6.3.1.11

  • CLEAN incorrect

    6.3.1.2 > 6.3.1.11 > 5.4.4.3 > 6.3.4.12 > 6.3.1.6 > 3.5.3.26 > 1.13.11.24 > 1.11.1.6 > 2.7.7.53 > 4.1.1.19

  • ChatGPT incorrect

    1.1.1.16

  • Foldseek incorrect

    6.3.1.2

  • Pika incorrect

    6.1.1.33

  • random incorrect

    3.2.1.204 > 3.4.16.4 > 7.2.2.11 > 4.1.1.87 > 1.1.1.40 > 1.14.14.160 > 2.7.1.92 > 1.8.99.2 > 1.1.3.48 > 1.14.14.39

Item 729% solve rateanswer: 2.8.3.12

glutaconyl-CoA + butyrate = butyryl-CoA + glutaconate

How subjects answered

  • CLIPZyme correct

    2.8.3.15 > 2.8.3.5 > 2.8.3.9 > 2.8.3.16 > 2.8.3.25 > 6.2.1.13 > 2.8.3.13 > 2.8.3.23 > 6.2.1.19 > 6.2.1.9

  • CREEP correct

    2.8.3.18 > 2.8.3.23 > 2.8.3.8 > 2.8.3.5 > 2.8.3.6 > 2.8.3.9 > 2.8.3.12 > 2.8.3.22 > 2.8.3.19 > 2.8.3.26

  • CREEP_text correct

    2.8.3.23 > 2.8.3.19 > 2.8.3.18 > 2.8.3.22 > 2.8.3.8 > 2.8.3.26 > 2.8.3.13 > 2.8.3.20 > 2.8.3.6 > 2.8.3.16

  • ChatGPT incorrect

    1.3.3.6

  • ChatGPT_text incorrect

    2.8.3.9

  • random incorrect

    4.1.1.120 > 1.11.1.20 > 4.1.1.51 > 4.1.1.55 > 6.2.1.44 > 2.1.2.5 > 3.4.24.31 > 5.4.99.51 > 3.2.1.93 > 3.1.3.70

Item 840% solve rateanswer: 1.1.99.14

MQTQLTEEMRQNARALEADSILRACVHCGFCTATCPTYQLLGDELDGPRGRIYLIKQVLEGNEVTLKTQEHLDRCLTCRNCETTCPSGVRYHNLLDIGRDIVEQKVKRPLPERILREGLRQVVPRPAVFRALTQVGLVLRPFLPEQVRAKLPAETVKAKPRPPLRHKRRVLMLEGCAQPTLSPNTNAATARVLDRLGISVMPANEAGCCGAVDYHLNAQEKGLARARNNIDAWWPAIEAGAEAILQTASGCGAFVKEYGQMLKNDALYADKARQVSELAVDLVELLREEPLEKLAIRGDKKLAFHCPCTLQHAQKLNGEVEKVLLRLGFTLTDVPDSHLCCGSAGTYALTHPDLARQLRDNKMNALESGKPEMIVTANIGCQTHLASAGRTSVRHWIEIVEQALEKE

How subjects answered

  • CLEAN correct

    1.1.99.14 > 1.1.99.39 > 2.1.1.80 > 1.8.98.3 > 1.2.1.101 > 1.7.1.4 > 2.1.1.272 > 4.2.1.91 > 1.7.7.1 > 1.16.1.1

  • Foldseek correct

    1.1.99.14

  • ChatGPT incorrect

    2.7.11.1

  • Pika incorrect

    1.8.99.1

  • random incorrect

    4.2.3.103 > 3.2.1.143 > 6.3.1.11 > 1.14.19.27 > 2.3.2.21 > 2.3.1.7 > 4.2.3.158 > 4.6.1.17 > 2.7.7.71 > 2.5.1.51

Item 950% solve rateanswer: 1.1.2.4;1.1.99.40

MLRNILVRSTGSNFKFAGRYMKSSALLGYYRRVNYYSTKIQTRLTSENYPDVHRDPRFKKLTSDDLNYFKSILSEQEILRASESEDLSFYNEDWMRKYKGQSKLVLRPKSVEKVSLILNYCNDEKIAVVPQGGNTGLVGGSVPIFDELILSLANLNKIRDFDPVSGILKCDAGVILENANNYVMEQNYMFPLDLGAKGSCHVGGVVATNAGGLRLLRYGSLHGSVLGLEVVMPNGQIVNSMHSMRKDNTGYDLKQLFIGSEGTIGIITGVSILTVPKPKAFNVSYLSVESFEDVQKVFVRARQELSEILSAFEFMDAKSQVLAKSQLKDAAFPLEDEHPFYILIETSGSNKDHDDSKLETFLENVMEEGIVTDGVVAQDETELQNLWKWREMIPEASQANGGVYKYDVSLPLKDLYSLVEATNARLSEAELVGDSPKPVVGAIGYGHVGDGNLHLNVAVREYNKNIEKTLEPFVYEFVSSKHGSVSAEHGLGFQKKNYIGYSKSPEEVKMMKDLKVHYDPNGILNPYKYI

How subjects answered

  • BLAST correct

    1.1.2.4; 1.1.99.40

  • CLEAN correct

    1.1.99.40 > 1.1.99.39 > 1.1.2.4 > 1.1.3.38 > 6.1.1.20 > 5.1.3.13 > 1.3.8.4 > 2.7.1.6 > 6.3.2.3 > 2.5.1.60

  • Foldseek correct

    1.1.2.4; 1.1.99.40

  • ChatGPT incorrect

    1.1.1.40

  • Pika incorrect

    1.1.99.6

  • random incorrect

    2.3.1.n6 > 2.4.2.45 > 2.6.1.108 > 2.3.1.109 > 3.5.1.133 > 4.2.3.163 > 2.1.1.246 > 1.5.1.1 > 2.3.1.178 > 3.11.1.1

Item 1050% solve rateanswer: 1.14.11.15

MSSVTQLFKNNPVNRDRIIPLDFTNTKTLPDSHVWSKPEPETTSGPIPVISLSNPEEHGLLRQACEEWGVFHITDHGVSHSLLHNVDCQMKRLFSLPMHRKILAVRSPDESTGYGVVRISMFYDKLMWSEGFSVMGSSLRRHATLLWPDDHAEFCNVMEEYQKAMDDLSHRLISMLMGSLGLTHEDLGWLVPDKTGSGTDSIQSFLQLNSYPVCPDPHLAMGLAPHTDSSLLTILYQGNIPGLEIESPQEEGSRWIGVEPIEGSLVVIMGDLSHIISNGQFRSTMHRAVVNKTHHRVSAAYFAGPPKNLQIGPLTSDKNHPPIYRRLIWEEYLAAKATHFNKALTLFRC

How subjects answered

  • BLAST correct

    1.14.11.15

  • CLEAN correct

    1.14.11.15 > 2.1.1.328 > 3.1.7.11 > 3.5.2.19 > 4.2.3.121 > 1.14.11.24 > 4.2.3.109 > 1.2.1.78 > 4.2.3.56 > 1.14.20.6

  • Foldseek correct

    1.14.11.15

  • ChatGPT incorrect

    1.3.1.47

  • Pika incorrect

    1.14.11.1

  • random incorrect

    1.5.3.18 > 1.1.1.178 > 1.5.1.16 > 1.1.1.395 > 3.1.3.79 > 2.5.1.91 > 3.4.24.29 > 5.4.99.62 > 3.4.23.38 > 3.1.1.65

Item 1150% solve rateanswer: 3.7.1.5;4.1.1.112

MAASRPLSRFWEWGKNIVCVGRNYADHVREMQSAAPSEPVLFLKPSTAYAPEGSPVLVPAYTRNLHHELELAVVMGKRCRAVSEAAAMDYVAGYALCLDMTARDVQDECKKKGLPWTLAKSFTASCPVSAFVPKEKIPDPHNLKLWLKVNGELRQEGETSSMIFSIPYIISYVSKIMTLEEGDIILTGTPKGVGPVKENDEIQAGIHGVLSMKFKVERPEY

How subjects answered

  • BLAST correct

    3.7.1.5; 4.1.1.112

  • CLEAN correct

    3.7.1.5 > 4.1.1.68 > 4.1.1.77 > 5.1.3.13 > 3.7.1.20 > 4.3.2.3 > 1.13.11.54 > 1.13.11.53 > 6.3.2.47 > 5.1.3.27

  • Foldseek correct

    3.7.1.5; 4.1.1.112

  • ChatGPT incorrect

    1.1.1.237

  • Pika incorrect

    4.2.1.22

  • random incorrect

    4.1.2.57 > 4.3.2.7 > 2.1.1.173 > 2.4.1.217 > 2.5.1.20 > 2.8.1.7 > 2.4.1.353 > 2.1.1.359 > 2.6.1.42 > 2.7.1.66

Item 1267% solve rateanswer: 5.3.4.1

MMFDRRFFALVVLLCVSAVRSTEDASDDELNYEMDEGVVVLTDKNFDAFLKKNPSVLVKFYAPWCGHCKHLAPEYEKASSKVSIPLAKVDATVETELGKRFEIQGYPTLKFWKDGKGPNDYDGGRDEAGIVEWVESRVDPNYKPPPEEVVTLTTENFDDFISNNELVLVEFYAPWCGHCKKLAPEYEKAAQKLKAQGSKVKLGKVDATIEKDLGTKYGVSGYPTMKIIRNGRRFDYNGPREAAGIIKYMTDQSKPAAKKLPKLKDVERFMSKDDVTIIGFFATEDSTAFEAFSDSAEMLREEFKTMGHTSDPAAFKKWDAKPNDIIIFYPSLFHSKFEPKSRTYNKAAATSEDLLAFFREHSAPLVGKMTKKNAATRYTKKPLVVVYYNADFSVQYREGSEYWRSKVLNIAQKYQKDKYKFAVADEEEFAKELEELGLGDSGLEHNVVVFGYDGKKYPMNPDEFDGELDENLEAFMKQISSGKAKAHVKSAPAPKDDKGPVKTVVGSNFDKIVNDESKDVLIEFYAPWCGHCKSFESKYVELAQALKKTQPNVVLAKMDATINDAPSQFAVEGFPTIYFAPAGKKSEPIKYSGNRDLEDLKKFMTKHGVKSFQKKDEL

How subjects answered

  • BLAST correct

    5.3.4.1

  • CLEAN correct

    5.3.4.1 > 1.8.1.8 > 1.8.4.2 > 3.4.24.39 > 3.4.24.24 > 1.8.1.9 > 3.2.2.6 > 3.4.24.22 > 3.4.21.102 > 5.2.1.8

  • Foldseek correct

    5.3.4.1

  • Pika correct

    5.3.4.1

  • ChatGPT incorrect

    1.1.1.88

  • random incorrect

    3.2.1.74 > 2.6.1.59 > 2.7.1.157 > 6.5.1.4 > 1.14.13.215 > 1.2.1.60 > 2.5.1.100 > 2.7.1.21 > 6.1.1.24 > 2.8.3.10

Subjects

The models, agents, and reward models evaluated.

11 subjects, ranked by mean response (accuracy) across this benchmark's items.

  1. 1BLAST0.8156
  2. 2Foldseek0.7219
  3. 3CLEAN0.6842
  4. 4Pika0.2817
  5. 5CREEP_text0.2183
  6. 6Similarity0.187
  7. 7ChatGPT_text0.1453
  8. 8CREEP0.1421
  9. 9CLIPZyme0.049
  10. 10ChatGPT0.02
  11. 11random0.0004
Full data on Hugging FaceBack to the gallery