Mathematics
Cascade Routing
Per-(model x item) binary correctness released with the ICML 2025 paper "A Unified Approach to Routing and Cascading for LLMs", spanning SWE-Bench, Minerva Math, LiveCodeBench, ARC-Challenge, MMLU-Pro, MixEval and GSM8k.
16,552items
32subjects
40%observed
apache-2.0license
software_engineeringdomain
mathematicsdomain
knowledgedomain
reasoningdomain
textmodality
Response matrix
Fit to width. Hover for subject & item; click a cell for details.
Correct (1)Incorrect (0)Unobserved
Scale: 1 = correct · 0 = incorrect
Subjects
- 1meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo0.8184
- 2Qwen/Qwen2.5-72B-Instruct-Turbo0.7912
- 3meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo0.7596
- 4Qwen/Qwen2.5-7B-Instruct-Turbo0.7068
- 5mistralai/Mixtral-8x22B-Instruct-v0.10.6898
- 6google/gemma-2-27b-it0.6828
- 7google/gemma-2-9b-it0.6443
- 8meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo0.6247
- 9mistralai/Mixtral-8x7B-Instruct-v0.10.5857
- 1020241202_agentless-1.5_claude-3.5-sonnet-202410220.5091
- 1120241106_navie-2-gpt4o-sonnet0.4883
- 12mistralai/Mistral-7B-Instruct-v0.30.4875
- 1320240824_gru0.4569
- 14Qwen2.5-Coder-7B-Ins0.4078
- 1520241029_epam-ai-run-claude-3-5-sonnet0.3995
- 1620241028_agentless-1.5_gpt4o0.3916
- 17Qwen2.5-Math-1.5B-Ins0.3764
- 18Qwen2.5-Math-7B-Ins0.372
- 19microsoft/Phi-3.5-mini-instruct0.3699
- 20Qwen/Qwen2.5-3B-Instruct0.3648
- 2120241007_nfactorial0.3368
- 22Qwen/Qwen2.5-1.5B-Instruct0.3233
- 2320241016_epam-ai-run-gpt-4o0.2898
- 2420241001_nfactorial0.2585
- 2520240820_epam-ai-run-gpt-4o0.2454
- 26Qwen/Qwen2.5-0.5B-Instruct0.2167
- 27google/gemma-2b-it0.185
- 2820240620_sweagent_claude3.5sonnet0.1816
- 29Qwen2.5-Coder-1.5B-Ins0.1703
- 3020240402_sweagent_gpt40.1259
- 3120240728_sweagent_gpt4o0.1241
- 3220240402_sweagent_claude3opus0.0926