Science
AtmosSci-Bench
AtmosSci-Bench: a symbolic-template benchmark of atmospheric-science problems (MCQ and open-ended) spanning hydrology, atmospheric dynamics, atmospheric physics, geophysics, and physical oceanography. 25 LLMs across instruction, reasoning, math, and domain-specific categories are evaluated by accuracy with symbolic-perturbation robustness analysis.
670items
29subjects
CC-BY-4.0license
sciencedomain
reasoningdomain
textmodality
item-level responses released
Saturation status: Unknown
Response matrix
Fit to width. Hover for subject & item; click a cell for details.
Correct (1)Incorrect (0)Unobserved
Scale: 1 = correct · 0 = incorrect
Subjects
- 1deepseek-r10.8851
- 2Qwen3-235B-A22B-fp8-tput0.8821
- 3gpto10.8642
- 4o3-mini0.8164
- 5gemini-2.0-flash-thinking-exp-12-190.7701
- 6gemini-2.0-flash-thinking-exp-01-210.6776
- 7QwQ-32B-Preview0.6649
- 8gemini-2.0-flash-exp0.6418
- 9deepseek-v30.6328
- 10Qwen2.5-Math-72B-Instruct0.5836
- 11gpt4o0.5821
- 12Qwen2.5-72B-Instruct-Turbo0.5701
- 13Llama-3.1-405B-Instruct-Turbo0.5433
- 14Qwen2.5-32B-Instruct0.5284
- 15Llama-3.3-70B-Instruct0.5151
- 16DeepSeek-R1-Distill-Llama-70B0.4571
- 17gpt4o-mini0.4567
- 18Qwen2.5-7B-Instruct0.4343
- 19Qwen2.5-Coder-32B-Instruct0.4313
- 20gemma-2-27b-it0.3791
- 21Qwen2.5-72B-GeoGPT0.3493
- 22Qwen2.5-Math-7B-Instruct0.3478
- 23Qwen2.5-3B-Instruct0.3149
- 24deepseek-math-7b-instruct0.309
- 25Qwen2.5-Math-1.5B-Instruct0.3
- 26climategpt-70b0.2731
- 27deepseek-math-7b-rl0.2463
- 28climategpt-7b0.2015
- 29gemma-2-9b-it0.1851