Reasoning
Potemkin Understanding
Potemkin Understanding benchmark: 32 concepts across literary techniques, game theory and psychological biases. LLMs are scored on define / classify / generate / edit / incoherence tasks; each response is binary (1 = correct application/classification, 0 = incorrect).
4,023items
11subjects
unknownlicense
reasoningdomain
textmodality
item-level responses released
Saturation status: Unknown
Response matrix
Fit to width. Hover for subject & item; click a cell for details.
Correct (1)Incorrect (0)Unobserved
Scale: 1 = correct · 0 = incorrect
Subjects
- 1o3-mini0.9862
- 2gpt-4.5-preview0.9697
- 3o1-mini0.9284
- 4deepseek-ai/DeepSeek-R10.8195
- 5gemini-2.0-flash-exp0.8056
- 6deepseek-ai/DeepSeek-V30.8056
- 7meta-llama/Llama-3.3-70B-Instruct-Turbo0.7877
- 8mistralai/Mistral-7B-Instruct-v0.20.7769
- 9Qwen/Qwen2-VL-72B-Instruct0.7497
- 10claude-3-5-sonnet-202410220.7327
- 11gpt-4o0.7072