Knowledge
HiST-LLM
HiST-LLM: expert-level global history knowledge benchmark of 36,577 four-choice multiple-choice questions derived from the Seshat Global History Databank. Each question asks whether a historical characteristic was Present / Inferred Present / Inferred Absent / Absent for a named polity over a stated time frame.
36,577items
7subjects
CC-BY-NC-SA-4.0license
knowledgedomain
textmodality
item-level responses released
Saturation status: Unknown
Response matrix
Fit to width. Hover for subject & item; click a cell for details.
Correct (1)Incorrect (0)Unobserved
Scale: 1 = correct · 0 = incorrect
Subjects
- 1Llama-3.1-70B-Instruct-Turbo0.5318
- 2GPT-4-Turbo (2024-04-09)0.5295
- 3Llama-3-70B-chat0.5272
- 4GPT-4o (2024-05-13)0.5091
- 5Gemini-1.5-flash0.4802
- 6GPT-3.5-Turbo-01250.4436
- 7Llama-3.1-8B-Instruct-Turbo0.3894