Cybersecurity
CTIBench
CTIBench: a benchmark for evaluating LLMs on cyber threat intelligence tasks, including multiple-choice CTI knowledge (CTI-MCQ), CVE-to-CWE root-cause mapping (CTI-RCM), CVSS vulnerability severity prediction (CTI-VSP), and threat-actor attribution (CTI-TAA). Each task provides per-(model, item) predictions for five LLMs graded against gold answers.
5,483items
5subjects
MITlicense
cybersecuritydomain
knowledgedomain
textmodality
item-level responses released
Saturation status: Unknown
Response matrix
Fit to width. Hover for subject & item; click a cell for details.

Correct (1)Incorrect (0)Unobserved
Scale: 1 = correct · 0 = incorrect
Subjects
- 1gpt-4-turbo0.6189
- 2Gemini-1.50.5834
- 3Llama-3-70B0.5673
- 4gpt-3.5-turbo0.5092
- 5Llama-3-8B0.4559