Knowledge
WikiContradict
WikiContradict: real-world knowledge-conflict QA mined from Wikipedia. Each item is a question whose answer is contradicted across two Wikipedia passages; LLM answers are graded by humans as Correct / Partially correct / Incorrect under several prompt (RAG) templates.
48items
5subjects
Apache-2.0license
knowledgedomain
textmodality
item-level responses released
Saturation status: Unknown
Response matrix
Fit to width. Hover for subject & item; click a cell for details.
lowhighUnobserved
Scale: {0, 0.5, 1} (Incorrect / Partially correct / Correct)
Subjects
- 1llama-3-70b-instruct0.6542
- 2mixtral-8x7b-instruct-v0-10.6229
- 3gpt-40.6083
- 4llama-2-70b-chat0.5917
- 5mistral-7b-instruct-v0-20.5813