Stanford UniversityAI Measurement ScienceAIMS — AI Measurement Science

Software Engineering

VideoGameQA-Bench

VideoGameQA-Bench: evaluating VLMs for video game quality assurance. Per-item verdict dumps for 4 tasks (image/video glitch detection, image/video bug-report generation) over 16+ VLMs, parsed from the project artifacts dashboards. response is the upstream per-item Correct?/judge-Match boolean.

2,200items
17subjects
CC-BY-4.0license
software_engineeringdomain
imagemodality
videomodality
textmodality
item-level responses released
Saturation status: Unknown
Original sourcePaper← All benchmarks

Response matrix

Fit to width. Hover for subject & item; click a cell for details.

VideoGameQA-Bench response matrix: AI models (rows) against items (columns)
Correct (1)Incorrect (0)Unobserved

Scale: 1 = correct · 0 = incorrect

Subjects

  1. 1GPT-4o0.8148
  2. 2GPT-4.10.7639
  3. 3o30.7303
  4. 4Gemini-2.5-Pro0.7294
  5. 5GPT-4.1-mini0.7104
  6. 6o4-mini0.7096
  7. 7Claude-3.5-Sonnet0.6956
  8. 8Claude-3.7-Sonnet0.628
  9. 9Gemini-2.5-Flash0.6174
  10. 10Qwen-2.5-VL-72B0.5892
  11. 11Gemini-2.0-Flash0.5782
  12. 12Mistral-Small-3.1-24B0.5773
  13. 13Llama-4-Scout0.5714
  14. 14Llama-4-Maverick0.5342
  15. 15GPT-4.1-nano0.5
  16. 16Gemma-3-27B0.455
  17. 17Qwen-VL-Max0.1508
Full data on Hugging FaceBack to the gallery