Software Engineering
LiveCodeBench
LiveCodeBench tests whether models can solve competitive programming problems by writing a program that produces the expected output on the problem's test cases.
1,055items
72subjects
CC-BY-4.0license
software_engineeringdomain
textmodality
item-level responses released
Saturation status: No
Response matrix
leaderboard-only rows remain aggregate pass@1
Loading session strips…
lowhighUnobserved
Scale: {0, 1}