Stanford UniversityAI Measurement ScienceAIMS — AI Measurement Science

General

BFCL

Build a per-task binary response matrix and extract per-task error types from BFCL score files

4,133items
93subjects
Apache-2.0license
agents_and_tool_usedomain
textmodality
item-level responses released
Saturation status: Unknown
Original sourcePaper← All benchmarks

Response matrix

Fit to width. Hover for subject & item; click a cell for details.

BFCL response matrix: AI models (rows) against items (columns)
Correct (1)Incorrect (0)Unobserved

Scale: 1 = correct · 0 = incorrect

Subjects

  1. 1watt-ai_watt-tool-70B0.7826
  2. 2gpt-4o-2024-11-200.7817
  3. 3gpt-4o-2024-11-20-FC0.7617
  4. 4gpt-4-turbo-2024-04-09-FC0.755
  5. 5watt-ai_watt-tool-8B0.7474
  6. 6o1-2024-12-170.7474
  7. 7gemini-2.0-flash-exp0.7344
  8. 8meetkai_functionary-medium-v3.1-FC0.7241
  9. 9gemini-1.5-pro-0020.7186
  10. 10o1-mini-2024-09-120.7161
  11. 11gpt-4o-mini-2024-07-18-FC0.7142
  12. 12MadeAgents_Hammer2.1-7b0.7141
  13. 13Qwen_Qwen2.5-72B-Instruct0.714
  14. 14gemini-2.0-flash-exp-FC0.7119
  15. 15gemini-1.5-pro-0010.7083
  16. 16gpt-4o-mini-2024-07-180.7072
  17. 17Team-ACE_ToolACE-8B0.7057
  18. 18gemini-exp-1206-FC0.7057
  19. 19gemini-1.5-pro-002-FC0.7053
  20. 20gemini-1.5-pro-001-FC0.6984
  21. 21Qwen_Qwen2.5-32B-Instruct0.6965
  22. 22MadeAgents_Hammer2.1-3b0.6947
  23. 23gemini-1.5-flash-0020.6872
  24. 24Qwen_Qwen2.5-14B-Instruct0.6851
  25. 25gemini-1.5-flash-002-FC0.6834
  26. 26Salesforce_xLAM-8x22b-r0.678
  27. 27meetkai_functionary-small-v3.1-FC0.6773
  28. 28deepseek-ai_DeepSeek-Coder-V2-Instruct-07240.6742
  29. 29BitAgent_GoGoAgent0.6735
  30. 30gemini-1.5-flash-001-FC0.6706
  31. 31claude-3-5-sonnet-20241022-FC0.6689
  32. 32claude-3-opus-20240229-FC0.6683
  33. 33mistral-large-2407-FC0.6662
  34. 34DeepSeek-V30.6628
  35. 35mistral-medium-23120.6586
  36. 36gpt-4-turbo-2024-04-090.6567
Full data on Hugging FaceBack to the gallery