Stanford UniversityAI Measurement ScienceAIMS — AI Measurement Science

Agents & Tool Use

BFCL

Build a per-task binary response matrix and extract per-task error types from BFCL score files

4,133items
93subjects
Apache-2.0license
agents_and_tool_usedomain
textmodality
item-level responses released
Saturation status: No
Original sourcePaper← All benchmarks

Response matrix

Fit to width. Hover for subject & item; click a cell for details.

BFCL response matrix: AI models (rows) against items (columns)
Correct (1)Incorrect (0)Unobserved

Scale: 1 = correct · 0 = incorrect

Subjects

  1. 1watt-ai_watt-tool-70B0.7823
  2. 2gpt-4o-2024-11-200.7814
  3. 3gpt-4o-2024-11-20-FC0.7616
  4. 4gpt-4-turbo-2024-04-09-FC0.7548
  5. 5watt-ai_watt-tool-8B0.747
  6. 6o1-2024-12-170.747
  7. 7gemini-2.0-flash-exp0.734
  8. 8meetkai_functionary-medium-v3.1-FC0.7239
  9. 9gemini-1.5-pro-0020.7182
  10. 10o1-mini-2024-09-120.7156
  11. 11gpt-4o-mini-2024-07-18-FC0.714
  12. 12MadeAgents_Hammer2.1-7b0.7137
  13. 13Qwen_Qwen2.5-72B-Instruct0.7135
  14. 14gemini-2.0-flash-exp-FC0.7114
  15. 15gemini-1.5-pro-0010.7078
  16. 16gpt-4o-mini-2024-07-180.7068
  17. 17gemini-1.5-pro-002-FC0.7057
  18. 18gemini-exp-1206-FC0.7053
  19. 19Team-ACE_ToolACE-8B0.7053
  20. 20gemini-1.5-pro-001-FC0.6984
  21. 21Qwen_Qwen2.5-32B-Instruct0.696
  22. 22MadeAgents_Hammer2.1-3b0.6943
  23. 23gemini-1.5-flash-0020.6868
  24. 24Qwen_Qwen2.5-14B-Instruct0.6847
  25. 25gemini-1.5-flash-002-FC0.683
  26. 26Salesforce_xLAM-8x22b-r0.6777
  27. 27meetkai_functionary-small-v3.1-FC0.6771
  28. 28deepseek-ai_DeepSeek-Coder-V2-Instruct-07240.6739
  29. 29BitAgent_GoGoAgent0.6731
  30. 30gemini-1.5-flash-001-FC0.6701
  31. 31claude-3-5-sonnet-20241022-FC0.6686
  32. 32claude-3-opus-20240229-FC0.6678
  33. 33mistral-large-2407-FC0.6659
  34. 34DeepSeek-V30.6629
  35. 35mistral-medium-23120.6583
  36. 36gpt-4-turbo-2024-04-090.6566
Full data on Hugging FaceBack to the gallery