LiveMCPBench: Can Agents Navigate an Ocean of MCP Tools?

LiveMCPBench introduces the first large-scale benchmark evaluating AI agents' ability to navigate real-world tasks using Model Context Protocol (MCP) tools across multiple servers. The benchmark reveals significant performance gaps, with top model Claude-Sonnet-4 achieving 78.95% success while most models only reach 30-50%, identifying tool retrieval as the primary bottleneck.

• →LiveMCPBench evaluates 95 real-world tasks across 70 servers with 527 tools, addressing gaps in current AI agent evaluation methods.
• →Claude-Sonnet-4 leads with 78.95% task success rate while most state-of-the-art models achieve only 30-50% success.
• →Retrieval errors account for nearly half of all agent failures, making it the dominant performance bottleneck.
• →Active tool composition strongly correlates with task success, highlighting the importance of multi-tool coordination.
• →The benchmark provides the first reproducible, large-scale diagnosis of MCP agent capabilities with publicly available code and data.