Tool-Schema Compression Enables Agentic RAG Under Constrained Context Budgets

Researchers demonstrate that tool-schema compression reduces token consumption by 44-50%, enabling large language model agents to function under tight context constraints. Testing across 14 models shows compressed schemas restore RAG functionality with +20.5 percentage point exact-match improvements at 8K tokens, while frontier models can now handle 800+ tools instead of ~494.

This research addresses a fundamental bottleneck in agentic AI systems: the competition between tool definitions and retrieval context within limited token budgets. As language models are deployed with dozens or hundreds of available tools, their schemas can consume prohibitive amounts of the context window, degrading performance in retrieval-augmented generation tasks. The study reveals a sharp binary effect—at 8K token budgets, uncompressed JSON schemas overflow completely, reducing exact-match accuracy to near-zero levels (2.6%), while the proposed TSCG compression technique restores functionality with dramatic improvements averaging 20.5 percentage points across tested models.

The research validates findings across a comprehensive evaluation spanning 14 models from 1.5B to 32B parameters plus frontier API models, testing against 6,566 controlled API calls. This scale of evaluation establishes credibility for the practical impact claims. The external validation on HotpotQA multi-hop reasoning tasks shows +48 percentage point improvement under overflow conditions, suggesting these gains transfer beyond the controlled test environment.

For developers building agentic systems, this work identifies schema compression as essential infrastructure rather than an optimization. Organizations deploying language models with large tool sets in resource-constrained environments—whether running local models or optimizing API costs—directly benefit from these techniques. The demonstration that frontier models can operationalize 800+ tools with compression versus 494 uncompressed opens new possibilities for complex agent architectures.

Looking forward, tool-schema compression likely becomes a standard preprocessing layer in agent frameworks. The publicly available code and checkpoints accelerate adoption, potentially influencing how major AI platforms design their tool-use abstractions.

• →Schema compression reduces tool definition tokens by 44-50%, enabling RAG functionality under 8K token budgets where uncompressed schemas completely overflow.
• →Testing across 14 models shows +20.5 pp average exact-match improvement, with some models gaining +24.7 pp when compression enables full functionality.
• →Compressed schemas support 800+ tools on frontier models versus ~494 with standard JSON schemas, expanding practical agent complexity limits.
• →At larger budgets (32K tokens), performance delta between compressed and uncompressed schemas is ≤1 pp, confirming benefits are purely budget-driven.
• →Research includes public code, data, and checkpoints, positioning schema compression as foundational infrastructure for constrained-context agent deployment.