Indexing the Unreadable: LLM-Native Recursive Construction and Search of Service Taxonomies

Researchers propose A2X, an LLM-native service discovery system that organizes thousands of callable services into hierarchical taxonomies to solve the context-window limitation problem facing AI agents. The approach achieves 20+ point improvements in retrieval accuracy while reducing token consumption to one-ninth compared to baseline methods, enabling scalable orchestration of distributed services.

The emergence of the Internet of Agents represents a fundamental shift in how AI systems will operate, moving from monolithic models to distributed networks of specialized services. However, this architectural transition exposes a critical technical bottleneck: language models cannot efficiently manage discovery across large service registries because context windows are finite and models exhibit documented attention deficiencies in long sequences. A2X addresses this by leveraging LLMs themselves to construct hierarchical taxonomies of services, enabling progressive disclosure during query time rather than dumping all service descriptions into prompts.

This research builds on years of work in information retrieval and prompt optimization, extending insights about context scarcity into the agent orchestration domain. The hierarchical approach mirrors successful patterns from traditional search and database systems, but optimizes specifically for LLM constraints. By reducing each query to a small, highly-relevant candidate set, the system decouples effective context availability from registry size—a crucial property as service ecosystems grow.

The practical implications are significant for developers building agentic systems. Token efficiency translates directly to reduced API costs and faster inference, while improved Hit Rate accuracy means better service selection and fewer failed agent executions. This technology becomes particularly valuable as organizations expose hundreds or thousands of internal services through MCP servers and A2A protocols.

The competitive advantage gained by embedding taxonomic intelligence directly into the LLM pipeline, rather than relying purely on external embedding systems, suggests a broader trend: successful agentic infrastructure will combine LLM reasoning with optimized information architecture. Future development should focus on taxonomy maintenance, cross-domain service discovery, and handling dynamic service registries where endpoints appear and disappear.

• →A2X solves the context-window bottleneck for service discovery by organizing services hierarchically and querying layer-by-layer instead of concatenating all descriptions
• →The system achieves 6.2-point Hit Rate improvement over full-context methods while using only 11% of the token budget
• →Performance gains exceed embedding-based baselines by over 20 points, indicating LLM-native approaches outperform pure vector-similarity methods for service discovery
• →Hierarchical taxonomies decouple effective context from registry size, enabling scalable agent orchestration across thousands of services
• →Token efficiency improvements directly reduce API costs and inference latency for agentic AI systems in production