Declarative Data Services: Structured Agentic Discovery for Composing Data Systems

Researchers propose Declarative Data Services (DDS), a structured framework for using AI agents to discover and compose multi-system data backends more reliably than unbounded agentic search. The approach decomposes the complex search problem into typed layers with explicit knowledge flow, demonstrating convergence on working solutions where previous methods failed.

The paper addresses a fundamental limitation in applying large language model-driven discovery to real-world infrastructure composition. While LLMs have shown promise in finding novel algorithms and code under controlled conditions, deploying these agents to compose heterogeneous data systems presents unique challenges: the search space becomes combinatorially complex, success criteria shift from benchmark performance to actual runtime functionality, and the relevant knowledge for system composition remains sparsely represented in training data.

Traditional unbounded agentic discovery—where coding agents iterate on failure logs—fails consistently because the feedback loop lacks structure. DDS solves this by introducing four typed contracts across successive abstraction layers: user intent, operator directed acyclic graphs, per-system capabilities, and runtime attribution. This architecture enables specialized sub-agents to search within bounded spaces while maintaining knowledge flow through inline skill citations and error signals routed backward as typed messages.

For the infrastructure and data engineering community, this research demonstrates that structured AI-assisted composition outperforms naive agent iteration. The trading-backend proof-of-concept shows practical applicability beyond theoretical benchmarks. As organizations increasingly orchestrate complex data pipelines across multiple systems, tools that reliably compose these stacks using AI assistance could significantly reduce engineering overhead.

The work suggests a broader principle: unbounded agent exploration fails when the problem space is heterogeneous and feedback is high-dimensional. Future development likely focuses on how these typed layers can be standardized across different data-system families and whether the framework generalizes beyond trading infrastructure to other multi-system domains.

• →Structured agentic discovery with typed contracts converges reliably where unbounded agent iteration fails on data-system composition.
• →The framework decomposes complex multi-system search into four typed layers with explicit knowledge flow and error routing.
• →DDS demonstrates practical viability on trading-backend workloads, with runtime failures becoming inline skill patches for future deployments.
• →The approach addresses a critical gap: LLM agents struggle with heterogeneous search spaces where success requires actual runtime validation.
• →This work suggests that AI-assisted infrastructure composition requires architectural constraints and typed communication channels for reliable convergence.