Agents All the Way Down; A Methodology for Building Custom AI Agents from Substrate to Production
Researchers present 'Agents All the Way Down,' a framework-agnostic methodology for building custom AI agents from development through production. The approach combines preconditions (substrate setup and building blocks) with three iterative practices (prototyping, CLI deployment via the Turtle pattern, and agent-driven testing), offering developers a structured path to create specialized agents tailored to specific applications rather than relying on general-purpose models.
The paper addresses a significant gap in AI development practice: while tools for building custom agents exist scattered across APIs and protocols, no cohesive methodology has been formally documented. This work consolidates tacit knowledge from blogs, podcasts, and system prompts into a repeatable engineering process. The methodology establishes two foundational conditions—framing the LLM as a software component with proper tooling and abstraction, and assembling discrete building blocks like function calling and the Model Context Protocol—then applies three practices iteratively: prototyping with general-purpose agents, harvesting results into CLI-deployable units (the Turtle pattern), and using agent-driven testing to validate behavior.
This matters because it democratizes custom agent development. Previously, building production-ready agents required deep implicit knowledge; now engineers have explicit guidelines. The framework-agnostic approach ensures applicability across languages and platforms, reducing vendor lock-in and increasing adoption potential. The methodology's elegance—particularly how multi-agent orchestration emerges naturally from CLI composition—suggests agents are becoming standard infrastructure components rather than experimental research tools.
For developers and organizations, this represents a shift toward practical AI integration. The case study of the AAC agent for LAMB, built in ten days by one developer with AI assistance, demonstrates feasibility at scale. The agent-tests-agent practice is particularly valuable, complementing classical testing without replacing it and addressing a real gap in AI quality assurance. As custom agents proliferate across applications, standardized development practices become critical for maintainability and security.
- →A formal methodology for custom AI agent development addresses the current gap between scattered tools and production-ready systems
- →The framework-agnostic design ensures broad applicability across programming languages and AI platforms without vendor dependencies
- →Multi-agent orchestration emerges naturally from CLI composition, simplifying complex system architecture
- →Agent-driven testing (agent-tests-agent) provides a novel quality assurance approach complementary to traditional testing methodologies
- →The ten-day production deployment of the AAC agent demonstrates the methodology enables rapid development with minimal team resources