Self-Healing Agentic Orchestrators for Reliable Tool-Augmented Large Language Model Systems

Researchers present a self-healing orchestration framework for tool-augmented large language models that treats reliability as a bounded runtime control problem, achieving 98.8% task success by mapping failure signals to recovery actions and verifying results. The approach outperforms retry-only and full-replanning baselines across multiple benchmarks, particularly excelling when recovery budgets are constrained.

This research addresses a critical bottleneck in production AI systems: the reliability of orchestration layers that coordinate LLM planning, tool invocation, and recovery workflows. While LLM capabilities have advanced rapidly, deployed systems fail not just from model inference errors but from infrastructure-level issues including tool timeouts, malformed arguments, and stale context—problems that conventional retry logic handles poorly. The self-healing framework treats these failures as a bounded control problem, systematically mapping observable failure signals to specific recovery actions and enforcing explicit resource budgets. This design philosophy shifts reliability engineering from reactive (retry everything) to diagnostic (understand what failed and why).

The experimental validation is rigorous: a 100-task controlled fault-injection benchmark reveals that self-healing achieves 98.8% success versus 94.5% for retry-only approaches and 93.8% for full replanning. More importantly, under single-attempt recovery budgets, the gap widens to 94.0% versus 85.3% and 88.2%, demonstrating that the framework's targeted recovery actions use constrained resources more efficiently than brute-force methods. The verifier-guided variant eliminates silent failures (wrong-but-plausible outputs) entirely in semantic failure scenarios, addressing a particularly dangerous failure mode in production systems where incorrect answers presented confidently can propagate downstream.

For the AI infrastructure industry, this work establishes that orchestration-layer reliability engineering can be formalized and measured. As enterprises deploy more complex tool-calling LLM systems for financial analysis, customer service, and autonomous research, the ability to recover gracefully from partial failures becomes economically significant. The compact model-in-the-loop validation demonstrates practical applicability, suggesting the framework could integrate into existing tool-calling pipelines without requiring model retraining.

• →Self-healing orchestrators achieve 98.8% task success by mapping failure signals to targeted recovery actions, outperforming retry-only approaches by 4.3 percentage points.
• →Under single-attempt recovery budgets, self-healing reaches 94.0% success versus 85.3% for retry-only and 88.2% for full replanning, proving efficiency under resource constraints.
• →Verifier-guided recovery reduces silent failures to 0.0% in controlled semantic failure scenarios, eliminating the risk of confident wrong answers in production systems.
• →The framework formalizes reliability engineering for tool-augmented LLMs as a bounded runtime control problem with explicit observability traces and recovery verification.
• →Model-in-the-loop validation confirms the approach works with live tool-calling models, suggesting practical integration into existing production AI pipelines.