From Flat Logs to Causal Graphs: Hierarchical Failure Attribution for LLM-based Multi-Agent Systems

Researchers introduce CHIEF, a new framework that improves failure analysis in LLM-powered multi-agent systems by transforming execution logs into hierarchical causal graphs. The system uses oracle-guided backtracking and counterfactual attribution to better identify root causes of failures, outperforming existing methods on benchmark tests.

• →CHIEF framework addresses the fragility and opaque failure mechanisms in LLM-powered Multi-Agent Systems.
• →Traditional failure attribution methods treat execution logs as flat sequences, missing complex causal relationships.
• →The new approach uses hierarchical causal graphs and virtual oracles to efficiently identify true root causes.
• →CHIEF outperformed eight state-of-the-art baselines on both agent-level and step-level accuracy metrics.
• →The research focuses on improving observability and responsibility attribution in complex multi-agent AI systems.