Beyond Final Answers: Auditing Trajectory-Level Hallucinations in Multi-Agent Industrial Workflows

Researchers introduce Trajel, a dataset and evaluation framework for detecting hallucinations in multi-step LLM agent workflows, revealing that existing benchmarks miss intermediate failures. The framework defines five hallucination types and shows that trajectory-level detection outperforms traditional post-hoc verification, highlighting critical gaps in current AI safety evaluation methodologies.

The deployment of large language models as autonomous agents represents a significant shift in AI application, yet existing evaluation methods remain inadequate for production environments. Trajel addresses a fundamental blind spot: most hallucination benchmarks assess only final outputs, ignoring failures embedded within multi-step reasoning processes where agents plan, execute tools, and observe results. This oversight has real consequences for industrial workflows where intermediate errors can compound or trigger cascading failures.

The research emerges from growing concern about AI reliability in mission-critical applications. As organizations increasingly automate complex processes using agentic systems, the need for granular failure detection becomes urgent. The five-type taxonomy—factual, referential, logical, procedural, and scope-based hallucinations—provides a structured framework for understanding failure modes that occur during intermediate steps rather than only at completion. The finding that nearly half of hallucinated trajectories involve multiple hallucination types simultaneously underscores the complexity of agent behavior.

The implications for stakeholders span multiple dimensions. Developers deploying LLM agents now have evidence that standard verification approaches miss subtle but critical errors. Organizations building on agentic platforms must implement trajectory-aware monitoring rather than relying on final-answer validation. The research demonstrates that high binary accuracy in detection models can mask systematic misclassification of subtle hallucination types, suggesting current production safeguards may offer false confidence.

Moving forward, the adoption of trajectory-level auditing frameworks becomes essential for safe agentic deployment. Organizations must shift from endpoint validation to continuous monitoring across reasoning steps. This work establishes that comprehensive safety evaluation requires taxonomy-grounded approaches, positioning trajectory awareness as a baseline requirement rather than optional enhancement.

• →Existing hallucination benchmarks miss intermediate failures in multi-step agent workflows by focusing only on final outputs
• →Nearly 50% of hallucinated trajectories involve multiple hallucination types, requiring comprehensive detection beyond single-category classification
• →Automated detectors with high binary accuracy still systematically misclassify subtle hallucination types, indicating false confidence in current systems
• →Trajectory-aware detection significantly outperforms standard post-hoc verification for identifying agent reasoning failures
• →Industrial deployment of autonomous agents requires taxonomy-grounded evaluation frameworks rather than endpoint-focused assessment methods