JustDiag!: A Diagnostic Justification Engine for Accountable Root Cause Analysis

Researchers introduce JustDiag, an AI-powered diagnostic justification engine that improves root cause analysis (RCA) by maintaining explicit process states, competing hypotheses, and evidence tracking rather than relying solely on fluent final answers. Evaluated on 66 real-world incidents, the system demonstrates stronger accountability and process quality in high-stakes operational environments where transparency and calibrated uncertainty matter more than confident completion.

JustDiag addresses a critical gap in how large language models conduct root cause analysis for incident response. While LLMs excel at generating fluent narratives, operators in high-stakes environments—critical infrastructure, financial systems, healthcare—require explicit documentation of reasoning pathways, contradictions, and evidentiary support. This distinction between fluent output and accountable diagnostics represents a maturation of AI deployment in operational contexts.

The research reflects broader industry recognition that AI reasoning systems must become transparent and auditable for enterprise adoption. In incident response, engineers currently face a choice: trust a model's confident but unexplained diagnosis, or spend substantial time reconstructing its logic. JustDiag inverts this by making justification the primary artifact. The two-layer evaluation protocol—scoring both final-answer quality and diagnostic process—establishes methodological rigor often missing in LLM benchmarking.

For enterprise AI deployment, this work validates that accepting lower task completion rates in favor of calibrated uncertainty and documented reasoning produces superior real-world outcomes. Organizations relying on AI-assisted diagnostics for incident response can now demand transparency standards rather than settling for confident fluency. The 66-incident validation set, while limited, suggests practical applicability across diverse failure modes.

Looking forward, this framework extends beyond incident response to any domain requiring high-stakes diagnostic reasoning—cybersecurity threat analysis, medical diagnostics, or financial fraud detection. As AI systems increasingly support critical decision-making, the demand for explicitly justified outputs will likely become compliance requirements rather than nice-to-have features, establishing new benchmarks for AI system evaluation in regulated industries.

• →JustDiag achieves stronger accountability by maintaining explicit process states tracking evidence, hypotheses, conflicts, and uncertainty rather than relying on fluent final answers alone.
• →The system demonstrates higher quality outcomes while accepting lower completion rates, validating that calibrated uncertainty improves real-world incident response effectiveness.
• →Two-layer evaluation protocols measuring both answer quality and diagnostic process quality establish new methodological standards for assessing AI reasoning systems.
• →Transparent, documented reasoning pathways become critical for enterprise adoption of AI in high-stakes operational and compliance-sensitive environments.
• →The framework extends beyond incident response to any domain requiring auditable diagnostic reasoning, from cybersecurity to medical analysis to fraud detection.