Evaluating Explainability in Safety-Critical ATR Systems: Limitations of Post-Hoc Methods and Paths Toward Robust XAI

A peer-reviewed study evaluates explainability methods in AI systems used for automatic target recognition in safety-critical applications, revealing that popular post-hoc explanation techniques have significant limitations including spurious explanations and vulnerability to manipulation. The research argues that current XAI approaches are insufficient for deployment in high-stakes environments and calls for more robust, causally-grounded methods that prioritize system assurance over visual plausibility.

This research addresses a critical gap between AI model performance and trustworthiness in defense and safety-critical systems. While machine learning has achieved impressive accuracy metrics in target recognition tasks, the ability to explain and validate these decisions remains underdeveloped. The paper systematically deconstructs four major explainability paradigms—saliency-based, attention-based, surrogate, and detection-aware methods—exposing how each fails under scrutiny when deployed in mission-critical contexts.

The study's findings carry substantial implications for military and aerospace applications where model decisions directly impact human safety and national security. By documenting failure modes such as spurious explanations that appear convincing but lack causal grounding, the authors demonstrate that visual plausibility can mask underlying algorithmic fragility. This distinction matters enormously: a defense contractor might deploy a system that appears interpretable to human operators while remaining vulnerable to adversarial inputs or distribution shifts.

For the AI development community, this research signals a maturation of thinking around explainability—moving beyond checkbox compliance toward genuine assurance engineering. Organizations building safety-critical systems face pressure to adopt XAI methods, but this analysis suggests many current approaches provide false confidence rather than genuine understanding. The path forward requires integrating causal inference, physics-informed constraints, and formal verification methods rather than relying solely on explanation visualizations.

The paper essentially repositions XAI from a user-facing feature to a validation mechanism, fundamentally changing how organizations should evaluate and deploy these tools in high-stakes environments.

• →Popular post-hoc XAI methods exhibit systematic vulnerabilities including spurious explanations and instability under perturbations unsuitable for safety-critical deployment.
• →Visual plausibility of explanation outputs can create false confidence and overtrust in AI systems, masking underlying algorithmic fragility.
• →Current explainability paradigms fail adequately across four critical dimensions: interpretability, robustness, manipulation resistance, and validation suitability.
• →Effective XAI for safety-critical systems requires causal grounding and physical constraints rather than purely visual or attention-based explanation methods.
• →Organizations deploying AI in defense and aerospace contexts need to treat explainability as an assurance engineering problem, not a feature to add post-deployment.