EnTrust: Modeling Inter-Modal Conflict for Trustworthy Multimodal Medical Image Analysis

EnTrust is a new framework for multimodal medical image analysis that treats disagreement between imaging modalities as a direct source of predictive uncertainty rather than averaging it away. The approach combines feature decomposition, diffusion-based segmentation, and calibrated uncertainty estimation to help clinicians understand not just where predictions are uncertain, but why, achieving state-of-the-art accuracy across multiple medical imaging domains.

EnTrust addresses a fundamental problem in medical imaging: when different imaging modalities (CT, MRI, PET, ultrasound) provide conflicting information about pathological regions, current AI models either obscure this disagreement through averaging or estimate uncertainty post-hoc, disconnected from the fusion process. This matters clinically because understanding why a prediction is unreliable directly impacts diagnostic confidence and patient safety.

The framework's innovation lies in its structured decomposition approach. Rather than treating multimodal fusion as a black box, EnTrust explicitly separates shared anatomical consensus from modality-specific cues and conflict signals through enforced statistical independence. This separation feeds into SegDiff, a diffusion-based generative model that naturally captures prediction uncertainty through sampling variability in disagreement regions. The TrustMap module then converts this divergence into interpretable, calibrated pixel-wise uncertainty metrics.

Performance metrics demonstrate substantial clinical value: 40% reduction in calibration error compared to existing methods and single-model performance matching 5x deep ensembles at half the memory cost. This efficiency gain matters for deployment in resource-constrained clinical settings. Testing across brain, cardiac, lesion, and oncology domains validates generalizability beyond single-application scenarios.

The work represents meaningful progress in trustworthy AI for healthcare by making uncertainty estimation interpretable and clinically actionable. Rather than replacing radiologist judgment, this approach enhances it by flagging specifically where and why the model is uncertain, enabling more informed clinical decision-making. The open-source release suggests potential for adoption in clinical research pipelines.

• →EnTrust treats inter-modal conflict as the primary source of uncertainty rather than averaging disagreements away, addressing a core clinical limitation in current medical imaging AI.
• →The framework achieves 40% lower calibration error and matches 5x ensemble performance using a single model with half the memory footprint, improving practical deployability.
• →Structured feature decomposition enforces statistical independence between shared anatomy, modality-specific signals, and conflict indicators, enabling interpretable uncertainty estimation.
• →State-of-the-art segmentation accuracy across four medical domains (brain, cardiac, lesion, oncology) demonstrates broad applicability beyond single-indication use cases.
• →Open-source code availability positions EnTrust for adoption in clinical research and potentially influences how medical AI handles multimodal uncertainty going forward.