When Does Multimodal Learning Help in Healthcare? A Benchmark on EHR and Chest X-Ray Fusion

Researchers conducted a systematic benchmark study on multimodal fusion between Electronic Health Records (EHR) and chest X-rays for clinical decision support, revealing when and how combining data modalities improves healthcare AI performance. The study found that multimodal fusion helps when data is complete but benefits degrade under realistic missing data scenarios, and released an open-source benchmarking toolkit for reproducible evaluation.

• →Multimodal fusion between EHR and chest X-rays improves clinical prediction performance when both data modalities are complete.
• →Benefits concentrate in diseases requiring complementary information from both structured health records and medical imaging.
• →Multimodal advantages rapidly degrade under realistic scenarios where some data modalities are missing unless models are specifically designed for incomplete inputs.
• →Cross-modal learning mechanisms capture clinically meaningful dependencies beyond simple data concatenation approaches.
• →Multimodal fusion does not inherently improve algorithmic fairness, with disparities arising from unequal sensitivity across demographic groups.