Efficient Multimodal Clinical Question Answering for Pulmonary Embolism Risk Assessment

Researchers have developed a benchmark for evaluating efficient multimodal language models on pulmonary embolism diagnosis and risk assessment using a dataset of 23,248 CTPA studies. The study demonstrates that compact models like Gemma4 perform significantly better when combining imaging and electronic health record data, with diagnostic tasks outperforming prognostic predictions.

This research addresses a critical gap in clinical AI by evaluating how compact multimodal models perform on real-world medical tasks that require integration of imaging and longitudinal patient data. Pulmonary embolism represents an ideal test case—the condition demands rapid diagnosis combining radiological imaging with patient history, mirroring workflows in thousands of hospitals. The creation of INSPECT, a substantial multimodal dataset with 23,248 studies, provides the medical AI community with a valuable benchmark for measuring progress.

The findings reveal important patterns about model capabilities. Gemma4's improved performance with combined CTPA and EHR data suggests that efficient models benefit substantially from multimodal fusion, contradicting assumptions that only large-scale models excel at complex reasoning. The gap between diagnostic and prognostic task performance is particularly instructive—models struggle more with prediction tasks than with pattern recognition in imaging, indicating that temporal clinical reasoning remains a frontier challenge.

For healthcare institutions, these results validate investment in efficient multimodal models that can run on-premise with reasonable computational requirements. The zero-shot and few-shot evaluation protocols mean hospitals could implement such systems with minimal retraining on proprietary data. However, the performance ceiling on readmission prediction suggests current approaches cannot yet replace clinical judgment for complex prognostic decisions.

The research trajectory indicates that practical clinical AI increasingly emphasizes compact, efficient models over scale. Future work should investigate why prognostic tasks underperform relative to diagnostics and whether architectural improvements can better capture temporal patient trajectories from EHR data.

• →Efficient multimodal models like Gemma4 significantly improve clinical performance when combining imaging and EHR data versus single-modality inputs.
• →The INSPECT dataset of 23,248 pulmonary embolism CTPA studies provides a new benchmark for evaluating multimodal medical AI systems.
• →Diagnostic tasks achieved higher performance than prognostic tasks, particularly for readmission prediction, highlighting remaining challenges in temporal clinical reasoning.
• →Compact models demonstrate practical viability for hospital deployment without requiring massive computational infrastructure.
• →Zero-shot and few-shot prompting approaches enable rapid clinical implementation without extensive model retraining on institutional data.