Attention Consistent Longitudinal Medical Visual Question Answering Guided by Vision Foundation Models

Researchers propose a novel attention-guided encoder-decoder architecture for longitudinal medical visual question answering using chest X-rays, incorporating affine registration and vision foundation models (DINO) to identify anatomical changes over time. The approach combines saliency masking with multimodal transformer decoding and auxiliary learning objectives, achieving strong benchmark performance while providing interpretable visual explanations for clinical reasoning.

This research addresses a specialized challenge in medical AI: comparing medical images across time points to identify meaningful anatomical changes. The paper demonstrates how vision foundation models, originally developed for general computer vision tasks, can be adapted for clinical applications requiring temporal reasoning. By integrating lightweight affine registration to reduce motion artifacts before attention masking, the authors solve a practical problem that has hindered longitudinal medical image analysis.

The technical innovation lies in combining multiple learning objectives simultaneously—supervised VQA training alongside unsupervised representation learning objectives borrowed from DINO-v3, including mask rebuilding and Gram-style consistency losses. This multi-objective approach appears to stabilize training and enhance the model's ability to isolate clinically relevant changes from visual noise. The frozen DINO backbone paired with adaptive masking creates an interpretability mechanism absent in black-box medical AI systems.

For the medical AI industry, this work validates a broader paradigm: vision foundation models pretrained on general images can be effectively transferred to medical domains through targeted architectural modifications and sophisticated loss functions. The strong benchmark performance on Medical-Diff-VQA suggests clinical deployment potential for automated longitudinal monitoring in radiology workflows. The intrinsic interpretability through saliency masks addresses regulatory and safety concerns around AI decision-making in healthcare.

The framework's emphasis on simultaneous optimization of supervised and unsupervised objectives may influence how researchers approach other medical vision tasks. Future applications could extend to CT scans, MRI comparisons, and multi-organ assessment, potentially reducing radiologist workload for routine longitudinal reviews while maintaining clinical transparency.

• →Lightweight affine registration combined with attention masking effectively isolates anatomical changes in longitudinal chest X-ray analysis
• →Vision foundation models like DINO can be adapted for medical VQA through frozen backbone plus trainable adaptive components
• →Multi-objective learning combining supervised VQA loss with unsupervised consistency losses improves model stability and change detection
• →The approach provides interpretable saliency masks showing which image regions influenced the model's clinical reasoning
• →Strong benchmark results suggest practical deployment potential for automated longitudinal medical image assessment in clinical workflows