CADRE: Stable, Parameter Efficient Adaptation of Medical Vision Language Models with Bounded Forgetting and Prior Drift

Researchers present CADRE, a parameter-efficient adaptation framework for medical vision-language models that addresses catastrophic forgetting and model drift when updating deployed systems. By combining low-rank adaptation with elastic weight consolidation and prior-anchoring penalties, CADRE reduces forgetting sevenfold while training only 0.23% of parameters, demonstrating improved stability across different medical imaging modalities.

CADRE tackles a critical safety challenge in deploying adaptable medical AI systems. When clinical services need to update vision-language models for new imaging modalities, naive fine-tuning creates two patient-safety hazards: the model forgets previously mastered modalities (catastrophic forgetting), and it drifts from its trustworthy pretrained behavior toward shortcuts specific to new data. This represents a fundamental tension between model improvement and clinical safety that standard accuracy-focused benchmarks miss entirely.

The research addresses this through a frozen-backbone architecture that combines low-rank adaptation with mathematical constraints. The elastic weight consolidation term bounds competence loss on previously learned tasks, while an anchor-to-prior penalty prevents embedding drift from the original pretrained model. Crucially, the framework includes scale-invariance properties that eliminate fragility sources found in vanilla elastic weight consolidation, making the approach more robust across different training orders and initialization scales.

The experimental validation uses breast cancer detection across three maximally dissimilar modalities—histopathology, ultrasound, and chest radiography—creating a stress test more realistic than standard domain-adaptation benchmarks. Results show CADRE achieves the lowest forgetting rates while maintaining highest backward transfer, meaning previously learned capabilities actually improve slightly rather than degrade during adaptation.

This work matters because it reframes medical AI safety around stability properties rather than leaderboard rankings. Parameter efficiency enables practical deployment in resource-constrained clinical environments. However, the authors appropriately scope claims, noting that robustness against distribution shift and adversarial attacks remains out of scope, preventing false security claims in high-stakes medical settings.

• →CADRE reduces catastrophic forgetting sevenfold compared to baseline methods while training only 0.23% of model parameters
• →The framework combines low-rank adaptation with elastic weight consolidation and prior-anchoring penalties for clinical safety
• →Scale-invariance properties eliminate fragility sources in standard elastic weight consolidation across different training orders
• →Medical AI safety requires stability metrics beyond accuracy, preventing silent failures that harm patients across modalities
• →Positive backward transfer achieved where all baselines show negative transfer, indicating improved retention of learned capabilities