Silent Sabotage During Fine-Tuning: Few-Shot Rationale Poisoning of Compact Medical LLMs

Researchers discovered a new stealth poisoning attack method targeting medical AI language models during fine-tuning that degrades performance on specific medical topics without detection. The attack injects poisoned rationales into training data, proving more effective than traditional backdoor attacks or catastrophic forgetting methods.

• →A novel poisoning attack targets medical LLM reasoning processes during supervised fine-tuning, unlike detectable backdoor attacks.
• →The attack injects poisoned rationales into few-shot training data, causing stealthy performance degradation on targeted medical topics.
• →Knowledge overwriting proved ineffective while rationale poisoning significantly reduced accuracy on target subjects.
• →The attack requires a minimum number of poisoned samples and works only when no correct samples of the target subject exist in the dataset.
• →This research highlights critical security vulnerabilities in medical AI systems during the training phase.