Risk Awareness Injection: Calibrating Vision-Language Models for Safety without Compromising Utility

Researchers propose Risk Awareness Injection (RAI), a lightweight, training-free framework that enhances vision-language models' ability to recognize unsafe content by amplifying risk signals in their feature space. The method maintains model utility while significantly reducing vulnerability to multimodal jailbreak attacks, addressing a critical security gap in VLMs.

Vision-language models represent a significant advancement in AI capabilities, extending LLM reasoning to image and video inputs. However, this multimodal expansion has introduced new security vulnerabilities—attackers can exploit visual inputs to bypass safety mechanisms that remain intact for text-only interactions. The core problem stems from a fundamental asymmetry: underlying language models retain inherent safety recognition capabilities, but the addition of visual processing dilutes these risk signals, making models susceptible to jailbreak attacks that wouldn't work in text-only settings.

The Risk Awareness Injection framework addresses this vulnerability through an elegant solution that doesn't require expensive retraining. Rather than fine-tuning entire models or aggressively manipulating tokens (approaches that degrade performance), RAI works by constructing an unsafe prototype subspace from language embeddings and selectively modulating high-risk visual tokens. This targeted approach reactivates the safety-critical signals that vision inputs previously suppressed, essentially restoring the model's native ability to recognize dangerous content while preserving legitimate semantic reasoning.

For developers deploying VLMs in production environments, this research carries substantial implications. The training-free nature of RAI makes it immediately implementable without infrastructure investment or performance degradation—a critical advantage over existing defensive measures. The experimental validation across multiple jailbreak and utility benchmarks suggests the method scales effectively. As VLMs become increasingly prevalent in real-world applications from content moderation to autonomous systems, the ability to defend against multimodal attacks without sacrificing model utility becomes essential for maintaining both security and user experience.

• →Risk Awareness Injection is a training-free defense mechanism that restores safety recognition in vision-language models without requiring model retraining or performance compromise.
• →The framework addresses the core vulnerability where visual inputs dilute safety signals inherent in underlying language models.
• →RAI achieves safety improvements by amplifying risk-related tokens in the cross-modal feature space while preserving semantic integrity.
• →Experimental results demonstrate substantial reductions in jailbreak attack success rates across multiple benchmarks without degrading task performance.
• →The lightweight, deployable nature of RAI makes it practically implementable for existing VLM systems in production environments.