LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shifts

Researchers have identified a new vulnerability in large language models called 'natural distribution shifts' where seemingly benign prompts can bypass safety mechanisms to reveal harmful content. They developed ActorBreaker, a novel attack method that uses multi-turn prompts to gradually expose unsafe content, and proposed expanding safety training to address this vulnerability.

• →Large language models are vulnerable to 'natural distribution shifts' where benign prompts semantically related to harmful content can bypass safety mechanisms.
• →ActorBreaker, a new attack method based on actor-network theory, outperforms existing approaches in diversity, effectiveness, and efficiency across aligned LLMs.
• →The vulnerability stems from LLMs' exposure to potentially harmful data during pre-training and their susceptibility to gradual prompt escalation.
• →Researchers propose expanding safety training to cover broader semantic spaces of toxic content as a mitigation strategy.
• →Fine-tuning models on the proposed multi-turn safety dataset improves robustness but comes with some trade-offs in utility.