Depth Charge: Jailbreak Large Language Models from Deep Safety Attention Heads

Researchers have developed SAHA (Safety Attention Head Attack), a new jailbreak framework that exploits vulnerabilities in deeper attention layers of open-source large language models. The method improves attack success rates by 14% over existing techniques by targeting insufficiently aligned attention heads rather than surface-level prompts.

• →SAHA targets deeper attention layers in LLMs, revealing vulnerabilities that shallow-level defenses miss.
• →The framework uses Ablation-Impact Ranking to identify the most vulnerable layers for unsafe output generation.
• →Layer-Wise Perturbation enables minimal changes to attention mechanisms while maintaining semantic relevance.
• →SAHA achieves 14% higher attack success rate compared to state-of-the-art baseline methods.
• →Open-source LLMs remain vulnerable to sophisticated attacks even after safety alignment procedures.