Safety Anchor: Defending Harmful Fine-tuning via Geometric Bottlenecks

Researchers propose Safety Bottleneck Regularization (SBR), a defense mechanism against harmful fine-tuning attacks on large language models. The approach anchors a model's unsafe responses to safe outputs via the unembedding layer, reducing harmful capabilities while maintaining performance on legitimate tasks.

Large language models face persistent security challenges despite safety alignment efforts. Current defenses constraining parameters, gradients, or internal representations have proven circumventable because attackers exploit the redundancy inherent in high-dimensional parameter spaces, finding optimization paths orthogonal to existing safeguards. This research identifies a critical vulnerability in defensive architecture: the assumption that constraining one dimension of a massive model sufficiently prevents adversarial behavior. The proposed Safety Bottleneck Regularization shifts defensive strategy from the parameter space to the unembedding layer, treating it as a geometric chokepoint through which all model outputs must flow. By anchoring harmful query responses to safety-aligned outputs at this bottleneck, the defense operates at a structural level rather than attempting to constrain individual parameters. Empirically, SBR achieves harmful score reductions below 10 using a single safety anchor while preserving model utility. This approach matters significantly for AI deployment in production systems where fine-tuning attacks represent realistic threats from both malicious actors and well-resourced competitors seeking to compromise model safety. The defense mechanism has implications for organizations deploying LLMs in sensitive domains. As models become more accessible and fine-tuning techniques mature, architectural defenses become increasingly valuable. The bottleneck approach suggests future safety mechanisms may benefit from identifying natural structural constraints in model architecture rather than attempting to police vast parameter spaces. Continued research should examine whether multiple bottlenecks strengthen defenses and how adaptive attackers might target this vulnerability.

• →Current parameter-space defenses fail against persistent harmful fine-tuning because high-dimensional redundancy enables attackers to exploit orthogonal optimization paths
• →Safety Bottleneck Regularization anchors harmful queries to safe outputs at the unembedding layer, creating a geometric constraint difficult to circumvent
• →The approach reduces harmful scores below 10 using single safety anchors while maintaining competitive performance on benign tasks
• →Architectural defenses targeting natural bottlenecks may prove more robust than parameter-level constraints as fine-tuning attacks become more sophisticated
• →This work highlights the importance of structural design in AI safety beyond algorithmic constraints