THINKSAFE: Self-Generated Safety Alignment for Reasoning Models

Researchers introduce ThinkSafe, a self-generated safety alignment framework that improves AI reasoning models' resistance to harmful prompts without relying on external teacher models. The approach leverages models' latent safety knowledge through lightweight refusal steering, achieving superior safety outcomes compared to existing methods while preserving reasoning capabilities and reducing computational costs.

ThinkSafe addresses a critical vulnerability in large reasoning models (LRMs) that emerges from reinforcement learning optimization. As these models undergo extensive training to improve reasoning performance, they often become overoptimized for compliance at the expense of safety mechanisms, leaving them susceptible to adversarial or harmful prompts. Traditional safety alignment solutions rely on external teacher distillation, but this introduces distributional mismatches that degrade the native reasoning capabilities researchers worked to develop.

The technical innovation lies in recognizing that safety knowledge remains latent within models even after compliance-focused optimization. Rather than importing external safety standards, ThinkSafe uses lightweight refusal steering to guide models into generating their own safety reasoning traces in a manner consistent with their training distribution. This self-generated alignment approach then fine-tunes the models on these responses, effectively realigning safety without introducing the distributional discrepancy associated with external teacher methods.

For the AI development community, this work carries significant practical implications. Testing on DeepSeek-R1-Distill and Qwen3 demonstrates that ThinkSafe achieves safety improvements comparable to or exceeding GRPO (a more computationally expensive alternative) while maintaining reasoning proficiency. The reduced computational overhead makes safety alignment more accessible to organizations with limited resources, potentially accelerating the deployment of safer AI systems across the industry. The availability of code, models, and datasets enables broader adoption and validation of the approach.

The framework addresses a fundamental challenge in scaling reasoning models: the safety-capability tradeoff. As LRMs become more sophisticated and widely deployed, ensuring they resist misuse becomes increasingly critical. ThinkSafe's efficiency and effectiveness suggest the field is moving toward more elegant solutions that preserve model capabilities while enhancing robustness.

• →ThinkSafe enables safety alignment without external teacher models by leveraging models' latent safety knowledge through refusal steering.
• →The approach achieves superior safety outcomes to existing methods while preserving reasoning capabilities and reducing computational costs.
• →Self-generated safety reasoning traces minimize distributional shift that typically degrades native reasoning performance.
• →Tested successfully on DeepSeek-R1-Distill and Qwen3, with comparable or better results than computationally expensive alternatives.
• →Open-source availability of code, models, and datasets accelerates adoption of efficient safety alignment across the AI development community.