Reasoned Safety Alignment: Ensuring Jailbreak Defense via Answer-Then-Check

Researchers introduce Answer-Then-Check, a novel safety alignment approach for large language models that enables them to evaluate response safety before outputting to users. The method uses a new 80K-sample dataset called Reasoned Safety Alignment (ReSA) and demonstrates improved jailbreak defense while maintaining general reasoning capabilities.

• →Answer-Then-Check method allows LLMs to internally generate responses then evaluate their safety before providing final answers to users.
• →The approach achieves superior safety capabilities while reducing over-refusal rates compared to existing methods.
• →Fine-tuned models maintain performance on standard benchmarks like MMLU, MATH500, and HumanEval.
• →Research shows that even 500 samples can yield performance comparable to the full 80K dataset, suggesting data-efficient safety training is possible.
• →The study demonstrates that inference-time strategies alone are insufficient and safety training is necessary for robust jailbreak defense.