SafeDPO: A Simple Approach to Direct Preference Optimization with Enhanced Safety

Researchers have developed SafeDPO, a simplified approach to training large language models that balances helpfulness and safety without requiring complex multi-stage systems. The method uses only preference data and safety indicators, achieving competitive safety-helpfulness trade-offs while eliminating the need for reward models and online sampling.