DOG-DPO:Dynamic Optimization in Geometry for Safety Alignment

Researchers introduce DOG-DPO, a training-free data selection framework that optimizes safety alignment for large language models by treating preference pairs as geometric signals. The method achieves comparable safety performance using only 11% of preference data, significantly reducing computational costs and redundancy in alignment datasets.

The development of DOG-DPO addresses a fundamental inefficiency in how AI safety researchers prepare data for large language model alignment. Current approaches to preference-based training rely on massive datasets that contain substantial redundancy—many preference pairs encode similar safety concepts without adding unique information. This research identifies that traditional data selection methods fail to capture the directional structure of safety preferences, instead reducing rich multi-dimensional information into single scalar scores.

The geometric approach underlying DOG-DPO represents a conceptual shift in thinking about alignment data. By mapping preference pairs into representation space and decomposing them into global safety directions and dataset-specific residuals, the framework enables more intelligent subset selection. This mathematical perspective reveals why scaling preference data alone proves inefficient; duplicate directions consume training resources without improving safety boundaries.

For the AI development ecosystem, this work has significant practical implications. The ability to achieve 89% data reduction while maintaining safety performance directly translates to lower computational requirements, faster iteration cycles, and reduced environmental impact for safety-critical AI development. The method's teacher-free and training-free properties make it immediately applicable to existing workflows without requiring new infrastructure or additional model training.

Looking forward, this geometric data selection approach could extend beyond safety alignment to other preference-learning domains where efficiency and coverage matter. The framework's success across multiple benchmarks and model architectures suggests the principles generalize, potentially influencing how researchers approach data efficiency across machine learning broadly.

• →DOG-DPO reduces required preference data from 100% to 11% while maintaining safety alignment performance
• →The framework treats preference pairs as geometric structures rather than independent samples, enabling smarter selection
• →Method operates training-free and teacher-free, integrating into existing alignment pipelines without additional overhead
• →Approach decomposes multi-dataset preferences into shared safety directions and dataset-specific risks for targeted coverage
• →Results demonstrate significant computational efficiency gains applicable to large-scale AI safety research