AINeutralarXiv – CS AI · 7h ago6/10
🧠
Differentially Private Preference Data Synthesis for Large Language Model Alignment
Researchers introduce DPPrefSyn, an algorithm for generating differentially private synthetic preference data to train large language models while protecting user privacy. The method combines the Bradley-Terry preference model with DP-PCA to create synthetic training data from private datasets, achieving competitive alignment performance with formal privacy guarantees.