AIBullisharXiv – CS AI · 6h ago6/10
🧠
PACZero: PAC-Private Fine-Tuning of Language Models via Sign Quantization
PACZero introduces a novel PAC-private fine-tuning mechanism for large language models that achieves usable utility while maintaining zero mutual information leakage, surpassing traditional differential privacy approaches. Using sign quantization of zeroth-order gradients, the method exploits moments of unanimous agreement across candidate subsets to eliminate privacy costs, demonstrating competitive performance on benchmark tasks like SST-2 and SQuAD.