#zeroth-order-optimization News & Analysis

3 articles tagged with #zeroth-order-optimization. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

3 articles

AINeutralarXiv – CS AI · May 296/10

🧠

Turning Stale Gradients into Stable Gradients: Coherent Coordinate Descent with Implicit Landscape Smoothing for Lightweight Zeroth-Order Optimization

Researchers propose Coherent Coordinate Descent (CoCD), a deterministic zeroth-order optimization method that improves sample efficiency for scenarios where backpropagation is unavailable. The approach reframes stale gradients as computational assets and demonstrates that larger finite-difference step sizes create implicit landscape smoothing, achieving superior convergence stability compared to existing randomized methods across neural network architectures.

AINeutralarXiv – CS AI · May 116/10

🧠

Ensemble Distributionally Robust Bayesian Optimisation

Researchers propose a novel Ensemble Distributionally Robust Bayesian Optimisation algorithm that addresses context distributional uncertainty in zeroth-order optimization. The method achieves sublinear regret bounds while remaining computationally tractable, improving upon existing state-of-the-art approaches.

AIBullisharXiv – CS AI · May 96/10

🧠

PACZero: PAC-Private Fine-Tuning of Language Models via Sign Quantization

PACZero introduces a novel PAC-private fine-tuning mechanism for large language models that achieves usable utility while maintaining zero mutual information leakage, surpassing traditional differential privacy approaches. Using sign quantization of zeroth-order gradients, the method exploits moments of unanimous agreement across candidate subsets to eliminate privacy costs, demonstrating competitive performance on benchmark tasks like SST-2 and SQuAD.