#zeroth-order News & Analysis

2 articles tagged with #zeroth-order. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

2 articles

AINeutralarXiv – CS AI · May 297/10

🧠

Aligned but Fragile: Enhancing LLM Safety Robustness via Zeroth-Order Optimization

Researchers propose a novel framework using zeroth-order optimization to enhance the robustness of safety alignment in large language models against perturbations like parameter noise and quantization. The hybrid approach combines standard first-order safety alignment with zeroth-order refinement steps, demonstrating that weak safety mechanisms can be significantly strengthened while maintaining model utility with minimal computational overhead.

AIBullisharXiv – CS AI · May 47/10

🧠

AdaMeZO: Adam-style Zeroth-Order Optimizer for LLM Fine-tuning Without Maintaining the Moments

Researchers introduce AdaMeZO, a new zeroth-order optimizer that combines the memory efficiency of MeZO with Adam-style moment estimation for fine-tuning large language models. The method achieves faster convergence than MeZO while reducing GPU memory requirements and requiring up to 70% fewer forward passes.