AIBullisharXiv – CS AI · 6h ago7/10
🧠
Stabilizing LLM Supervised Fine-Tuning via Explicit Distributional Control
Researchers propose Anchored Learning, a new fine-tuning method that prevents catastrophic forgetting in large language models by controlling distributional drift through a dynamically evolving reference anchor. The technique achieves near-optimal performance gains while reducing degradation from over 53% to under 5% on benchmark tasks.