AINeutralarXiv – CS AI · 18h ago6/10
🧠
Phantom transitions in language model fine-tuning
Researchers discovered that language models fail silently when fine-tuned on contexts with near-synonym competitors, exhibiting apparent phase transitions that are actually artifacts of the softmax readout rather than genuine geometric changes. The study identifies two failure modes and demonstrates that apparent discontinuities persist even under LoRA fine-tuning where embedding matrices remain frozen, revealing the phenomenon occurs entirely in the output layer.