AINeutralarXiv – CS AI · 14h ago6/10
🧠
How LoRA Remembers? A Parametric Memory Law for LLM Finetuning
Researchers introduce the Parametric Memory Law, a power law framework quantifying how Large Language Models store information through Low-Rank Adaptation (LoRA) finetuning. The study reveals a deterministic phase transition at the token level and proposes MemFT, an optimization strategy that improves memory fidelity by dynamically redistributing training resources toward undertrained tokens.