AINeutralarXiv – CS AI · 7h ago6/10
🧠
What Makes a Strong Model? A Unified Spectral Analysis of Knowledge Transfer over High-dimensional Linear Regression
Researchers present a unified theoretical framework analyzing knowledge transfer (KT) in machine learning through spectral analysis of SGD dynamics. The study reveals two distinct mechanisms—Spectral Horizon Expansion in knowledge distillation and Spectral Denoising in weak-to-strong generalization—explaining how knowledge transfer efficiency is governed by implicit regularization and heterogeneous spectral learning speeds.