AINeutralarXiv – CS AI · 14h ago6/10
🧠
A Minimal Model of Representation Collapse: Frustration, Stop-Gradient, and Dynamics
Researchers present a minimal mathematical model demonstrating how representation collapse occurs in self-supervised learning when frustrated (misclassified) samples exist, and show that stop-gradient techniques prevent this failure mode. The work provides closed-form analysis of gradient-flow dynamics and fixed points, offering theoretical insights into why modern embedding-based learning systems sometimes lose discriminative power.