y0news
AnalyticsDigestsSourcesTopicsRSSAICrypto

#gradient-dynamics News & Analysis

2 articles tagged with #gradient-dynamics. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

2 articles
AINeutralarXiv – CS AI · Apr 77/10
🧠

Grokking as Dimensional Phase Transition in Neural Networks

Researchers identify neural network 'grokking' as a dimensional phase transition where effective dimensionality shifts from sub-diffusive to super-diffusive during the memorization-to-generalization transition. The study reveals this transition reflects gradient field geometry rather than network architecture, offering new insights into overparameterized network trainability.

$AVAX
AINeutralarXiv – CS AI · Apr 146/10
🧠

A Minimal Model of Representation Collapse: Frustration, Stop-Gradient, and Dynamics

Researchers present a minimal mathematical model demonstrating how representation collapse occurs in self-supervised learning when frustrated (misclassified) samples exist, and show that stop-gradient techniques prevent this failure mode. The work provides closed-form analysis of gradient-flow dynamics and fixed points, offering theoretical insights into why modern embedding-based learning systems sometimes lose discriminative power.