AINeutralarXiv – CS AI · 10h ago7/10
🧠
All Routes Lead to Collapse
Researchers demonstrate that attention sinks, representation collapse, and norm stratification—previously thought to be transformer-specific problems—are universal behaviors of content-based routing systems with mismatched metrics. The study reveals this collapse pattern occurs across diverse architectures including softmax attention, graph attention, state-space models, and recurrent mixers, suggesting the issue stems from fundamental routing mechanics rather than transformer design.