AINeutralarXiv – CS AI · Apr 67/10
🧠
On the Geometric Structure of Layer Updates in Deep Language Models
Researchers analyzed the geometric structure of layer updates in deep language models, finding they decompose into a dominant tokenwise component and a geometrically distinct residual. The study shows that while most updates behave like structured reparameterizations, functionally significant computation occurs in the residual component.