AINeutralarXiv – CS AI · 7h ago6/10
🧠
PermDoRA -- Understanding Adapter Interference in Language Models: Limits of Parameter-Space Geometry
Researchers challenge the conventional wisdom that adapter interference in language models stems from parameter-space geometry by testing whether orthogonal or directionally independent updates reduce cross-domain interference. Their findings using DoRA-RBAC on multiple LLMs show geometry-aware merging provides no consistent advantage, suggesting interference mechanisms operate in shared nonlinear representations rather than linear parameter space.