AINeutralarXiv – CS AI · 10h ago6/10
🧠
Subspace-Constrained Federated Learning with Low-Rank Adaptation
Researchers propose a subspace-regularized federated learning approach for low-rank adaptation (LoRA) that addresses geometric misalignment issues when training large language models across distributed clients with heterogeneous data. The method achieves superior performance on RoBERTa-large while demonstrating near-perfect basis overlap (0.9999) across multiple models and random seeds, outperforming existing federated learning baselines.