HO-SFL: Hybrid-Order Split Federated Learning with Backprop-Free Clients and Dimension-Free Aggregation

Researchers propose HO-SFL (Hybrid-Order Split Federated Learning), a new framework that enables memory-efficient fine-tuning of large AI models on edge devices by eliminating backpropagation on client devices while maintaining convergence speed comparable to traditional methods. The approach significantly reduces communication costs and memory requirements for distributed AI training.

• →HO-SFL eliminates memory-intensive backpropagation on client devices while servers perform precise first-order updates
• →The framework achieves convergence rates comparable to traditional first-order methods despite using zeroth-order optimization on clients
• →Communication costs are drastically reduced through dimension-free model aggregation
• →Theoretical analysis demonstrates the approach mitigates dimension-dependent convergence slowdown of zeroth-order optimization
• →Extensive experiments across vision and language tasks validate the framework's effectiveness