LAYUP: Asynchronous decentralized gradient descent with LAYer-wise UPdates
Researchers present LayUp, an asynchronous decentralized gradient descent algorithm that enables faster distributed training of deep learning models through layer-wise updates and gossip-based communication. The method demonstrates 32% faster convergence than synchronous training while maintaining robustness to stragglers and requiring no extra buffering.