FedNSAM:Consistency of Local and Global Flatness for Federated Learning

Researchers propose FedNSAM, a new federated learning algorithm that improves global model performance by addressing the inconsistency between local and global flatness in distributed training environments. The algorithm uses global Nesterov momentum to harmonize local and global optimization, showing superior performance compared to existing FedSAM approaches.

• →FedNSAM addresses the problem where local flatness optimization doesn't guarantee global model flatness in federated learning.
• →The algorithm introduces global Nesterov momentum to align local training with global optimization objectives.
• →Researchers prove tighter convergence bounds compared to existing FedSAM algorithms through Nesterov extrapolation.
• →Comprehensive experiments on CNN and Transformer models demonstrate superior performance and efficiency.
• →The solution tackles data heterogeneity issues that typically lead to sharper global minima in federated learning.