NCSAM Noise-Compensated Sharpness-Aware Minimization for Noisy Label Learning

Researchers propose NCSAM, a novel optimization-based approach to learning from noisy labels that theoretically connects label noise to Sharpness-Aware Minimization's behavior. The method uses noise-compensated perturbations to reduce memorization of corrupted annotations while maintaining optimization simplicity, demonstrating competitive performance against existing noisy-label learning methods.

This research addresses a persistent problem in machine learning: training on datasets with corrupted or mislabeled annotations. Rather than adopting the conventional approaches of correcting labels or selecting clean samples, the authors take an optimization-focused perspective. They identify how label noise biases the perturbation mechanism in Sharpness-Aware Minimization, a popular technique for finding flatter loss landscapes that typically generalize better. By theoretically establishing this connection, they develop NCSAM to compensate for noise-induced distortions in SAM's perturbations.

The significance extends beyond academic interest. Real-world datasets inevitably contain annotation errors—from crowdsourced labeling, OCR mistakes, or genuine ambiguity. Existing solutions often require explicit noise modeling or sample filtering, adding computational overhead and complexity. NCSAM's optimization-centric approach maintains the simplicity of SAM while directly addressing noise effects, making it practical for practitioners. The method's ability to reduce memorization of noisy labels during training directly improves model robustness without requiring architectural changes or multi-stage training pipelines.

For the broader machine learning community, this represents incremental but meaningful progress in a challenging domain. The theoretical analysis connecting label noise to SAM behavior provides new insights for optimization research. Practitioners working with imperfect datasets gain a tool that integrates seamlessly with existing SAM-based workflows. The experimental validation on both synthetic and real-world benchmarks suggests the approach generalizes across different noise types and dataset characteristics. Future work might extend this framework to other optimization methods or combine it with complementary label-correction techniques for even stronger robustness.

• →NCSAM theoretically connects label noise to Sharpness-Aware Minimization's optimization behavior, enabling noise-compensated perturbations.
• →The method reduces memorization of corrupted labels while maintaining the simplicity of optimization-based learning approaches.
• →Experimental results show NCSAM consistently outperforms SAM baselines and remains competitive with dedicated noisy-label methods.
• →The optimization perspective avoids explicit label correction overhead, making it practical for real-world datasets with inherent annotation errors.
• →The approach integrates seamlessly into existing SAM-based training workflows without architectural modifications.