New Smooth Loss functions for Robust Regression that Closely Approximate Absolute Error and Provide Improved Performance on Datasets With Significant Outliers

Researchers introduce two new differentiable loss functions—Square Root Loss (SRL) and Smooth Mean Absolute Error (SMAE)—that better approximate Mean Absolute Error while improving robustness in regression tasks with outlier-heavy datasets. These functions address limitations of existing approaches like MSE and MAE by providing superior mathematical properties and training stability.

This research addresses a fundamental challenge in machine learning: training robust models when datasets contain significant outliers. Traditional loss functions present a trade-off between mathematical convenience and practical performance. Mean Squared Error (MSE) dominates training optimization due to its differentiability but overweights large errors, making it vulnerable to outlier corruption. Mean Absolute Error (MAE) handles outliers more gracefully but lacks differentiability at zero, causing gradient-based optimization challenges and training instability.

The paper builds on prior attempts to bridge this gap through functions like Huber and Log-Cosh losses, which smooth MAE's behavior. However, the authors demonstrate that their proposed SRL and SMAE functions achieve closer approximations to MAE while maintaining infinite differentiability and desirable convexity properties. SRL is proven strictly convex, ensuring unique global minima, while SMAE is strictly quasi-convex, providing similar optimization guarantees.

For practitioners, these functions have immediate applications across industries reliant on regression models—from financial forecasting to sensor data processing to medical diagnostics. Improved robustness to outliers reduces preprocessing overhead and model retraining costs. The introduction of robust linear regression variants extends benefits to foundational statistical models.

The research demonstrates empirical superiority across multiple benchmarks, suggesting broader adoption potential. Key challenges ahead involve computational efficiency comparisons, integration into popular machine learning frameworks, and validation across diverse domain-specific datasets. Practitioners should monitor academic implementations and framework updates for accessibility.

• →Two new infinitely differentiable loss functions (SRL and SMAE) provide better robustness to outliers than existing alternatives like Huber and Log-Cosh losses
• →SRL is proven strictly convex and SMAE is strictly quasi-convex, offering mathematical guarantees for optimization stability
• →Empirical testing across multiple benchmarks demonstrates superior performance on datasets with significant outliers compared to traditional loss functions
• →New robust linear regression models are introduced, extending improved outlier handling to foundational statistical techniques
• →The advancement reduces the MSE-MAE trade-off, enabling better model training without sacrificing mathematical tractability or convergence properties