Changing the Training Data Distribution to Reduce Simplicity Bias Improves In-distribution Generalization

Researchers developed USEFUL, a new training method that modifies data distribution to reduce simplicity bias in machine learning models. The approach clusters examples early in training and upsamples underrepresented data, achieving state-of-the-art performance when combined with optimization methods like SAM on popular image classification datasets.