SpurAudio: A Benchmark for Studying Shortcut Learning in Few-Shot Audio Classification

Researchers introduce SpurAudio, a new benchmark for evaluating few-shot audio classification that reveals how state-of-the-art models exploit spurious correlations between foreground content and background noise. The study demonstrates that even large pretrained audio foundation models suffer significant performance degradation when background contexts shift, exposing a critical vulnerability in current evaluation methodologies that has been largely overlooked in audio research.

The SpurAudio benchmark addresses a fundamental blind spot in few-shot audio classification research. While computer vision researchers have extensively studied shortcut learning—where models exploit spurious correlations rather than learning genuine concepts—audio classification has remained largely unexamined on this dimension. This oversight matters because real-world audio rarely exists in isolation; speech recognition systems encounter varying acoustic environments, and sound event detection models face unpredictable background noise. The benchmark leverages audio's natural separability between foreground events and background environments, enabling controlled evaluation of how models perform when contextual cues shift between training and testing scenarios.

The findings present a sobering assessment of current methods. Even large pretrained foundation models, which typically demonstrate strong generalization across tasks, show marked vulnerability to background distribution shifts. This reveals that model capacity alone does not solve the shortcut learning problem—the issue runs deeper into how representations are learned and how classifiers make decisions at inference time. The research demonstrates that methods appearing equivalent under standard benchmarks exhibit vastly different sensitivities to spurious correlations, suggesting that current evaluation protocols mask important algorithmic differences.

For the audio AI community, these results highlight the urgent need for more rigorous evaluation frameworks. Developers deploying audio models in production environments cannot assume robustness to contextual variations that naturally occur in real data. The benchmark provides researchers with a tool to identify which architectural choices, training procedures, and classifier designs offer genuine resilience versus those merely exploiting convenient spurious patterns. This work establishes a foundation for developing more reliable audio classification systems.

• →State-of-the-art few-shot audio models suffer severe performance drops when background contexts shift despite achieving high accuracy under standard evaluation
• →Large pretrained audio foundation models remain vulnerable to spurious correlations, indicating capacity alone cannot solve shortcut learning
• →Methods appearing equivalent under conventional benchmarks show markedly different sensitivities to background distribution shifts
• →SpurAudio benchmark enables controlled multi-level evaluation of contextual shifts in foreground-background audio separation
• →Current audio classification evaluation protocols fail to probe context dependence, masking critical algorithmic vulnerabilities