Rethinking Weak Supervision in Anomaly Detection: A Comprehensive Benchmark

Researchers introduce WSADBench, the first unified benchmark for weakly supervised anomaly detection (WSAD) that evaluates 36 algorithms across 4 modalities and over 700K experiments. The study reveals that specialized WSAD methods only outperform in extreme label-scarcity scenarios, while general foundation models and classification approaches dominate with increased supervision, fundamentally challenging current research isolation.

WSADBench addresses a critical fragmentation in anomaly detection research by unifying three previously isolated supervision paradigms: incomplete, inexact, and inaccurate labels. This benchmark matters because anomaly detection underpins critical infrastructure across fraud detection, cybersecurity, and medical diagnostics—domains where perfect labels rarely exist. The research provides empirical evidence that current WSAD specialization may be premature optimization.

The field has evolved organically with researchers pursuing narrow technical improvements within isolated problem settings without assessing whether different weak supervision types share fundamental mechanics. This benchmark reveals strong intrinsic correlations between these scenarios, suggesting researchers have been solving variations of the same core problem rather than distinct challenges. The finding that tabular foundation models outperform specialized algorithms as label availability increases indicates a paradigm shift toward general-purpose models.

The practical implications are significant for practitioners deploying anomaly detection systems. Organizations with extreme data scarcity may benefit from specialized WSAD algorithms, but most real-world deployments will likely achieve better results by applying general foundation models and classification techniques. The inconsistent utility of unlabeled data challenges assumptions underlying semi-supervised approaches, suggesting label quality refinement yields stronger returns than simply acquiring more unlabeled data.

Future research should focus on understanding why foundation models generalize better across weak supervision scenarios and developing hybrid approaches that combine specialized techniques for truly scarce-label regimes with general models for practical deployments. The release of WSADBench as open-source infrastructure accelerates this transition by providing standardized evaluation protocols.

• →Specialized WSAD algorithms only dominate in extreme label-scarcity scenarios but are outperformed by general foundation models with increased supervision
• →Strong correlations exist between incomplete, inexact, and inaccurate supervision types, challenging the premise of treating them as isolated research directions
• →Unlabeled data provides inconsistent utility across settings with marginal gains compared to investing in label quality improvement
• →WSADBench unifies evaluation across weak supervision paradigms through 700K+ experiments across 36 algorithms and 4 modalities
• →Models exhibit asymmetric sensitivity to different label noise types, requiring tailored noise-handling strategies rather than one-size-fits-all approaches