Beyond Waveform Robustness: Robust Feature-Vocoder Adversarial Attacks on Automatic Speech Recognition

Researchers have developed a new adversarial attack method against automatic speech recognition systems that operates in feature space rather than directly on audio waveforms, achieving significantly higher transfer rates to black-box ASR models and bypassing existing defenses. The attack uses self-supervised learning representations and vocoders to reconstruct adversarial signals, revealing critical vulnerabilities in current ASR robustness evaluation protocols.

This research identifies a fundamental blindspot in how automatic speech recognition systems are evaluated for security. Rather than adding noise directly to audio signals—the conventional adversarial attack approach—the researchers shift the attack vector to intermediate feature representations learned by self-supervised models. This methodological shift is significant because it targets more generalizable acoustic properties rather than system-specific waveform artifacts, enabling attacks trained on one model to transfer effectively to entirely different ASR systems.

The work builds on growing concerns about adversarial robustness in machine learning systems deployed at scale. As ASR technology becomes embedded in critical applications from healthcare to financial services, understanding attack surfaces becomes increasingly important. Previous defenses focused on mitigating additive noise in the waveform domain, but this research demonstrates that defenders face an asymmetric problem—they must protect against perturbations in multiple representation spaces simultaneously.

The practical implications extend across the AI industry. Organizations deploying ASR for sensitive tasks face a two-pronged challenge: existing defenses may provide false confidence, and the attack surface is broader than previously understood. The 26.6 WER improvement over prior baselines indicates substantial degradation in system performance, potentially affecting transcription accuracy in production environments.

Looking forward, this research likely catalyzes renewed focus on defense mechanisms operating at feature-space levels rather than input-only protections. The community will need to develop evaluation protocols that account for attacks operating across multiple representation layers. ASR developers should reassess threat models and consider ensemble defenses combining input-space, feature-space, and decision-level protections.

• →Feature-space adversarial attacks transfer more effectively to black-box ASR systems than waveform-based attacks
• →Current ASR defenses tailored to waveform perturbations are ineffective against feature-vocoder reconstructed adversarial signals
• →SSL representation-based attacks generalize across different ASR models, suggesting fundamental architectural vulnerabilities
• →The attack achieves 36.2 WER improvement over baselines even against trained defenses
• →Current ASR robustness evaluation protocols have significant gaps in coverage