Backdoor Attacks on Speech Emotion Recognition via TTS-Generated Poisoning

Researchers demonstrate the first systematic study of poisoning-based backdoor attacks on Speech Emotion Recognition (SER) systems using text-to-speech generated audio. The study reveals that modern SER models can be reliably compromised with imperceptible acoustic triggers while maintaining normal performance on benign inputs, exposing critical vulnerabilities in AI systems that process voice data.

This research exposes a fundamental security gap in speech emotion recognition systems that are increasingly deployed in healthcare, customer service, and security applications. The attack leverages TTS technology to create poisoned training data with stealthy acoustic triggers—sounds imperceptible to human listeners—that cause models to misclassify emotions. The threat is particularly acute because self-supervised learning models, which dominate modern SER architectures, show heightened susceptibility to these backdoor patterns.

The vulnerability emerges from the intersection of two trends: the growing reliance on self-supervised pre-training for speech tasks and the commoditization of high-quality TTS systems. Previous security research focused on image and text domains, leaving voice systems understudied despite their critical role in authentication, mental health monitoring, and accessibility tools. The paper's demonstration that backdoors transfer across different models amplifies the risk—an attacker needs only to poison shared training datasets or foundational models to compromise multiple downstream systems simultaneously.

For AI developers and organizations deploying SER systems, this research signals urgent mitigation requirements. Industries relying on voice-based emotion detection for clinical assessment or security purposes face potential liability if systems make decisions based on manipulated emotional signals. The low poisoning ratio required for successful attacks—maintaining near-perfect benign accuracy while achieving high attack success—makes detection through standard validation techniques extremely challenging.

Immediate priorities include developing robust defenses against acoustic backdoors, establishing certified dataset provenance tracking, and implementing anomaly detection for out-of-distribution triggers. The research motivates stronger regulatory frameworks for AI training data security and third-party auditing of voice-processing systems before deployment in sensitive applications.

• →SER models can be reliably compromised with imperceptible acoustic triggers while maintaining normal benign performance, creating a severe detection challenge
• →Text-to-speech technology dramatically lowers barriers to effective backdoor attacks by enabling scalable creation of poisoned training data
• →Self-supervised learning representations, dominant in modern voice systems, are particularly vulnerable to learning and retaining backdoor patterns
• →Backdoor patterns exhibit strong cross-model transferability, meaning poisoning shared datasets or foundation models compromises multiple downstream applications
• →Current SER pipelines lack dedicated defenses, exposing critical vulnerabilities in systems used for healthcare, security, and accessibility applications