#speech-models News & Analysis

7 articles tagged with #speech-models. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

7 articles

AINeutralarXiv – CS AI · May 287/10

🧠

KVoiceBench, KOpenAudioBench, and KMMAU: Agent-Driven Korean Speech Benchmarks for Evaluating SpeechLMs

Researchers introduce three new Korean speech benchmarks (KVoiceBench, KOpenAudioBench, and KMMAU) totaling 12,345 samples to evaluate multilingual speech language models, addressing the gap in non-English evaluation. The study reveals significant performance disparities between English and Korean across eight SpeechLMs, exposing weaknesses invisible to English-only testing.

AINeutralarXiv – CS AI · Jun 256/10

🧠

SpeechEQ: Benchmarking Emotional Intelligence Quotient in Socially Aware Voice Conversational Models

Researchers introduce SpeechEQ, a benchmarking framework that evaluates how well voice-based AI models understand emotional intelligence through multi-turn dialogue. The dataset of 2,265 dialogues reveals that current speech-language models fail to fully process paralinguistic cues, relying instead on text shortcuts and exhibiting contextual memory gaps.

🏢 Hugging Face

AINeutralarXiv – CS AI · Jun 196/10

🧠

IHBench: Evaluating Post-Interruption Recovery in Voice Agents with Structured Workflows

Researchers introduce IHBench, a benchmark for evaluating how voice agents recover from user interruptions while executing multi-step workflows in enterprise settings. Testing 27 model configurations reveals closed-weight models (OpenAI, Google) significantly outperform open-weight alternatives in handling interruptions, recovering 3.3x more gracefully and maintaining task completion rates.

🏢 OpenAI

AINeutralarXiv – CS AI · Jun 116/10

🧠

Fast Speech Foundation Model Distillation Using Interleaved Stacking

Researchers propose interleaved stacking, a novel training method for distilling large speech foundation models into efficient student models while accelerating training speed. The technique maintains consistent layer positions during progressive depth expansion, addressing performance degradation issues in existing stacking approaches and demonstrating effectiveness on the SUPERB benchmark.

AINeutralarXiv – CS AI · Jun 16/10

🧠

Performance and Complexity Trade-off Optimization of Speech Models During Training

Researchers propose a novel reparameterization technique using feature noise injection that enables joint optimization of speech model performance and computational complexity during training via gradient descent. Unlike post-hoc methods like pruning or quantization, this approach dynamically optimizes model size without heuristic weight-selection criteria, demonstrated through voice activity detection and audio anti-spoofing applications.

AINeutralarXiv – CS AI · Apr 146/10

🧠

Efficient Training for Cross-lingual Speech Language Models

Researchers introduce Cross-lingual Speech Language Models (CSLM), an efficient training method for building multilingual speech AI systems using discrete speech tokens. The approach achieves cross-modal and cross-lingual alignment through continual pre-training and instruction fine-tuning, enabling effective speech LLMs without requiring massive datasets.

AINeutralarXiv – CS AI · Apr 106/10

🧠

In-Context Learning in Speech Language Models: Analyzing the Role of Acoustic Features, Linguistic Structure, and Induction Heads

Researchers investigate in-context learning (ICL) in speech language models, revealing that speaking rate significantly affects model performance and acoustic mimicry, while induction heads play a causal role identical to text-based ICL. The study bridges the gap between text and speech domains by analyzing how models learn from demonstrations in text-to-speech tasks.