Bridging the Stability-Expressivity Gap: Synthetic Data Scaling and Preference Alignment for Low-Resource Spoken Language Models

Researchers address a critical limitation in Spoken Language Models (SLMs) for low-resource languages by identifying a fundamental trade-off called the Stability-Expressivity Gap, where synthetic data improves phonetic accuracy but suppresses prosodic variability. The proposed self-alignment frameworks—DGSA and TDSC—recover expressivity while maintaining stability, achieving performance comparable to commercial systems and enabling zero-shot voice cloning for Lao.

This research tackles a fundamental challenge in speech synthesis that has significant implications for language technology accessibility. The core problem—that synthetic data necessary for scaling SLMs in low-resource languages inadvertently suppresses natural prosodic variation—reveals an overlooked cost of data augmentation strategies widely used across machine learning. The phenomenon termed 'Synthetic Erosion' demonstrates how quantity gains in training data can paradoxically reduce output quality in dimensions not explicitly optimized during training.

The breakthrough comes through two complementary approaches: Disentanglement-Guided Self-Alignment exploits the separation between prosody and timbre to recover natural expressivity, while Temperature-Driven Self-Critique uses automated exploration and filtering when authentic reference data remains scarce. This dual-framework approach addresses different constraint regimes, making the solution practical across varying resource levels.

The achievement of outperforming established commercial systems like ElevenLabs and Gemini Pro signals maturation in open research for speech synthesis. The specific application to Lao—a language with minimal commercial speech synthesis support—demonstrates how advances in self-alignment techniques democratize high-quality voice technology for underserved language communities. This has direct implications for digital inclusion and economic opportunity in Southeast Asia.

The research suggests future work should examine whether similar stability-expressivity trade-offs exist in other synthetic data scaling scenarios beyond speech, potentially affecting computer vision, music generation, and multimodal systems. The self-critique methodology may offer broader applications across domains where synthetic data dominance risks information collapse in unexplored dimensions.

• →Synthetic data scaling for speech models creates a fundamental trade-off between phonetic stability and prosodic expressivity, causing 'Synthetic Erosion' in low-resource settings.
• →Disentanglement-Guided Self-Alignment and Temperature-Driven Self-Critique frameworks effectively recover natural expressivity while maintaining phonetic accuracy.
• →The approach outperforms commercial competitors including ElevenLabs and Gemini Pro on speech quality metrics.
• →Zero-shot voice cloning for Lao represents first commercial-grade capability for a language historically lacking synthetic speech support.
• →Self-alignment techniques may address similar synthetic-data-induced quality collapse across other AI domains beyond speech synthesis.