DASB -- Discrete Audio and Speech Benchmark

Researchers introduce DASB, a comprehensive benchmark framework for evaluating discrete audio tokens across speech, audio, and music domains. The study reveals that discrete representations lag behind continuous features and require significant tuning, with semantic tokens outperforming acoustic ones, establishing standardized evaluation protocols for multimodal AI systems.

The emergence of discrete audio tokenization represents a critical inflection point for multimodal AI development. Traditional continuous audio representations have dominated machine learning, but discrete tokens enable audio integration into language models through shared token vocabularies—a crucial requirement for unified multimodal systems. DASB addresses a fragmentation problem where inconsistent evaluation methodologies have obscured true performance comparisons across tokenization approaches.

This research builds on accelerating interest in audio-language bridging technologies, evidenced by recent advances in models like AudioLM, Wav2Vec, and commercial applications in conversational AI. The benchmark's findings reveal fundamental tradeoffs: discrete systems sacrifice robustness for computational efficiency and compatibility with transformer-based language models. The persistent gap between discrete and continuous representations indicates that current tokenization methods discard meaningful information during the discretization process.

For developers and AI companies, DASB provides actionable insights about architecture choices and hyperparameter sensitivity. Organizations building multimodal systems must weigh integration advantages against performance costs, particularly for applications requiring high-fidelity audio understanding like music generation or speaker identification. The public leaderboard structure mirrors successful benchmark practices in NLP and computer vision, likely accelerating community contributions.

Future research directions center on improving semantic token design and understanding what information is irreversibly lost during discretization. The gap closure requires innovations in tokenizer efficiency, vocabulary size optimization, and potentially hybrid approaches combining discrete and continuous representations for domain-specific applications.

• →Discrete audio tokens enable audio-language model integration but exhibit lower robustness than continuous representations.
• →Semantic tokens outperform acoustic tokens, suggesting information content preservation matters more than spectral accuracy.
• →Model architecture, data size, learning rate, and capacity significantly impact discrete token performance—no universal optimal configuration exists.
• →DASB provides standardized evaluation framework addressing inconsistent benchmarking practices across audio tokenization research.
• →A persistent performance gap exists between discrete tokens and continuous features, highlighting incomplete solutions for audio understanding.