Unbiased Sliced Wasserstein Kernels for High-Quality Audio Captioning

Researchers developed an unbiased sliced Wasserstein RBF kernel with rotary positional embedding to improve audio captioning systems by addressing exposure bias and temporal relationship issues. The method shows significant improvements in caption quality and text-to-audio retrieval accuracy on AudioCaps and Clotho datasets, while also enhancing audio reasoning capabilities in large language models.

• →New USW-RBF kernel with rotary positional embedding addresses exposure bias in audio captioning systems.
• →The approach preserves temporal relationships between acoustic and linguistic modalities more effectively than existing contrastive methods.
• →Extensive testing on AudioCaps and Clotho datasets shows significant improvements in caption quality and lexical diversity.
• →The kernel enhances reasoning capabilities of large audio language models with 4% accuracy improvement on MMAU-test-mini benchmarks.
• →The solution offers computational efficiency through stochastic gradient optimization for real-world applications.