AIBullisharXiv – CS AI · Jun 106/10
🧠
Spatial-Omni: Spatial Audio Understanding Integration in Multimodal LLMs via FOA Encoding
Researchers introduce Spatial-Omni, a method that integrates First-Order Ambisonics (FOA) spatial audio into multimodal large language models, enabling them to understand sound localization and spatial scene reasoning. The approach includes new datasets and benchmarks with 400K audio clips and 2.1M QA pairs, demonstrating improved performance on spatial audio tasks while maintaining general audio understanding.