SCENEBench: An Audio Understanding Benchmark Grounded in Assistive and Industrial Use Cases

Researchers introduce SCENEBench, a new benchmark for evaluating Large Audio Language Models (LALMs) beyond speech recognition, focusing on real-world audio understanding including background sounds, noise localization, and vocal characteristics. Testing of five state-of-the-art models revealed significant performance gaps, with some tasks performing below random chance while others achieved high accuracy.

• →SCENEBench addresses the gap in audio understanding evaluation beyond automatic speech recognition for LALMs.
• →The benchmark focuses on four categories: background sound understanding, noise localization, cross-linguistic speech understanding, and vocal characterizer recognition.
• →Testing reveals critical performance variations across tasks, with some models performing below random chance on certain audio understanding tasks.
• →The benchmark is grounded in real-world applications for accessibility technology and industrial noise monitoring.
• →Results provide direction for targeted improvements in Large Audio Language Model capabilities.