AVI-Bench: Toward Human-like Audio-Visual Intelligence of Omni-MLLMs

Researchers introduce AVI-Bench, a comprehensive benchmark for evaluating audio-visual intelligence in multimodal large language models across perception, understanding, and reasoning tasks. The study reveals significant limitations in current models and proposes a taxonomy to guide development of more robust audio-visual AI systems.

AVI-Bench addresses a critical gap in AI evaluation methodology by establishing the first systematic benchmark for assessing audio-visual capabilities in Omni-MLLMs. As these models increasingly integrate multiple modalities, the absence of rigorous evaluation frameworks has allowed capabilities to be assumed rather than verified. The benchmark's three-stage architecture—perception, understanding, and reasoning—mirrors human cognitive processing, enabling precise identification of where models excel or fail in cross-modal interpretation tasks.

The research builds on a broader trend of multimodal AI development where vision and language integration has matured substantially, but audio-visual reasoning remains nascent. Most current MLLMs treat audio as secondary to visual input, limiting their real-world applicability in scenarios requiring genuine cross-modal synthesis. The introduction of AVI-Bench-PriSe, which tests models with unfamiliar, low-semantic stimuli, represents a methodological innovation that moves beyond benchmark saturation—a common problem where models memorize patterns specific to popular datasets.

For the AI development community, these findings signal that current Omni-MLLMs are less capable than marketing materials suggest, particularly in genuine audio-visual reasoning rather than simple audio transcription plus image analysis. This impacts product roadmaps for companies developing multimodal AI systems, indicating where architectural improvements and training methodologies require investment. Developers working on real-world applications involving audio-visual understanding—such as video analysis, autonomous systems, or accessibility tools—must account for these documented limitations.

The four-level AVI taxonomy provides a shared language for describing model capabilities, potentially standardizing future development efforts. As audio-visual AI moves toward commercial deployment, benchmarks like this become infrastructure investments that accelerate progress across the industry.

• →AVI-Bench provides the first comprehensive evaluation framework for audio-visual intelligence in multimodal large language models across three cognitive stages.
• →Current Omni-MLLMs demonstrate substantial limitations in genuine cross-modal audio-visual reasoning beyond simple transcription tasks.
• →The AVI-Bench-PriSe extension tests generalization with unfamiliar stimuli, revealing gaps between familiar and novel domain performance.
• →A four-level AVI taxonomy emerges from the research, establishing standards for describing audio-visual model capabilities.
• →Findings suggest significant development opportunities for companies and researchers advancing multimodal AI systems.