Do Joint Audio-Video Generation Models Understand Physics?

Researchers introduced AV-Phys Bench, a benchmark testing whether joint audio-video generation models truly understand physics or merely generate plausible outputs. Testing seven models across three scene categories, the study found all systems lack robust physical understanding, with performance collapsing on deliberately inconsistent prompts and transition-heavy scenarios.

The emergence of joint audio-video generation models has accelerated rapidly, with systems like Sora and Seedance 2.0 approaching production-quality outputs. However, this research exposes a critical limitation: these models generate convincing media without comprehending underlying physical laws. The AV-Phys Bench framework evaluates models across five dimensions—visual and audio semantics plus three types of physical commonsense—revealing that even leading proprietary systems fail catastrophically on Anti-AV-Physics prompts designed to request physically impossible scenarios.

This finding reflects a broader challenge in generative AI: scaling to photorealism has outpaced development of robust physical reasoning. Models trained on vast internet data absorb correlational patterns rather than causal understanding. The sharp performance drops on event and environment transitions indicate systems struggle with dynamic scene changes, suggesting their learned associations break down when standard patterns shift.

For developers building multimodal AI systems, this research identifies critical gaps requiring architectural innovation beyond current scaling approaches. The introduction of AV-Phys Agent—a ReAct-style evaluator combining multimodal language models with acoustic measurement tools—provides a practical methodology for assessing physical consistency, useful for quality control in production pipelines.

Looking ahead, the field must integrate physics-based constraints and reasoning modules into generation architectures. Current approaches may plateau without explicit physical grounding, creating opportunities for companies developing physics-aware training methods or constraint-based generation systems. The research suggests that next-generation systems require hybrid approaches combining learned patterns with deterministic physical simulation.

• →All tested audio-video generation models, including Seedance 2.0, fail to demonstrate robust physical understanding despite high visual and audio quality.
• →Models show dramatic performance degradation on transition-driven scenes and physically impossible scenarios, indicating surface-level pattern matching rather than causal reasoning.
• →AV-Phys Bench introduces a standardized evaluation framework measuring physical commonsense across visual, audio, and cross-modal dimensions with five evaluation criteria.
• →Physics-aware constraints and explicit reasoning modules represent critical missing components in current generative architecture approaches.
• →The research identifies cross-modal physical consistency as a key open challenge requiring fundamental architectural innovation beyond current scaling strategies.