InPhyRe Discovers: Large Multimodal Models Struggle in Inductive Physical Reasoning

Researchers introduced InPhyRe, a new benchmark showing that large multimodal models (LMMs) struggle with inductive physical reasoning—their ability to apply learned physical laws to novel, unseen scenarios. Testing 13 LMMs revealed critical weaknesses: models fail to generalize parametric knowledge, perform poorly with unseen physical laws, and exhibit language bias that causes them to ignore visual inputs, raising concerns about their reliability for safety-critical applications.

The InPhyRe research addresses a fundamental limitation in current large multimodal models that has significant implications for AI development and deployment. While LMMs have demonstrated impressive capabilities in encoding and recalling physical laws observed during training, the study reveals a critical gap: these models cannot reliably adapt their reasoning to novel physical environments—a capability humans possess naturally. This distinction between parametric knowledge (memorized physical laws) and inductive reasoning (applying laws to new situations) represents a crucial limitation for any AI system intended to operate in real-world, safety-critical domains.

The benchmark's findings emerge from a growing recognition that current LMM evaluations focus narrowly on established knowledge rather than generalization capacity. By testing models on algorithmically generated synthetic videos of collision events with unseen physical parameters, the researchers created controlled conditions revealing systematic failures. The discovery that models suffer from language bias and may ignore visual inputs fundamentally questions whether these systems genuinely understand visual information or merely pattern-match linguistic cues to training data.

For the AI industry, these results suggest that achieving trustworthy AI for autonomous systems, robotics, and safety-critical applications requires substantial architectural innovations beyond scaling. Current models appear to lack the adaptive reasoning mechanisms necessary for domains where physical understanding matters. Developers and researchers must prioritize inductive reasoning capabilities rather than merely expanding training datasets. The work establishes InPhyRe as a critical evaluation framework, potentially influencing how future LMMs are designed and assessed for real-world deployment.

• →LMMs encode physical laws as fixed parametric knowledge but fail to apply these laws to novel, unseen scenarios
• →13 tested models show weak inductive physical reasoning when encountering previously unobserved physical environments
• →Language bias causes models to potentially ignore visual inputs, questioning their visual understanding reliability
• →Current LMM evaluation benchmarks overlook inductive reasoning, focusing only on parametric knowledge assessment
• →Safety-critical applications requiring LMMs demand fundamental improvements in adaptive reasoning beyond current architectures