Can Image Models Imagine Time? ImageTime: A Novel Benchmark for Probing Visual World Modeling Through Spatiotemporal Consistency

Researchers introduce ImageTime, a diagnostic benchmark that evaluates whether image generation models can coherently imagine sequences of visual states over time. The benchmark requires models to generate four ordered keyframes representing an action's progression, revealing significant gaps in how current AI systems understand temporal consistency and causal relationships in visual narratives.

ImageTime addresses a critical blind spot in AI image generation evaluation. While models like DALL-E, Midjourney, and Stable Diffusion excel at producing individual high-quality images, their ability to maintain coherent visual narratives across multiple frames remains largely unmeasured. This distinction matters because real-world applications—from filmmaking and animation to architectural visualization and game design—depend on temporal consistency. A character's position should evolve logically; objects should maintain identity across frames; causal sequences should respect physical laws.

The benchmark's design elegantly sidesteps the complexity of dense video generation by focusing on four keyframes that represent the critical moments of an action. This middle ground between single-image generation and full video synthesis provides a targeted diagnostic tool. By decomposing evaluation into state predicates, temporal constraints, and forbidden violations, ImageTime moves beyond subjective assessment toward structured, interpretable capability scoring through GPT-4V-style VLM-as-judge protocols.

For the AI industry, this research exposes whether current models genuinely understand visual causality or merely pattern-match training data. The multi-family benchmarking approach will likely pressure developers to integrate temporal reasoning into their architectures. For creative professionals and enterprises, understanding these limitations informs tool selection and workflow design—knowing where models fail helps teams determine when human oversight remains essential.

Immediate implications include potential refinements to model training regimes to emphasize temporal coherence. Future work may push toward models that explicitly reason about action sequences, physics, and object persistence rather than treating each frame independently.

• →ImageTime benchmarks visual world modeling by requiring models to generate four ordered keyframes representing action progression, exposing gaps in temporal reasoning.
• →Current image generation systems struggle with spatiotemporal consistency, suggesting they lack robust understanding of causality and object identity over time.
• →The benchmark uses structured VLM evaluation to produce interpretable scores and diagnostic subscores, moving beyond subjective assessment toward quantifiable metrics.
• →Practical applications like storyboarding, video previsualization, and reference-guided editing critically depend on temporal consistency that current models cannot reliably provide.
• →Results suggest architectural changes may be needed to integrate temporal reasoning into generative models rather than treating frames as independent generation tasks.