Video generation models as world simulators

OpenAI introduces Sora, a large-scale text-conditional diffusion model capable of generating up to one minute of high-fidelity video content. The model uses transformer architecture on spacetime patches and represents a significant advancement toward building general purpose physical world simulators.

• →Sora can generate up to one minute of high-quality video from text prompts using diffusion models.
• →The model operates on variable durations, resolutions and aspect ratios for flexible video generation.
• →Uses transformer architecture applied to spacetime patches of video and image latent codes.
• →Training involves joint learning on both video and image data at large scale.
• →Results indicate scaling video generation models could lead to general purpose world simulators.