MOSS-Video-Preview: Toward Real-Time Video Understanding via Cross-Attention

Researchers introduce MOSS-Video-Preview, a cross-attention architecture enabling real-time video understanding where models process frames continuously and revise answers as new information arrives. The approach achieves 5x speedup in time-to-first-token and 2.7x higher decoding throughput compared to decoder-only models, while maintaining competitive offline performance.

MOSS-Video-Preview represents a meaningful shift in how vision-language models process video data. Rather than treating video understanding as a batch processing task—consuming an entire recording before generating responses—the architecture separates perception and generation into independent pathways. This dual-channel design using cross-attention prevents visual processing from blocking text generation, a bottleneck inherent in decoder-only approaches where visual tokens must join the autoregressive sequence.

The paradigm addresses a real limitation in current multimodal AI. Most production systems wait for frame buffering or rely on sequential processing, creating latency problems for applications requiring responsive video analysis. By allowing the model to output answers while continuously ingesting new frames and revising conclusions, MOSS-Video-Preview demonstrates a more human-like interaction pattern.

The performance metrics reveal substantial efficiency gains: achieving 5x improvement in time-to-first-token on a single H200 GPU with 256 frames indicates viable deployment for real-time applications without requiring massive computational scaling. The tradeoff comes in raw capability—the model trails Qwen2.5-VL-7B on comprehensive benchmarks, which researchers attribute to data scale and quality rather than architectural limitations.

For the AI development community, this work establishes architectural principles for real-time vision-language fusion beyond video. The cross-attention interface provides clean separation for independent compression of visual streams, opening possibilities for selective frame processing and adaptive bitrate handling. As video understanding becomes central to AI applications across surveillance, autonomous systems, and interactive media, efficiency-focused architectures like this likely influence production designs moving forward.

• →Cross-attention architecture separates perception and generation into non-blocking pathways, reducing computational bottlenecks in video processing.
• →Model achieves 5x faster time-to-first-token and 2.7x higher decoding throughput on single H200 GPU compared to decoder-only baselines.
• →Real-time paradigm enables models to revise answers, maintain silence when appropriate, and process continuous frame streams—behaviors absent in offline models.
• →Performance gap versus larger baselines stems from data and scale constraints rather than architectural design, suggesting improvement potential with additional training.
• →Clean channel-wise interface enables independent compression strategies for visual features, applicable beyond video to other vision-language fusion domains.