Wan-Streamer v0.1: End-to-end Real-time Interactive Foundation Models

Researchers introduce Wan-Streamer, a unified foundation model that handles real-time audio-visual interaction through a single Transformer architecture, eliminating the need for separate modules and achieving approximately 200ms model-side latency. The system enables sub-second duplex communication by integrating perception, reasoning, generation, and response timing within one end-to-end model.

Wan-Streamer represents a significant architectural shift in how foundation models approach multimodal interaction. Rather than chaining specialized components for voice activity detection, speech recognition, language processing, text-to-speech, and video generation, the system consolidates these functions into a single unified model. This monolithic approach reduces latency accumulation that typically plagues cascaded systems, achieving approximately 200ms model-side response times—a meaningful improvement for natural conversation flows.

The technical innovation centers on redesigning the entire processing stack for streamability. Block-causal attention mechanisms enable incremental token processing, while causal encoders and decoders support streaming units as short as 160 milliseconds at 25 frames per second. This engineering enables the system to process information incrementally rather than waiting for complete inputs, mimicking how humans communicate through overlapping, real-time exchanges.

The achievement of approximately 550 milliseconds total interaction latency, including 350 milliseconds of bidirectional network overhead, positions the system within practical thresholds for natural audio-visual dialogue. Current multimodal systems often suffer from noticeable delays that disrupt conversational flow; Wan-Streamer's performance suggests viable pathways toward genuinely responsive interactive AI.

For developers and researchers, this work demonstrates that unified architectures can outperform modular pipelines in both latency and error propagation. The framework's applicability extends beyond entertainment and avatar interactions to teleconferencing, accessibility tools, and real-time collaborative systems. Future development will likely focus on scaling these techniques while maintaining latency guarantees and improving multimodal synchronization robustness.

• →Wan-Streamer eliminates separate VAD, ASR, TTS, and video-generation modules by unifying perception, reasoning, and generation in a single Transformer model.
• →The system achieves approximately 200ms model-side latency with sub-second total interaction latency, enabling natural real-time audio-visual communication.
• →Block-causal attention and causal encoders/decoders enable streaming at 160ms intervals, processing information incrementally rather than waiting for complete inputs.
• →End-to-end joint learning reduces pipeline latency and error accumulation compared to traditional cascaded interactive systems.
• →The unified architecture demonstrates that monolithic multimodal designs can outperform modular approaches in both speed and cross-modal synchronization.