4 articles tagged with #streaming-inference. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers introduce StreamingVLM, a vision-language model designed to process infinite video streams in real-time without excessive computational costs. The model uses a compact KV cache and supervised fine-tuning on overlapped video chunks to maintain stable performance up to 8 FPS, outperforming GPT-4O mini on a new benchmark featuring videos over two hours long.
OmniMem is a new memory compression framework for audio-visual large language models that enables efficient long-form video understanding by using modality-aware memory allocation and perturbation-aware token selection. The approach achieves 2-4% accuracy improvements over existing compression methods while reducing memory requirements, with potential applications in real-time video AI systems.
Researchers introduce Chatterbox-Flash, a zero-shot text-to-speech model combining block-diffusion decoding with streaming capabilities. The system addresses token distribution bias through prior-calibrated scoring and early-decoding schedules, achieving high-fidelity speech synthesis with low latency comparable to autoregressive systems.
Researchers introduce S-MARC, a streaming framework for modeling conversational behavior in full-duplex dialogue systems that predicts communicative functions and interaction behaviors while capturing their causal relationships. The system generates interpretable reasoning chains and establishes benchmarks for conversational AI reasoning, advancing natural human-computer interaction capabilities.
