Response-G1: Explicit Scene Graph Modeling for Proactive Streaming Video Understanding

Response-G1 introduces a novel framework for real-time video understanding that uses explicit scene graphs to align video evidence with query-specific response conditions, enabling Video-LLMs to make more accurate timing decisions during streaming video analysis without requiring fine-tuning.

Response-G1 represents a meaningful advancement in streaming video understanding by addressing a fundamental limitation in existing Video-LLM approaches: the inability to proactively determine optimal response timing as video unfolds. Traditional methods rely on implicit, query-agnostic visual modeling, which creates ambiguity around when responses should occur. The framework's innovation centers on converting both accumulated video evidence and expected response conditions into a shared scene graph representation, establishing explicit structural alignment that improves interpretability and decision accuracy.

This work builds on growing recognition within the AI research community that structured representations enhance reasoning capabilities across multimodal tasks. Scene graphs, which decompose visual content into objects, attributes, and relationships, have proven effective for various vision-language tasks. Applying this approach specifically to the temporal dimension of streaming video introduces a novel dimension—query-guided graph generation at streaming scale, combined with memory-based retrieval of semantically relevant historical graphs.

The practical significance extends to applications requiring real-time video monitoring and analysis, such as surveillance systems, live event detection, and interactive video understanding platforms. The framework's fine-tuning-free design reduces computational barriers to deployment while maintaining competitive or superior performance against existing methods on both proactive and reactive benchmarks.

Looking forward, the validation of explicit scene graph modeling in streaming contexts could influence architectural decisions in subsequent Video-LLM development. Questions remain regarding scalability with extremely long video sequences and computational efficiency of continuous scene graph generation, areas where future iterations may focus refinement efforts.

• →Response-G1 uses scene graphs to create explicit alignment between video evidence and query response conditions, improving timing decision accuracy.
• →The framework operates without fine-tuning through three stages: online scene graph generation, historical graph retrieval, and retrieval-augmented trigger prompting.
• →Structured graph representations enable more interpretable decisions compared to implicit modeling approaches in Video-LLMs.
• →Benchmarks demonstrate superiority in both proactive streaming and reactive video understanding tasks.
• →The fine-tuning-free design reduces computational requirements while maintaining or exceeding existing method performance.