A Paradigm Shift: Fully End-to-End Training for Temporal Sentence Grounding in Videos

Researchers propose a fully end-to-end training paradigm for temporal sentence grounding in videos, introducing the Sentence Conditioned Adapter (SCADA) to better align video understanding with natural language queries. The method outperforms existing approaches by jointly optimizing video backbones and localization components rather than using frozen pre-trained encoders.

• →Current video grounding methods suffer from task discrepancy by using frozen visual encoders trained for classification rather than language-video alignment.
• →The proposed end-to-end paradigm jointly optimizes video backbones and localization heads for better performance.
• →SCADA adapter uses sentence features to adaptively train video backbone parameters with reduced memory requirements.
• →The method enables deployment of deeper network backbones while enhancing visual representation through linguistic embedding integration.
• →Experimental results on two benchmarks demonstrate superior performance compared to state-of-the-art approaches.