Breaking Contextual Inertia: Reinforcement Learning with Single-Turn Anchors for Stable Multi-Turn Interaction

Researchers introduce RLSTA (Reinforcement Learning with Single-Turn Anchors), a new training method that addresses 'contextual inertia' - a problem where AI models fail to integrate new information in multi-turn conversations. The approach uses single-turn reasoning capabilities as anchors to improve multi-turn interaction performance across domains.

• →Large Language Models suffer from 'contextual inertia' where they stick to previous reasoning even when new information is provided in multi-turn conversations.
• →RLSTA training method uses models' strong single-turn capabilities as internal anchors to provide reward signals for better multi-turn performance.
• →The approach shows strong cross-domain generalization, working effectively from math to code applications without external verifiers.
• →RLSTA significantly outperforms standard fine-tuning and abstention-based methods in experimental testing.
• →The method addresses a fundamental limitation in current AI systems' ability to adapt reasoning based on updated information.