Contextual Drag: How Errors in the Context Affect LLM Reasoning

Researchers have identified 'contextual drag' - a phenomenon where large language models (LLMs) generate similar errors when failed attempts are present in their context. The study found 10-20% performance drops across 11 models on 8 reasoning tasks, with iterative self-refinement potentially leading to self-deterioration.

• →Contextual drag causes LLMs to repeat structurally similar errors from previous failed attempts in their context.
• →Performance drops of 10-20% were observed across 11 proprietary and open-weight models on reasoning tasks.
• →Iterative self-refinement can collapse into self-deterioration when contextual drag is severe.
• →External feedback and self-verification methods fail to eliminate this error propagation effect.
• →Current mitigation strategies only provide partial improvements and cannot fully restore baseline performance.