Models Recall What They Violate: Constraint Adherence in Multi-Turn LLM Ideation

Researchers introduce DriftBench, a benchmark evaluating how well large language models maintain fidelity to original constraints during multi-turn iterative refinement. The study reveals a critical disconnect: models can accurately restate constraints while simultaneously violating them, with non-compliance rates ranging from 8% to 99% depending on the model.

This research exposes a fundamental limitation in how modern LLMs handle constrained reasoning during extended interactions. When users iteratively refine ideas with AI assistants, the models progressively drift from original objectives despite maintaining declarative knowledge of stated constraints. The study's most striking finding—the knows-but-violates phenomenon—suggests that constraint understanding and constraint adherence operate through different mechanisms in neural language models.

The research addresses a growing pain point as AI tools integrate deeper into scientific and creative workflows. Users increasingly rely on multi-turn conversations to develop complex ideas, trusting models to maintain guardrails around budget, scope, or methodological requirements. This work demonstrates that such trust may be misplaced. The DriftBench evaluation across 2,146 runs and seven models shows complexity inflation occurs reliably, indicating the problem isn't model-specific but structural to how LLMs process iterative refinement.

For practitioners and organizations deploying LLMs in high-stakes ideation tasks, these findings highlight the need for external verification mechanisms. Structured checkpointing provides partial mitigation but doesn't resolve the underlying dissociation. The research also reveals that LLM-based evaluation itself under-detects violations, meaning self-monitoring approaches will systematically underestimate drift.

This work suggests future LLM development should prioritize architectural changes or training methods that tighter coupling between constraint representation and behavioral execution. Until such improvements materialize, workflows requiring strict adherence to constraints should implement human review, external constraint tracking, or hybrid approaches combining human and machine judgment rather than relying solely on model self-regulation.

• →LLMs accurately recall constraints while violating them in practice, with non-compliance rates from 8% to 99% across models tested
• →Iterative refinement pressure increases structural complexity and reduces adherence to original constraints across all tested interaction conditions
• →LLM-based constraint evaluation under-detects violations, making reported adherence scores artificially conservative
• →Structured checkpointing partially mitigates constraint drift but fails to eliminate the recall-behavior dissociation
• →Results remain robust across temperature variations and pressure types, indicating the problem is fundamental rather than parameter-dependent