Calibration Drift Under Reasoning: How Chain-of-Thought Budgets Induce Overconfidence in Large Language Models

Researchers discover that Chain-of-Thought reasoning in large language models can paradoxically increase overconfidence when reasoning budgets exceed task-specific thresholds, a phenomenon called Calibration Drift Under Reasoning (CDUR). The study shows that while extended reasoning initially improves accuracy, it eventually produces internally consistent but incorrect explanations that mislead models into false confidence, with implications for safe LLM deployment.

This research exposes a critical vulnerability in how modern language models handle uncertainty during reasoning processes. As LLMs generate longer chains of thought, they can construct narratives that feel logically sound but lead to wrong conclusions with high confidence—a dangerous combination for safety-critical applications. The phenomenon mirrors human confirmation bias, where extended deliberation sometimes entrenches false beliefs rather than correcting them.

The study builds on growing recognition that chain-of-thought prompting, while generally beneficial for accuracy, requires careful calibration. Previous work established that CoT improves reasoning quality, but this research reveals the dark side: the autoregressive generation process can lock in early errors through self-reinforcing explanations. The Hypothesis Lock-In model provides theoretical grounding for why this occurs, showing the non-monotonic relationship between reasoning depth and calibration error.

For AI deployment in finance, healthcare, and other high-stakes domains, this finding carries significant weight. Systems relying on LLMs for decision support must account for the possibility that longer reasoning chains increase false confidence rather than reliability. The proposed CABStop solution—a calibration-aware stopping mechanism—suggests practitioners need active monitoring rather than passive reliance on reasoning budgets.

Limitations of the current work include modest sample sizes and inconclusive results on larger models like Llama-3.3-70B, leaving open questions about whether scaling mitigates CDUR. Future research should explore whether this phenomenon persists in production-scale models and whether architectural changes can prevent hypothesis lock-in during generation.

• →Extended chain-of-thought reasoning can increase model overconfidence on incorrect answers beyond optimal reasoning budgets, creating dangerous false certainty
• →The Hypothesis Lock-In mechanism explains how autoregressive generation commits early errors into internally consistent but fundamentally wrong explanations
• →Current reasoning-based AI systems require calibration monitoring and may benefit from stopping rules that detect confidence-accuracy divergence
• →Larger models show less pronounced calibration drift, suggesting scaling may partially mitigate the effect but requires further investigation
• →Safety-critical AI applications cannot assume longer reasoning chains improve reliability and must implement auxiliary accuracy estimation alongside reasoning processes