Hallucination as Trajectory Commitment: Causal Evidence for Asymmetric Attractor Dynamics in Transformer Generation

Researchers demonstrate through causal experiments that hallucinations in language models arise from early trajectory commitments governed by asymmetric attractor dynamics. Using controlled prompt bifurcation on Qwen2.5-1.5B, they show that 44% of test prompts diverge into factual or hallucinated outputs at the first token, with activation patterns revealing that corrupting correct trajectories is far easier than recovering hallucinated ones—suggesting hallucination represents a stable but difficult-to-escape attractor state.

This research addresses a fundamental problem in large language model reliability: understanding the mechanistic origins of hallucination. Rather than treating hallucination as random noise or uniform model failure, the authors employ novel causal methodology to demonstrate it operates as a deterministic dynamical system. The bifurcation experiments reveal that model trajectories split immediately, suggesting hallucination isn't a gradual drift but an early commitment to an alternative basin of attraction. The asymmetric patching results—where corrupting correct outputs proves far easier than recovering from hallucinated ones—provide direct evidence that hallucinated states occupy locally stable regions of the model's activation landscape. This asymmetry has profound implications: it explains why hallucinations persist and why simple interventions fail. The step-0 predictability (r=0.776) indicates that prompt encoding itself establishes which basin the model will occupy, suggesting hallucination propensity is largely determined before token generation begins. For practitioners, this implies that preventing hallucination requires upstream intervention during prompt processing rather than downstream correction. The identification of regime-like clusters organizing the basin structure suggests hallucination isn't uniformly distributed across prompts but follows discoverable patterns. This mechanistic understanding could enable targeted architectural improvements or inference-time interventions designed to destabilize hallucinated attractors.

• →Hallucinations commit to stable attractor basins at token 0, making them difficult to correct once initiated
• →Activation patching reveals 87.5% corruption success but only 33.3% recovery rate, demonstrating asymmetric dynamics
• →Step-0 residual states predict per-prompt hallucination rates with high correlation (r=0.776), indicating early determination
• →Correcting hallucinated trajectories requires sustained multi-layer intervention while corruption needs only single perturbations
• →Five identifiable regime clusters organize hallucination basins, with false-premise prompts concentrated in saddle-adjacent regions