Hallucination Basins: A Dynamic Framework for Understanding and Controlling LLM Hallucinations

Researchers introduce a geometric framework for understanding LLM hallucinations, showing they arise from basin structures in latent space that vary by task complexity. The study demonstrates that factual tasks have clearer separation while summarization tasks show unstable, overlapping patterns, and proposes geometry-aware steering to reduce hallucinations without retraining.

• →LLM hallucinations can be understood through geometric basin structures in latent space that depend on task complexity.
• →Factual question tasks show clearer basin separation while summarization and misconception-heavy tasks exhibit unstable overlapping patterns.
• →Researchers developed task-complexity and multi-basin theorems to formalize hallucination behavior across different scenarios.
• →The framework enables geometry-aware steering techniques that can reduce hallucination probability without requiring model retraining.
• →The study analyzed autoregressive hidden-state trajectories across multiple open-source models and benchmarks to validate the framework.