When Context Misleads: Surprisal, Energy and Attention Entropy as Metrics of Coherence Illusions in LLMs

Researchers discovered that Dutch language models exhibit coherence illusions similar to humans, where incoherent text appears coherent when a matching distractor precedes it. Using surprisal, attention entropy, and energy metrics, they identified shared mechanisms underlying these illusions across different model architectures.

This research extends psycholinguistic findings into the domain of large language models, revealing that neural networks process discourse coherence in ways that mirror human cognitive biases. The study examined how Dutch LLMs respond to texts containing anaphoric words like 'again' and 'too', discovering that models become less surprised by incoherent continuations when contextual distractors align with expected patterns. This finding challenges assumptions about LLM robustness and text understanding.

The research employs three complementary metrics to diagnose coherence illusions. Surprisal measurements tracked human acceptability judgments and eye-tracking data, establishing a quantifiable link between model behavior and human perception. Attention entropy analysis identified specific attention heads that behave distinctly under coherent versus incoherent conditions, while energy metrics—borrowed from associative memory theory—provided a novel framework for quantifying discourse coherence at scale.

For AI developers and researchers, these results have significant implications. The discovery that coherence illusions transfer across different experimental settings suggests a fundamental mechanism in how LLMs process discourse, rather than task-specific artifacts. This understanding could inform better training approaches, evaluation methods, and architectural improvements. The work also demonstrates the value of interpretability techniques in understanding model vulnerabilities that might not surface during standard benchmarking.

Future research should investigate whether these coherence illusions affect downstream applications like question-answering or information retrieval, and whether architectural modifications can mitigate them. Understanding these mechanisms becomes increasingly important as LLMs are deployed in high-stakes domains where coherence failures could have real consequences.

• →Dutch LLMs exhibit coherence illusions where incoherent text seems coherent when matching distractors appear in prior context.
• →Surprisal, attention entropy, and energy metrics effectively identify and measure coherence illusion mechanisms in neural language models.
• →Specific attention heads show consistent behavior patterns across coherence versus incoherence conditions, indicating shared underlying mechanisms.
• →Model vulnerabilities to coherence illusions transfer across different experimental settings, suggesting fundamental limitations rather than task artifacts.
• →These findings have implications for improving LLM robustness and developing better evaluation methods for discourse understanding.