Integrating Local and Global Entropy for Uncertainty Quantification in LLMs

Researchers propose Global-Local Uncertainty (GLU), a new method for quantifying uncertainty in large language models by combining hidden-state geometric entropy with token-level signals. The approach successfully identifies confident-but-wrong predictions that existing token-only methods miss, offering improved reliability assessment across multiple model families.

Large language models present a critical reliability challenge: they generate plausible-sounding incorrect responses with high confidence, a phenomenon known as hallucination. Current uncertainty quantification methods focus narrowly on token-level probabilities, missing structural patterns within the model's hidden representations. This research identifies that geometric complexity in intermediate hidden states captures distinct failure modes—specifically, cases where the model is confidently wrong but token probabilities appear high. The distinction matters because local and global uncertainty signals operate near-independently, each revealing different failure regimes. By treating the problem geometrically rather than purely probabilistically, the authors access information inaccessible through token analysis alone. GLU's multiplicative gating mechanism efficiently fuses these orthogonal signals without requiring additional training, making it architecture-agnostic and computationally efficient. The method's validation across three model families and six benchmarks demonstrates robustness, addressing a real deployment concern for enterprises relying on LLMs for high-stakes applications like healthcare, finance, or customer support. This work advances the broader trend toward interpretable and reliable AI systems, where understanding model confidence becomes as important as accuracy itself. For developers and enterprises, better uncertainty quantification reduces risks associated with silent failures—cases where incorrect outputs seem authoritative. The single-pass requirement and lack of fine-tuning dependency enable practical integration into existing inference pipelines without performance overhead.

• →GLU combines hidden-state geometry and token-level entropy to detect confident-but-wrong predictions that token-only methods miss.
• →Global and local uncertainty signals operate statistically near-orthogonal, capturing distinct LLM failure modes.
• →The method requires only a single forward pass with no training, making it lightweight and architecture-agnostic.
• →Validation across three model families and six benchmarks shows GLU matches or outperforms existing unsupervised baselines.
• →Better uncertainty quantification reduces deployment risk for LLMs in high-stakes applications.