Shared Doubt: Zero-shot Cross-Lingual Confidence Estimation for Language Models

Researchers demonstrate that multilingual large language models encode shared confidence features that transfer across languages without retraining. A lightweight linear probe trained on English can predict answer correctness in unseen languages with zero-shot generalization, suggesting confidence estimation mechanisms are language-universal in LLMs.

This research addresses a critical gap in LLM reliability assessment by revealing that confidence estimation—predicting whether a model's answer is correct—operates through shared, transferable mechanisms across languages. Most prior work focused exclusively on English, creating blind spots for the global multilingual deployment of LLMs. The study employs a minimal linear probe that learns from intermediate neural representations rather than requiring expensive model retraining, making it practically deployable.

The breakthrough finding is that confidence features concentrate in middle layers of transformer architectures across all tested languages, indicating a shared representational space for uncertainty quantification. This architecture-level consistency suggests confidence is not language-specific but rather a fundamental property of how these models process information. The zero-shot transfer capability—working on languages the probe never saw during training—demonstrates robustness that traditional machine learning approaches rarely achieve.

For practitioners deploying multilingual LLMs in production, this has immediate implications. Confidence estimation enables safer deployment by flagging unreliable outputs before users encounter them, reducing hallucinations and errors. The method avoids retraining costs while providing strong baselines compared to existing techniques. However, performance degrades gracefully based on typological distance from the source language, meaning Romance or Germanic languages transfer better than distantly related tongues.

The research opens pathways for more reliable multilingual AI systems across healthcare, legal, and financial applications where confidence estimates guide human-in-the-loop workflows. Future work should validate performance on non-Indo-European language families and investigate whether this confidence subspace correlates with actual factual accuracy beyond grammatical correctness.

• →Multilingual LLMs encode shared, language-universal confidence features in middle transformer layers that enable zero-shot cross-lingual transfer.
• →A lightweight linear probe provides competitive confidence estimation without retraining, significantly reducing deployment costs for multilingual systems.
• →The method's effectiveness depends on typological similarity between training and target languages, with diminishing returns for distant language families.
• →Confidence estimation can flag unreliable model outputs in production, improving safety for multilingual LLM deployments in high-stakes domains.
• →The shared confidence subspace discovery suggests confidence is a fundamental property of transformer architectures rather than language-specific behavior.