Weakly Supervised Distillation of Hallucination Signals into Transformer Representations

Researchers developed a weak supervision framework to detect hallucinations in large language models by distilling grounding signals into transformer representations during training. Using substring matching, sentence embeddings, and LLM judges, they created a 15,000-sample dataset and trained five probing classifiers that achieve hallucination detection from internal activations alone at inference time, eliminating the need for external verification systems.

This research addresses a critical challenge in large language model deployment: hallucination detection without external dependencies. Traditional approaches require real-time fact-checking against knowledge bases, retrieval systems, or auxiliary models—all computationally expensive and operationally complex. By encoding hallucination signals directly into model representations during training, this work enables inference-time detection through internal activation patterns alone.

The innovation lies in the weak supervision framework, which combines three complementary signals to label training data without human annotation. This approach scales efficiently compared to manual labeling while maintaining reasonable accuracy across grounding signals. The researchers constructed a substantial 15,000-sample dataset from SQuAD v2, providing rigorous validation across multiple architectures. Transformer-based probes—particularly the CrossLayerTransformer and HierarchicalTransformer variants—outperformed simpler architectures, suggesting that modeling inter-layer dependencies captures meaningful hallucination patterns.

For practitioners deploying large language models in production environments, this work has significant implications. The negligible probe latency (0.15-6.66 milliseconds) and maintained throughput (0.231 queries per second) demonstrate practical viability. End-to-end generation costs no meaningful performance penalty, making internal hallucination detection feasible for real-world applications. This could substantially reduce infrastructure costs by eliminating external verification systems while improving user-facing reliability.

Future research should explore generalization across different base models, domains, and hallucination types. Testing on models beyond LLaMA-2-7B and datasets beyond SQuAD would establish broader applicability. Additionally, understanding which layers encode hallucination signals most effectively could enable model-agnostic detection methods.

• →Hallucination detection can be distilled into transformer representations during training, enabling detection from internal activations without external verification
• →Weak supervision combining substring matching, sentence embeddings, and LLM judges creates reliable training labels without human annotation
• →Transformer-based probes significantly outperform simpler architectures, with CrossLayerTransformer and HierarchicalTransformer achieving best performance
• →Probe inference adds negligible latency (0.15-6.66 ms) and maintains practical throughput of 0.231 queries per second
• →Internal hallucination detection could reduce infrastructure costs by eliminating external fact-checking systems while improving deployment reliability