Cross-Model Disagreement as a Label-Free Correctness Signal

Researchers introduce cross-model disagreement as a training-free method to detect when AI language models make confident errors without requiring ground truth labels. The approach uses Cross-Model Perplexity and Cross-Model Entropy to measure how surprised a second verifier model is when reading another model's answers, significantly outperforming existing uncertainty-based methods across multiple benchmarks.

• →Cross-model disagreement provides a practical solution for detecting AI model errors without needing training data or ground truth labels.
• →The method addresses the critical problem of confident errors where models are wrong but certain of their answers.
• →Cross-Model Perplexity achieved 0.75 AUROC compared to 0.59 for traditional within-model entropy baselines on MMLU benchmark.
• →The approach can be integrated into existing production systems for model routing, deployment monitoring, and selective prediction.
• →The method requires only a single forward pass from a verifier model, making it computationally efficient for production use.