Know When to Abstain: Optimal Selective Classification with Likelihood Ratios

Researchers developed new selective classification methods using likelihood ratio tests based on the Neyman-Pearson lemma, allowing AI models to abstain from uncertain predictions. The approach shows superior performance across vision and language tasks, particularly under covariate shift scenarios where test data differs from training data.

• →New selective classification framework uses likelihood ratio tests to determine when AI models should abstain from making predictions.
• →The approach unifies existing post-hoc selection methods and motivates novel techniques for uncertain prediction handling.
• →Methods demonstrate consistent outperformance across vision, language, and vision-language model tasks.
• →Special focus on covariate shift scenarios where input distributions differ between training and testing phases.
• →Research includes publicly available code implementation for broader adoption and validation.