Mitigating Label Shift in Tabular In-Context Learning via Test-Time Posterior Adjustment

Researchers introduce DistPFN, a test-time adjustment method that improves TabPFN's vulnerability to label shift—a common problem where machine learning models overfit to majority classes. The solution rescales predicted probabilities without requiring architectural changes or retraining, demonstrating significant improvements across 250+ datasets while maintaining performance in standard settings.

TabPFN represents a significant advancement in foundation models for tabular data, leveraging in-context learning on synthetic datasets to achieve competitive performance. However, the model exhibits a critical weakness: susceptibility to label shift, where changes in class distribution between training and deployment cause performance degradation through majority class overfitting. This limitation matters because real-world tabular datasets frequently exhibit imbalanced class distributions, making the issue practically relevant for enterprises relying on foundation models for predictive analytics.

DistPFN addresses this through an elegant post-hoc adjustment mechanism that operates at inference time. By recalibrating predicted class probabilities to reduce dependence on the training distribution while emphasizing the model's learned posterior, the method achieves robustness without modifying TabPFN's underlying architecture. The introduction of DistPFN-T adds adaptive temperature scaling, allowing the adjustment strength to respond dynamically to prior-posterior divergence. This approach demonstrates thoughtful engineering—solving a fundamental problem through minimal intervention.

For practitioners deploying tabular foundation models in production, this work reduces a major deployment risk. Organizations handling imbalanced datasets across finance, healthcare, and e-commerce sectors gain a readily implementable solution that improves reliability without computational overhead. The evaluation across 250+ OpenML datasets provides confidence in generalizability.

Future developments may focus on extending posterior adjustment techniques to other foundation model architectures and exploring whether similar approaches benefit sequential or graph-structured tabular data. As foundation models increasingly serve as infrastructure for tabular machine learning, addressing their specific failure modes becomes essential for enterprise adoption.

• →DistPFN solves TabPFN's label shift vulnerability through test-time posterior probability adjustment without retraining
• →The method demonstrates substantial improvements across 250+ datasets while preserving performance in balanced settings
• →Temperature scaling variant (DistPFN-T) adaptively controls adjustment strength based on prior-posterior discrepancy
• →Post-hoc approach enables easy integration into existing TabPFN deployments with minimal engineering overhead
• →Addresses practical enterprise need for robust tabular models handling imbalanced real-world datasets