CRUMB: Efficient Prior Fitted Network Inference via Distributionally Matched Context Batching

CRUMB is a new inference wrapper that makes prior-fitted networks (PFNs) more practical for large datasets by clustering test queries and selecting distributionally matched training subsets using maximum mean discrepancy minimization. The technique is architecture-agnostic, requires no retraining, and demonstrates superior performance across multiple PFN models on tabular benchmarks.

CRUMB addresses a fundamental scalability bottleneck in prior-fitted networks, an emerging class of tabular foundation models that perform in-context learning by processing entire training datasets in a single forward pass. The quadratic scaling of self-attention mechanisms has made this approach computationally prohibitive for large datasets, limiting PFNs' practical applicability despite their strong empirical performance. By introducing an intelligent context selection mechanism based on maximum mean discrepancy (MMD) optimization, CRUMB enables efficient inference without requiring model retraining or architecture modifications.

The technical approach reflects broader trends in machine learning toward smarter inference optimization. Rather than attempting architectural overhauls, the researchers developed a modular wrapper that works across different PFN implementations, demonstrating results on three distinct architectures evaluated on 51 datasets. The MMD-based clustering strategy serves a dual purpose: it reduces computational overhead while naturally handling covariate drift—a persistent challenge in real-world applications where test data distributions diverge from training distributions.

For the tabular machine learning community, CRUMB significantly expands the practical deployment window for PFN models, which have shown promise as foundation models for structured data. The architecture-agnostic nature means existing PFN implementations can immediately benefit from efficiency gains. The resilience to distribution shift is particularly valuable for practitioners dealing with evolving datasets or external data sources. As tabular foundation models gain adoption in enterprise and research settings, techniques like CRUMB that enable practical large-scale inference become increasingly important for competitive deployment.

• →CRUMB reduces computational requirements for prior-fitted network inference through intelligent training subset selection without model retraining.
• →The method uses maximum mean discrepancy minimization to distributionally match training context to test query clusters.
• →Architecture-agnostic design enables immediate application across multiple PFN models (TabPFNv2, TabICLv1, TabICLv2).
• →Demonstrates superior performance to existing context selection strategies on 51-dataset TabArena benchmark.
• →Inherent robustness to covariate drift makes the approach practical for real-world applications with distribution shifts.