OmniTabBench: Mapping the Empirical Frontiers of GBDTs, Neural Networks, and Foundation Models for Tabular Data at Scale

OmniTabBench introduces the largest tabular data benchmark with 3,030 datasets to evaluate gradient boosted decision trees, neural networks, and foundation models. The comprehensive analysis reveals no universally superior approach, but identifies specific conditions favoring different model categories through decoupled metafeature analysis.

OmniTabBench addresses a critical gap in machine learning research by providing the largest empirical evaluation framework for tabular data—a domain where practical applications vastly outnumber those using unstructured data. Previous benchmarks with fewer than 100 datasets created selection bias concerns and limited generalizability of findings. This research consolidates 3,030 datasets from diverse sources and industries, enabling statistically robust conclusions about model performance across varying conditions.

The significance stems from the ongoing debate about which modeling paradigm dominates tabular tasks. Gradient boosted decision trees (GBDTs) like XGBoost and LightGBM have traditionally held supremacy, while deep learning advocates argued neural networks would eventually prevail. Foundation models introduce another contender, yet consensus remained elusive due to fragmented benchmarking practices. OmniTabBench's scale and rigor provide authoritative clarity on this strategic question.

For practitioners and organizations, the decoupled metafeature analysis offers actionable intelligence beyond winner-take-all declarations. By isolating specific dataset properties—size, feature composition, skewness, kurtosis—researchers can now match model selection to empirical conditions rather than relying on heuristics. This enables more efficient resource allocation and model development strategies across industries.

The research impacts AI/ML infrastructure investment decisions, model development priorities, and educational curriculum design. Organizations can now make data-driven choices about which frameworks to prioritize based on their specific datasets and use cases. Future work will likely extend this framework to multimodal tasks and investigate why certain properties favor specific paradigms, further refining the theoretical understanding of tabular learning.

• →OmniTabBench with 3,030 datasets is the largest tabular data benchmark, providing statistically robust evaluation across tree-based, neural, and foundation models.
• →No single model family dominates all tabular tasks, rejecting long-held assumptions about universal superiority.
• →Decoupled metafeature analysis identifies specific dataset properties that favor different modeling paradigms.
• →Selection bias in prior smaller benchmarks (under 100 datasets) is mitigated through comprehensive, industry-categorized data collection.
• →The findings enable practitioners to select models based on empirical dataset characteristics rather than generic best-practice assumptions.