Prediction Bottlenecks Don't Discover Causal Structure (But Here's What They Actually Do)

Researchers rigorously tested claims that Mamba state-space models can discover causal structure through prediction-only training, finding the method underperforms classical approaches like PCMCI and Granger causality. The apparent success in earlier experiments was largely attributable to sample-size confounds and non-standard intervention semantics rather than genuine architectural advantages.

A widely-circulated claim in machine learning suggested that neural architectures trained purely on prediction tasks could recover causal relationships without explicit causal inference methods. This paper systematically dismantles that claim through comprehensive benchmarking. The researchers constructed a reusable falsification framework encompassing synthetic datasets (VAR, Lorenz, CauseMe-style), multiple intervention semantics, and real-world datasets with known ground truth. Their staged analysis reveals critical methodological issues: the bottleneck readout mechanism performs no better than simple linear alternatives, tuned Lasso substantially outperforms it on standard benchmarks, and classical methods like PCMCI and Granger causality maintain superiority on the only test with unambiguous causal ground truth. The headline claim that interventional data provides advantage at p<10^-5 dissolves under scrutiny—approximately 60% reflects sample-size confounding, and the residual effect survives only under non-standard random-forcing interventions rather than canonical do-calculus formulations. Crucially, equivalent effects appear in classical bivariate Granger, indicating the phenomenon is method-agnostic rather than architecture-specific. This work exemplifies rigorous scientific practice by implementing staged controls and publishing negative results. For the broader AI research community, it demonstrates the importance of robust benchmarking protocols before promoting architectural innovations as fundamental breakthroughs. The lasting contribution is the reusable falsification benchmark itself, which standardizes causal discovery evaluation across methods and intervention types, enabling more reliable future comparisons.

• →Mamba's apparent causal discovery capability does not survive rigorous benchmarking against classical methods like PCMCI and Granger causality.
• →Most reported advantages from interventional data reflect sample-size confounds rather than genuine methodological superiority.
• →Simple linear bottlenecks and tuned Lasso outperform or match the neural architecture on causal discovery tasks.
• →The reusable falsification benchmark with standardized datasets and intervention semantics becomes the paper's most valuable contribution.
• →Results highlight the critical importance of staged validation and control arms before claiming novel causal discovery capabilities.