Does Your Neural Network Extrapolate? Feature Engineering as Identifiability Bias for OOD Generalization

Researchers demonstrate that neural networks fail at out-of-distribution (OOD) generalization not due to insufficient training data, but because the choice of feature representation fundamentally determines what extrapolation patterns a model can learn. The same architecture achieving identical in-distribution loss can differ by 520x out-of-distribution depending on how features are encoded, showing that correct feature engineering is necessary but not sufficient without appropriate model class constraints.

This research addresses a fundamental challenge in machine learning: why neural networks that perform excellently on training data often fail catastrophically when encountering new, out-of-distribution scenarios. The authors prove mathematically that OOD extrapolation is inherently non-identifiable from in-distribution data alone—infinitely many different data-generating processes can produce identical training results while diverging arbitrarily beyond it. This non-identifiability cannot be resolved by better optimization or more training data; it requires external structural commitments through feature engineering, model architecture, or domain knowledge.

The research provides concrete evidence across multiple domains: periodic functions become tractable when expressed in Fourier coordinates, chemical reaction prediction improves with mass-action kinetics, and exoplanet detection leverages Kepler's laws. A 264-run study spanning Transformers, Mamba, and S4D architectures confirms that positional encoding choices dramatically impact OOD performance. This reveals why pretrained models often generalize better—pretraining implicitly injects structural commitments about feature spaces that in-distribution training alone cannot discover.

For practitioners and researchers, this reframes OOD generalization as a feature representation problem rather than a scaling or optimization problem. The implications extend beyond academic AI: any system relying on neural networks for decision-making in novel contexts (financial forecasting, drug discovery, autonomous systems) faces this identifiability problem. Engineers cannot assume that models performing well on historical data will behave predictably on new data without explicit architectural commitments encoding domain knowledge about relevant features and their relationships.

• →OOD generalization is fundamentally limited by feature representation choice, not training data quantity or optimization quality
• →The same model architecture can vary by 520x in OOD performance depending only on how input features are encoded
• →Correct feature engineering requires external domain knowledge; in-distribution loss alone cannot identify the right representation
• →Pretraining and architectural choices (like positional encodings) succeed by implicitly encoding structural commitments about feature spaces
• →Feature maps and model class constraints are necessary but insufficient without transformed training data covering the relevant representation space