Excluding the Target Domain Improves Extrapolation: Deconfounded Hierarchical Physics Constraints

Researchers propose Deconfounded Hierarchical Gate (DHG), a novel approach to improve physics-constrained deep generative models' ability to extrapolate beyond training conditions. The method counterintuitively finds that excluding target-domain data during pretraining improves extrapolation performance by 39%, achieving 46% better results on lithium-ion battery temperature prediction benchmarks.

This research addresses a critical limitation in physics-informed machine learning: the inability of models trained on one set of conditions to reliably predict behavior in new environments. Traditional physics-constrained models apply physical laws uniformly across the generation process, missing the hierarchical nature of how different physical principles interact and failing to account for confounding variables that can distort constraint relationships.

The DHG framework introduces two key innovations. First, it applies counterfactual reasoning through causal inference techniques (the do-operator and backdoor adjustment) to disentangle true physical constraints from spurious correlations caused by temperature variations. Second, it implements progressive, hierarchical constraint application rather than static regularization, allowing the model to differentiate between genuine physical inconsistencies and artifacts of confounding variables.

The counterintuitive finding—that withholding target-domain data during pretraining yields superior results—reveals important principles about transfer learning in physics-informed contexts. When models train on the target domain, they may learn domain-specific patterns that generalize poorly to out-of-distribution scenarios. By excluding this data, the model focuses on extracting fundamental, generalizable physical relationships that transfer more effectively across temperature ranges.

The 46% improvement on lithium-ion battery extrapolation (from RMSE 0.397 to 0.215) demonstrates practical significance for energy storage applications, where thermal behavior prediction directly impacts safety and performance. This work extends beyond batteries to any domain requiring physics-constrained predictions under variable conditions, including materials science, climate modeling, and industrial process control.

• →Excluding target-domain data during pretraining improves extrapolation by 39%, suggesting domain-specific overfitting undermines generalization
• →DHG uses causal inference to remove confounding effects, enabling cleaner extraction of hierarchical physical constraints
• →The method achieves 46% error reduction on lithium-ion battery temperature extrapolation across a 39-degree Celsius range
• →Hierarchical constraint application outperforms static regularization by distinguishing genuine physical inconsistencies from spurious correlations
• →Results suggest fundamental rethinking of pretraining strategies for physics-informed machine learning models