Modeling Nonlinear Feature Interactions with Product-Unit Residual Networks

Researchers introduce Product-Unit Residual Networks (PURe), a neural architecture that explicitly models nonlinear feature interactions through multiplicative units combined with residual connections. The approach demonstrates improved interpretability, robustness to noise, and sample efficiency compared to standard MLPs across synthetic and real-world datasets.

Product-Unit Residual Networks address a fundamental limitation in deep learning: standard multilayer perceptrons implicitly capture feature interactions in ways that obscure how models make decisions. PURe networks use multiplicative product units—operations that explicitly multiply features together—to model cross-feature couplings directly, while residual connections stabilize training and prevent optimization degradation. This architectural choice produces more interpretable models without sacrificing predictive performance.

The research builds on decades of neural network development but tackles an increasingly important problem: as machine learning systems influence critical decisions in healthcare, finance, and engineering, interpretability matters as much as accuracy. Implicit feature interactions in standard MLPs create black-box behavior that regulatory compliance and safety verification struggle to address. PURe networks' explicit interaction modeling provides structural transparency through attention mechanisms like SHAP analysis, revealing which feature combinations drive predictions.

The empirical validation demonstrates competitive or improved accuracy while consistently outperforming baselines in robustness against Gaussian noise and performance with limited training data. Both real-valued and complex-valued variants show benefits, with PURe learning "more concentrated and structurally coherent interaction patterns." This matters for practitioners deploying models in data-scarce domains or where noise poses challenges.

Future applications likely extend to domains requiring certification and explainability—scientific discovery, clinical diagnostics, autonomous systems. The work suggests that architectural innovations emphasizing explicit computation over implicit representation can simultaneously improve both performance and trustworthiness, a trend increasingly valued across industries as AI systems face greater scrutiny.

• →PURe networks explicitly model nonlinear feature interactions through multiplicative units, improving interpretability compared to standard MLPs
• →The architecture demonstrates superior robustness to feature noise and better sample efficiency in low-data regimes
• →SHAP-based analysis reveals PURe learns more structured and concentrated interaction patterns than baseline approaches
• →Performance remains competitive or improved while gaining transparency benefits critical for regulated domains
• →Both real and complex-valued variants show benefits under matched parameter budgets