Reformulating Neural Operators in $d+1$ Dimensions for Embedding Evolution

Researchers introduce a reformulated Neural Operators framework that models embedding evolution in d+1 dimensions, using Fourier-based operators to improve function space mappings. The approach demonstrates superior performance across multiple benchmarks while reducing computational overhead compared to traditional embedding-scaling methods.

Neural Operators represent a significant advancement in machine learning for scientific computing, enabling models to learn mappings between infinite-dimensional function spaces. This research addresses a critical inefficiency in existing NO architectures: while most work optimizes kernel parameterizations on the physical domain, the evolution of lifted embeddings—the internal feature representations—has received minimal attention, forcing practitioners toward computationally expensive scaling solutions. The paper's core innovation introduces an auxiliary function dimension that treats embedding evolution as an operator problem itself, fundamentally reconceptualizing the pipeline from d to d+1 dimensions. This theoretical reframing yields practical benefits through basis-diversified auxiliary evolution modules that replace brute-force scaling approaches. Testing across physics-based benchmarks—from simple heat equations to complex 3D fluid dynamics—demonstrates consistent performance gains with lower relative L2 errors than competing methods. The robustness characteristics are particularly noteworthy: the model maintains accuracy under mixed-resolution training scenarios and generalizes to unseen temporal regimes without retraining. Such zero-shot generalization capabilities carry important implications for scientific computing applications where extrapolation beyond training conditions is essential. The controlled comparisons against scaled baselines ensure improvements stem from architectural innovation rather than parameter count increases. This work reflects broader trends in neural operators toward more principled, theoretically grounded designs that improve computational efficiency alongside predictive accuracy. For practitioners developing surrogate models for expensive physical simulations, this approach offers a pathway to better performance with reduced computational demands.

• →Neural Operators reformulated in d+1 dimensions separate kernel parameterization from embedding evolution, improving model efficiency and accuracy
• →Approach achieves lowest L2 errors across ten+ benchmarks ranging from 1D to 3D physics problems without increasing parameter budgets
• →Model demonstrates zero-shot generalization to unseen temporal regimes and robustness under mixed-resolution training scenarios
• →Fourier-based operators on joint physical-auxiliary domains provide basis-diversified evolution as alternative to expensive embedding scaling
• →Design choices for lifting and recovery operators are documented, enabling reproducibility and practical implementation