Multi-ResNets for Subspace Preconditioning in Constrained Optimization

Researchers propose MResOpt, a staged residual neural network architecture that solves constrained optimization problems by decomposing constraint satisfaction hierarchically. The method demonstrates improved performance on convex and non-convex optimization benchmarks, with particular applications to power flow problems in electrical grids.

MResOpt addresses a fundamental challenge in optimization: how neural networks can reliably satisfy multiple constraints with varying priority levels. Traditional approaches often treat all constraints equally or struggle to maintain feasibility during iterative solving. This work introduces a staged architecture that sequentially refines solutions while respecting constraint hierarchies, enabling domain-specific ordering when problem structure permits.

The theoretical foundation grounds the approach in Gaussian Process regression under infinite-width assumptions, providing interpretability alongside empirical performance gains. On synthetic benchmarks spanning quadratic programming (QP), quadratically-constrained quadratic programming (QCQP), and second-order cone programming (SOCP), the staged design consistently reduces high-priority constraint violations compared to reprojection baselines.

The power flow application demonstrates practical relevance. AC optimal power flow problems are notoriously difficult—non-convex and computationally expensive—making them ideal testbeds for neural network acceleration. By introducing physics-motivated constraint ordering, MResOpt enables iterates to remain on the equality manifold (representing physical balance equations) while satisfying inequality constraints. This division of labor between constraint types mirrors how domain experts solve these problems manually.

The work positions neural networks as solvers for critical infrastructure optimization, where computational efficiency and constraint satisfaction reliability both matter. The framework's modularity suggests applicability beyond power systems to logistics, control, and resource allocation problems where hierarchical constraint structures naturally arise. However, the infinite-width theoretical guarantees may not transfer to practical finite networks, leaving questions about real-world robustness.

• →MResOpt uses staged architecture to hierarchically satisfy constraints in optimization, improving reliability on high-priority violations.
• →Theoretical analysis shows the design behaves as sequential Gaussian Process regression under idealized conditions.
• →Demonstrates substantial improvements on AC optimal power flow, a critical application for electrical grid operations.
• →Physics-informed constraint ordering enables neural networks to maintain equality constraints while satisfying inequalities.
• →Framework generalizes across QP, QCQP, and SOCP problem classes with consistent performance gains over baselines.