Select-then-differentiate: Solving Bilevel Optimization with Manifold Lower-level Solution Sets

Researchers present HG-MS, a novel bilevel optimization method that handles cases where lower-level problems have multiple solutions along a manifold rather than a single optimum. The work provides theoretical guarantees for convergence while maintaining computational efficiency through pseudoinverse-based calculations, with practical applications demonstrated in LLM fine-tuning.

This research addresses a fundamental challenge in bilevel optimization—a mathematical framework increasingly relevant to machine learning and hyperparameter tuning. Traditional bilevel optimization assumes a unique lower-level solution, but many real-world problems exhibit multiple optimal solutions forming continuous manifolds. The authors prove that differentiability of the hyper-objective doesn't require a single solution; instead, uniqueness of the optimistic selection suffices, enabling practical computation through explicit pseudoinverse formulas.

Bilevel optimization underpins critical AI applications including meta-learning, hyperparameter optimization, and adversarial training. The theoretical contribution extends classical results by characterizing when the hyper-objective maintains smoothness properties despite manifold non-uniqueness, establishing conditions for Hölder regularity and identifying failure modes. This theoretical clarity addresses gaps in understanding when gradient-based methods can reliably optimize upper-level objectives.

The HG-MS algorithm demonstrates that computational complexity depends on the intrinsic dimensionality of the solution manifold rather than ambient dimension—a crucial insight for high-dimensional problems. Empirical validation on LLM source reweighting shows competitive performance on standardized benchmarks, suggesting practical viability beyond theoretical interest.

This work matters for AI researchers developing more sophisticated training procedures and for practitioners optimizing complex nested objectives where solutions aren't naturally unique. The intersection of manifold theory with bilevel optimization opens avenues for understanding and improving hyperparameter learning, particularly as models grow more complex and solution landscapes become less convex.

• →Bilevel optimization can handle non-unique lower-level solutions if the optimistic selection is unique, enabling practical hyper-gradient computation
• →Solution manifold intrinsic dimension governs convergence complexity rather than ambient dimension, improving scalability
• →Theoretical conditions establish when the hyper-objective maintains smoothness despite manifold non-convexity
• →HG-MS method achieves competitive LLM fine-tuning results while respecting select-then-differentiate principles
• →Framework extends classical bilevel optimization theory to realistic settings with multiple optimal solutions