Differential estimates for fast first-order multilevel nonconvex optimisation

With a view on bilevel and PDE-constrained optimisation, we develop iterative estimates $\widetilde{F’}(x^k)$ of $F’(x^k)$ for composite functions $F :=J \circ S$, where $S$ is the solution mapping of the inner optimisation problem or PDE. The idea is to form a single-loop method by interweaving updates of the iterate $x^k$ by an outer optimisation method, with updates of the estimate by single steps of standard optimisation methods and linear system solvers. When the inner methods satisfy simple tracking inequalities, the differential estimates can almost directly be employed in standard convergence proofs for general forward-backward type methods. We adapt those proofs to a general inexact setting in normed spaces, that, besides our differential estimates, also covers mismatched adjoints and unreachable optimality conditions in measure spaces. As a side product of these efforts, we provide improved convergence results for nonconvex Primal-Dual Proximal Splitting (PDPS).

💡 Research Summary

The paper addresses the computational bottleneck inherent in bilevel and PDE‑constrained optimization, where each outer iteration traditionally requires an exact solution of an inner problem (or a PDE) and its adjoint. The authors propose a unified single‑loop framework that simultaneously updates the outer variable, an inner‑problem iterate, and an adjoint estimate, thereby avoiding costly nested solves.

Formally, the objective is a composite function F(x)=J(S(x)) with S:X→U being the solution map of an inner optimization problem or a PDE defined implicitly by T(u,x)=0. The key contribution is the construction of an inexact differential estimator
\