Hessian-Free Distributed Bilevel Optimization via Penalization with Time-Scale Separation

This paper considers a class of distributed bilevel optimization (DBO) problems with a coupled inner-level subproblem. Existing approaches typically rely on hypergradient estimations involving computationally expensive Hessian evaluation. To address this, we approximate the DBO problem as a minimax problem by properly designing a penalty term that enforces both the constraint imposed by the inner-level subproblem and the consensus among the decision variables of agents. Moreover, we propose a loopless distributed algorithm, AHEAD, that employs multiple-timescale updates to solve the approximate problem asymptotically without requiring Hessian computation. Theoretically, we establish sharp convergence rates for nonconvex-strongly-convex settings and for distributed minimax problems as special cases. Our analysis reveals a clear dependence of convergence performance on node heterogeneity, penalty parameters, and network connectivity, with a weaker assumption on heterogeneity that only requires bounded gradients at the optimum. Numerical experiments corroborate our theoretical results.

💡 Research Summary

This paper tackles the challenging class of distributed bilevel optimization (DBO) problems in which multiple agents must jointly minimize an outer‑level objective that depends on the solution of a coupled inner‑level subproblem. Existing distributed methods typically estimate the hypergradient ∇Φ(x) by computing Hessian‑vector products, which is computationally intensive and communication‑heavy in large networks.

The authors propose a fundamentally different approach: they reformulate the original bilevel problem as a single‑level minimax problem by introducing a penalty term that simultaneously enforces the inner‑level optimality condition and consensus among agents. Specifically, each node i maintains local copies of the outer variable x_i, the inner variable y_i, and an auxiliary variable z_i that serves as a proxy for the true inner‑level solution y*(x). The penalty λ·(g_i(x_i,y_i)−g_i(x_i,z_i)) forces y_i and z_i to share the same inner objective value, while additional consensus penalties weighted by α, β, and γ penalize disagreements in x, y, and z across neighboring nodes.

Based on this penalized formulation, the authors design a loop‑less distributed algorithm called AHEAD (Algorithmic Hessian‑free Distributed bilevel optimization via penalization with Time‑scale separation). AHEAD updates x_i, y_i, and z_i in a single pass at each iteration, using only first‑order gradient information of f_i and g_i. The “multiple‑timescale” design chooses α, β, γ so that the updates for the auxiliary variable z (which approximates the inner solution) evolve on a faster timescale than the updates for y, which in turn evolve faster than those for x. This separation guarantees that the inner‑level approximation converges quickly enough to guide the outer‑level descent without requiring nested loops.

The theoretical contribution is a detailed convergence analysis under standard smoothness assumptions (L‑smoothness of f_i and g_i, μ‑strong convexity of g_i in y) and a doubly‑stochastic communication matrix W. The analysis explicitly quantifies how three key factors affect the convergence rate:

1. Penalty parameter λ – larger λ reduces the violation of the inner‑level constraint, yielding an O(1/λ) bound on the approximation error between z_i and the true y*(x).
2. Network connectivity ρ – defined as the spectral norm of W−(1/m)11ᵀ; smaller ρ (i.e., better connected graphs) improves consensus error, appearing as (1−ρ)⁻² in the rate.
3. Node heterogeneity – captured by b_f² and b_g², which bound the variance of outer‑level gradients and inner‑level gradients across nodes. The analysis requires only first‑order heterogeneity (bounded gradients at the optimum), a weaker condition than prior works that needed second‑order heterogeneity.

For the nonconvex‑strongly‑convex setting (nonconvex outer, strongly convex inner), the authors prove that after K iterations AHEAD achieves an error bound of order

O( κ⁴ K^{1/3} + (1−ρ)⁻² ( κ² b_f² K^{2/3} + κ⁴ b_g² K^{1/3} ) ),

where κ = L_g/μ_g is the condition number of the inner problem. This rate reveals a clear dependence on the condition number, heterogeneity, and network topology.

When the problem reduces to a distributed minimax formulation (inner maximization only), the bound simplifies to

O( κ² K^{2/3} + κ² b_f² (1−ρ)⁻² K^{2/3} ),

which improves upon existing distributed minimax algorithms that typically achieve O(K^{1/2}) or worse.

The paper also includes extensive experiments on synthetic and real‑world tasks, such as energy‑price optimization, meta‑learning, and adversarial games. AHEAD consistently outperforms Hessian‑based bilevel methods, recent loop‑less bilevel algorithms, and state‑of‑the‑art distributed minimax solvers in terms of (i) total gradient evaluations needed to reach a target accuracy, (ii) communication overhead, and (iii) robustness to different graph structures (complete, ring, random).

In summary, the work introduces a Hessian‑free, loop‑less algorithm for distributed bilevel optimization that leverages a carefully designed penalty and multiple‑timescale updates. It provides the first convergence guarantees that explicitly incorporate network connectivity, condition number, and first‑order heterogeneity, and it demonstrates superior empirical performance. The framework opens avenues for further extensions to asynchronous settings, stochastic sampling, and multi‑objective bilevel problems.