Achieving Better Local Regret Bound for Online Non-Convex Bilevel Optimization

Online bilevel optimization (OBO) has emerged as a powerful framework for many machine learning problems. Prior works have developed several algorithms that minimize the standard bilevel local regret or the window-averaged bilevel local regret of the OBO problem, but the optimality of existing regret bounds remains unclear. In this work, we establish optimal regret bounds for both settings. For standard bilevel local regret, we propose an algorithm that achieves the optimal regret $Ω(1+V_T)$ with at most $O(T\log T)$ total inner-level gradient evaluations. We further develop a fully single-loop algorithm whose regret bound includes an additional gradient-variation terms. For the window-averaged bilevel local regret, we design an algorithm that captures sublinear environmental variation through a window-based analysis and achieves the optimal regret $Ω(T/W^2)$. Experiments validate our theoretical findings and demonstrate the practical effectiveness of the proposed methods.

💡 Research Summary

This paper studies online bilevel optimization (OBO), a framework where at each round t the learner must choose a decision xₜ in a feasible set X while an inner problem determines y*ₜ(xₜ) = argmin_y gₜ(xₜ, y). The outer objective fₜ(x, y) is smooth but possibly non‑convex, whereas the inner function gₜ is μ_g‑strongly convex in y, guaranteeing a unique inner solution. Existing OBO methods rely on hyper‑gradient approximations via Approximate Implicit Differentiation (AID) or Iterative Differentiation (ITD) and evaluate performance using two notions of local regret: (i) standard bilevel local regret and (ii) window‑averaged bilevel local regret. While several algorithms have been proposed, it was unclear whether their regret bounds are optimal.

The authors make three major contributions. First, they introduce an adaptive‑inner‑loop algorithm (AOBO) that dynamically determines how many inner‑level gradient steps to take based on the current norm of ∇y gₜ. By stopping when the gradient norm falls below a tolerance δ, AOBO ensures that the inner iterate yₜ₊₁ is sufficiently close to yₜ(xₜ). The hyper‑gradient is then computed using AID with a fixed number M of projected gradient steps for solving the linear system that defines vₜ(x). With δ set to 1/√T, AOBO achieves a standard regret bound Reg(T) ≤ O(1 + V_T), where V_T = ∑{t=2}^T sup_x |F_t(x) − F_{t‑1}(x)| captures environmental variation. This matches the known lower bound Ω(1 + V_T), establishing optimality. The total number of inner‑level gradient evaluations is O(T log T), a modest overhead compared with earlier methods that required a fixed O(log T) inner steps per round but could not adapt to changing inner functions.

Second, the paper proposes a fully single‑loop algorithm (FSOBO) that performs exactly one inner‑level gradient step and one projected gradient step for the linear system at each round. Although this simplification introduces additional error terms, the authors control them by carefully choosing step sizes α, β, γ and by leveraging new variation measures H_{2,T}, E_{2,T}, P_T that quantify the drift of the inner optimum, the change in the Hessian inverse, and the change in the outer gradient, respectively. The resulting regret bound is Reg(T) ≤ O(1 + V_T + H_{2,T} + E_{2,T} + P_T) with only O(T) gradient queries, making the method highly practical for large‑scale or resource‑constrained settings.

Third, the authors address the window‑averaged regret, defined as Reg_W(T) = ∑{t=1}^T ‖G_X(x_t, (1/W)∑{i=0}^{W‑1}∇f_{t‑i}(·), γ)‖². They design the Window‑averaged Online Bilevel Optimizer (WOBO) that averages recent hyper‑gradients within a sliding window of size W. By analyzing the variation inside each window, they prove an upper bound O(T/W²) and a matching lower bound Ω(T/W²), showing that WOBO is optimal for this performance metric. Importantly, this result does not rely on sublinear environmental variation assumptions that were required in prior work.

The theoretical developments are complemented by extensive experiments on synthetic data, meta‑learning tasks (e.g., MAML), and hyper‑parameter tuning scenarios. AOBO consistently outperforms prior algorithms such as SOBO, OBBO, and SOGD, achieving lower regret while using fewer inner‑level gradient evaluations thanks to its adaptive stopping criterion. FSOBO, despite its single‑loop simplicity, attains comparable regret levels, confirming the practicality of the proposed trade‑off. WOBO demonstrates the predicted O(T/W²) behavior across different window sizes, whereas competing methods suffer performance degradation as W increases.

In summary, the paper delivers (1) optimal regret bounds for both standard and window‑averaged bilevel local regret, (2) novel adaptive and single‑loop algorithms that achieve these bounds with provably low computational cost, and (3) a thorough empirical validation. The work closes a gap in the OBO literature regarding the optimality of regret guarantees and opens avenues for future research on stochastic settings, multi‑level extensions, and distributed implementations.