On the Universality of Online Mirror Descent

We show that for a general class of convex online learning problems, Mirror Descent can always achieve a (nearly) optimal regret guarantee.

💡 Research Summary

The paper establishes a universal optimality result for Online Mirror Descent (OMD) in the broad class of convex online learning problems. The authors consider a setting where a learner repeatedly selects a predictor w_t from a closed convex set W (a subset of a Banach space B) and then suffers a convex loss f_t drawn from a class F. The sub‑gradients of all losses are assumed to lie in a convex, centrally‑symmetric set X ⊂ B*, which may be completely unrelated to W (i.e., W and X need not be dual balls). The performance measure is the average regret, and the value of the game V_n(W,X) is defined as the smallest worst‑case regret achievable by any algorithm against an adversarial sequence of losses.

The authors first generalize the standard analysis of Mirror Descent to this “non‑dual” situation. They introduce the notion of q‑uniform convexity of a regularizer Ψ with respect to the norm induced by X*. Lemma 2 shows that if Ψ is q‑uniformly convex on W and the step size η is chosen as (sup_{w∈W}Ψ(w)/n)^{1/p} (with p = q/(q‑1)), then for any sequence of losses whose sub‑gradients have norm at most one, OMD guarantees regret ≤ 2·(sup_{w∈W}Ψ(w))^{1/q}. This yields the quantities MD_p (the best regret constant achievable by OMD) and D_p (the optimal uniform‑convexity constant), and establishes V_p ≤ MD_p ≤ 2·D_p.

Next, the paper extends the classical concept of martingale type of a Banach space to a pair (W*,X). Definition 2 says that (W*,X) has martingale type p if, for every martingale difference sequence (x_n) taking values in X and any initial vector x_0, the p‑th moment of the W*‑norm of the cumulative sum is bounded by a constant C_p times the p‑th moment of the X‑norms. Theorem 4 (from prior work) relates V_n(W,X) to the supremum of such martingale expressions, and Lemma 5 shows that a sub‑linear decay of V_n (specifically V_n ≤ D·n^{-(1‑1/r)}) implies that (W*,X) possesses martingale type p for all p < r, with explicit constants. Corollary 6 translates this into a quantitative bound C_{p’} ≤ 1104·V_p·(p‑p’)^{-2}.

The crucial bridge between martingale type and uniform convexity is built in Lemma 7. It proves that if (W*,X) has martingale type p, then there exists a convex function Ψ that is q‑uniformly convex with respect to the X*‑norm (where q = p/(p‑1)) and satisfies 1/q·‖w‖{X*}^q ≤ Ψ(w) ≤ C_q·‖w‖{W}^q. The construction uses a dual formulation: Ψ*q(x) = sup{(x_n)}