Optimization, Learning, and Games with Predictable Sequences

We provide several applications of Optimistic Mirror Descent, an online learning algorithm based on the idea of predictable sequences. First, we recover the Mirror Prox algorithm for offline optimization, prove an extension to Holder-smooth functions, and apply the results to saddle-point type problems. Next, we prove that a version of Optimistic Mirror Descent (which has a close relation to the Exponential Weights algorithm) can be used by two strongly-uncoupled players in a finite zero-sum matrix game to converge to the minimax equilibrium at the rate of O((log T)/T). This addresses a question of Daskalakis et al 2011. Further, we consider a partial information version of the problem. We then apply the results to convex programming and exhibit a simple algorithm for the approximate Max Flow problem.

💡 Research Summary

This paper studies Optimistic Mirror Descent (OMD), an online learning algorithm that exploits “predictable sequences” – auxiliary vectors Mₜ that can be computed before the loss gradient of round t is revealed. The authors demonstrate that OMD unifies and extends several important methods across optimization, game theory, and convex programming.

First, they show that when the predictable vector is chosen as the previous gradient, Mₜ = ∇G(gₜ₋₁), OMD reduces exactly to Nemirovski’s Mirror Prox algorithm. Using the standard OMD regret bound (Lemma 1) together with smoothness of G, they recover the classic O(1/T) convergence rate of Mirror Prox for Lipschitz‑smooth functions. They then generalize this analysis to α‑Hölder smooth objectives, proving that the average iterate satisfies G( \bar f_T ) − min G ≤ 8 H R^{1+α} T^{-(1+α)/(2+α)} (Lemma 3). This interpolates between the O(1/√T) rate (α = 0, no predictability) and the O(1/T) rate (α = 1, fully predictable gradients).

Next, the paper tackles structured saddle‑point problems of the form G(f)=supₓ φ(f,x) where φ is convex in f and concave in x. By letting two players run OMD on their respective variables (with possibly different regularizers R₁,R₂ and step sizes), the authors obtain a coupled regret bound (Lemma 4) where the positive regret terms of one player cancel the negative terms of the other. Under Hölder‑smoothness assumptions on φ, Corollary 5 shows that the saddle‑point error decays as O(T^{-(1+γ)/(2+γ)}), where γ is the smallest Hölder exponent among the four partial gradients of φ.

The most striking contribution concerns zero‑sum matrix games. The authors answer an open question by Daskalakis et al. (2011): can a “simple” uncoupled algorithm achieve O(log T / T) convergence to the minimax equilibrium while retaining the usual O(√T) regret against arbitrary opponents? They propose a variant of OMD based on the negative entropy regularizer (i.e., Exponential Weights) with two key modifications: (i) the most recent loss vector receives a factor‑2 weight, and (ii) a tiny uniform mixture (β = 1/√T) is added to prevent any probability from becoming too small. When both players follow this protocol, the average strategies ( \bar f_T, \bar x_T ) form an O((log n+log m)/T)‑approximate equilibrium (Proposition 6). If only one player uses the algorithm, its regret against any opponent is O(log n · √T), matching the standard bound, and it even yields intermediate rates when the opponent’s actions are “stable”.

The authors also treat a partial‑information setting where each player observes only the scalar payoff fₜᵀA xₜ rather than the full loss vectors. By using the same OMD framework with Mₜ set to the previous scalar payoff, they retain the O(log T / T) convergence guarantee, showing that the method is robust to limited feedback.

Finally, the paper applies the predictable‑sequence idea to convex programming, specifically to the approximate maximum‑flow problem. By formulating max‑flow as a linear program with convex constraints and running OMD on the dual variables, they obtain an ε‑approximation in time \tilde O(d^{3/2}/ε), where d is the number of edges. This matches the best known bound (previously achieved via a more involved “excessive gap” technique) but with a far simpler algorithmic structure.

Overall, the work presents a unifying perspective: predictable sequences allow OMD to adapt to the underlying structure of the problem, yielding faster rates for smooth optimization, accelerated convergence in saddle‑point and game settings, and efficient algorithms for classic combinatorial problems. The results bridge online learning theory with practical algorithm design, offering both theoretical insights and concrete, implementable methods.