Slow Learners are Fast

Online learning algorithms have impressive convergence properties when it comes to risk minimization and convex games on very large problems. However, they are inherently sequential in their design which prevents them from taking advantage of modern multi-core architectures. In this paper we prove that online learning with delayed updates converges well, thereby facilitating parallel online learning.

💡 Research Summary

The paper addresses a fundamental tension in modern machine learning: online learning algorithms enjoy strong theoretical guarantees for risk minimization and convex games, yet their classic designs are inherently sequential, preventing efficient exploitation of today’s multi‑core and distributed hardware. The authors propose and rigorously analyze a “delayed‑update” variant of online learning, showing that allowing updates to be applied after a bounded or stochastic lag does not destroy the algorithm’s convergence properties.

First, the authors formalize the delayed model. At iteration t the learner receives a loss function ℓ_t and computes a sub‑gradient g_t, but the parameter vector w is updated only using a sub‑gradient from τ steps earlier: w_{t+1}=w_t−η_t g_{t−τ}. Under the standard assumptions that each loss is convex and L‑Lipschitz, and that the delay τ is either a known constant or has a finite expectation, they derive a regret bound of the form

R_T ≤ (D²)/(2η) + η L² T + L D τ,

where D is the diameter of the feasible set and η is the learning rate. By choosing η≈D/(L√T) the bound simplifies to O(√T + τ). Consequently, even with a non‑zero delay, the average regret converges to zero at the same √T rate as the classic, instantaneous‑update algorithm.

The analysis is then extended to stochastic delays. Assuming τ_t are independent random variables with E