Online Learning: Beyond Regret

We study online learnability of a wide class of problems, extending the results of (Rakhlin, Sridharan, Tewari, 2010) to general notions of performance measure well beyond external regret. Our framework simultaneously captures such well-known notions as internal and general Phi-regret, learning with non-additive global cost functions, Blackwell’s approachability, calibration of forecasters, adaptive regret, and more. We show that learnability in all these situations is due to control of the same three quantities: a martingale convergence term, a term describing the ability to perform well if future is known, and a generalization of sequential Rademacher complexity, studied in (Rakhlin, Sridharan, Tewari, 2010). Since we directly study complexity of the problem instead of focusing on efficient algorithms, we are able to improve and extend many known results which have been previously derived via an algorithmic construction.

💡 Research Summary

The paper presents a unifying theoretical framework for online learning that goes far beyond the traditional notion of external regret. Building on the sequential Rademacher complexity introduced by Rakhlin, Sridharan, and Tewari (2010), the authors identify three fundamental quantities that together determine whether a learning problem is “learnable” under any performance measure: (1) a martingale convergence term that guarantees the average of the observed loss sequence behaves like a zero‑mean martingale, (2) a term that captures the ability to perform optimally if the future loss sequence were known (an oracle‑optimality term), and (3) a generalized sequential Rademacher complexity that measures the richness of the hypothesis class in an adaptive, time‑varying setting.

The central theorem states that if all three quantities are uniformly bounded (or converge to zero at an appropriate rate), then for any performance metric—whether it is internal regret, Φ‑regret, Blackwell approachability, calibration error, adaptive regret, or any non‑additive global cost—there exists an online strategy whose average excess loss relative to the best benchmark vanishes as the horizon grows. Importantly, the result is existential: it does not construct an explicit algorithm but proves that a learning algorithm must exist whenever the complexity measures are controlled.

To illustrate the power of the framework, the authors systematically map a wide variety of previously studied problems onto the three‑term decomposition. Internal regret is shown to be a special case of Φ‑regret where the transformation Φ swaps actions; when Φ is Lipschitz, the sequential Rademacher term behaves exactly as in the external‑regret setting. Blackwell approachability, which concerns steering a vector‑valued loss into a convex target set, reduces to showing that the martingale term and the complexity term vanish for the associated loss vectors. Calibration of probabilistic forecasters is modeled as a global cost that penalizes the discrepancy between predicted probabilities and empirical frequencies; the same three‑term analysis yields a calibration guarantee without constructing a specific calibrated algorithm. Adaptive regret, which requires low regret on every contiguous sub‑interval, is handled by partitioning the horizon and bounding the martingale and complexity contributions on each segment, leading to tighter bounds than those obtained by algorithmic constructions.

A notable methodological contribution is the shift from algorithm‑centric proofs to a complexity‑centric perspective. Traditional online learning literature often designs a concrete update rule (e.g., Follow‑the‑Regularized‑Leader, Mirror Descent) and then proves regret bounds for that rule. In contrast, this work first asks whether the problem class itself possesses low sequential complexity and favorable martingale properties; if so, learnability follows automatically. This approach not only recovers all known regret bounds as corollaries but also extends them to settings where algorithmic techniques are currently lacking, such as non‑additive global loss functions or multi‑objective criteria.

The paper also discusses the implications for algorithm design. Since the existence of a learner is guaranteed once the three quantities are bounded, future work can focus on constructing computationally efficient algorithms that achieve the theoretical limits identified here. Moreover, the framework suggests new avenues for research: exploring tighter estimates of sequential Rademacher complexity for specific function classes, developing martingale concentration tools tailored to online environments, and investigating how additional structural assumptions (e.g., curvature, strong convexity) affect the three‑term decomposition.

In summary, the authors provide a comprehensive, unified theory of online learnability that subsumes a broad spectrum of performance measures. By reducing all such measures to the control of a martingale convergence term, an oracle‑optimality term, and a sequential Rademacher complexity term, they demonstrate that the essence of online learning lies in the intrinsic complexity of the problem rather than in the specifics of any particular algorithm. This insight both clarifies the relationships among many previously disparate results and opens the door to new, more general learning guarantees in online settings.