On Theorem 2.3 in "Prediction, Learning, and Games" by Cesa-Bianchi and Lugosi

The note presents a modified proof of a loss bound for the exponentially weighted average forecaster with time-varying potential. The regret term of the algorithm is upper-bounded by sqrt{n ln(N)} (uniformly in n), where N is the number of experts and n is the number of steps.

💡 Research Summary

The paper revisits Theorem 2.3 from Cesa‑Bianchi and Lugosi’s monograph “Prediction, Learning, and Games,” which is a cornerstone result for the exponentially weighted average (EWA) forecaster. The original theorem states that, for N experts and any horizon n, the regret of the EWA algorithm is bounded by O(√(n ln N)). The classic proof assumes a fixed learning rate η that must be tuned to the (unknown) horizon n, which limits its applicability in truly online settings where n is not known in advance.

To overcome this limitation, the authors propose a modified analysis that employs a time‑varying learning rate ηₜ = √(8 ln N / t) and a corresponding time‑varying potential function
Φₜ = ∑{i=1}^N exp(−∑{s=1}^{t‑1} η_s ℓ_{s,i}),
where ℓ_{s,i} denotes the loss suffered by expert i at round s. The key technical steps are as follows:

1. Log‑Potential Transformation – By taking the logarithm of Φₜ, the multiplicative weight updates become additive, which simplifies the analysis and makes it amenable to concentration inequalities.

2. Pointwise Inequality – Using Hoeffding’s Lemma, the authors show that for each round t,
   ln Φ_{t+1} − ln Φ_t ≤ ηₜ · ℓ_t + ηₜ²/8,
   where ℓ_t is the loss incurred by the algorithm’s own prediction.

3. Summation Over Rounds – Summing the above inequality from t = 1 to n yields
   ln Φ_{n+1} − ln Φ_1 ≤ ∑{t=1}^n ηₜ ℓ_t + ∑{t=1}^n ηₜ²/8.
   Since Φ_1 = N (all experts start with equal weight), ln Φ_1 = ln N.

4. Link to the Best Expert – By definition, Φ_{n+1} ≥ exp(−∑{t=1}^n ηₜ ℓ{t,i*}) for the best expert i*. Taking logs and rearranging gives
   ∑{t=1}^n ηₜ ℓ_t − ∑{t=1}^n ηₜ ℓ_{t,i*} ≤ ln N + ∑_{t=1}^n ηₜ²/8.

5. Explicit Summations – With ηₜ = √(8 ln N / t), we have
   ∑{t=1}^n ηₜ = √{8 ln N} · ∑{t=1}^n 1/√t ≤ 2 √{n ln N},
   and
   ∑{t=1}^n ηₜ² = 8 ln N · ∑{t=1}^n 1/t ≤ 8 ln N · (1 + ln n).

6. Regret Bound – Substituting these expressions and dividing by the smallest ηₜ (which is η_n) yields the final regret bound:
   Regret = L_A − L_* ≤ √{n ln N} + O(ln n).
   The additive O(ln n) term is dominated by the √{n ln N} term for all n ≥ 2, so the bound can be succinctly written as Regret ≤ √{n ln N} uniformly in n.

Two methodological innovations distinguish this proof from the classical one. First, the log‑potential technique converts multiplicative weight updates into additive quantities, allowing a clean application of Hoeffding’s Lemma without extra approximations. Second, the time‑varying learning rate eliminates the need for prior knowledge of the horizon; the algorithm automatically adapts its aggressiveness, being more responsive early on (large ηₜ) and more conservative later (small ηₜ).

The authors also address practical concerns. Direct computation of Φₜ can overflow for large N or long horizons, so they recommend maintaining ln Φₜ instead of Φₜ itself. This log‑space implementation is numerically stable and aligns naturally with the theoretical analysis.

Beyond the immediate result, the paper suggests that the same framework can be extended to other online learning schemes that rely on potential functions, such as adaptive Hedge variants (AdaHedge, FlipFlop) and even certain bandit algorithms where a similar exponential weighting is employed. By showing that a simple modification—making the potential time‑dependent—preserves the optimal √{n ln N} regret without horizon knowledge, the work provides a robust tool for designing truly online algorithms.

In summary, the note delivers a concise yet rigorous alternative proof of Theorem 2.3, clarifies the role of a time‑varying potential, and demonstrates that the classic √{n ln N} regret bound holds uniformly for any horizon. This contribution strengthens the theoretical foundations of the EWA forecaster and broadens its applicability to realistic, horizon‑agnostic online prediction scenarios.