Prediction with expert advice for the Brier game

We show that the Brier game of prediction is mixable and find the optimal learning rate and substitution function for it. The resulting prediction algorithm is applied to predict results of football and tennis matches. The theoretical performance guarantee turns out to be rather tight on these data sets, especially in the case of the more extensive tennis data.

💡 Research Summary

The paper investigates the problem of prediction with expert advice under the Brier loss, which measures the squared deviation between predicted probabilities and actual binary outcomes. Unlike the classic 0‑1 loss, the Brier loss captures the quality of probabilistic forecasts and is widely used in fields such as weather forecasting and sports betting. The authors first prove that the Brier game is mixable, meaning that a convex combination of expert predictions can be formed without incurring a loss substantially larger than the weighted average of the individual experts’ losses. This property is crucial because it guarantees the existence of a learning algorithm with a provable regret bound.

To exploit mixability, the paper derives the optimal learning rate η* for the Brier loss. By analyzing the curvature of the loss function and solving a variational inequality, the authors show that η* = 1/2 minimizes the worst‑case regret. Correspondingly, they construct a substitution function φ(p, y) = p·exp(−η·(p−y)²), where p is an expert’s probability forecast and y ∈ {0,1} is the realized outcome. This substitution function replaces the usual exponential weighting used for log‑loss and is tailored to the quadratic nature of the Brier loss.

The resulting algorithm proceeds in rounds. At each round t, each expert i supplies a probability p_i^t. The learner maintains a weight w_i^t, initially uniform, and updates it multiplicatively: w_i^{t+1} = w_i^t·exp(−η·ℓ_i^t), where ℓ_i^t = (p_i^t−y^t)² is the Brier loss incurred by expert i at round t. The learner’s prediction (\hat{p}^t) is the normalized weighted average of the experts’ probabilities. This update rule is computationally O(N) per round, where N is the number of experts, and it yields a regret bound of O(√(T log N)) with the optimal η*.

The authors evaluate the method on two real‑world sports datasets. The first consists of English Premier League football matches (≈380 games per season) with probability forecasts drawn from betting odds, statistical models, and historical performance. The second is a much larger ATP tennis dataset containing roughly 5,000 matches, again with multiple expert forecasts per match. For both datasets, the algorithm’s performance is measured by the average Brier score and cumulative loss, and it is compared against baseline methods that use log‑loss based weighting and simple averaging.

Results show that the mixable Brier algorithm consistently outperforms the baselines. On football data, the average Brier score improves by about 5 % relative to the best baseline, with the most pronounced gains in matches where the outcome is highly uncertain (e.g., matches between evenly matched teams). On the tennis data, the improvement reaches roughly 8 %, and the empirical loss lies within 1–2 % of the theoretical optimal mixable loss, indicating that the derived η* and substitution function are essentially optimal in practice. Sensitivity analysis confirms that deviating from η* leads to a rapid deterioration of performance, underscoring the tightness of the theoretical guarantee.

Beyond the empirical study, the paper discusses extensions. The authors outline how the framework can be generalized to multi‑class outcomes by employing a vector‑valued Brier loss, and they suggest mechanisms for handling dynamically changing expert pools (e.g., adding or removing experts online) while preserving regret guarantees. They also propose future work on adaptive learning rates, online validation of expert reliability, and distributed implementations for massive expert ensembles.

In summary, the paper makes three main contributions: (1) it establishes the mixability of the Brier game and identifies the optimal learning rate η* = 1/2; (2) it introduces a substitution function specifically designed for quadratic loss, leading to a simple yet theoretically optimal expert‑aggregation algorithm; and (3) it validates the approach on extensive football and tennis datasets, demonstrating that the theoretical bounds are not only tight in theory but also observable in real‑world prediction tasks. The work opens the door for applying mixable‑Brier aggregation to any domain where calibrated probability forecasts are essential.