Kelly Betting as Bayesian Model Evaluation: A Framework for Time-Updating Probabilistic Forecasts

This paper proposes a new way of evaluating the accuracy and validity of probabilistic forecasts that change over time (such as an in-game win probability model, or an election forecast). Under this approach, each model to be evaluated is treated as a canonical Kelly bettor, and the models are pitted against each other in an iterative betting contest. The growth or decline of each model’s bankroll serves as the evaluation metric. Under this approach, market consensus probabilities and implied model credibilities can be updated real time as each model updates, and do not require one to wait for the final outcome. Using a simulation model, it will be shown that this method is in general more accurate than traditional average log-loss and Brier score methods at distinguishing a correct model from an incorrect model. This Kelly approach is shown to have a direct mathematical and conceptual analogue to Bayesian inference, with bankroll serving as a proxy for Bayesian credibility.

💡 Research Summary

The paper introduces a novel framework for evaluating probabilistic forecasts that evolve over time, such as in‑game win‑probability models or election forecasts. Traditional evaluation metrics—log‑loss, Brier score, calibration tests—require waiting until the final outcome and ignore the order in which predictions are updated. The authors propose to treat each forecasting model as a canonical Kelly bettor and to pit the models against each other in a continuous betting contest.

The core idea is to use the Kelly criterion, which prescribes the optimal fraction of a bettor’s bankroll to wager given an estimated probability p and offered odds o. The classic formula f = (p − (1‑p)/o) is extended to accommodate existing positions (win shares w) by f = p − (1‑p)/o · 1/(1 + w/b). Here w represents the net amount a bettor would win (or lose) if the event occurs, and b is the current bankroll. This generalization allows the betting process to be iterative: after each round of information, each model updates its probability estimate, the market clearing probability is recomputed, and the optimal bet size is recalculated.

The market clearing probability (the “consensus” probability) is derived as

 market probability = ∑ p_i b_i / (1 − ∑ p_i w_i)

where the sums run over all n bettors. This expression reduces to the bankroll‑weighted average of the participants’ forecasts when no prior bets exist, a result previously reported by Beygelzimer et al. (2012). The denominator term corrects for existing win‑share exposures, ensuring that the consensus reflects both current beliefs and outstanding positions.

With the consensus probability in hand, the algorithm proceeds through six steps: (1) compute the market clearing probability; (2) translate it into odds and apply the generalized Kelly formula to obtain each model’s bet fraction; (3) optionally compute each model’s “marked‑to‑market” bankroll, which serves as a real‑time proxy for Bayesian credibility; (4) update bankrolls and win shares based on the placed bets; (5) repeat as new information arrives; (6) settle all bets once the event resolves, distributing win shares accordingly.

The authors illustrate the method with a simple basketball example involving two forecasters, Bob and Alice, who update their win‑probability estimates each quarter. By following the Kelly‑based betting process, Bob’s bankroll declines because he consistently lags Alice’s updates, while Alice’s bankroll grows. Notably, the final outcome of the game does not affect the relative bankroll changes; the dynamics are driven entirely by the timing and magnitude of probability updates.

The framework is then generalized to multinomial outcomes (e.g., win/lose/draw). Instead of separating “bankroll” and “bet,” each model’s position is expressed as a vector of win‑shares w_i for each possible outcome i. The updated win‑share after a betting round follows

 w′_i = p_i · m_i · (∑ m_j w_j) / (∑ m_j)

where m_i is the market‑implied probability for outcome i. This formula extends Kelly’s original result (which assumes fair odds) to the case where participants already hold positions. The market clearing odds for multiple outcomes and multiple bettors are obtained by solving an eigenvector problem: (p w^T) m = m, where the matrix p w^T aggregates each bettor’s probability estimates and win‑share holdings.

A simulation study compares the Kelly‑based evaluation to average log‑loss and Brier scores. Two synthetic models, one that quickly converges to the true probability and another that does so later, are generated. While traditional scores assign similar performance to both models (because they average over all time points), the Kelly‑based metric clearly rewards the early correct model by increasing its bankroll and penalizes the lagging model. This demonstrates superior discriminative power for identifying more skillful forecasters in a time‑updating context.

Key advantages of the proposed approach include:

1. Real‑time assessment – model performance can be tracked continuously without waiting for the final event.
2. Intuitive interpretation – bankroll growth or decline maps directly onto a tangible notion of “money won or lost,” making the metric accessible to non‑technical audiences.
3. Bayesian correspondence – treating bankroll as a proxy for posterior credibility aligns the method with Bayesian model comparison, allowing prior beliefs (equal bankrolls) to be updated by observed betting outcomes.

Limitations are also acknowledged. The method assumes a liquid betting market; in practice, virtual bankrolls must be chosen, and results can be sensitive to the initial bankroll size. Computational complexity grows with the number of outcomes and participants because of matrix operations required for market clearing. Finally, the framework presumes independence among models; correlated forecasts could inflate or deflate bankroll changes, suggesting a need for extensions that model forecast dependence.

In summary, the paper offers a compelling synthesis of Kelly betting theory and Bayesian inference to create a dynamic, interpretable, and theoretically sound metric for evaluating time‑varying probabilistic forecasts. By converting forecast updates into a betting contest, it provides immediate feedback on model quality and a clear pathway for integrating multiple forecasters into a coherent, market‑driven credibility system.