Freezing and Sleeping: Tracking Experts that Learn by Evolving Past Posteriors

A problem posed by Freund is how to efficiently track a small pool of experts out of a much larger set. This problem was solved when Bousquet and Warmuth introduced their mixing past posteriors (MPP) algorithm in 2001. In Freund’s problem the experts would normally be considered black boxes. However, in this paper we re-examine Freund’s problem in case the experts have internal structure that enables them to learn. In this case the problem has two possible interpretations: should the experts learn from all data or only from the subsequence on which they are being tracked? The MPP algorithm solves the first case. Our contribution is to generalise MPP to address the second option. The results we obtain apply to any expert structure that can be formalised using (expert) hidden Markov models. Curiously enough, for our interpretation there are \emph{two} natural reference schemes: freezing and sleeping. For each scheme, we provide an efficient prediction strategy and prove the relevant loss bound.

💡 Research Summary

Freund’s “tracking a small pool of experts” problem asks how to efficiently follow a few useful experts drawn from a much larger set. Bousquet and Warmuth answered this in 2001 with the Mixing Past Posteriors (MPP) algorithm, which mixes the posterior distributions of all experts over time. Their solution treats each expert as a black box that learns from the entire data stream.

The present paper revisits Freund’s problem under a more realistic assumption: experts have internal learning mechanisms (e.g., Bayesian updates, online regression models) and therefore can be described by hidden Markov models (HMMs). This internal structure raises a crucial question: should an expert be allowed to learn from all observed outcomes, or only from the subsequence on which it is actually being tracked? The first interpretation corresponds exactly to the original MPP setting. The second interpretation—learning only from the tracked subsequence—has not been addressed before and is the focus of this work.

To formalize experts with internal dynamics, the authors introduce the Expert Hidden Markov Model (EHMM). An EHMM consists of a finite set of hidden states, a transition matrix that updates the internal state based on past observations, and an emission distribution that generates predictions. In this framework, the posterior distribution of an expert at time (t) is a deterministic function of its previous posterior and the loss incurred on the most recent outcome. Thus, the “mixing past posteriors” idea of MPP can be naturally extended to EHMMs.

When learning only from the tracked subsequence, two natural reference schemes emerge:

1. Freezing – Once an expert is selected, its internal state stops evolving; the posterior computed at the selection moment is “frozen” and used for all future predictions of that expert, regardless of subsequent data.
2. Sleeping – An expert that is not currently selected remains in a “sleeping” mode; its internal state does not change until the expert is selected again, at which point it resumes updating from the frozen posterior.

Both schemes prevent an expert from being contaminated by data that it never actually helped to predict, thereby aligning the learning process with the tracking objective.

The authors design efficient prediction algorithms for each scheme by extending the MPP update rule. At each round they maintain a weight vector over all experts, compute a mixture prediction, observe the loss, and then update the weights. The key modification is that the posterior update for each expert follows the rule dictated by the chosen scheme (freezing or sleeping). Because each expert’s internal state space is finite, the per‑round computational cost is (O(|\mathcal{E}|\cdot|S|)), where (|\mathcal{E}|) is the number of experts and (|S|) the size of an expert’s state space.

The main theoretical contributions are loss bounds that mirror those of the original MPP. For both freezing and sleeping, the cumulative log‑loss of the algorithm satisfies

\