Sequential Change Detection Under Markov Setup With Unknown Prechange And Postchange Distributions

In this work we extend the results developed in 2022 for a sequential change detection algorithm making use of Page’s CUSUM statistic, the empirical distribution as an estimate of the pre-change distribution, and a universal code as a tool for estimating the post-change distribution, from the i.i.d. case to the Markov setup.

💡 Research Summary

The paper addresses the problem of sequential change detection when both the pre‑change and post‑change probability distributions are unknown, extending previous work that was limited to independent and identically distributed (i.i.d.) observations. The authors consider a finite‑alphabet, stationary, ergodic, first‑order Markov source that satisfies Berk’s mixing conditions. Under this model, the classic Page‑CUSUM test, which requires exact knowledge of the likelihood ratio, cannot be applied directly.

To overcome this limitation, the authors propose a modified CUSUM procedure that replaces the unknown post‑change log‑likelihood with the code length of a strongly pointwise universal lossless compressor (e.g., Lempel‑Ziv or Context‑Tree Weighting). Universal codes enjoy the property that their per‑symbol redundancy converges almost surely to the Kullback‑Leibler (KL) divergence between the true source and the code’s implicit model, guaranteeing a 1‑quick convergence of the surrogate log‑likelihood.

The pre‑change distribution is estimated empirically from an initial training block of length (n_{0}). Transition frequencies ( \hat{\mu}{0}(b|a) ) and marginal frequencies ( \hat{\mu}{0}(b) ) are computed, and under the assumption that each symbol and each admissible transition appears at least once, the empirical estimate converges to the true (\mu_{0}) as (n_{0}\to\infty).

The detection statistic at time (n) is defined as

\