Markov Chain Order Estimation and Relative Entropy

We use the $f-divergence$ also called relative entropy as a measure of diversity between probability densities and review its basic properties. In the sequence we define a few objects which capture relevant information from the sample of a Markov Chain to be used in the definition of a couple of estimators i.e. the Local Dependency Level and Global Dependency Level for a Markov chain sample. After exploring their properties we propose a new estimator for the Markov chain order. Finally we show a few tables containing numerical simulation results, comparing the performance of the new estimator with the well known and already established AIC and BIC estimators.

💡 Research Summary

The paper introduces a novel approach for estimating the order of a Markov chain by exploiting the f‑divergence, specifically the Kullback‑Leibler (KL) divergence, as a measure of discrepancy between empirical and theoretical transition distributions. After reviewing the basic properties of f‑divergences—non‑negativity, zero‑only when the two distributions coincide, and the chain rule—the authors construct two new statistics: the Local Dependency Level (LDL) and the Global Dependency Level (GDL).

LDL quantifies, for a given context length k, the KL‑divergence between the observed conditional probabilities of the next state and the true conditional probabilities implied by a Markov model of order k. Formally, LDL_k is the sum over all k‑tuples of the empirical joint probability multiplied by the log‑ratio of empirical to true conditional probabilities. The key theoretical insight is that if the true order of the chain is r, then for k ≥ r the expected LDL_k decays to zero at a rate O(1/n), while for k < r it remains bounded away from zero. This creates a sharp “knee” in the LDL curve at the true order.

GDL aggregates LDL across all examined context lengths, typically by averaging: GDL_m = (1/m) Σ_{k=1}^{m} LDL_k. By smoothing the local fluctuations, GDL provides a global criterion that is minimized at the true order. The authors prove convergence results for both LDL and GDL under standard regularity conditions, showing that the proposed estimator
\