Maximum Likelihood Estimation for Markov Chains

A new approach for optimal estimation of Markov chains with sparse transition matrices is presented.

💡 Research Summary

The paper addresses the problem of estimating the transition matrix of a Markov chain when the matrix is highly sparse, i.e., most state‑to‑state transitions never occur in practice. Traditional maximum‑likelihood estimation (MLE) simply normalizes observed transition counts, which works well for dense matrices and large datasets but suffers from severe bias and high variance in the sparse regime. Unobserved or rarely observed transitions are often estimated as zero, leading to under‑representation of genuine low‑probability dynamics and degrading downstream predictive performance.

To overcome these limitations, the authors propose a novel MLE framework that couples a Laplacian‑based regularization term with an Expectation‑Maximization (EM) algorithm. The key ideas are:

1. Structural Sparsity via Laplacian Penalty – The state space is treated as a graph whose nodes are states and edges represent possible transitions. The graph Laplacian (L) captures the smoothness of transition probabilities across neighboring states. By adding the quadratic penalty (\lambda \theta^{\top} L \theta) to the log‑likelihood, the estimator encourages similar probabilities for adjacent states while simultaneously shrinking many entries toward zero. The regularization strength (\lambda) controls the trade‑off between fidelity to the data and enforcement of sparsity.

2. EM for Latent Transition Counts – In many real‑world scenarios only partial transition information is available; some transitions are never observed even though they have non‑zero probability. The EM scheme treats the full transition count matrix (N) as latent. In the E‑step, given the current estimate (\theta^{(t)}), the expected counts (\mathbb{E}