Sparsistent Estimation of Time-Varying Discrete Markov Random Fields

Network models have been popular for modeling and representing complex relationships and dependencies between observed variables. When data comes from a dynamic stochastic process, a single static network model cannot adequately capture transient dependencies, such as, gene regulatory dependencies throughout a developmental cycle of an organism. Kolar et al (2010b) proposed a method based on kernel-smoothing l1-penalized logistic regression for estimating time-varying networks from nodal observations collected from a time-series of observational data. In this paper, we establish conditions under which the proposed method consistently recovers the structure of a time-varying network. This work complements previous empirical findings by providing sound theoretical guarantees for the proposed estimation procedure. For completeness, we include numerical simulations in the paper.

💡 Research Summary

The paper addresses the problem of estimating the evolving structure of a discrete Markov random field (DMRF) when observations are collected over time. Traditional static network models fail to capture transient dependencies that arise in dynamic biological, neurological, or social systems, such as gene‑regulatory interactions that change throughout development. Kolar et al. (2010b) introduced a practical procedure that combines kernel smoothing with ℓ₁‑penalized logistic regression to produce a time‑varying estimate of the underlying graph. However, their work was primarily empirical; no rigorous proof of consistency was provided.

In this study the authors fill that gap by establishing sufficient conditions under which the Kolar‑type estimator is “sparsistent”—that is, it simultaneously recovers the true sparsity pattern (the set of edges) and converges to the correct parameter values as the number of observations grows. The model assumes binary variables X₁(t),…,X_p(t) whose joint distribution at each time t follows a pairwise Markov random field with conditional probabilities given by logistic functions. The edge‑specific parameters θ_{ij}(t) are allowed to vary smoothly over time; smoothness is formalized by requiring a bounded second derivative and a Lipschitz constant.

The estimation algorithm proceeds as follows. For each target time point t a kernel K_h(·) (typically Gaussian) assigns weights to the observed samples {X(t_k)} based on their temporal distance |t_k−t|. A weighted ℓ₁‑penalized logistic regression is then solved for each node i, yielding a vector of estimated coefficients β̂_i(t). An edge (i,j) is declared present at time t if either |β̂_i,j(t)| or |β̂_j,i(t)| exceeds zero after penalization. The penalty level λ_n and bandwidth h are the two tuning parameters that control sparsity and temporal smoothing, respectively.

The theoretical contribution consists of two main theorems. The first, a local consistency result, shows that with bandwidth h≈n^{‑1/5} and penalty λ_n≈C√