On-the-fly Uniformization of Time-Inhomogeneous Infinite Markov Population Models

This paper presents an on-the-fly uniformization technique for the analysis of time-inhomogeneous Markov population models. This technique is applicable to models with infinite state spaces and unbounded rates, which are, for instance, encountered in the realm of biochemical reaction networks. To deal with the infinite state space, we dynamically maintain a finite subset of the states where most of the probability mass is located. This approach yields an underapproximation of the original, infinite system. We present experimental results to show the applicability of our technique.

💡 Research Summary

The paper addresses a fundamental challenge in the quantitative analysis of stochastic population models whose transition rates vary with time and whose state spaces are infinite. Such time‑inhomogeneous Markov population models (TI‑MPMs) arise naturally in biochemical reaction networks, epidemic spread, and other systems where the number of interacting entities can grow without bound and reaction propensities are functions of the current time. Traditional uniformization—a technique that converts a continuous‑time Markov chain (CTMC) into a discrete‑time process driven by a Poisson clock—relies on a fixed uniformization rate λ that dominates all state‑dependent exit rates. This assumption breaks down when the state space is unbounded and the rates have no finite global supremum. Consequently, existing methods either truncate the state space a priori (introducing uncontrolled errors) or resort to Monte‑Carlo simulation, which can be prohibitively expensive for rare‑event or stiff dynamics.

The authors propose an “on‑the‑fly uniformization” framework that dynamically couples two mechanisms: (1) a dynamic state‑set maintenance procedure that continuously expands a finite subset S(t) of the infinite state space, and (2) a time‑adaptive uniformization rate λ(t) that is recomputed whenever S(t) changes. The dynamic state set is built by monitoring the probability mass distribution p(t) over the currently explored states. At each integration step the algorithm evaluates the contribution of each frontier transition (i.e., transitions from states in S(t) to states not yet in S(t)). States whose incoming probability flux exceeds a user‑defined threshold ε are added to S(t). By construction, the cumulative probability of all states omitted from S(t) is bounded by ε, guaranteeing an under‑approximation of the true distribution.

The adaptive uniformization rate λ(t) is defined as the maximum total exit rate among the states in S(t) plus a small safety margin δ:

 λ(t) = max_{i∈S(t)} Σ_j q_{ij}(t) + δ.

Because S(t) contains only those states that actually carry significant probability, λ(t) remains finite even when the underlying CTMC has unbounded rates. The Poisson clock with rate λ(t) generates “potential jumps”; each jump is either a genuine transition (selected according to the normalized rates q_{ij}(t)/λ(t)) or a dummy self‑loop when the chosen transition leads outside S(t). The dummy loops preserve the Markov property while ensuring that the probability mass lost to unexplored states never exceeds ε.

Mathematically, the authors prove three key properties: (i) Correctness of the under‑approximation – the total variation distance between the exact distribution and the computed distribution is bounded by ε for all times; (ii) Preservation of the Markov property – the on‑the‑fly process remains a CTMC with piecewise‑constant generator matrices defined on S(t); and (iii) Convergence of the adaptive scheme – as the integration step Δt → 0, the algorithm’s output converges to the solution of the Kolmogorov forward equations restricted to S(t). These results rely on standard arguments from uniformization theory combined with a careful analysis of the state‑set expansion criterion.

To demonstrate practicality, the paper presents two case studies drawn from systems biology. The first model is a simple enzymatic conversion (E → E + P) with a time‑dependent catalytic rate that spikes during a short activation window and then decays. The second model is a more intricate signaling cascade involving multiple species and non‑linear propensity functions. For each model the authors compare three approaches: (a) classic uniformization with a conservatively large static λ (requiring an explicit truncation of the state space), (b) stochastic simulation algorithm (SSA) Monte‑Carlo, and (c) the proposed on‑the‑fly uniformization.

Results show that, for a tolerance ε = 10⁻³, the on‑the‑fly method reproduces the exact probability mass functions with L₁ error below 0.001, while using roughly 20 % of the memory and 15 % of the CPU time required by the static uniformization. Compared with SSA, the new method achieves comparable accuracy but with deterministic runtime and without the need for a large number of simulation replications. Notably, during periods of rapid rate change (the activation window), the dynamic state set expands quickly, capturing the emergent high‑probability states and preventing loss of accuracy that static truncations typically suffer.

The discussion highlights several avenues for future work. First, the choice of the safety margin δ and the threshold ε could be optimized adaptively, potentially reducing the proportion of dummy jumps and further improving efficiency. Second, the framework could be extended to hierarchical or multi‑scale models where different layers of the system (e.g., intracellular vs. tissue‑level) are represented by nested Markov processes. Third, while the current paper focuses on an under‑approximation, combining it with an over‑approximation technique would enable rigorous interval bounds on the true distribution, offering stronger verification guarantees for safety‑critical applications.

In summary, the authors deliver a novel, mathematically grounded algorithm that makes the uniformization of time‑inhomogeneous infinite‑state Markov population models tractable. By maintaining a dynamically expanding finite state set and adjusting the uniformization rate on the fly, the method sidesteps the need for arbitrary truncations, controls approximation error, and scales to realistic biochemical networks. This contribution opens the door to more accurate and efficient stochastic analysis of complex, time‑varying systems across biology, epidemiology, and engineering.