Decisive Markov Chains

We consider qualitative and quantitative verification problems for infinite-state Markov chains. We call a Markov chain decisive w.r.t. a given set of target states F if it almost certainly eventually reaches either F or a state from which F can no longer be reached. While all finite Markov chains are trivially decisive (for every set F), this also holds for many classes of infinite Markov chains. Infinite Markov chains which contain a finite attractor are decisive w.r.t. every set F. In particular, this holds for probabilistic lossy channel systems (PLCS). Furthermore, all globally coarse Markov chains are decisive. This class includes probabilistic vector addition systems (PVASS) and probabilistic noisy Turing machines (PNTM). We consider both safety and liveness problems for decisive Markov chains, i.e., the probabilities that a given set of states F is eventually reached or reached infinitely often, respectively. 1. We express the qualitative problems in abstract terms for decisive Markov chains, and show an almost complete picture of its decidability for PLCS, PVASS and PNTM. 2. We also show that the path enumeration algorithm of Iyer and Narasimha terminates for decisive Markov chains and can thus be used to solve the approximate quantitative safety problem. A modified variant of this algorithm solves the approximate quantitative liveness problem. 3. Finally, we show that the exact probability of (repeatedly) reaching F cannot be effectively expressed (in a uniform way) in Tarski-algebra for either PLCS, PVASS or (P)NTM.

💡 Research Summary

The paper introduces the notion of a decisive Markov chain with respect to a set of target states F: a chain is decisive if, with probability one, every execution eventually either reaches F or reaches a state from which F is no longer reachable. While this property holds trivially for all finite Markov chains, the authors investigate its presence in several classes of infinite‑state stochastic systems and show that it can be leveraged to obtain strong verification results.

The first major contribution is a structural characterization: any infinite‑state Markov chain that possesses a finite attractor—a finite set A such that every run visits A infinitely often with probability one—is decisive for every possible target set F. This observation immediately yields decisiveness for three well‑studied models. Probabilistic lossy channel systems (PLCS) have a finite attractor formed by the set of configurations with empty channels; probabilistic vector addition systems (PVASS) are globally coarse, a property that also guarantees a finite attractor; and probabilistic noisy Turing machines (PNTM) become trapped in a finite “error‑saturated” region when the noise probability is positive. Consequently, all three models are decisive.

Having established decisiveness, the authors reformulate the classic safety and liveness verification problems in abstract terms. The safety problem asks whether the probability of eventually hitting F equals 1, while the liveness problem asks whether the probability of hitting F infinitely often equals 1. For each of the three models they provide an almost complete decidability picture. Safety is decidable for PLCS, PVASS and PNTM. Liveness is decidable for PLCS and PVASS; for PNTM the problem remains open, reflecting a genuine gap in current techniques.

On the quantitative side, the paper revisits the path‑enumeration algorithm originally proposed by Iyer and Narasimha. This algorithm enumerates all finite paths that lead to F, computes their probabilities, and sums them to obtain an ε‑approximation of the reachability probability. The authors prove that, for any decisive Markov chain, the enumeration process necessarily terminates, guaranteeing that the algorithm yields a correct approximation for the safety problem. By a modest modification—allowing the algorithm to detect cycles that can be traversed infinitely often—they extend the method to approximate the liveness probability as well. Thus, the classic algorithm, previously known to be incomplete for general infinite‑state chains, becomes a reliable tool for all decisive systems.

The final contribution is a negative result concerning exact expressibility. The authors show that the exact probability of (repeatedly) reaching a target set cannot be uniformly expressed in the first‑order theory of real closed fields (Tarski algebra) for any of the three models. In other words, there is no algorithm that, given a PLCS, PVASS, or (P)NTM and a target set F, produces a closed‑form algebraic expression for the exact probability. This non‑expressibility underscores the intrinsic difficulty of exact quantitative analysis in infinite‑state stochastic models and justifies the focus on approximation techniques.

Overall, the paper makes three intertwined advances: (1) it identifies a robust structural property—decisiveness—shared by several important infinite‑state probabilistic models; (2) it exploits this property to obtain near‑complete decidability results for safety and liveness, and to guarantee termination of a classic approximation algorithm; and (3) it delineates the limits of exact symbolic computation by proving non‑expressibility in Tarski algebra. These contributions deepen our theoretical understanding of infinite‑state probabilistic verification and provide practical guidance for building analysis tools that can handle lossy channels, vector addition systems, and noisy Turing machines.