On the connections between PCTL and Dynamic Programming

Probabilistic Computation Tree Logic (PCTL) is a well-known modal logic which has become a standard for expressing temporal properties of finite-state Markov chains in the context of automated model checking. In this paper, we give a definition of PCTL for noncountable-space Markov chains, and we show that there is a substantial affinity between certain of its operators and problems of Dynamic Programming. After proving some uniqueness properties of the solutions to the latter, we conclude the paper with two examples to show that some recovery strategies in practical applications, which are naturally stated as reach-avoid problems, can be actually viewed as particular cases of PCTL formulas.

💡 Research Summary

The paper addresses a fundamental gap in the literature on Probabilistic Computation Tree Logic (PCTL) by extending its semantics from finite‑state Markov chains to Markov processes with uncountable state spaces. The authors begin by formalising a measurable‑theoretic framework: the state space is equipped with a σ‑algebra and a probability kernel that defines the transition dynamics. Within this setting they reinterpret the core PCTL operators—next (X) and until (U)—so that they remain well‑defined for continuous‑state Markov processes.

A central contribution of the work is the identification of a deep structural correspondence between several PCTL constructs and classic dynamic programming (DP) problems. The “probability‑at‑least” reachability formula P≥p