A Counterexample Guided Abstraction-Refinement Framework for Markov Decision Processes

The main challenge in using abstractions effectively, is to construct a suitable abstraction for the system being verified. One approach that tries to address this problem is that of {\it counterexample guided abstraction-refinement (CEGAR)}, wherein one starts with a coarse abstraction of the system, and progressively refines it, based on invalid counterexamples seen in prior model checking runs, until either an abstraction proves the correctness of the system or a valid counterexample is generated. While CEGAR has been successfully used in verifying non-probabilistic systems automatically, CEGAR has not been applied in the context of probabilistic systems. The main issues that need to be tackled in order to extend the approach to probabilistic systems is a suitable notion of ``counterexample’’, algorithms to generate counterexamples, check their validity, and then automatically refine an abstraction based on an invalid counterexample. In this paper, we address these issues, and present a CEGAR framework for Markov Decision Processes.

💡 Research Summary

**
The paper introduces a Counterexample‑Guided Abstraction‑Refinement (CEGAR) framework for verifying properties of Markov Decision Processes (MDPs), extending a technique that has been highly successful for non‑probabilistic systems. The authors first identify a suitable fragment of PCTL (both safety and liveness) that can be interpreted over MDPs, noting that existing fragments for DTMCs do not capture the nuances introduced by nondeterminism. They then examine three previously proposed notions of counterexample—multisets of executions, tree‑like structures, and DTMC‑based examples—and prove that none of them can represent counterexamples for the full safety fragment of PCTL on MDPs. Consequently, they define a counterexample as a pair (E, R) where E is a small MDP that violates the property and R is a simulation relation showing that the abstract model simulates E. By making the simulation relation part of the counterexample, validity checking reduces to a standard MDP‑simulation test with the additional constraint that the witnessing relation is contained in R.

The authors analyze the computational complexity of generating counterexamples. They show that finding a minimum counterexample (fewest states/edges) is NP‑complete and unlikely to admit good approximation algorithms. Nevertheless, they present a polynomial‑time algorithm that computes a minimal counterexample—one that becomes valid only when no further state or transition can be removed. This algorithm iteratively removes components while preserving property violation, guaranteeing minimality without incurring exponential cost.

Abstraction is built on an equivalence relation over concrete states; each equivalence class becomes an abstract state, and its outgoing transitions are derived by aggregating the transitions of all concrete members. The refinement step uses the invalid counterexample to split equivalence classes precisely where the abstraction was too coarse. The refinement algorithm is a natural generalization of the classic non‑probabilistic CEGAR refinement, but it respects the probabilistic semantics and nondeterministic choices of MDPs.

A special case is treated for a “weak safety” fragment, for which tree‑structured counterexamples do exist. The paper provides an on‑the‑fly unrolling algorithm that expands a minimal counterexample into a tree, often outperforming the generic simulation‑based validity check in practice.

Overall, the contributions are: (1) a formal definition of counterexamples for MDPs that includes simulation relations; (2) proof of expressive inadequacy of earlier counterexample notions; (3) complexity results for counterexample generation and a practical minimal‑counterexample algorithm; (4) algorithms for validity checking and abstraction refinement; and (5) a demonstration that weak‑safety properties admit more tractable counterexamples. This work bridges the gap between CEGAR and probabilistic model checking, offering both theoretical insights and concrete algorithms for automated verification of MDP‑based systems.