Obligation Blackwell Games and p-Automata

We recently introduced p-automata, automata that read discrete-time Markov chains. We used turn-based stochastic parity games to define acceptance of Markov chains by a subclass of p-automata. Definition of acceptance required a cumbersome and complicated reduction to a series of turn-based stochastic parity games. The reduction could not support acceptance by general p-automata, which was left undefined as there was no notion of games that supported it. Here we generalize two-player games by adding a structural acceptance condition called obligations. Obligations are orthogonal to the linear winning conditions that define winning. Obligations are a declaration that player 0 can achieve a certain value from a configuration. If the obligation is met, the value of that configuration for player 0 is 1. One cannot define value in obligation games by the standard mechanism of considering the measure of winning paths on a Markov chain and taking the supremum of the infimum of all strategies. Mainly because obligations need definition even for Markov chains and the nature of obligations has the flavor of an infinite nesting of supremum and infimum operators. We define value via a reduction to turn-based games similar to Martin’s proof of determinacy of Blackwell games with Borel objectives. Based on this definition, we show that games are determined. We show that for Markov chains with Borel objectives and obligations, and finite turn-based stochastic parity games with obligations there exists an alternative and simpler characterization of the value function. Based on this simpler definition we give an exponential time algorithm to analyze finite turn-based stochastic parity games with obligations. Finally, we show that obligation games provide the necessary framework for reasoning about p-automata and that they generalize the previous definition.

💡 Research Summary

The paper addresses a fundamental limitation in the theory of p‑automata, which are automata that read discrete‑time Markov chains. In earlier work the authors could only define acceptance for a restricted subclass of p‑automata by reducing the problem to a sequence of turn‑based stochastic parity games. This reduction was cumbersome, could not be applied to general p‑automata, and required a notion of game that could express “obligations” – statements that player 0 can guarantee a certain probability from a given configuration.

To overcome this, the authors introduce Obligation Blackwell Games (OBG), a new class of two‑player games that extend classical Blackwell games with an orthogonal structural acceptance condition called an obligation. An obligation is a declaration that from a particular configuration player 0 can achieve a specified value; if the declaration is fulfilled the value of that configuration is set to 1. Obligations are independent of the linear winning condition (e.g., a Borel set) that determines ordinary victory. Because obligations must be defined even on pure Markov chains, the usual definition of a game’s value as the supremum over player 0 strategies of the infimum over player 1 strategies of the measure of winning paths no longer works. The presence of obligations induces an infinite nesting of sup‑inf operators.

The authors resolve this by adapting Martin’s proof of determinacy for Blackwell games. They reduce an OBG to a turn‑based deterministic game in which each obligation becomes an explicit choice for player 0. This reduction allows the value of the original game to be defined as the value of the deterministic counterpart, preserving both the probabilistic transitions and the linear winning condition while handling obligations as additional strategic moves.

The main theoretical contributions are:

1. Determinacy – Every OBG is determined; one of the two players possesses an optimal strategy, and the game’s value is well defined.
2. Value for Markov chains with Borel objectives and obligations – For a pure Markov chain equipped with a Borel winning set and a set of obligations, the value function coincides with the standard measure‑theoretic value on paths, except that any configuration whose obligation is satisfied receives value 1.
3. Alternative characterization for finite turn‑based stochastic parity games with obligations – When the underlying game is a finite stochastic parity game, the value can be expressed by a simpler recursive characterization that first checks whether an obligation can be fulfilled and, if not, falls back to the usual parity‑value computation.

Based on this alternative characterization the authors devise an exponential‑time algorithm for solving finite OBGs. The algorithm proceeds in two phases: (i) a dynamic‑programming pass computes, for every configuration, whether its obligation can be satisfied; configurations with satisfied obligations are assigned value 1, (ii) the remaining sub‑game is a standard stochastic parity game, which is solved using known techniques (e.g., Zielonka’s algorithm). The overall running time is O(2ⁿ) where n is the size of the game graph, matching the worst‑case complexity of solving ordinary stochastic parity games but with a much clearer structure.

Finally, the paper shows that OBGs provide exactly the missing framework for reasoning about general p‑automata. Each state of a p‑automaton can be equipped with an obligation that captures the automaton’s requirement on the probability of reaching accepting states. By interpreting the execution of a p‑automaton on a Markov chain as a play of an OBG, the acceptance problem reduces to checking whether player 0 can meet all obligations. Consequently, the acceptance notion previously defined only for a subclass now extends to all p‑automata, and the determinacy and algorithmic results for OBGs immediately yield decision procedures for p‑automata acceptance.

In summary, the paper makes three intertwined contributions: (i) it introduces a novel game model that cleanly separates structural obligations from linear winning conditions; (ii) it proves determinacy and provides an alternative, algorithmically tractable characterization of the value function for both infinite‑state Markov chains and finite stochastic parity games; (iii) it demonstrates that this model precisely captures the semantics of p‑automata, thereby generalizing earlier work and opening the way to efficient verification of probabilistic systems.