On Probabilistic Alternating Simulations

On Probabilistic Alternating Sim ulations Chen yi Zhang and Jun Pang F acult y of Sciences, T ec hnology and Comm unication Univ ersity of Luxem bourg 6, rue Ric hard Coudenho ve-Kalergi, L-1359 Luxem b ourg Abstract. This pap er presen ts sim ulation-based relations for proba- bilistic game structures. The first relation is called probabilistic alter- nating sim ulation, and the second called probabilistic alternating for- w ard simulation, follo wing the naming conv ention of Segala and Lync h. W e study these relations with resp ect to the preserv ation of prop erties sp ecified in probabilistic alternating-time temp oral logic. 1 In tro duction Sim ulation relations [Mil89] hav e pro ved to be useful for comparing the behavior of concurren t systems, whic h can b e formally interpreted as lab eled transition systems. The study of logic characterization of sim ulation is to build its connec- tion to a modal or temp oral logic whic h can b e used to form ulate some in terest- ing prop erties. Soundness of logic ch aracterization requires sim ulation preserv e the satisfaction of logic formulas, while completeness sho ws the relation has the same strength as the logic. Intuitiv ely , the fact that one state s 1 sim ulates an- other state s 2 can be used to establish the relation that an y possible behavior of s 1 is also p ossible on s 2 . Thus it can preserv e certain desirable prop erties 1 form ulated in temporal logics lik e CTL [Eme90]. Simulation relations hav e set up the foundations for constructing correct abstractions. R elate d work. Segala and Lynch [SL95] extend the classical notions of sim ula- tion for probabilistic automata [Seg95b], a general extension of labeled transi- tion systems which admits b oth probabilistic and nondeterministic b eha viors. Their main idea is to relate probabilit y distributions o ver states, instead of re- lating individual states. They show soundness of the logical characterization of probabilistic sim ulation, whic h preserv es probabilistic CTL formulas [Han94] without negation and existential quan tification. Segala introduces the notion of probabilistic forward simulation, which relates states to probability distribu- tions ov er states and is sound and complete for trace distribution precongru- ence [Seg95a,LSV07]. Logic c haracterization of strong and w eak probabilistic bisim ulation has b een studied in [DGJP02,PS07]. Alur, Henzinger and Kupferman [AHK97,AHK02] define A TL (alternating- time temp oral logic) to generalize CTL for game structures by requiring each 1 F or example, safety prop erties stating “nothing bad can happ en”. path quan tifier to b e parametrized with a set of agents. Game structures are more general than L TS, in the sense that they allo w both collab orative and adv ersarial b ehaviors of individual agen ts in a system, and A TL can b e used to express prop erties like “a set of agen ts can enforce a sp ecific outcome of the system”. Alternating refinemen t relations, in particular alternating sim ulation, are introduced later in [AHKV98]. Alternating sim ulation is a natural game- theoretic interpretation of the classical simulation in tw o-pla yer games. Logic c haracterization of this simulation concen trates on a subset of A TL  form ulas where negations are only allow ed at prop osition level and all path quan tifiers are parametrized by a prefixed set of agen ts A . This sublogic of A TL  con tains all formulas expressing the prop erties the agen ts in A can enforce no matter what the other agents do. Alur et al. [AHKV98] hav e prov ed b oth soundness and completeness of the logic characterization. Our c ontribution. In this app er, w e in tro duce t w o notions of sim ulation for prob- abilistic game structures – probabilistic alternating sim ulation and forward sim- ulation, follo wing the aforementioned results [Seg95a,SL95,AHKV98]. W e prov e the soundness of logical c haracterization of probabilistic alternating sim ulation relations, by showing that they preserv e a fragment of a probabilistic extension of A TL. Structur e of the p ap er. The rest of the pap er is organized as follo ws. W e briefly explain some basic notations that are used throughout the pap er in Sect. 2. Sect. 3 in tro duces the notion of probabilistic game structures and the defini- tion of probabilistic executions. In Sect. 4 we present P A TL an extension of the alternating-time temp oral logic [AHK02] for probabilistic systems, and roughly discuss its mo del c hecking problem. W e define probabilistic alternating sim ula- tion and forward simulation in Sect. 5, and sho w their soundness for preserving prop erties sp ecified in P A TL in Sect. 6. Probabilistic alternating bisimulation is shortly discussed in Sect. 7. W e conclude the pap er with some future research topics in Sect. 8. 2 Preliminaries This section con tains basic notions that are used in the tec hnical part. Let S b e a set. A discrete probabilistic distribution ∆ ov er S is a function of t yp e S → [0 , 1], satisfying P s ∈ S ∆ ( s ) = 1. W e write D ( S ) for the set of all suc h distributions. F or a set S 0 ⊆ S , define ∆ ( S 0 ) = P s ∈ S 0 ∆ ( s ). Giv en t wo distributions ∆ 1 , ∆ 2 and p ∈ [0 , 1], ∆ 1 ⊕ p ∆ 2 is a function of t yp e S → [0 , 1] defined as ∆ 1 ⊕ p ∆ 2 ( s ) = p · ∆ 1 ( s )+(1 − p ) · ∆ 2 ( s ) for all s ∈ S . Obviously , ∆ 1 ⊕ p ∆ 2 is also a distribution. W e further extend this notion b y combining a set of distributions { ∆ i } i ∈ I ordered b y an indexed set { p i } i ∈ I in to a distribution P i ∈ I p i ∆ i , where p i ∈ [0 , 1] for all i ∈ I and P i ∈ I p i = 1. s is called a point distribution satisfying s ( s ) = 1 and s ( t ) = 0 for all t 6 = s . Let ∆ ∈ D ( S ), write d ∆ e for the supp ort of ∆ as the set { s ∈ S | ∆ ( s ) > 0 } . Let S = S 1 × S 2 × · · · × S n , then s ∈ S is a vector of length n . W e may also write s = h s 1 , s 2 , . . . , s n i , with s ( i ) = s i ∈ S i . Giv en a finite sequence α = s 1 s 2 . . . s n ∈ S ∗ , write last ( α ) for s n . Let S 0 ⊆ S , then α | S 0 is a subsequence of α with exactly the elements not in S 0 remo ved. Given L ⊆ S ∗ , write L | S 0 for the set { ( α | S 0 ) | α ∈ L } . 3 Probabilistic Game Structures Assume a set of play ers Σ = { 1 , 2 , . . . , k } . A probabilistic game structure (PGS) G is defined as a tuple h S, s 0 , L , Act , δ i , where – S is a finite set of states, with s 0 the initial state, – Act = Act 1 × Act 2 × · · · × Act k is a set of joint actions, where Act i is the set of actions for play er i = 1 , . . . , k , – L : S → 2 Prop is the lab elling function, – δ : S × Act → D ( S ) is a transition function. A pla y ρ is a (finite or infinite) sequence s 0 a 1 s 1 a 2 s 2 . . . , such that a i ∈ Act and δ ( s i − 1 , a i )( s i ) > 0 for all i . W rite | ρ | for the length of a run ρ , which is the num b er of transitions in ρ , and | ρ | = ∞ if ρ is infinite. W e write ρ ( i ) for the i -th state in ρ starting from 0, and ρ [ i, j ] for the subsequence starting from i -th state and ending at the j -th state, pro vided 0 ≤ i ≤ j ≤ | ρ | . Note that the play ers c ho ose their next mo v es sim ultaneously , but their mo ves may or ma y not b e coop erativ e. If on s tate s eac h play er i performs action a i , then δ ( s, h a 1 , a 2 , . . . a k i ) is the distribution for the next reachable states. In the follo wing discussion, w e fix a probabilistic game structure G . W e assume that the transition relation is total on the set Act . Note that this do es not p ose any limitation on the expressiveness of the mo del. If an action c ∈ Act i of play er i is not supp osed to b e enabled on state s for play er i , we ma y find another action c 0 ∈ Act i and define c to hav e the same effect as c 0 on s . Since play er i knows the current state, he also kno ws the set of actions a v ailable to him, so that as a rational pla yer he will not choose actions that are not enabled. This allo ws such models to express systems in whic h on some states the av ailable (join t) actions are prop er subsets of Act . 2 W e may ev en disable a particular play er on a state. A play er i is disabled on s if δ ( s, a ) = δ ( s, a 0 ) for all action vectors a , a 0 ∈ Act satisfying a ( j ) = a 0 ( j ) for all j 6 = i . A PGS is turn- b ase d if all but one play er is disabled on s for all s ∈ S . A probabilistic game structure can b e regarded as a generalization of a concurren t game structure of [AHK02]. F rom a state s ∈ S , each play er i may c ho ose an action from Act i and together they resolv e the nondeterminism. On the other hand, a PGS is more stratified on external actions than some of the existing mo dels. 3 2 In the literature some authors enco de a v ailable actions for play er i as a function of t yp e S → 2 Act i \ {∅} . 3 F or example, a one-play er PGS resembles a reactive system of [vGSS95], and a tw o- pla yer turn-based PGS (assuming they alternately act) lo osely simulates a simple probabilistic automaton [Seg95b], in the wa y that one play er p erforms external ac- tions and the other resolves nondeterminism after the previous play er’s mo ve is done. A strategy of a play er i ∈ Σ is a function of type S + → D ( Act i ). W e write Π G i for the set of strategies of pla yer i in G . 4 A play ρ is compatible with an i -strategy π i , if a k ( i ) ∈ d π i ( ρ [0 , k − 1] | S ) e for all k ≤ | ρ | . Giv en a v ector of strategies π ∈ Π G 1 × Π G 2 × · · · × Π G | Σ | , a run ρ is compatible with π if a k ( i ) ∈ d π ( i )( ρ [0 , k − 1] | S ) e for all k ≤ | ρ | and i = 1 , . . . , k . W rite G ( π , s ) for the set of infinite pla ys compatible with every strategy in π starting from s ∈ S , and G ∗ ( π , s ) the set of finite plays in G that are compatible with π starting from s . The set of finite pla ys compatible to a strategy v ector π is also called a set of c ones [Seg95b], with eac h finite play α representing the set of infinite plays prefixed b y α . Giv en a state s 0 ∈ S , w e can derive the probabilit y for every mem- b er in S + compatible with π , by recursively defining a function P r G ( π ,s 0 ) from S + to [0 , 1] as follows. This function P r G ( π ,s 0 ) can b e further generalized as the probabilit y measure to the σ -field F G , π ,s 0 ⊆ G ( π , s 0 ) whic h is a unique extension from the set of cones G ∗ ( π , s ) closed by countable union and complementation, in a w ay similar to [Seg95b]: – P r G ( π ,s 0 ) ( s 0 ) = 1, – P r G ( π ,s 0 ) ( α · s ) = P r G ( π ,s 0 ) ( α ) · δ ( last ( α ) , h π (1)( α ) , π (2)( α ) , . . . , π ( k )( α ) i )( s ), where δ ( s, h ∆ 1 , ∆ 2 , . . . , ∆ k i ) is a distribution ov er states derived from δ and the v ector of action distributions defined b y δ ( s, h ∆ 1 , . . . , ∆ k i ) = X i ∈{ 1 ,..., k } ,a i ∈d ∆ i e ∆ 1 ( a 1 ) · . . . · ∆ k ( a k ) · δ ( s, h a 1 , . . . , a k i ) . Giv en A ⊆ Σ , sometimes w e write π ( A ) for a vector of | A | strategies { π i } i ∈ A , and Π ( A ) for the set of all suc h strategy vectors. W rite A for Σ \ A . Giv en A ∩ A 0 = ∅ , strategy v ectors π ∈ Π ( A ) and π 0 ∈ Π ( A 0 ), π ∪ π 0 is the vector of strategies { π i } i ∈ A ∪ { π 0 j } j ∈ A 0 that combines π and π 0 . W e also define strategies of finite depth b y restricting the size of their do- mains, by writing π ∈ Π G ,n i as a level- n strategy , i.e., π is a function from traces of states with length up to n (i.e., the set S m ∈{ 1 , 2 ,...,n } S m ) to D ( Act i ). Given a set of strategies { π i } i ∈ I of the same domain, and { p i } i ∈ I with P i ∈ I p i = 1, let π = P i ∈ I p i · π i b e a (combined) strategy , by letting π ( γ ) = P i ∈ I p i · π i ( γ ) for all γ in the domain. W e ov erload the function δ as from a state in S and a v ector of strategies (of any depth n ) π ∈ Π G ,n 1 × Π G ,n 2 × · · · × Π G ,n | Σ | to D ( S ), b y δ ( s, π ) = δ ( s, a ), where a ( i ) = π ( i )( s ) for all i ∈ Σ . Note eac h a ( i ) is a distribution ov er Act i . W e further lift δ to b e a transition function from state distributions and strategy v ectors to state distributions, by δ ( ∆, π ) = X s ∈d ∆ e ∆ ( s ) · δ ( s, π ) 4 Sometimes we omit G , if it is clear from the context. Probabilistic Executions W e settle the nondeterminism in a probabilistic game structure b y fixing the b eha viours of all pla yers represented as strategies. Let G = h S, s 0 , L , Act , δ i b e a PGS, define a pr ob abilistic exe cution E as in the form of h E , ∆, L E , δ E i , where – E ⊆ S + is the set of finite plays starting form a state in the initial dis- tribution and compatible with δ E , i.e., s 0 s 1 . . . s n ∈ E if s 0 ∈ d ∆ e , and δ E ( s 0 . . . s i )( s 0 . . . s i +1 ) > 0 for all 0 ≤ i < n , – ∆ ∈ D ( S ) an (initial) distribution, – L E is the lab elling function defined as L E ( e ) = L ( last ( e )) for all e ∈ E , – δ E : E → D ( E ) is a (deterministic) transition relation, satisfying for all e ∈ E there exists a (level 1) strategy vector π e , such that δ E ( e )( e · t ) = δ ( last ( e ) , π e )( t ) if t ∈ d δ ( last ( e ) , π e ) e , and 0 otherwise. A probabilistic execution of G can b e uniquely determined by a strategy v ector π starting from a state distribution. Given ∆ ∈ D ( S ), define E ( G , π , ∆ ) as the probabilistic execution h E π , ∆, L π , δ π i , with E π = S s ∈d ∆ e G ∗ ( π , s ) | S for the set of compatible finite plays, L π defined as L π ( e ) = L ( last ( e )) for all e ∈ E π , and δ π ( e ) = δ ( last ( e ) , π e ) for all e ∈ E π , where π e ( i ) = π ( i )( e ) for all i ∈ Σ . In tuitively , a probabilistic execution resembles the notion of the same name prop osed by Segala and Lynch [Seg95b,SL95], and in this case the strategies of the play ers altogether represent a single adv ersary of Segala and Lynch. 4 Probabilistic Alternating-Time T emp oral Logic In this section w e introduce a probabilistic version of alternating-time temp oral logic [AHK02], whic h focuses on the pla yers abilit y to enforce a prop ert y with an exp ected probabilit y . Let Prop b e a nonempty set of propositions. Probabilistic alternating-time temp oral logic (P A TL) form ulas [CL07] are defined as follo ws. φ := p | ¬ φ | φ 1 ∧ φ 2 | h h A i i  α ψ ψ :=  φ | φ 1 U ≤ k φ 2 where A ⊆ Σ is a set of play ers,   ∈ { <, >, ≤ , ≥} , k ∈ N ∪ {∞} , p ∈ Prop , and α ∈ [0 , 1]. W e also write ψ 1 U ψ 2 for ψ 1 U ≤∞ ψ 2 as ‘unbounded until’. The sym b ols φ, φ 1 , φ 2 are state formulas, and ψ is a path formula. W e omit the syntactic sugars in our definition, such as true ≡ p ∨ ¬ p and false ≡ p ∧ ¬ p for some p ∈ Prop , φ 1 ∨ φ 2 ≡ ¬ ( ¬ φ 1 ∧ ¬ φ 2 ) for state formulas. The path mo dalit y R can b e expressed by U without introducing negations into path formulas, as we will show later in this section. One may also define 2 ≤ k ψ ≡ false R ≤ k ψ , and 3 ≤ k ψ ≡ true U ≤ k ψ , where k ∈ N ∪ {∞} . The set of P A TL formulas L are the set of state formulas as defined ab o ve. W e hav e the semantics of the path form ulas and the state formulas defined as follows. – ρ | = φ iff G , ρ (0) | = φ where φ is a state form ula, – ρ | =  φ iff ρ (1) | = φ , – ρ | = φ 1 U ≤ k φ 2 iff there exists i ≤ k suc h that ρ ( j ) | = φ 1 for all 0 ≤ j < i and ρ ( i ) | = φ 2 , – G , s | = p iff p ∈ L ( s ), – G , s | = ¬ φ iff G , s 6| = φ , – G , s | = φ 1 ∧ φ 2 iff G , s | = φ 1 and G , s | = φ 2 , – G , s | = h h A i i  α ψ iff there exists a vector of strategies π ∈ Π ( A ), such that for all vectors of strategies π 0 ∈ Π ( A ) for pla y ers in A , we hav e P r G ( π ∪ π 0 ,s ) ( { ρ ∈ G ( π ∪ π 0 , s ) | ρ | = ψ } )   α , where ρ is an infinite play in G , α ∈ [0 , 1], φ , φ 1 , φ 2 are state form ulas, and ψ is a path formula. Equiv alently , giv en S the state space of a probabilistic game structure G , we write J φ K for { s ∈ S | s | = φ } for all P A TL (state) formulas φ . F or ∆ ∈ D ( S ), we write ∆ | = φ iff d ∆ e ⊆ J φ K . In tuitively , G , s | = h h A i i ≥ α ψ describ es the ability of play ers in A to co operatively enforce ψ with probability at least α in s . The follo wing lemma is directly from the P A TL seman tics. If a group of users A can enforce a linear-time temp oral logic form ula ψ to hold with probability at least α with strategies π ∈ Π ( A ), then at the same time π enforces the formula ¬ ψ to hold with probability at most 1 − α . T o simplify the notation, we let ‘ ∼ ’ denote changes on directions of the symbols in { <, >, ≤ , ≥} , e.g., sym b ol e ≥ for ≤ , e ≤ for ≥ , e > for < , and e < for > . Lemma 1. G , s | = h h A i i  α ψ iff G , s | = h h A i i e   1 − α ¬ ψ Pr o of. (sketc h) F or all π ∈ Π ( A ) and π 0 ∈ Π ( A ), s ∈ S and ψ a path formula, w e ha ve P r G ( π ∪ π 0 ,s ) ( { ρ ∈ G ( π ∪ π 0 , s ) | ρ | = ψ } ) + P r G ( π ∪ π 0 ,s ) ( { ρ ∈ G ( π ∪ π 0 , s ) | ρ | = ¬ ψ } ) = 1. Therefore P r G ( π ∪ π 0 ,s ) ( { ρ ∈ G ( π ∪ π 0 , s ) | ρ | = ψ } )   α iff P r G ( π ∪ π 0 ,s ) ( { ρ ∈ G ( π ∪ π 0 , s ) | ρ | = ¬ ψ } ) e   1 − α , for all α ∈ [0 , 1] and   ∈ { <, >, ≤ , ≥} . u t Therefore, the path quantifier R (release) can be expressed by the existing P A TL syn tax, in the wa y that h h A i i  α φ 1 R ≤ k φ 2 ≡ h h A i i e   1 − α ( ¬ φ 1 ) U ≤ k ( ¬ φ 2 ), where b oth ¬ φ 1 and ¬ φ 2 are state form ulas. On Model Chec king of P A TL In this section we briefly survey the results in the literature related to P A TL mo del chec king. Given a P A TL formula in the form of h h A i i  α ψ ( φ 1 , . . . , φ n ), regarding J φ 1 K , . . . , J φ n K as the sets of states satisfying state formulas φ 1 , . . . , φ n , a standard wa y to solv e this problem is to determine the maximal or minimal probabilit y that the play ers in A can enforce the L TL formula ψ ( φ 1 , . . . , φ n ). In the following we write ψ for ψ ( φ 1 , . . . , φ n ) without further confusions. L TL properties are sp ecial cases of ω -regular winning ob jectives [Tho91] in t wo-pla yer concurrent (zero-sum) games [dAM04,CdAH06]. In such games one ma y group a set of play ers A ⊆ Σ into a single protagonist and A into a single an tagonist. Given an ω -regular winning ob jectiv e ξ and starting from a state Fig. 1. An example sho wing that play er I can guaran tee to satisfy 3 φ with probabilit y α for all 0 ≤ α < 1, but he cannot ensure that property with probability 1. s ∈ S , the protagonist pla ys with a strategy trying to maximize the probabil- it y for a play to satisfy ξ while the antagonist tries to minimize the probabil- it y . In suc h a game there alwa ys exists a unique v alue in [0 , 1], on which both pla yers ha ve strategies to guaran tee (or infinitely approac h) their b est p erfor- mances, regardless of the strategies pla y ed by their opponents. Such a supre- m um v alue (or infin um v alue, as for the antagonist) is called the value of the game [Mar98,dAM04]. In a probabilistic multi-pla yer game, we let a group of pla yers A ⊆ Σ b e a single play er, and A b e the other, and the supremal proba- bilit y for A to enforce such an L TL formula ψ starting from a given state s ∈ S can b e uniquely determined. This v alue is defined as h A i ψ ( s ) = G π ∈ Π ( A ) l π 0 ∈ Π ( A ) P r G ( π ∪ π 0 ,s ) ( { ρ ∈ G ( π ∪ π 0 , s ) | ρ | = ψ } ) Example 1. Fig. 1 gives a PGS with t wo pla yers { I , I I } , initial state s 0 , Act I = { a 1 , a 2 } and Act I I = { b 1 , b 2 } . Note that this PGS is deterministic, i.e, no prob- abilities in its transitions. W e assume that the only a v ailable transitions from s 1 and s 2 are self-lo ops, and the other transition relations are as depicted in the graph. Suppose play er I w ants to maximize the probability to enforce the prop ert y 3 φ , and play er I I aims to minimize it. Since the strategies applied on s 1 and s 2 do not matter, we fo cus on the c hoices of actions from b oth play ers on s 0 . W e first fo cus on memoryless strate- gies, and let play er I ’s strategy π 1 giv es π 1 ( γ )( a 1 ) = p and π 1 ( γ )( a 2 ) = 1 − p for all γ ∈ S + . Similarly we let I I assign probability q to b 1 and 1 − q to b 2 all the time. This pro duces an infinite tree, on whic h w e write x s 0 ( I ) for the actual probabilit y I can achiev e 3 φ from s 0 , given the ab o ve memoryless strategies. (Note that x s 1 ( I ) = 0 and x s 2 ( I ) = 1 in all cases.) This establishes an equation whic h further derives x s 0 ( I ) = (1 − p )+(2 p − 1) q (1 − p )+ pq . A simple analysis sho ws that when p approac hes 1, the minimal v alue of x s 0 ( I ) approac hes 1 as well, for all c hoices of q . That is, there exists a strategy for play er I to enforce 3 φ with probability 1 − ε for all ε > 0. How ever, if pla y er I chooses p = 1, play er I I may set q = 0 so that a play will b e trapp ed in s 0 for ever that yields x s 0 ( I ) = 0. The result of [dAM04] shows that in this case play er I cannot do b etter even with general (history dependent) strategies. In fact there are no strategies for pla yer I to enforce 3 φ with probability 1. u t Indeed, h A i ψ ( s ) can be almost the best, i.e., we hav e G , s | = h h A i i ≥ h A i ψ ( s ) − ε ψ for all ε > 0 [dAHK98]. Nevertheless, the quantitativ e version of determi- nacy [Mar98] ensures that for all L TL form ulas ψ and s ∈ S , we hav e h A i ψ ( s ) + h A i¬ ψ ( s ) = 1 The P A TL mo del chec king problems can b e solv ed by calculating the v alues h A i ψ s ( s ) for eac h state s , where each lo cal ob jective ψ s related to s migh t b e distinct. The algorithms of [dAM04] define monotonic functions of type ( S → [0 , 1]) → ( S → [0 , 1]) to arbitrarily approach a v ector {h A i ψ s ( s ) } s ∈ S in a game structure with finite state space S with resp ect to an ω -regular win- ning ob jective ψ . Within each step one has to go through O ( | S | ) matrix games, and each iteration pro duces a unique fixed p oin t. The algorithms on safet y and reac hability ob jectives are sp ecial cases of solving stochastic games [RF91]. More complex properties can b e expresse d as nested fixed points [dAM04]. Therefore, the upp er bound complexities b ecome exp onen tial to the size of the winning ob jectives translated from L TL form ulas. More recently , alternative algorithms prop osed in [CdAH06] pro ve that for quantitativ e games with ω -regular winning ob jectives expressed as parity conditions, whether the v alues of a game is within [ r − , r +  ] can b e decided in N P ∩ coN P for all rational r ∈ [0 , 1] and  > 0, whic h impro ves the theoretical upp er b ound for estimating the optimal v alues. Optimal Strategies It has b een shown in [dAM04] that for safety games there alwa ys exist optimal strategies for the protagonists, ho w ever for r e achability games it is not alw ays the case. As sho wn in example 1, pla yer I has no optimal strategy to enforce 3 φ with probabilit y 1 on s 0 ev en though h I i φ ( s 0 ) = 1. Based on similar proof strategies applied in [dAM04], we examine the existence of optimal strategies on winning ob jectiv es expressed as path formulas of P A TL on a state. Lemma 2. L et s b e a state, ψ b e a p ath formula, and A the set of pr otagonists. 1. If ψ is of the form  φ , φ 1 U ≤ k φ 2 , φ 1 R ≤ k φ 2 , or φ 1 R φ 2 with k ∈ N , ther e always exists a joint optimal str ate gy for A that enfor c es ψ on s with pr ob ability at le ast h A i ψ ( s ) . 2. If ψ is of the form φ 1 U φ 2 , ther e always exists a joint  -optimal str ate gy for A that enfor c es ψ on s with pr ob ability at le ast h A i ψ ( s ) −  , for al l  > 0 . F or the prov e of Lemma 2 we rely on the representation of a solution for a winning ob jective in quan titative game µ -calculus [dAM04]. F or the sake of readabilit y w e lea ve the whole pro of in the app endix. The next result pro v es the existence of a join t A strategy to enforce an P A TL path form ula with probability greater than α if there exists a join t strategy to enforce that formula with probability greater than α against an optimal A strategy . Lemma 3. L et ψ b e a P A TL p ath formula and π 0 b e a joint optimal str ate gy for the antagonists A on state s , if ther e exists a joint str ate gy π for the pr otagonists A such that P r G ( π ∪ π 0 ,s ) ( { ρ ∈ G ( π ∪ π 0 , s ) | ρ | = ψ } ) > α , then G , s | = h h A i i >α ψ . Pr o of. Since π 0 is the optimal strategy for the an tagonists, w e ha ve for all join t strategies π 00 , P r G ( π 00 ∪ π 0 ,s ) ( { ρ ∈ G ( π 00 ∪ π 0 , s ) | ρ | = ψ } ) ≤ h A i ψ ( s ), then we ha ve h A i ψ ( s ) > α . If there exists an optimal joint strategy for A then we ha ve s | = h h A i i ≥h A i ψ ( s ) ψ , which implies s | = h h A i i >α ψ . Otherwise b y Lemma 2 there exists an  -optimal join t strategy for A with small  > 0 to enforce ψ with probabilit y at least h A i ψ ( s ) −  > α . This also gives us s | = h h A i i >α ψ . u t This result do es not hold if w e replace the op erator “ > ” b y “ ≥ ” for unbounded un til U . This is b ecause if there do es not exist a join t optimal strategy for A to enforce φ 1 U φ 2 with probabilit y ≥ α , we hav e no space to insert a tin y  > 0 as w e did in the abov e proof. F or the fragment of path form ulas without unbounded un til, w e extend the results for ≥ , by the fact that optimal joint strategies for A alw ays exist for these path mo dalities, as shown by Lemma 2. Lemma 4. F or p ath formulas ψ in the form of  φ or φ 1 U ≤ k φ 2 and optimal str ate gies π 0 for the antagonists A on state s , if ther e exists a joint str ate gy π for the pr otagonists A such that P r G ( π ∪ π 0 ,s ) ( { ρ ∈ G ( π ∪ π 0 , s ) | ρ | = ψ } )   α , then G , s | = h h A i i  α ψ , wher e k ∈ N and   ∈ { >, ≥} . Pr o of. Since there exists joint strategies for A against A ’s optimal strategies, w e hav e h A i¬ ψ ( s ) e   1 − α , therefore h A i ψ ( s )   α by determinacy . By Lemma 2 there alwa ys exist optimal strategies for A to enforce ψ with probability   α if ψ is in the form of  φ or φ 1 U ≤ k φ 2 . u t A -P A TL W e define a sublogic of P A TL b y fo cusing on a particular set of pla yers. Similar to the approac h of [AHKV98], we only allo w negations to app ear on the lev el of prop ositions. Let A ⊆ Σ , an A -P A TL formula φ is a state formula defined as follows: φ := p | ¬ p | φ 1 ∧ φ 2 | φ 1 ∨ φ 2 | h h A 0 i i  α  φ | h h A 0 i i  α φ 1 U ≤ k φ 2 | h h A 0 i i >α φ 1 U φ 2 where k ∈ N ,   ∈ { >, ≥} and A 0 ⊆ A . W rite L A for the set of A -P A TL formulas. An A -P A TL formula describ es a prop ert y that play ers in A are able to ensure with a minimal exp ectation by their joint strategies. Note that we only allow ‘ > α ’ in the construction of unbounded un til. 5 Probabilistic Alternating Sim ulation Relations W e define probabilistic versions of alternating sim ulation [AHKV98]. An alter- nating simulation is a tw o-step simulation. F or a sketc h, suppose state s is sim- ulated by state t . In the first step the protagonists c ho ose their actions on t to simulate the behaviour of the protagonists on s , and in the second step the an tagonists choose actions on s to resp ond the behaviour of the antagonists on t . This somehow results in a sim ulation-like relation, so that for a certain prop ert y the protagonists can enforce on s , they can also enforce it on t . T o this end w e split Σ into tw o groups of play ers — one group of protagonist and the other group of antagonist. Subsequently , w e consider only the tw o-pla yer case in a probabilistic game structure — play er I for the protagonist and play er I I for the an tagonist, since what we can ac hiev e in the tw o-pla yer case naturally extends to a result in systems with tw o complementary sets of pla yers, i.e., A ∪ A = Σ . F or readabilit y w e also write the transition functions as δ ( s, a 1 , a 2 ) and δ ( s, π 1 , π 2 ) for δ ( s, h a 1 , a 2 i ) and δ ( s, h π 1 , π 2 i ), resp ectiv ely . Let S, T be tw o sets and R ⊆ S × T be a relation, then R ⊆ D ( S ) × D ( T ) is defined by ∆ R Θ if there exists a weigh t function w : S × T → [0 , 1] satisfying – P t ∈ T w ( s, t ) = ∆ ( s ) for all s ∈ S , – P s ∈ S w ( s, t ) = Θ ( t ) for all t ∈ T , – s R t for all s ∈ S and t ∈ T with w ( s, t ) > 0. Note in this definition, it is equiv alen t to ha ve P t ∈d Θ e w ( s, t ) = ∆ ( s ) for all s ∈ S , and P s ∈d ∆ e w ( s, t ) = Θ ( t ) for all t ∈ T . Since w can only assign non-zero v alues to the states in the supp ort of ∆ or Θ . If w ( s, t ) > 0 for some s 6∈ d ∆ e and t ∈ T , then w e w ould ha v e P t ∈ T w ( s, t ) > 0 = ∆ ( s ), whic h is a contradiction. The followings are several prop erties of lifted relations. Lemma 5. (inverse) L et R − 1 ⊆ T × S b e the inverse of R ⊆ S × T , then for al l ∆ ∈ D ( S ) and Θ ∈ D ( T ) , ∆ R Θ iff Θ R − 1 ∆ . Pr o of. By taking the inv erse of the weigh t function. u t Lemma 6. L et ∆ ∈ D ( S ) , ∆ 0 ∈ D ( S 0 ) , and R a r elation on S . If ∆ R ∆ 0 , then 1. If ther e exist ∆ 1 , ∆ 2 , · · · ∈ D ( S ) and an index set { p i } I satisfying P i ∈ I p i = 1 and ∆ = P i ∈ I p i · ∆ i , then ther e exist ∆ 0 1 , ∆ 0 2 · · · ∈ D ( S 0 ) such that ∆ 0 = P i ∈ I p i · ∆ 0 i , and ∆ i R ∆ 0 i for al l i ∈ I . 2. If ther e exist ∆ 0 1 , ∆ 0 2 , · · · ∈ D ( S 0 ) and an index set { p i } I satisfying P i ∈ I p i = 1 and ∆ 0 = P i ∈ I p i · ∆ 0 i , then ther e exist ∆ 1 , ∆ 2 · · · ∈ D ( S ) such that ∆ = P i ∈ I p i · ∆ i , and ∆ i R ∆ 0 i for al l i ∈ I . Pr o of. W e prov e the second part, and the first part is similar. Let ∆ 0 = P i ∈ I p i · ∆ 0 i , then define ∆ i for each i ∈ I by ∆ i ( s ) = P s 0 ∈ S 0 w ( s, s 0 ) · ∆ 0 i ( s 0 ) ∆ 0 ( s 0 ) for all s ∈ S . No w w e can chec k that P i ∈ I p i · ∆ i ( s ) = ∆ ( s ) for all s , i.e., ∆ = P i ∈ I p i · ∆ i . T o show that ∆ i R ∆ 0 i , we define a weigh t function w i : S × S 0 → [0 , 1] b y for all s ∈ S and s 0 ∈ S 0 , w i ( s, s 0 ) = w ( s, s 0 ) · ∆ 0 i ( s 0 ) ∆ 0 ( s 0 ) . Consider the follo wing conditions. 1. w i ( s, s 0 ) > 0 implies w ( s, s 0 ) > 0, therefore s R s 0 . 2. F or all s ∈ S , we hav e P s 0 ∈ S 0 w i ( s, s 0 ) = P s 0 ∈ S 0 w ( s, s 0 ) · ∆ 0 i ( s 0 ) ∆ 0 ( s 0 ) = ∆ i ( s ), 3. F or all s 0 ∈ S 0 , we hav e P s ∈ S w i ( s, s 0 ) = P s ∈ S w ( s, s 0 ) · ∆ 0 i ( s 0 ) ∆ 0 ( s 0 ) = ∆ 0 i ( s 0 ) ∆ 0 ( s 0 ) · P s ∈ S w ( s, s 0 ) = ∆ 0 i ( s 0 ) ∆ 0 ( s 0 ) · ∆ 0 ( s 0 ) = ∆ 0 i ( s 0 ). u t Lemma 7. L et R b e a r elation on S and { p i } i ∈ I b e an index set satisfying P i ∈ I p i = 1 and ∆ i R ∆ 0 i for distributions ∆ i , ∆ 0 i ∈ D ( S ) for al l i , then P i ∈ I p i · ∆ i R P i ∈ I p i · ∆ 0 i . Pr o of. W.l.o.g., let ∆ i ∈ D ( S ) and ∆ 0 i ∈ D ( S 0 ) for all i , and let w i b e the w eight function for ∆ i R ∆ 0 i . Define a new weigh t function w : S × S 0 → [0 , 1], b y w ( s, s 0 ) = P i ∈ I p i · w i ( s, s 0 ). – w ( s, s 0 ) > 0, then P i ∈ I p i · w i ( s, s 0 ) > 0, i.e, there exists some i ∈ I such that w i ( s, s 0 ) > 0, which giv es s R s 0 . – F or all s ∈ S , P s 0 ∈ S 0 w ( s, s 0 ) = P s 0 ∈ S 0 P i ∈ I p i · w i ( s, s 0 ) = P i ∈ I p i · P s 0 ∈ S 0 w i ( s, s 0 ) = P i ∈ I p i · ∆ i ( s ). – T o show that for all s 0 ∈ S 0 , P s ∈ S w ( s, s 0 ) = P i ∈ I p i · ∆ 0 i ( s 0 ) is similar. This gives P i ∈ I p i · ∆ i R P i ∈ I p i · ∆ 0 i . u t Based on the notion of lifting, we define the probabilistic alternating sim- ulation relation for play er I that extends the alternating simulation relation of [AHKV98]. The definition for play er I I can b e made in a similar wa y . Definition 1. Consider G , G 0 as two pr ob abilistic game structur es. A pr ob abilis- tic alternating I -simulation v⊆ S × S 0 is a r elation satisfying if s v s 0 , then – L ( s ) = L 0 ( s 0 ) , – for al l π 1 ∈ Π G , 1 I , ther e exists π 0 1 ∈ Π G 0 , 1 I , such that for al l π 0 2 ∈ Π G 0 , 1 I I , ther e exists π 2 ∈ Π G , 1 I I , such that δ ( s, π 1 , π 2 ) v δ 0 ( s 0 , π 0 1 , π 0 2 ) . Let R ⊆ S × S 0 and R 0 ⊆ S 0 × S 00 b e t wo relations, then R · R 0 is a relation on S × S 00 defined by s ( R · R 0 ) s 00 if there exists s 0 ∈ S 0 suc h that s R s 0 and s 0 R 0 s 00 . Lemma 8. (T r ansitivity of alternating simulation) Consider G , G 0 and G 00 b e thr e e pr ob abilistic game structur es. If v ⊆ S × S 0 and v 0 ⊆ S 0 × S 00 ar e pr ob- abilistic alternating I -simulations, then v · v 0 is a pr ob abilistic alternating I - simulation on S × S 00 . Pr o of. (sketc h) Let s v · v 0 s 00 , then b y definition there exists s 0 ∈ S 0 suc h that s v s 0 and s 0 v 0 s 00 . Therefore L ( s ) = L ( s 0 ) = L ( s 00 ). Let π 1 ∈ Π G , 1 I , then by definition there exists π 0 1 ∈ Π G 0 , 1 I suc h that for all π 0 2 ∈ Π G 0 , 1 I I there exists π 2 ∈ Π G , 1 I I suc h that δ ( s, h π 1 , π 2 i ) v δ 0 ( s 0 , h π 0 1 , π 0 2 i ). By s 0 v 0 s 00 , there exists π 00 1 ∈ Π G 00 , 1 I suc h that for all π 00 3 ∈ Π G 00 , 1 I I , there exists π 0 3 ∈ Π G 0 , 1 I I suc h that δ 0 ( s 0 , h π 0 1 , π 0 3 i ) v 0 δ 00 ( s 00 , h π 00 1 , π 00 3 i ). Then from ab o ve there also exists π 3 ∈ Π G , 1 I suc h that δ ( s, h π 1 , π 3 i ) v δ 0 ( s 0 , h π 0 1 , π 0 3 i ). W rite ∆ = δ ( s, h π 1 , π 3 i ), ∆ 0 = δ 0 ( s 0 , h π 0 1 , π 0 3 i ) and ∆ 00 = δ 00 ( s 00 , h π 00 1 , π 00 3 i ). w e need to show that ∆ v · v 0 ∆ 00 . Let w 1 b e a weigh t function for ∆ v ∆ 0 and w 2 a weigh t function for ∆ 0 v 0 ∆ 00 , define a new weigh t function w : S × S 00 → [0 , 1], by w ( s, s 00 ) = P s 0 ∈ S 0 w 1 ( s,s 0 ) · w 2 ( s 0 ,s 00 ) ∆ 0 ( s 0 ) . Let s ∈ S and s 00 ∈ S 00 . – If w ( s, s 00 ) > 0 then exists s 0 ∈ d ∆ 0 e such that w 1 ( s, s 0 ) > 0 and w 2 ( s 0 , s 00 ) > 0, which implies s v s 0 and s 0 v 0 s 00 . Therefore, s v · v 0 s 00 . – P s ∈ S w ( s, s 00 ) = P s ∈ S P s 0 ∈ S 0 w 1 ( s,s 0 ) · w 2 ( s 0 ,s 00 ) ∆ 0 ( s 0 ) = P s 0 ∈ S 0 w 2 ( s 0 ,s 00 ) ∆ 0 ( s 0 ) · P s ∈ S w 1 ( s, s 0 ) = P s 0 ∈ S 0 w 2 ( s 0 ,s 00 ) ∆ 0 ( s 0 ) · ∆ 0 ( s 0 ) = P s 0 ∈ S 0 w 2 ( s 0 , s 00 ) = ∆ 00 ( s 00 ) – Sho wing P s 00 ∈ S 00 w ( s, s 00 ) = ∆ ( s ) is similar. u t Lemma 8 can also be derived from the transitivity of probabilistic alternating forw ard sim ulation (Corollary 1) and the fact that ev ery probabilistic alternating sim ulation is also a probabilistic alternating forward simulation (Lemma 11). Based on the probabilistic forw ard sim ulation of Segala [Seg95a], and the alternating simulation of Alur et al. [AHKV98], we prop ose the notion of proba- bilistic alternating forward simulation. A forward simulation relates a state to a distribution of states, whic h requires a different wa y of lifting. Let R ⊆ S × D ( S ) b e a relation, write R for the smallest relation satisfying ∆ R Θ if there exists an index set { p i } i ∈ I satisfying Σ i ∈ I p i = 1, suc h that ∆ = Σ i ∈ I p i · s i , Θ = Σ i ∈ I p i · Θ i and s i R Θ i for all i . W e call R the forwar d lifting of R . F orward lifting has the follo wing similar prop erties as the previous lifting. Lemma 9. L et R b e a r elation on S ×D ( S ) and { p i } i ∈ I b e an index set satisfying P i ∈ I p i = 1 and ∆ i R ∆ 0 i for distributions ∆ i , ∆ 0 i ∈ D ( S ) for al l i , then P i ∈ I p i · ∆ i R P i ∈ I p i · ∆ 0 i , wher e R is the forwar d lifting of R . Lemma 10. L et ∆ ∈ D ( S ) , ∆ 0 ∈ D ( S 0 ) , and R a r elation on S . If ∆ R ∆ 0 , and ther e exist ∆ 1 , ∆ 2 , · · · ∈ D ( S ) and an index set { p i } I satisfying P i ∈ I p i = 1 and ∆ = P i ∈ I p i · ∆ i , then ther e exist ∆ 0 1 , ∆ 0 2 · · · ∈ D ( S 0 ) such that ∆ 0 = P i ∈ I p i · ∆ 0 i , and ∆ i R ∆ 0 i for al l i ∈ I , wher e R is the forwar d lifting of R . No w w e define the probabilistic alternating forward sim ulation relation for pla y er I , and the definition for play er I I can b e made in a similar wa y . Definition 2. Consider two pr ob abilistic game structur es G = h S, s 0 , L , Act , δ i and G 0 = h S 0 , s 0 0 , L 0 , Act 0 , δ 0 i . A pr ob abilistic alternating forwar d I -simulation v f ⊆ S × D ( S 0 ) is a r elation satisfying if s v f ∆ 0 , then – L ( s ) = L 0 ( s 0 ) for al l s 0 ∈ d ∆ 0 e , Fig. 2. An example showing that probabilistic alternating forw ard sim ulation is strictly w eaker than probabilistic alternating simulation. – for al l π 1 ∈ Π G , 1 I , ther e exists π 0 1 ∈ Π G 0 , 1 I , such that for al l π 0 2 ∈ Π G 0 , 1 I I , ther e exists π 2 ∈ Π G , 1 I I , such that δ ( s, π 1 , π 2 ) v f δ 0 ( ∆ 0 , π 0 1 , π 0 2 ) . Lemma 11. s v t implies s v f t . This lemma says that every probabilistic alternating sim ulation is a probabilis- tic forward sim ulation with a p oin t distribution on the right hand side of the relation. The other wa y do es not hold, i.e., probabilistic alternating forward sim ulation relates strictly more game structures than probabilistic alternating sim ulation. In Fig. 2, we assume Act I and Act I I are b oth singleton sets. One ma y find that there are no states in the set { s 0 2 , s 0 3 , s 0 4 , s 0 5 } in Fig. 2(b) that can sim ulate states s 3 and s 5 in Fig. 2(a). Therefore, we cannot establish a prob- abilistic alternating sim ulation from s 1 to s 0 1 . Ho wev er, s 1 is related to s 0 1 b y probabilistic alternating forward simulation, since s 3 ( s 5 ) can b e related to a uniform distribution o ver s 0 2 and s 0 3 ( s 0 4 and s 0 5 ). Before proceeding to the next step w e introduce the follo wing several auxil- iary lemmas. Lemma 12. Given { p i } i ∈ I with P i ∈ I p i = 1 , s ∈ S , { π i } i ∈ I ⊆ Π G , 1 I and π ∈ Π G , 1 I I , we have δ ( s, P i ∈ I p i · π i , π ) = P i ∈ I p i · δ ( s, π i , π ) . Pr o of. Let t ∈ S , then we hav e δ ( s, P i ∈ I p i · π i , π )( t )= δ ( s, P i ∈ I p i · π i ( s ) , π ( s ))( t ) = P a 1 ∈ Act 1 P a 2 ∈ Act 2 P i ∈ I p i · π i ( a 1 ) · π ( a 2 ) · δ ( s, a 1 , a 2 )( t ) = P i ∈ I p i · [ P a 1 ∈ Act 1 P a 2 ∈ Act 2 π i ( a 1 ) · π ( a 2 ) · δ ( s, a 1 , a 2 )( t )] = P i ∈ I p i · δ ( s, π i ( s ) , π ( s ))( t ) = P i ∈ I p i · δ ( s, π i , π )( t ) u t Lemma 13. L et G and G 0 b e two game structur es, { ∆ i } i ∈ I b e a set of distri- butions, { π i } i ∈ I a set of level 1 I -str ate gies, π ∈ Π G 0 , 1 I I and { p i } i ∈ I satisfies P i ∈ I p i = 1 , we have δ ( P i ∈ I p i · ∆ i , π 0 , π ) = P i ∈ I p i · δ ( ∆ i , π i , π ) , wher e π 0 is a level 1 I str ate gy define d by π 0 ( s ) = X i ∈ I p i · ∆ i ( s ) · π i ( s ) P i ∈ I p i · ∆ i ( s ) Pr o of. W rite LH S = δ ( P i ∈ I p i · ∆ i , π 0 , π ) and RH S = P i ∈ I p i · δ ( ∆ i , π i , π ), w e need to show for all t ∈ S , LH S ( t ) = R H S ( t ). W rite ∆ for the distribution P i ∈ I p i · ∆ i . Let t ∈ S , then b y definition LH S ( t )= P s ∈d ∆ e P i ∈ I p i · ∆ i ( s ) δ ( s, π 0 , π )( t ) = P s ∈d ∆ e ∆ ( s ) · δ ( s, π 0 , π )( t ) = P s ∈d ∆ e ∆ ( s ) · δ ( s, π 0 ( s ) , π ( s ))( t ) = P s ∈d ∆ e ∆ ( s ) · δ ( s, P j ∈ I p j · ∆ j ( s ) π j ( s ) ∆ ( s ) , π ( s ))( t ) = P s ∈d ∆ e ∆ ( s ) · P j ∈ I p j · ∆ j ( s ) ∆ ( s ) δ ( s, π j ( s ) , π ( s ))( t ) By Lemma 12 = P s ∈d ∆ e · P j ∈ I p j · ∆ j ( s ) · δ ( s, π j ( s ) , π ( s ))( t ) = P j ∈ I p j · P s ∈d ∆ e · ∆ j ( s ) · δ ( s, π j ( s ) , π ( s ))( t ) = P j ∈ I p j · P s ∈d ∆ j e · ∆ j ( s ) · δ ( s, π j ( s ) , π ( s ))( t ) = P j ∈ I p j · δ ( ∆ i , π i , π )( t ) By definition = RH S ( t ) u t The next result sho ws that the definition of forward sim ulation also works on the lifted relation. Lemma 14. If ∆ v f Θ , then for al l π 1 ∈ Π G , 1 I , ther e exists π 2 ∈ Π G 0 , 1 I , such that for al l π 0 2 ∈ Π G 0 , 1 I I , ther e exists π 0 1 ∈ Π G , 1 I I , such that δ ( ∆, π 1 , π 0 1 ) v f δ ( Θ, π 2 , π 0 2 ) . Pr o of. By definition there exists a set { p i } i ∈ I suc h that ∆ = P i ∈ I p i · s i , Θ = P i ∈ I p i · Θ i and s i v f Θ i . Let π 1 ∈ Π G , 1 I b e a (mixed) I -strategy . Then b y definition, for all i ∈ I there exists π i ∈ Π G 0 , 1 I suc h that for all π 0 i ∈ Π G 0 , 1 I I , there exists π 00 i ∈ Π G , 1 I I suc h that δ ( s i , π 1 , π 00 i ) v f δ ( s i , π i , π 0 i ). No w we tak e π 2 defined b y π 2 ( s ) = P i ∈ I p i Θ i ( s ) Θ ( s ) π i ( s ) for all s b e the required (lev el 1) I -strategy . Let π 0 2 ∈ Π G 0 , 1 I I , we pro ve as follo ws the existence of another I I -strategy π 0 1 ∈ Π G , 1 I I that satisfies δ ( ∆, π 1 , π 0 1 ) v f δ ( Θ, π 2 , π 0 2 ). F or each i ∈ I , by s i v f Θ i , there exists π 00 i ∈ Π G , 1 I I satisfying δ ( s i , π 1 , π 00 i ) v f δ ( Θ i , π 2 , π 0 2 ). Then we ha ve P i ∈ I p i · δ ( s i , π 1 , π 0 i ) v f P i ∈ I p i · δ ( Θ i , π 2 , π 0 2 ), b y Lemma 7. The required (mixed) I I -strategy π 0 1 is defined by π 0 1 ( s ) = P i ∈ I p i s i ( s ) ∆ ( s ) π 00 i ( s ) for all s , and the result follo ws from Lemma 13. u t Consequen tly , w e are able to show that lifted probabilistic alternating forward sim ulations are transitive. Corollary 1. (T r ansitivity of alternating forwar d simulation) L et v f b e a pr ob- abilistic alternating forwar d I -simulation, then ∆ 1 v f ∆ 2 and ∆ 2 v f ∆ 3 implies ∆ 1 v f ∆ 3 . 6 F orward I -Sim ulation is Sound for I -P A TL This section establishes the main result of the pap er: a relationship b et w een probabilistic forward I -simulation and I -P A TL formulas. Recall that a I -P A TL form ula has only strategy mo dalities h h I i i and h h∅i i , and negations are only al- lo wed to app ear immediately before the propositions. F or readabilit y w e write h h I i i for h h{ I }i i . Let G and G 0 b e tw o PGSs, ∆ ∈ D ( S ) and ∆ 0 ∈ D ( S 0 ) such that ∆ v f ∆ 0 b y a probabilistic alternating forward I -simulation. W e need to sho w that ∆ | = φ implies ∆ 0 | = φ for all I -P A TL form ula φ . Our pro of relies on the existence of pla yer I I ’s optimal strategies for path form ulas as winning ob jectives (as shown in Sect. 4). Supp ose π 1 is a I strategy that enforces φ , w e construct another I strategy π 0 1 that sim ulates π all along the w ay , in the sense that pro vided the optimal I I strategy π 0 2 there exists another I I strategy π 2 suc h that the probabilistic e xecution E ( G , h π 1 , π 2 i , ∆ ) will b e “simu- lated” b y the probabilistic execution E ( G 0 , h π 0 1 , π 0 2 i , ∆ 0 ). Since π 1 enforces φ , the E ( G , h π 1 , π 2 i , ∆ ) satisfies φ , we sho w that it is also the case of E ( G 0 , h π 0 1 , π 0 2 i , ∆ 0 ). Let E = h E , ∆, L E , δ E i and E 0 = h E 0 , ∆ 0 , L E 0 , δ E 0 i b e probabilistic executions of G and G 0 , resp ectively . Also let v f ⊆ S × D ( S 0 ) b e a probabilistic alternating forw ard I -simulation. W e say the pair ( E , E 0 ) is an instanc e of simulation , by writing E v E 0 , if there exists a (simulation) relation v 0 ⊆ E × D ( E 0 ), such that – ∆ v 0 ∆ 0 , – if e v 0 Θ then last ( e ) v f last ( Θ ), – if e v 0 Θ then δ E ( e ) v δ E 0 ( Θ ), where last ( Θ ) is a distribution satisfying last ( Θ )( s ) = P last ( e )= s Θ ( e ). A few prop erties of the relation v 0 are as follo ws. Lemma 15. 1. ∆ v 0 Θ implies δ E ( ∆ ) v 0 δ E 0 ( Θ ) . 2. ∆ v 0 Θ and ∆ = ∆ 1 ⊕ α ∆ 2 with α ∈ [0 , 1] , then ther e exist Θ 1 , Θ 2 such that ∆ 1 v 0 Θ 1 , ∆ 2 v 0 Θ 2 , and Θ = Θ 1 ⊕ α Θ 2 . A proof of part (1) is b y definition of v 0 and Lemma 9, and part (2) holds b y Lemma 10. Let ∆ b e a state distribution of G , ∆ 0 b e a state distribution of G 0 , and ∆ v f ∆ 0 . Supp ose π 1 is a I strategy in G that enforces φ with probability at least α , and π 0 2 is a I I strategy in G 0 , step-b y-step w e establish a I strategy π 0 1 and a I I strategy π 2 , so that the probabilistic executution decided by π 1 and π 2 from ∆ will b e sim ulated b y the probabilistic executution decided by π 1 and π 2 from ∆ 0 . Lemma 16. L et G = h S, s 0 , L , Act , δ i and G 0 = h S 0 , s 0 0 , L 0 , Act 0 , δ 0 i b e two PGSs. If ∆ v f ∆ 0 , then for al l π 1 ∈ Π G I and π 0 2 ∈ Π G 0 I I , ther e exists π 0 1 ∈ Π G 0 I and π 2 ∈ Π G I I , such that E ( G , h π 1 , π 2 i , ∆ ) v E 0 ( G 0 , h π 0 1 , π 0 2 i , ∆ 0 ) . Pr o of. W e construct π i 1 and π i 2 as a level 1 strategies of play er I and I I for all i ∈ N , and define π 0 1 ( γ · s ) = π | γ | +1 1 ( s ) for all γ ∈ S ∗ and s ∈ S . And π 2 ( γ · s ) = π | γ | +1 2 ( s ) for all γ ∈ ( S 0 ) ∗ and s ∈ S 0 . Since ∆ v f ∆ 0 , then b y Lemma 14, there exists π 1 1 ∈ Π G 0 , 1 I , such that for all π 00 2 ∈ Π G 0 , 1 I I there exists π 000 2 ∈ Π G , 1 I I suc h that δ ( ∆, π 1 , π 000 2 ) v f δ ( ∆ 0 , π 1 1 , π 00 2 ). So if w e tak e the first level of π 0 2 , there exists π 1 2 ∈ Π G , 1 I I , such that δ ( ∆, π 1 , π 1 2 ) v f δ ( ∆ 0 , π 1 1 , π 0 2 ). W e define ∆ 2 ∈ D ( S 2 ) b y ∆ 2 ( s 1 s 2 ) = ∆ ( s 1 ) · δ ( s 1 , π 1 , π 1 2 )( s 2 ), and ∆ 0 2 ∈ D (( S 0 ) 2 ) in a similar wa y . W e also ‘truncate’ the strategy π 1 b y defining π 1 (2) ∈ Π G , 1 I in the wa y that π 1 (2)( s ) = P s 0 ∈d ∆ e ∆ ( s 0 ) · π 1 ( s 0 s ). And we define π 0 2 (2) in a similar wa y . Supp ose we ha ve ∆ n , ∆ 0 n ∈ D ( S n ), and π 1 ( n ) ∈ Π G , 1 I , and π 0 2 ( s ) ∈ Π G 0 , 1 I I , in the similar wa y to ab o ve, we construct π n 1 and π n 2 , such that δ ( ∆, π 1 ( n ) , π n 2 ) v f δ ( ∆ 0 , π n 1 , π 0 2 ( n )). Then w e define ∆ n +1 ∈ D ( S n +1 ) b y ∆ n +1 ( s 1 . . . s n s n +1 ) = ∆ n ( s 1 . . . s n ) · δ ( s n , π 1 ( n ) , π n 2 )( s n +1 ), and ∆ 0 n +1 ∈ D (( S 0 ) n +1 ) by ∆ 0 n +1 ( s 1 . . . s n s n +1 ) = ∆ 0 n ( s 1 . . . s n ) · δ ( s 0 n , π n 1 , π 2 ( n ))( s n +1 ). W e then define π 1 ( n + 1) ∈ Π G , 1 I b y π 1 ( n + 1)( s ) = P γ ∈d ∆ n e ∆ n ( γ ) · π 1 ( γ · s ), and π 0 2 ( n +1) ∈ Π G 0 , 1 I I b y π 0 2 ( n +1)( s ) = P γ ∈d ∆ 0 n e ∆ 0 n ( γ ) · π 0 2 ( γ · s ). It is easily verifiable that w e hav e established tw o probabilistic executions satisfying E ( G , h π 1 , π 2 i , ∆ ) v E 0 ( G 0 , h π 0 1 , π 0 2 i , ∆ 0 ), by taking a probabilistic alter- nating forward simulation as v 0 . u t In order to measure the probabilit y of a path formula to b e satisfied when the strategies from b oth pla yer I and pla yer I I are fixed, w e define a relation | =  α for probabilistic executions. Definition 3. L et G b e a pr ob abilistic game structur e, E ( ∆ ) = h E , ∆, L E , δ E i a pr ob abilistic exe cution determine d by a str ate gy ve ctor π E , and ψ a p ath formula, define E ( ∆ ) | =  α ψ iff P r ∆ E ( { ρ ∈ [ s ∈d ∆ e G ( π E , s ) | ρ | = ψ } )   α It is conceiv able that in a probabilistic execution every finite or infinite trace in E ∗ ∪ E ω maps to a trace in G , in the w ay that ρ = e 1 e 2 e 3 . . . is a trace in E implies that pr oj ( ρ ) = last ( e 1 ) last ( e 2 ) last ( e 3 ) . . . is a play in G , where the function pr oj pro jects every finite sequence of states in E in to its last state in S . Consequen tly , we let P r ∆ E b e a probabilistic measure ov er E ω , suc h that for the cone sets (of finite traces), w e hav e P r ∆ E ( e ) = ∆ ( last ( e )), and P r ∆ E ( γ · e 1 · e 2 ) = P r ∆ E ( γ · e 1 ) · δ E ( e 1 )( e 2 ), for γ ∈ E ∗ and e 1 , e 2 ∈ E . Let ρ b e an infinite trace in E , we write ρ | = ψ iff proj ( ρ ) | = ψ . Similarly , for a state form ula φ and e ∈ E , write e ∈ J φ K iff last ( e ) ∈ J φ K . In the following we study the prop erties of the satisfaction relation for a probabilistic execution to satisfy a I -P A TL path form ula b y means of unfolding. Lemma 17. L et φ , φ 1 and φ 2 b e I -P A TL (state) formulas, and   ∈ { >, ≥} then 1. E ( ∆ ) | =  α  φ iff ther e exists α 0   α , such that δ E ( ∆ ) = ∆ 1 ⊕ α 0 ∆ 2 with d ∆ 1 e ∩ d ∆ 2 e = ∅ , and ∆ 1 | = φ . 2. E ( ∆ ) | =  α φ 1 U ≤ k φ 2 iff ther e exists a finite se quenc e of triples {h ( ∆ i, 0 , α i, 0 ) , ( ∆ i, 1 , α i, 1 ) , ( ∆ i, 2 , α i, 2 ) i} 0 ≤ i ≤ j for some j ≤ k , with d ∆ i, e ∩ d ∆ i, 0 e = ∅ for al l distinct ,  0 ∈ { 0 , 1 , 2 } and 0 ≤ i ≤ j , such that (1) X i ∈ [0 ...j ]   α i, 1 · Y i 0 ∈ [0 ...i − 1] α i 0 , 0     α, (2) ∆ = P  ∈{ 0 , 1 , 2 } α 0 , · ∆ 0 , , and δ E ( ∆ i, 0 ) = P  ∈{ 0 , 1 , 2 } α i +1 , · ∆ i +1 , for al l 0 ≤ i < j , (3) ∆ i, 0 | = φ 1 and ∆ i, 1 | = φ 2 for al l 0 ≤ i ≤ j . 3. E ( ∆ ) | =  α φ 1 U φ 2 iff ther e exists a finite or infinite se quenc e of triples {h ( ∆ i, 0 , α i, 0 ) , ( ∆ i, 1 , α i, 1 ) , ( ∆ i, 2 , α i, 2 ) i} 0 ≤ iα φ 1 U φ 2 , and the pro of metho ds for the other P A TL path constructors are just similar. Since for all t ∈ d ∆ 0 e there exists an optimal strategy π t for the winning ob jective ¬ φ 1 R¬ φ 2 b y Lemma 2(1), and w e com bine these strategies in to a single strategy π 0 2 satisfying π 0 2 ( t · α ) = π t ( t · α ) for all t ∈ d ∆ 0 e and α ∈ S ∗ . Then π 0 2 is optimal for ¬ φ 1 R¬ φ 2 on ∆ 0 . Then b y Lemma 16, there exist π 2 ∈ Π G I I and π 0 1 ∈ Π G 0 I suc h that E ( G , h π 1 , π 2 i , ∆ ) v E 0 ( G 0 , h π 0 1 , π 0 2 i , ∆ 0 ). Since π 1 enforces φ 1 U φ 2 with probability greater than α , we ha ve E ( ∆ ) | = >α φ 1 U φ 2 . Then b y Lemma 17(3) there exists a finite or infinite sequence of triples {h ( ∆ i, 0 , α i, 0 ) , ( ∆ i, 1 , α i, 1 ) , ( ∆ i, 2 , α i, 2 ) i} 0 ≤ i α , (2) ∆ 0 = P  ∈{ 0 , 1 , 2 } α 0 , · ∆ 0 0 , , and δ E ( ∆ 0 i, 0 ) = P  ∈{ 0 , 1 , 2 } α i +1 , · ∆ 0 i +1 , for all 0 ≤ i < j , (3) ∆ i, 0 v f ∆ 0 i, 0 and ∆ i, 1 v f ∆ 0 i, 1 for all 0 ≤ i < j . By induction h yp othesis w e hav e ∆ 0 i, 0 | = φ 1 and ∆ 0 i, 1 | = φ 2 for all 0 ≤ i < j . Therefore E ( ∆ 0 ) | = >α φ 1 U φ 2 b y Lemma 17(3). Since π 0 2 is an optimal strategy of I I , we ha ve ∆ 0 | = h h I i i >α φ 1 U φ 2 b y Lemma 3. F or a form ula h h∅i i  α ψ w e apply the same proof strategies as for h h I i i  α ψ , except that pla yer I do es not need to enforce ψ with a certain probabilit y   α since every probabilistic execution generated by a pair of I and I I strategies will enforce ψ with that probability . u t 7 Probabilistic Alternating Bisim ulation If a probabilistic alternating simulation is symmetric, we call it a probabilistic alternating bisimulation. Definition 4. Consider two pr ob abilistic game structur es G = h S, s 0 , L , Act , δ i and G 0 = h S 0 , s 0 0 , L 0 , Act 0 , δ 0 i . A pr ob abilistic alternating I -bisimulation '⊆ S × S 0 is a symmetric r elation satisfying if s ' s 0 , then – L ( s ) = L 0 ( s 0 ) , – for al l π 1 ∈ Π G , 1 I , ther e exists π 0 1 ∈ Π G 0 , 1 I , such that for al l π 0 2 ∈ Π G 0 , 1 I I , ther e exists π 2 ∈ Π G , 1 I I , such that δ ( s, π 1 , π 2 ) ' δ 0 ( s 0 , π 0 1 , π 0 2 ) , wher e ' is a lifting of ' by weight functions. Since every probabilistic alternating I -simulation is also a probabilistic al- ternating forward I -sim ulation b y treating the righ t hand side state as a p oin t distribution (Lemma 11), the lifted probabilistic alternating I -simulation is also a lifted probabilistic alternating forw ard I -simulation. This fact extends for bisim- ulation. A probabilistic alternating I -bisim ulation also preserv es form ulas in L I . Moreo ver w e write L + I for the set of formulas defined as follo ws, whic h allows negations to app ear an ywhere in a formula, and further w e are able to show that probabilistic alternating bisim ulation preserv es all prop erties expressed in L + I . φ := p | ¬ φ | φ 1 ∧ φ 2 | h h A 0 i i  α  φ | h h A 0 i i  α φ 1 U ≤ k φ 2 | h h A 0 i i >α φ 1 U φ 2 Theorem 2. L et G = h S, s 0 , L , Act , δ i and G 0 = h S 0 , s 0 0 , L 0 , Act 0 , δ 0 i b e two PGSs, '⊆ S × S 0 is a pr ob abilistic alternating I -bisimulation. F or al l s ∈ S and s 0 ∈ S 0 with s ' s 0 and φ ∈ L + I , we have G , s | = φ iff G 0 , s 0 | = φ . The pro of methodology basically follo ws that of Theorem 1, b esides that when- ev er ∆ ' ∆ 0 and ∆ | = ¬ φ , we sho w that if there were s 0 ∈ d ∆ e 0 suc h that G 0 , s 0 | = φ then w e w ould also ha ve G , s | = φ for some s ∈ d ∆ e , whic h is a contradiction. And from that we hav e ∆ 0 | = ¬ φ as well. 8 Conclusion and F uture W ork W e rep ort our first results on probabilistic alternating simulation relations. W e ha ve introduced tw o notions of simulation for probabilistic game structures – probabilistic alternating simulation and probabilistic alternating forward simu- lation, follo wing the seminal works of Segala and Lynch [Seg95a,SL95] on proba- bilistic sim ulation relations and the work of Alur et al. [AHKV98] on alternating refinemen t relations for non-probabilistic game structures. Our main effort has b een devoted to a logical c haracterization for probabilistic alternating sim ulation relations, by showing that they preserve a fragmen t of P A TL formulas. On our wa y to the main result, we find that the pro of strategy accommo- dated in [AHKV98] no longer applies, due to the failure in reconstructing a strategy from sub-strategies with the existence of probabilistic behaviors. Note that alternating sim ulations rely on mimicking b ehaviors b y strategies of depth one, while enforcing a P A TL prop ert y needs to fix a general strategy (of infinite depth) from one party regardless of any strategies of the other. W e circumv en t this problem by incorp orating the results of probabilistic determinacy [Mar98] and the existence of optimal strategies [dAM04] in sto c hastic games. There are sev eral wa ys to proceed. W e w ant to study the completeness of log- ical characterization for probabilistic alternating forw ard simulation. It is also of our in terest to in vestigate the complexit y for chec king probabilistic alternating sim ulation relations by studying the results in the literature [AHKV98,BEMC00]. Our w ork w as partially motiv ated b y the paper [ASW09], where P A TL is used to formalize a b alanc e d property for a probabilistic con tract signing protocol. Here, a balanced proto col means that a dishonest participant never has a strategy to unilaterally determine the outcome of the proto col. It is in teresting to see how m uch the developmen t of simulation relations for probabilistic game structures can help the verification of such kind of security protocols. References [AHK97] R. Alur, T. A. Henzinger, and O. Kupferman. Alternating-time tempo- ral logic. In Pr o c. 38th Annual Symp osium on F oundations of Computer Scienc e , pages 100–109. IEEE Computer So ciet y , 1997. [AHK02] R. Alur, T. A. Henzinger, and O. Kupferman. Alternating-time temporal logic. Journal of ACM , 49(5):672–713, 2002. [AHKV98] R. Alur, T. A. Henzinger, O. Kupferman, and M. Y. V ardi. Alternating refinemen t relations. In Pr oc. 9th Confer enc e on Concurr ency The ory , v ol- ume 1466 of L e ctur e Notes in Computer Scienc e , pages 163–178. Springer, 1998. [ASW09] M. Aizatulin, H. Schnoor, and T. Wilke. Computationally sound analysis of a probabilistic contract signing proto col. In Pr o c. 14th Europ e an Sympo- sium on R ese ar ch in Computer Se curity , volume 5789 of L e ctur e Notes in Computer Scienc e , pages 571–586. Springer, 2009. [BEMC00] C. Baier, B. Engelen, and M. E. Ma jster-Cederbaum. Deciding bisimilarit y and similarity for probabilistic pro cesses. Journal of Computer and System Scienc es , 60(1):187–231, 2000. [CdAH06] K. Chatterjee, L. de Alfaro, and T. A. Henzinger. The complexity of quan- titativ e concurrent parity games. In Pr o c. 17th A nnual A CM-SIAM Sym- p osium on Discrete Algorithm , pages 678–687. ACM, 2006. [CL07] T. Chen and J. Lu. Probabilistic alternating-time temp oral logic and mo del c hecking algorithm. In Pr o c. 4th Confer enc e on F uzzy Systems and Know l- e dge Disc overy , pages 35–39. IEEE Computer Society , 2007. [dAHK98] L. de Alfaro, T. A. Henzinger, and O. Kupferman. Concurren t reachabilit y games. In Pr o c. 39th A nnual IEEE Symp osium on F oundations of Computer Scienc e , pages 564–575. IEEE Computer So ciet y , 1998. [dAM04] L. de Alfaro and R. Ma jumdar. Quantitativ e solution of omega-regular games. Journal of Computer and System Scienc es , 68(2):374–397, 2004. [DGJP02] J. Desharnais, V. Gupta, R. Jagadeesan, and P . Panangaden. W eak bisim- ulation is sound and complete for PCTL ? . In Pr o c. 13th Confer enc e on Concurr ency The ory , v olume 2421 of L e ctur e Notes in Computer Scienc e , pages 355–370. Springer, 2002. [Eme90] E. A. Emerson. T emp oral and mo dal logic. In Handb o ok of The or etic al Computer Scienc e (B) , pages 955–1072. MIT Press, 1990. [Han94] H. Hansson. Time and Pr ob ability in F ormal Design of Distribute d Systems . Elsevier, 1994. [LSV07] N. A. Lync h, R. Segala, and F. W. V aandrager. Observing branching structure through probabilistic con texts. SIAM Journal of Computing , 37(4):977–1013, 2007. [Mar98] D. A. Martin. The determinacy of Blackw ell games. Journal of Symb olic L o gic , 63(4):1565–1581, 1998. [Mil89] R. Milner. Communic ation and Concurr ency . Prentice Hall, 1989. [PS07] A. P arma and R. Segala. Logical c haracterizations of bisimulations for discrete probabilistic systems. In Pr o c. 10th Confer enc e on F oundations of Softwar e Scienc e and Computational Structur es , v olume 4423 of L e ctur e Notes in Computer Scienc e , pages 287–301. Springer, 2007. [RF91] T. E. S. Ragha v an and J. A. Filar. Algorithms for sto chastic games – A surv ey . Mathematic al Metho ds of Oper ations R ese arch , 35(6):437–472, 1991. [Seg95a] R. Segala. A compositional trace-based semantics for probabilistic au- tomata. In Pr o c. 6th Confer enc e on Concurr ency Theory , volume 962 of L e ctur e Notes in Computer Scienc e , pages 234–248. Springer, 1995. [Seg95b] R. Segala. Mo deling and V erific ation of R andomize d Distribute d Re al-Time Systems . PhD thesis, Massac husetts Institute of T echnology , 1995. [SL95] R. Segala and N. A. Lynch. Probabilistic sim ulations for probabilistic pro- cesses. Nor dic Journal of Computing , 2(2):250–273, 1995. [Tho91] W. Thomas. Automata on infinite ob jects. In Handb o ok of The or etical Computer Scienc e, (V ol. B): F ormal Mo de ls and Sematics , pages 133–192. Elsevier, 1991. [vGSS95] R. J. v an Glabb eek, S. A. Smolk a, and B. Steffen. Reactiv e, generative, and stratified models of probabilistic pro cesses. Information and Computation , 121:59–80, 1995. [vNM47] J. von Neumann and O. Morgenstern. The ory of Games and Ec onomic Behavior . Princeton Unviersit y Press, 1947. A A pro of of Lemma 2 The pro of relies on the represen tation of a solution of an L TL path form ula as a winning ob jectiv e in quan titative game µ -calculus [dAM04] for a t w o-play er ( I and I I ) game. Its grammar is defined as follows. φ := Q | x | φ 1 ∨ φ 2 | φ 1 ∧ φ 2 | P pre I ( φ ) | P pre I I ( φ ) | µx.φ | ν x.φ The semantics of suc h form ulas map each form ula in to F , the function space S → [0 , 1]. A mem b er f ∈ F gives an exp ected v alue f ( s ) for pla yer I to win the game on every state s ∈ S . There is a partial order defined on F in the w ay that given tw o functions f , g ∈ F , f ≤ g if f ( s ) ≤ g ( s ) for all s ∈ S . F or Q ⊆ S , it represen ts a function that Q ( s ) = 1 if s ∈ Q and Q ( s ) = 0 otherwise. F or conjunction and disjunction, they are defined as ( f ∧ g )( s ) = min { f ( s ) , g ( s ) } for all s ∈ S , and ( f ∨ g )( s ) = max { f ( s ) , g ( s ) } for all s ∈ S . The quantitativ e predecessor op erator P pr e I for play er I and for every f ∈ F is b y P pre I ( f )( s ) = F π 1 ∈ Π I d π 2 ∈ Π I I P s 0 ∈d δ ( s, h π 1 ,π 2 i ) e δ ( s, h π 1 , π 2 i )( s 0 ) f ( s 0 ) for all s ∈ S . The op erator P pre I I can b e defined in a similar wa y . Intuitiv ely , based on f , P pr e i ( f ) gives the maximal exp ectation of play er i on each state s after one mo ve, and the existence of such maximal strategy and v alues are guaran teed by the minimax theorem [vNM47]. Finally , µx.φ ( x ) = d { f ∈ F | φ ( f ) ≤ f } and ν x.φ ( x ) = F { f ∈ F | φ ( f ) ≥ f } . The existence of the optimal strategy for pla y er I on an L TL ob jectiv e can b e sk etched as follows. – F or  φ , we construct the optimal strategy from P pre I ( φ ) by solving a ma- trix game on each state s ∈ S on reac hing states in J φ K . In this case w e only need to construct a lev el 1 strategy on every state, with its existence guaran teed b y the minimax theorem. – F or b ounded un til φ 1 U ≤ k φ 2 , w e do the following construction recursiv ely and pro ve the prop erty by induction. φ 1 U ≤ 0 φ 2 ≡ J φ 2 K w orks for every strat- egy in a state in J φ 2 K . F or k > 0, we interpret φ 1 U ≤ k φ 2 as φ 2 ∨ ( φ 1 ∧ P pre I ( φ 1 U ≤ k − 1 φ 2 )). Then supp ose there exists an optimal strategy for φ 1 U ≤ k − 1 φ 2 , w e only need to prolong the optimal strategy b y one additional level, based on the exp ected v alue already computed for φ 1 U ≤ k − 1 φ 2 . – The case of bounded release φ 1 R ≤ k φ 2 it can be shown in a similar w a y as the ab ov e case, by letting φ 1 R ≤ 0 φ 2 as J φ 2 K , and φ 1 R ≤ k φ 2 as φ 2 ∧ ( φ 1 ∨ P pre I ( φ 1 R ≤ k − 1 φ 2 )) for eac h k > 0. – F or un b ounded release φ 1 R φ 2 , our argument resembles the pro of of [dAM04, Lemma 2] on safety games. The v alue of the game for play er I as the protag- onist is in terpreted as the function f = ν x.φ 2 ∧ ( φ 1 ∨ P pr e I ( x )), and there ex- ists a memoryless strategy π 1 ∈ Π G , 1 I for pla y er I so that on eac h state s ∈ S , π 1 ( s ) ∈ D ( Act I ) is the best c hoice (in the matrix game on s ) play er I can mak e according to the greatest fixed point f , i.e., for all memoryless pla yer I I strategies π 2 ∈ Π G , 1 I I , w e hav e P s 0 ∈ S δ ( s, π 1 , π 2 )( s 0 ) · f ( s 0 ) ≥ f ( s ). W e sho w that π 1 is the strategy that guaran tees f ( s ) on eac h state s in the general sense. Let π 2 ∈ Π G I I b e an arbitrary play er I I strategy , and s ∈ S , we are go- ing to show that in the probabilistic execution E ( G , π 1 ∪ π 2 , s ) = h E , s, L , δ i , w e ha ve E ( s ) | = ≥ f ( s ) φ 1 R φ 2 . In order to do so, w e giv e the following in ter- mediate result that E ( s ) | = ≥ f ( s ) φ 1 R n φ 2 for all n ∈ N . W e prov e this by induction on n ∈ N that E ( e ) | = ≥ f ( last ( e )) φ 1 R ≤ n φ 2 for all e ∈ E . By abuse of the notation we treat π 1 also as a general strategy such that π 1 ( γ ) = π 1 ( last ( γ )) for all γ ∈ S + . W e also write π e 2 as the “truncated” strategy of π 2 , by defining π e 2 ( s · γ ) = π 2 ( e · γ ) for all s ∈ S and γ ∈ S + . T o simplify notation we write P r e E ( π 1 , π 2 , ψ ) for P r e E ( { ρ ∈ G (( π 1 , π 2 ) , last ( e )) | ρ | = ψ } ). Base case: let e ∈ E , n = 0 and φ 1 R ≤ 0 φ 2 ≡ φ 2 , we hav e P r e E ( π 1 , π e 2 , φ 1 R ≤ 0 φ 2 ) = 1 ≥ f ( last ( e )) = 1 if last ( e ) ∈ J φ 2 K , and P r e E ( π 1 , π e 2 , φ 1 R ≤ 0 φ 2 ) = 0 ≥ f ( last ( e )) = 0 otherwise. Supp ose this holds up to lev el n , we need to show the case of n + 1. • If last ( e ) ∈ J ¬ φ 2 K , then P r e E ( π 1 , π e 2 , φ 1 R ≤ n +1 φ 2 ) = 0 ≥ f ( last ( e )). • If last ( e ) ∈ J φ 2 K ∩ J φ 1 K , then P r e E ( π 1 , π e 2 , φ 1 R ≤ n +1 φ 2 ) = 1 ≥ f ( last ( e )). • If last ( e ) ∈ J φ 2 K ∩ J ¬ φ 1 K , then P r e E ( π 1 , π e 2 , φ 1 R ≤ n +1 φ 2 ) = P e 0 ∈d δ ( e ) e δ ( e )( e 0 ) · P r e 0 E ( π 1 , π e 0 2 , φ 1 R ≤ n φ 2 ). By I.H., P r e 0 E ( π 1 , π e 0 2 , φ 1 R ≤ n φ 2 ) ≥ f ( last ( e 0 )), we ha ve P r e E ( π 1 , π e 2 , φ 1 R ≤ n +1 φ 2 ) ≥ P e 0 ∈d δ ( e ) e δ ( e )( e 0 ) · f ( last ( e 0 )). By definition w e ha ve f ( last ( e )) = d π 0 ∈ Π G , 1 I I P s 0 ∈ δ ( last ( e ) ,π 1 ,π 0 ) δ ( last ( e ) , π 1 , π 0 )( s 0 ) f ( s 0 ) ≤ P s 0 ∈ δ ( last ( e ) ,π 1 ,π 2 ) δ ( last ( e ) , π 1 , π 2 )( s 0 ) f ( s 0 ) = P e 0 ∈d δ ( e ) e δ ( e )( e 0 ) · f ( last ( e 0 )). The last equiv alence is b y definition of E ( G , π 1 ∪ π 2 , s ). The result im- mediately follows. P r s E ( π 1 , π 2 , φ 1 R φ 2 ) = lim n →∞ P r s E ( π 1 , π 2 , φ 1 R n φ 2 ), we hav e P r s E ( π 1 , π 2 , φ 1 R φ 2 ) ≥ f ( s ). Since π 2 is arbitrarily c hosen, w e ha ve π 1 is a pla y er I strategy that enforces φ 1 R φ 2 with probability at least f ( s ). Moreov er, since s is arbitrarily c hosen, π 1 is optimal on s for all s ∈ S . The existence of  -optimal strategies for φ 1 U φ 2 for all  > 0 is guaranteed by the existence of  -optimal strategies for all ω -regular winning ob jectiv es, as sho wn in [dAM04]. B A pro of sk etc h of Lemma 17 1. Define the set { s ∈ d δ E ( ∆ ) e | s | = φ } . Now it is obvious that for all infinite run ρ ∈ E ω , ρ | =  φ iff ρ (1) | = φ . The result immediately follows. 2. The pro of this item is similar to the b elo w in its finite case. 3. Giv en the probabilistic execution E ( ∆ ), intuitiv ely , each e ∈ E represen ts a finite run within E ( ∆ ). W e construct as follows a maximal sequence of triples {h ( ∆ i, 0 , α i, 0 ) , ( ∆ i, 1 , α i, 1 ) , ( ∆ i, 2 , α i, 2 ) i} for i ∈ N . F or all e ∈ E , w e define E i, 1 = { e ∈ E | e = s 1 s 2 . . . s i , s i | = φ 2 , s j | = φ 1 for all j < i } , and E i, 0 = { e ∈ E | e = s 1 s 2 . . . s i , s j | = φ 1 for all j ≤ i } . In tuitively , E i, 1 are the prefixes of those runs that satisfy φ 1 U φ 2 , and E i, 1 are the prefixes of the runs that might satisfy φ 1 U φ 2 . F urther we define ∆ 0 = ∆ and E 0 , 2 = d ∆ 0 e \ ( E 0 , 0 ∪ E 0 , 1 ), and α 0 , = ∆ 0 ( E 0 , ) for  ∈ { 0 , 1 , 2 } . Then for each i ∈ N , recursively define ∆ i +1 = δ ( ∆ i, 0 ), E i +1 , 2 = d ∆ i +1 e \ ( E i +1 , 0 ∪ E i +1 , 1 ), α i +1 , = ∆ i +1 ( E i +1 , ) for all  ∈ { 0 , 1 , 2 } . Consequen tly , w e hav e ∆ i, ( e ) = ∆ i ( e ) /α i, if e ∈ E i, and 0 otherwise, for all i ∈ N and  ∈ { 0 , 1 , 2 } , pro vided α i, 6 = 0. If α i, = 0 w e let ∆ i, b e empt y , i.e., it assigns every e ∈ E to 0. It is easily v erifiable that d ∆ i, e ∩ d ∆ i, 0 e = ∅ for all distinct ,  0 ∈ { 0 , 1 , 2 } , since E i, ∩ E i, 0 = ∅ . Also for all i ∈ N we hav e (2) δ E ( ∆ i, 0 ) = P  ∈{ 0 , 1 , 2 } α i +1 , · ∆ i +1 , , and (3) ∆ i, 0 | = φ 1 and ∆ i, 2 | = φ 2 for all i . F or every infinite run ρ ∈ E ω , w e hav e ρ | = φ 1 U φ 2 iff there exists a prefix e ∈ E i, 1 for some i ∈ N . Therefore we ha ve P r ∆ E ( { ρ ∈ E ω | ρ | = φ 1 U φ 2 } )   α iff P i ∈ N ( α i, 1 · Q 0 ≤ j , ≥} , which is (1). That is, the collection of infinite traces satisfying φ 1 U φ 2 are those with prefix e 0 e 1 . . . e i with e i | = φ 2 and e j | = φ 1 for all 0 ≤ j < i . Therefore, suppose E ( ∆ ) | =   φ 1 U φ 2 , we hav e that the ab o v e sequence of triples {h ( ∆ i, 0 , α i, 0 ) , ( ∆ i, 1 , α i, 1 ) , ( ∆ i, 2 , α i, 2 ) i} i ∈ N satisfies the required conditions (1), (2) and (3). Supp ose there exists a sequence of triples satisfying (1), (2) and (3) with resp ect to   α , due to a similar w ay of reasoning, we already collected enough infinite runs that satisfy φ 1 U φ 2 with probability   α for   ∈ { >, ≥} .