Qualitative Logics and Equivalences for Probabilistic Systems

Logical Methods in Computer Science V ol. ? (?:?) 2???, ? pages www .lmcs-online.org Submitted date Published date QUALIT A TIVE LOGICS AND EQUIV ALENCES F OR PR OBABILISTIC SYSTEMS ∗ KRISHN ENDU CHA TTERJEE a , LUCA DE ALF ARO b , MARCO F AELLA c , AND AXEL LEGA Y d a Universit y of California, Baskin School of Engineering, Santa Cruz, US A e-mail addr ess : c krish@eecs.berkeley .edu b Universit y of California, Baskin School of Engineering, Santa Cruz, US A e-mail addr ess : luca@soe.ucsc.edu c Universit` a di Napoli “F ederico I I”, Italy e-mail addr ess : mfaella@na. infn.it d Carnegie Mellon Universi ty , Computer Science Department, Pittsburgh, U SA e-mail addr ess : alega y @ cs.cm u.edu Abstra ct. W e inv estigate logics and equiv alence relations that capture the qualitative b eha vior of Ma rko v D ecision Processes (MDPs). W e presen t Qualitative Randomized C tl ( Qrc tl ): form ulas of this log ic can ex press the fact that certain temp oral prop erties hold o ver all paths, or with p robabilit y 0 or 1, but they d o not distinguish among intermediate probabilit y v alues. W e present a symbolic, polynomial time mo del-chec king algorithm for Qr ctl on MDPs. The logic Qr ctl induces an equ iv alence relation ov er states of an MDP that we call qualitative e quivalenc e: informally , tw o states are qualitativ ely equiv alent if the sets of form ulas that hold with probabil ity 0 or 1 at t h e tw o states are the same. W e show that for finite alternating MDPs, where n ond eterministic and probabilistic choices o ccur in dif- feren t states, qu alitative equ iva lence coincides with alternating bisim ulation, and can th us b e computed v ia efficien t partition-refinement algorithms. On the other hand, in non- alternating MDPs the equiv alence relations cannot be computed via parti tion-refinement algorithms, bu t rather, they require non-lo cal computation. Finally , we consider Qr ctl ∗ , that extends Qr ctl with nested temporal operators in the sa me manner in whic h Ctl ∗ ex- tends Ctl . W e show that Qr ctl and Qr ctl ∗ induce the same qualitativ e equiva lence on alternating MDPs, while on non-alternati ng MDPs, the equiv alence arising from Qr ctl ∗ can b e strictly finer. W e also provide a f ull charac terization of the rel ation b etw een qual- itativ e equiv alence, b isimulation, and alternating bisim ulation, according to whether the MDPs are finite, and to whether their transitio n rel ations are finitely-b ranching. 1998 ACM Subje ct Class ific ation: F.4.1. Key wor ds and phr ases: Game T heory , Mark o v Deci sion Processes, Qualitativ e Analysis, Mo del Checking, Qualitativ e Probabilistic Logic, Qualitative Equiva lences. ∗ A preliminary v ersion of this p aper appeared i n the proceedings of the 4th In ternational Conference on the Quantita tive Ev aluation of Systems (QEST 2007). LOGICAL METHODS IN COMPUTER SCIENCE DOI:10.216 8/LMCS-??? c  K. Chatterjee , L. ˜ de Alfaro, M . Faella, and A. Legay Creative Commons 2 K. CHA TTERJEE, L. ˜ DE ALF ARO, M. F AELLA, AND A. LEGA Y 1. I n tro duction Mark ov decisio n pro cesses (MDPs) pro vide a mo del for systems exhibiting b o th p robabilis- tic and nondeterministic b eha vior. MDPs we re original ly in tro duced t o mo d el and solve con trol pr oblems for sto chastic s y s tems: there, nondeterminism represen ted the freedom in the c h oice of c on trol action, while the probabilistic comp onent of the b eha vior describ ed the system’s resp o nse to the con trol action [Ber95]. MDPs w ere later adopted as mo dels for concurrent probabilistic systems, probabilistic systems op erating in op en en vironment s [Seg95], and und er-sp ecified probabilistic systems [BdA95, d A97a]. Giv en an MDP and a prop ert y of in terest, w e can ask tw o kin ds of ve rification questions: quantitative and qualita tive questions. Quanti tativ e questions relate to the n umerical v alue of the pr obabilit y with whic h the pr op ert y holds in the system; qualitativ e questions ask whether the prop ert y holds with p r obabilit y 0 or 1. Examples of quan titativ e questions include the computation of the maximal and minimal pr obabilitie s with whic h the MDP satisfies a safet y , r eac habilit y , or in general, ω -regular prop ert y [BdA95]; the co rresp onding qualitativ e questions asks whether said p rop erties hold with p robabilit y 0 or 1. While m uc h recen t w ork on p robabilistic v erification has f o cused on answe ring quanti - tativ e questions, the in terest in qualitativ e ve rification questions predates the one in quan- titativ e ones. Answ ering qualit ativ e questions ab out MD Ps is useful in a wide range of applications. In t he analysis of randomized algo rithms, it is natural to require that the correct b eha vior arises with prob ability 1, and not just with probabilit y at least p for some p < 1. F or instance, wh en analyzing a randomized embedd ed sc heduler, we are in terested in whether ev ery th r ead pr ogresses w ith probabilit y 1 [dAFMR05]. Su c h a qualitati v e question is muc h easier to study , and to justify , than its quantita tiv e v ersion; ind eed, if we ask ed for a lo wer b ound p < 1 for the probabilit y of progress, the c h oice of p wo uld n eed to b e justified b y an analysis of ho w m uch failure probabilit y is acc eptable in the final system, an analysis that is generally not easy to accomplish. F or the same reason, the co rrect- ness of randomized distributed algorithms is often established with resp ect to qualitativ e, rather than quan titativ e, criteria (see, e.g., [PSL00, KNP00, Sto02]). F ur thermore, since qualitativ e answ ers can generally b e computed more efficient ly than quan titativ e ones, they are often used as a useful pre-pr o cessing step. F or instance, when computing the maximal probabilit y of reac hing a set of target states T , it is c on v enien t to first pr e-compute the set of states T 1 ⊇ T that can reac h T with pr obabilit y 1, and then compute the maximal probabilit y of reac hing T : this reduces the num b er of states where the quan titati v e quest ion needs to b e answered, and leads to more efficien t algorithms [dAKN + 00]. L astly , w e remark that qu alitativ e answers, unlik e quan titativ e ones, are more robust to p erturbations in the n umerical v alues of transition p robabilities in the MDP . Thus, whenever a system can b e mo deled only within some appr o x imation, qualitativ e v erification questions yield in f orma- tion ab out the s ys tem that is m ore robust with r esp ect to m o deling errors, and in man y w a ys , more basic in n ature. In this paper, w e pro vide lo gics for the sp ecificatio n of qualita tiv e prop erties of Marko v decision pro cesses, along with mo del-c hecking algorithms for such l ogics, and we study the equiv alence relations arising f rom such logics. Our starting p oin t for the logics is provi ded by the probabilistic logics pCtl and pCtl ∗ [HJ94, ASB + 95, BdA95]. T hese lo gics are able to express b ounds o n the probabilit y of eve n ts: the logic pCtl is deriv ed from Ctl b y a dding to its path quan tifiers ∀ (“ for all paths”) and ∃ (“for at least one path”) a probabilistic quan tifier P. F or a b ound q ∈ [0 , 1], an in equalit y ⊲ ⊳ ∈ { <, ≤ , ≥ , > } , and a path formula ϕ , QUALIT A TIVE LOGICS AND EQUIV ALENCES FOR PROBABILISTIC SYSTEMS 3 the pCtl form ula P ⊲ ⊳q ϕ holds at a state if the path formula ϕ holds fr om that state with probabilit y ⊲ ⊳ q . The logic pCtl ∗ is similarly deriv ed from Ctl ∗ . In order to obtain logics for qualitati v e prop erties, we consider the subsets of pCtl and pCtl ∗ where ∀ , ∃ hav e b een dropp ed, and where the b ound q aga inst w hic h probabilities are compared can assume only the t wo v alues 0, 1. W e call the resulting logi cs Qrctl a nd Qrctl ∗ , for Qualitative R andomize d Ctl and Ctl ∗ . W e provi de sym b olic mo del-c hec kin g algorithms for the logic Qrctl ; these algorithms can b e easily extended to Qrctl ∗ , since for MDPs the verificat ion of general temp oral-logic prop erties can b e r educed to reac habilit y questions [CY95, dA97a]. As usual, the mo del- c h ec king alg orithms for Qrctl pro ceed b y induction on the structur e of a form ula. The cases for some of the op erators are kno wn; for others, w e giv e new algorithms, completing the picture of the sym b olic algorithms required for Qrctl mo del c hec king. W e then pro ceed to s tu d y the equiv alence r elatio ns that arise from Qrctl . F or t wo states s and t of an MDP , we write s ≈ > 0 t if the state s s, t satisfy the same Qr ctl form ulas; similarly , Qr ctl ∗ induces the relat ion ≈ > 0 ∗ . I nformally , s ≈ > 0 t holds if the set of prop erties that hold with probabilit y 0, p ositiv e, and 1, at s and t coi ncide. These relations are th us strictly coarser than standard probabilistic bisim ulation [SL94], whic h relates states only when the precise p robabilit y v alues coincide. Other works ([DGJP99]) ha v e in tro duced distanc es whic h qu an tify the difference in the probabilistic b eha vior of t wo MDPs. When the distance b et w een s and t is zero, s and t are probabilistica lly bisimilar, and so th ey are also qualit ativ ely bisimilar. Aside f rom that, the distance b et wee n t wo states is in general unrelated to the states b eing qualitativ ely equiv alen t or not. The app eal o f the relati ons ≈ > 0 and ≈ > 0 ∗ lies in t heir abilit y to relate implemen tations and sp ecificatio ns in a qu alitati v e w a y , abstracting a w a y from precise pr obabilit y v alues. The relations, and their asymmetrical counterparts relate d to sim ulation, are particularly w ell-suited to the study of refin emen t and imp lemen tation of r andomized algorithms, where the p rop erties to b e p reserv ed are most often probabilit y-1 prop erties. F or instance, when implemen ting a randomized thread scheduler [dAFMR05], the implemen tation needs to guaran tee that eac h thread is scheduled in fi nitely often with probabilit y 1; it is n ot im- p ortan t that the implemen tation realiz es exactly the same probabilit y of sc h eduling eac h thread as the sp ecification. Our qualitat iv e relations can al so b e used as a help to analyze qualitativ e pr op erties o f systems, simil arly to h o w bisim u lation r eductions c an help in ve ri- fication. G iv en a system, the relations enable the construction o f a minimized, qu alitati v ely equiv alen t system, on wh ic h all qualita tiv e questions ab out the original sy s tem can be an- sw ered. W e will sho w that our qu alitativ e equiv alences a re computable b y efficien t discrete graph-theoretic algorithms that d o not refer to numerical computation. W e distinguish b et wee n alternating MDPs, where probabilistic and nondeterministic c h oices o ccur at different states, from the general case of non-alternating MDPs, where b oth c h oices can o ccur at the same state. Our first result is that on finite, alternating MDPs, the relation ≈ > 0 coincides with alt ernating bisimulatio n [AHKV98] on the MDP regarded as a t w o-pla yer game of probabilit y vs. nondeterminism. This result enables the computation of ≈ > 0 via the efficien t partition-refinemen t algorithms develo p ed for al ternating bisimula- tion. W e sho w that the corresp ond ence b et w een ≈ > 0 and alternating bisimulati on br eaks do wn b oth f or infinite MD Ps, and for finite, b ut non-alternating, MD Ps. In d eed, we sho w that on non-alternating MDPs, the rela tion ≈ > 0 cannot b e computed b y an y partition- refinemen t algorithm that is lo c al, in the sense that partitions are refined b y lo oking only at 1-neigh b ourho o d s of s tates (the classical partition-refinemen t algorithms for sim ulation 4 K. CHA TTERJEE, L. ˜ DE ALF ARO, M. F AELLA, AND A. LEGA Y and bisim ulation are lo cal). These results are surprising. One is tempted to consider al- ternating and non-alternating MDPs as equiv alen t, since a non-al ternating MDP can b e translated into an alternating one b y splitting its states in to multiple alternating ones. The difference b et ween the alternating and non-alternating mo dels w as already noted in [S T 05] for strong and we ak “precise” simulatio n, and in [BS01] f or axio matizatio ns. O u r results indicate that th e difference b et w een the alte rnating and n on-alternating mo del is ev en more mark ed for ≈ > 0 , whic h is a lo cal rela tion on alternating mo dels, and a non-local relation i n non-alternating ones. More surp r ises fol lo w wh en examining the roles of the  (“ next”) and U (“until” ) op erators, and the distinction b et ween Q rctl and Qrctl ∗ . F or Ctl , it is known that the  op erator alone suffices to charact erize b isim ulation; the U op erator do es not add distinguishing p o wer. Th e same is true for Qr ctl on fi nite, alternating MD Ps. On the other hand, w e sho w that for non-alternating, or infinite, MDPs, U adds distinguishing p o we r to the logic. Similarly , the relations induced by Qrctl a nd Qr ctl ∗ coincide o n finite, alternating MDPs, but Qr ctl ∗ has greater distinguishin g p o wer, and induces th us finer relations, on non-alternating or infinite MDPs. In summary , we establish that on finite, alte rnating MDPs, qualitativ e equiv alence can b e computed effici en tly , and enjo ys man y canonical prop erties. W e also sh o w that the situation b ecomes more complex as so on as infi nite or non-alternating MDPs are considered. In all cases, w e pro vide sharp b oundaries for the classes of MDPs on whic h our stat emen ts apply , distinguishing also b etw een finitely and infinitely- branching MDPs. Ou r results also indicate ho w the distinction b et w een a lternating and non-alternating MD Ps, while often o verlook ed, is in fact of great imp ortance where the logic al p rop erties of the MDPs are concerned. Our organization of the pap er is as follo ws: in sect ion 2 we present the formal d efinitions of MDPs and the logics Qr ctl ∗ and Qrctl . In sect ion 3 w e pr esen t a mo del c hec king algorithm for MDPs with t he logic Qr ctl . In sec tion 4 w e charac terize the equiv alence relations of MDPs with resp ect to Qrctl . In section 5 w e presen t algorithms to compute the equiv alence rel ations. Finally , in sectio n 6 w e discuss the role s of the u n til and w ait-for op erators in the logics, a nd in sec tion 7 w e consider th e role of linear-time nesting ( i.e., the equiv alences for the logic Q rctl ∗ ). 2. D efinitions 2.1. Marko v Decision Pro cesses A probabilit y distribu tion on a coun table set X is a function f : X 7→ [0 , 1] su c h that P x ∈ X f ( x ) = 1; w e d enote the set of all probabilit y distributions on X by D ( X ). Giv en f ∈ D ( X ), we define Supp ( f ) = { x ∈ X | f ( x ) > 0 } to b e the supp ort of f . W e consider a fixed set AP of atomic p rop ositions, whic h includes the distinguished prop osition turn . Giv en a set S , we denote S + (resp ectiv ely S ω ) the set of finite (resp. infinite) sequences of elemen ts of S . A Markov de cision pr o c ess (MDP) G = ( S, A , Γ , δ, [ · ]) co nsists of the follo wing comp o- nen ts: • a countable set of states S ; • a fi nite set of actions A ; QUALIT A TIVE LOGICS AND EQUIV ALENCES FOR PROBABILISTIC SYSTEMS 5 • an action assignment Γ : S 7→ 2 A \ ∅ , w hic h asso ciates with eac h sta te s ∈ S the set Γ( s ) of actions that can b e c hosen at s ; • a transition function δ : S × A 7→ D ( S ), wh ic h asso ciates w ith eac h state s and act ion a a next-state probab ility distribution δ ( s, a ); • a lab eling function [ · ] : S 7→ 2 AP , wh ic h lab els all s ∈ S with the set [ s ] of ato mic prop ositions true at s . F or s ∈ S and a ∈ Γ( s ), w e let Dest ( s, a ) = Supp ( δ ( s, a )) b e the set of p ossible d estinations when the act ion a is chosen at the state s . The MDP G is finite if the state s p ace S is finite, and it is finitely-br anching if for all s ∈ S and a ∈ Γ( s ), the set Dest ( s, a ) is finite. A play or p ath is an infinite sequence ~ ω = h s 0 , s 1 , . . . i ∈ S ω of state s of the MDP . F or s ∈ S and q ∈ AP , w e sa y that s is a q -state iff q ∈ [ s ]. W e define an e dge r elation E = { ( s, t ) ∈ S × S | ∃ a ∈ Γ( s ) . t ∈ Dest ( s, a ) } ; for s ∈ S , we let E ( s ) = { t | ( s, t ) ∈ E } . An MDP G is a Markov chain if | Γ( s ) | = 1 for all s ∈ S ; in this case, for all s, t ∈ S w e write δ ( s )( t ) rather than δ ( s, a )( t ) for the unique a ∈ Γ( s ). Interpr e tations. W e in terpret an MDP in t wo distinct w ays: as a 1 1 / 2 -pla yer ga me, and as a 2-pla y er game. In the 1 1 / 2 -pla yer inte rpretation, probabilistic c h oice is resolv ed proba- bilisticall y: at a state s ∈ S , pla y er 1 c ho oses a n ac tion a ∈ Γ( s ), a nd the MDP mov es to the successor state t ∈ S with probabilit y δ ( s, a )( t ). I n the 2-pla y er in terpr etatio n, we regard pr obabilistic c h oice as adv ersarial, and w e treat the MDP as a game b et ween play er 1 and play er p ( p f or “probabilit y”): at a state s , pla yer 1 c ho oses an action a ∈ Γ( s ), and pla y er p c ho oses a destinatio n t ∈ Dest ( s, a ). T he 1 1 / 2 -pla yer inte rpretation is the classical one [Der70]. The 2-pla y er inte rpretation will b e used to relate the qualitati v e equiv alence relations for the MDP with the alternating relations of [AHKV98], and thereby deriv e al- gorithms for computing the qualita tiv e equiv alence relati ons. Str ate gies. A player-1 str ate gy is a function σ : S + 7→ D ( A ) that prescrib es the p r obabilit y distribution σ ( ~ w ) o ver actions to b e p la yed, giv en the past sequence ~ w ∈ S + of states visited in the pla y . W e require th at if a ∈ Supp ( σ ( ~ w · s )), then a ∈ Γ( s ) for all a ∈ A , s ∈ S , and ~ w ∈ S ∗ . W e denote by Σ the set of all pla yer-1 strategie s. A play er- p str ate gy is a function π : S + × A 7→ D ( S ). The s trategy must b e suc h that , for all s ∈ S , ~ w ∈ S ∗ , and a ∈ Γ( s ), we ha ve that Supp ( π ( ~ w · s, a )) ⊆ Supp ( δ ( s, a )). Pla y er p follo ws the strategy π if, whenever pla y er 1 chooses mo ve a after a history of pla y ~ w , she c h o oses the destination s tate with probabilit y d istribution π ( ~ w , a ). Thus, in the 2-pla y er in terpretation, nondeterminism pla ys first, and probabilit y sec ond. W e denote by Π the set of all play er- p strategies. The 2-player interpr etation. In the 2-pla yer in terpretation, once a starting state s ∈ S and t wo strat egies σ ∈ Σ and π ∈ Π h a ve b een chosen, the game is reduced to an ordinary sto c h astic pro cess, and it is p ossible to d efine the p robabilities of eve nts, where an event A ⊆ S ω is a measurable set of paths. W e denote the probabilit y of ev en t A , starting from s ∈ S , u nder strategies σ ∈ Σ and π ∈ Π by Pr σ ,π s ( A ): note that the probabilit y of ev ents giv en strategies σ and π do not dep en d on the transition p robabilities of the MDP as th e strateg y π can c hose an y probabilit y distribution at eac h step. Giv en s ∈ S and σ ∈ Σ, π ∈ Π, a p la y h s 0 , s 1 , . . . i is f e asible if for every k ∈ N , there is a ∈ Γ( s k ) su c h that 6 K. CHA TTERJEE, L. ˜ DE ALF ARO, M. F AELLA, AND A. LEGA Y σ ( s 0 , s 1 , . . . , s k )( a ) > 0 and π ( s 0 , s 1 , . . . , s k , a )( s k +1 ) > 0. W e denote by Outc( s, σ , π ) ⊆ S ω the set of feasible pla ys th at start from s giv en strategies σ and π . The 1 1 / 2 -player interpr e tation. In the 1 1 / 2 -pla yer in terpretation, we fix for pla y er p the strategy π ∗ that c ho oses the next sta te with the distribu tion prescrib ed by δ . Precisely , for all ~ w ∈ S ∗ , s ∈ S , and a ∈ Γ( s ), w e let π ∗ ( ~ w · s, a ) = δ ( s, a ). W e then write Pr σ s ( A ) and Outc( s, σ ) instead of Pr σ ,π ∗ s ( A ) and Outc( s, σ, π ∗ ), r esp ectiv ely , to underline the fact that these p robabilities a nd set of outcomes a re functions only of the initial stat e and of the strategy of pla y er 1. Alter nating MD Ps. An alternating M DP (AMDP) is an MD P G = ( S, A , Γ , δ, [ · ]) along with a partition ( S 1 , S p ) of S suc h that: (1) If s ∈ S 1 , then turn ∈ [ s ] and, for all a ∈ Γ( s ), | Dest ( s, a ) | = 1. (2) If s ∈ S p , then turn 6∈ [ s ] and | Γ( s ) | = 1. The state s in S 1 are the play er-1, or nondeterministic states, and the states in S p are the p la yer- p , or pr ob abilistic states. The predicate turn ensures that the MDP is visibly alternating: th e difference b et w een pla yer-1 and pla y er- p s tates is obvio us to the play ers, and w e wan t it to b e obvi ous to the logi c to o. Alternating MDPs can b e represen ted more succinctly (a nd more in tuitiv ely) by pro viding, along with the partitio n ( S 1 , S p ) of S , the edge relatio n E ⊆ S × S , and a probabilistic transition function ˜ δ : S p 7→ D ( S ). The pr obabilistic transition fu nction is defin ed, for s ∈ S p , t ∈ S , and a ∈ Γ( s ), by ˜ δ ( s )( t ) = δ ( s, a )( t ). A non-alternating MDP is a ge neral (alternating or not) MDP . W e represen t M DPs b y g raphs: v ertices co rresp ond to no des, and eac h action a fr om a state s is d ra wn as a hyp eredge from s to De st ( s, a ). 2.2. Logics W e consider t wo logics for the sp ecification of M DP prop erties. Th e first, Qrctl ∗ , is a logic that captures qualitative prop erties of MDPs, and is a qualitativ e v ersion of pCtl ∗ [HJ94, ASB + 95, BdA95]. The logi c is defined with resp ect to the cla ssical, 1 1 / 2 -pla yer seman tics of MDPs. The second logic , A tl ∗ , is a game logic defined with resp ect to the 2-pla yer seman tics of MDPs as in [AHK02]. Syntax. The syn tax of b oth logic s is giv en by d efining the s et of p ath formula s ( ϕ ) and state formulas ( ψ ) via the follo wing inductiv e clauses: path form ulas: ϕ ::= ψ | ¬ ϕ | ϕ ∨ ϕ |  ϕ | ϕ U ϕ | ϕ W ϕ ; state formulas: ψ ::= tt | q | ¬ ψ | ψ ∨ ψ | PQ ( ϕ ); where q ∈ AP is an atomic prop osition, t t is the b o olean co nstan t with v alue true, and PQ is a p ath quantifier. The op erators U , W and  are temp oral op erators. The logics A tl ∗ and Qrctl ∗ differ in the p ath quan tifiers: • The p ath qu an tifiers in Qrctl ∗ are: ∃ al l , ∀ al l , ∃ some , ∀ some , ∃ 1 , ∀ 1 , ∃ > 0 and ∀ > 0 . • The p ath qu an tifiers in A tl ∗ are: h h 1 i i , h h p i i , h h 1 , p i i , h h∅i i . QUALIT A TIVE LOGICS AND EQUIV ALENCES FOR PROBABILISTIC SYSTEMS 7 The fragment s A tl of A tl ∗ and Qrctl of Qrctl ∗ consist of form ulas w here ev ery temp oral op erator is immediately preceded b y a path quanti fier. In the follo wing, when w e refer to a “formula” of a logic , without sp ecifying whether it is a state or path form ula, we alw ays mean a state form u la. As usu al, w e define ✷ ϕ and ♦ ϕ to b e abbreviations for ϕ W ( ¬ tt ) and tt U ϕ , resp ectiv ely . Semantics. F or a pla y ~ ω = h s 0 , s 1 , . . . i we denote by ~ ω [ i ] the pla y starting from th e i -th state of ~ ω , i.e., ~ ω [ i ] = h s i , s i +1 , . . . i . The se man tics for the path formulas is d efined a s follo ws, for path form ulas ϕ , ϕ 1 , ϕ 2 : ~ ω | = ϕ 1 ∨ ϕ 2 iff ~ ω | = ϕ 1 or ~ ω | = ϕ 2 ~ ω | = ¬ ϕ iff ~ ω 6| = ϕ ~ ω | =  ϕ iff ~ ω [1] | = ϕ ~ ω | = ϕ 1 U ϕ 2 iff ∃ j ∈ N .~ ω [ j ] | = ϕ 2 and ∀ 0 ≤ i < j. ~ ω [ i ] | = ϕ 1 ~ ω | = ϕ 1 W ϕ 2 iff  ∀ j ∈ N . ~ ω [ j ] | = ϕ 1  or ∃ j ∈ N . ~ ω [ j ] | = ϕ 2 and ∀ 0 ≤ i ≤ j. ~ ω [ i ] | = ϕ 1 . Observe that ¬ ( ψ 1 U ψ 2 ) = ✷ ( ¬ ψ 2 ) ∨ ( ¬ ψ 2 U ( ¬ ψ 1 ∧ ¬ ψ 2 )) = ¬ ψ 2 W ¬ ψ 1 . Finally , we ha ve ~ ω | = ψ iff s 0 | = ψ . Giv en a path formula ϕ we denote by [ [ ϕ ] ] = { ~ ω | ~ ω | = ϕ } the set of pla ys that satisfy ϕ . The seman tics of the state formulas of A tl ∗ and Qr ctl ∗ is defined as follo ws, for a state s , path formula ϕ , and state form ulas ψ 1 and ψ 2 : s | = tt s | = q iff q ∈ [ s ] s | = ¬ ψ 1 iff s 6| = ψ 1 s | = ψ 1 ∨ ψ 2 iff s | = ψ 1 or s | = ψ 2 s | = ∃ al l ( ϕ ) iff ∃ σ ∈ Σ . Outc( s, σ ) ⊆ [ [ ϕ ] ] s | = ∀ al l ( ϕ ) iff ∀ σ ∈ Σ . Outc( s, σ ) ⊆ [ [ ϕ ] ] s | = ∃ 1 ( ϕ ) iff ∃ σ ∈ Σ . Pr σ s ([ [ ϕ ] ]) = 1 s | = ∀ 1 ( ϕ ) iff ∀ σ ∈ Σ . Pr σ s ([ [ ϕ ] ]) = 1 s | = ∃ > 0 ( ϕ ) iff ∃ σ ∈ Σ . Pr σ s ([ [ ϕ ] ]) > 0 s | = ∀ > 0 ( ϕ ) iff ∀ σ ∈ Σ . Pr σ s ([ [ ϕ ] ]) > 0 s | = ∃ some ( ϕ ) iff ∃ σ ∈ Σ . Outc( s, σ ) ∩ [ [ ϕ ] ] 6 = ∅ s | = ∀ some ( ϕ ) iff ∀ σ ∈ Σ . Outc( s, σ ) ∩ [ [ ϕ ] ] 6 = ∅ s | = h h 1 i i ( ϕ ) iff ∃ σ ∈ Σ . ∀ π ∈ Π . Outc( s, σ, π ) ⊆ [ [ ϕ ] ] s | = h h p i i ( ϕ ) iff ∃ π ∈ Π . ∀ σ ∈ Σ . Outc( s, σ, π ) ⊆ [ [ ϕ ] ] s | = h h 1 , p i i ( ϕ ) iff ∃ σ ∈ Σ . ∃ π ∈ Π . Outc( s, σ, π ) ⊆ [ [ ϕ ] ] s | = h h∅i i ( ϕ ) iff ∀ σ ∈ Σ . ∀ π ∈ Π . Outc( s, σ, π ) ⊆ [ [ ϕ ] ] . Giv en an A tl ∗ or Qrctl ∗ form ula ϕ and an MDP G = ( S, A , Γ , δ, [ · ]), we denote by [ [ ψ ] ] G = { s ∈ S | s | = ϕ } the set of states that sat isfy the state formula ψ , and w e omit 8 K. CHA TTERJEE, L. ˜ DE ALF ARO, M. F AELLA, AND A. LEGA Y s t 1 / 2 1 / 2 1 Figure 1: A simp le Marko v c hain. the sub script G when obvio us from the con text. F or all path formulas ϕ of Qrctl , the follo wing dualities hold: [ [ ∃ al l ϕ ] ] = [ [ ¬ ( ∀ some ( ¬ ϕ ))] ] [ [ ∃ some ϕ ] ] = [ [ ¬ ( ∀ al l ( ¬ ϕ ))] ] [ [ ∃ > 0 ϕ ] ] = [ [ ¬ ( ∀ 1 ( ¬ ϕ ))] ] [ [ ∃ 1 ϕ ] ] = [ [ ¬ ( ∀ > 0 ( ¬ ϕ ))] ] . (2.1) W e no w presen t a simple example to illustrate the d ifference b etw een the satisfact ion of a path form ula with probabilit y 1 and for all paths. Example 2.1. Consider the simple Mark o v c hain shown in Figure 1. Let the p rop ositions true at states s and t b e q and r , resp ectiv ely . Let u s consider the starting state as s , and the form ula ♦ r (ev ent ually r ). The form ula holds at state s with probabilit y 1, since the only closed recurrent set o f states in the Mark o v c hain is t he state t (lab eled with p rop osition r ). Hence ♦ r holds in state s with probabilit y 1. Ho we v er, there is a path (n amely , s ω ) that violate s the prop ert y ev entually r , b ut the probabilit y mea sure for the set { s ω } of paths is 0. Th us the state s d o es not satisfy that all on all p aths w e ha v e even tually r , though it satisfies the p rop erty even tually r with pr obabilit y 1. If w e consider the prop erty even tually q , then for a ll paths starting from s the prop ert y h olds (hence the prop erty also holds with probabilit y 1). The follo wing lemma e stablishes a relatio nship betw een Qrctl and A tl , pr oving that the Q rctl quant ifiers with sup erscript al l and some are equiv alen t to the A tl quan tifiers. Lemma 2.2. F or al l p ath formulas ϕ , the fol lowing e qu i v alenc es hold . [ [ h h 1 i i ϕ ] ] = [ [ ∃ al l ϕ ] ] [ [ h h 1 , p i i ϕ ] ] = [ [ ∃ some ϕ ] ] [ [ h h p i i ϕ ] ] = [ [ ∀ some ϕ ] ] [ [ h h∅i i ϕ ] ] = [ [ ∀ al l ϕ ] ] Pr o of. Let G = ( S, A , Γ , δ , [ · ]) b e an MDP and let s ∈ S . W e pro v e the first state men t. Assume s | = h h 1 i i ϕ . By definition, there exists σ ∗ ∈ Σ suc h that: ∀ π ∈ Π . Ou tc( s, σ ∗ , π ) ⊆ [ [ ϕ ] ] . Let π ∗ ∈ Π b e the strate gy of pla y er p that chooses the next stat e acc ording to δ (i.e., the natur al strategy of pla y er p in G ). W e hav e: Outc( s, σ ∗ ) = Outc( s, σ ∗ , π ∗ ) ⊆ [ [ ϕ ] ] . (2.2) Therefore, s | = ∃ al l ϕ . Con v ersely , assume s | = ∃ al l ϕ . Then, there exists σ ∗ ∈ Σ suc h that (2.2) h olds. Let π b e an y strategy of pla y er p . W e ha v e that Outc( s, σ ∗ , π ) ⊆ Outc( s, σ ∗ , π ∗ ), b ecause π ∗ is QUALIT A TIVE LOGICS AND EQUIV ALENCES FOR PROBABILISTIC SYSTEMS 9 the most lib eral strategy f or pla y er p , i.e., n o pla y er- p strategy can ev er choose a successor state that is not among th ose that are c hosen by π ∗ . Th erefore, Outc( s, σ ∗ , π ) ⊆ [ [ ϕ ] ] and s | = h h 1 i i ϕ . Next, w e prov e the second stateme nt . The remaining s tatemen ts follo w b y dualit y . Assume s | = h h 1 , p i i ϕ . Then, there exi st σ • ∈ Σ and π • ∈ Π such that Outc( s, σ • , π • ) ⊆ [ [ ϕ ] ]. Let π ∗ b e the natural strategy for pla y er p in G . By the previous argumen t, Outc( s, σ • , π • ) ⊆ Outc( s, σ • , π ∗ ). Therefore, Outc( s, σ • , π ∗ ) ∩ [ [ ϕ ] ] 6 = ∅ and s | = ∃ some ϕ . Finally , assume s | = ∃ some ϕ . By defin ition, there exists σ ∗ ∈ Σ suc h that Outc( s, σ ∗ , π ∗ ) ∩ [ [ ϕ ] ] 6 = ∅ , where π ∗ is the natural strategy f or p la yer p in G . Let ~ ω b e a pla y in Outc( s, σ ∗ , π ∗ ) ∩ [ [ ϕ ] ]. Define σ • and π • as the deterministic strategi es that giv e as only outcome ~ ω . W e ha ve: Outc( s, σ • , π • ) = { ~ ω } ⊆ [ [ ϕ ] ] . Therefore, s | = h h 1 , p i i ϕ . Finally , the foll o wing lemma p ro v es the equiv alence of some Qrctl form ulas. Lemma 2.3. F or al l atomic pr op ositions q , r , and for al l MDPs, we ha ve: [ [ ∃ > 0  q ] ] = [ [ ∃ some  q ] ] [ [ ∃ 1  q ] ] = [ [ ∃ al l  q ] ] [ [ ∃ > 0 q U r ] ] = [ [ ∃ some q U r ] ] (2.3) [ [ ∃ 1 q W r ] ] = [ [ ∃ al l q W r ] ] . Pr o of. The fir st t wo state men ts are obvio us by definition. The third stat emen t follo ws by noting that s | = ∃ some q U r iff there is a finite path in ( S, E ) from s to an r -state, and all states of the path, except p ossibly the last , are q -states. If suc h a path exists, there is certainly a strategy of pla ye r 1 that follo ws it with p ositiv e p robabilit y . F or the la st stat emen t, the “ ⊇ ” inclusion is ob vious b y definition. F or the other in- clusion, assume b y con tradiction that s ∈ [ [ ∃ 1 q W r ] ], but all strategies of pla yer 1 ensuring q W r with probabilit y o ne also exhibit a path violating it . T hen, s ∈ [ [ ∀ some ¬ ( q W r )] ] = [ [ ∀ some ¬ r U ¬ q ] ]. F ollo wing an argum en t similar to th e one for the third statemen t, w e obtain that s ∈ [ [ ∀ > 0 ¬ r U ¬ q ] ] = [ [ ∀ > 0 ¬ ( q W r )] ], which is a con tradiction. 2.3. Equiv alence Relat ions Giv en an MDP G = ( S, A , Γ , δ , [ · ]), we consider the equiv alence r elations induced ov er its state space by v arious synta ctic subsets o f the lo gics Qrctl and A tl . Define t he follo wing fragmen ts of Qrctl : • Qrctl > 0 is the syn tacti c fr agmen t of Qrctl con taining only the path quan tifiers ∃ > 0 and ∀ > 0 ; • Qrctl al l is the syntac tic fragmen t o f Qr c tl con taining only the path quantifiers ∃ al l and ∀ al l . Note that, b ecause of the dualities (2.1 ), we do not need to consider the fragmen ts for ∀ 1 , ∃ 1 , ∀ some , ∃ some . T he r elations indu ced by Qrctl > 0 and Qrctl al l pro vide us w ith a notion of qualitative equiv alence b et w een states. ≈ > 0 = { ( s, s ′ ) ∈ S × S | ∀ ψ ∈ Qrctl > 0 , s | = ψ iff s ′ | = ψ } ≈ al l = { ( s, s ′ ) ∈ S × S | ∀ ψ ∈ Qrctl al l , s | = ψ iff s ′ | = ψ } . 10 K. CHA TTERJEE, L. ˜ DE ALF ARO, M. F AELLA, AND A. LEGA Y ≈ A TL ≈  ,> 0 ≈ > 0 ≈ Game ≈ > 0 ∗ finite finite finite branc hing ≈ TS finite branc hing finite ≈ al l Figure 2: Relationship b et ween equiv alence relations for AMDPs. W e denote by ≈ > 0 ,  b e the equiv alence relation defin ed b y Qrctl > 0 , with  as the only temp oral o p erator. W e also define the equiv alences ≈ > 0 ∗ and ≈ al l ∗ as th e Qrctl ∗ -v ers ion of ≈ > 0 and ≈ al l , resp ectiv ely . The syn tactic su bset of A tl wh ic h u ses only the path quanti fiers h h 1 , p i i and h h∅i i induces the u sual notio n of b isim ulation [Mi l90]: indeed, quan tifiers h h 1 , p i i and h h∅i i corresp ond to quan tifiers ∃ and ∀ of Ctl [CE81], resp ectiv ely . The syn tactic subset of A tl whic h uses only the path quanti fiers h h 1 i i and h h p i i induces alterna ting bisimulation [AHKV98 ]. W e ha v e: ≈ TS = { ( s, s ′ ) ∈ S × S | for all A tl fo rmulas ψ with h h 1 , p i i , h h∅i i as path quan tifiers, s | = ψ iff s ′ | = ψ } ; ≈ Game = { ( s, s ′ ) ∈ S × S | for all A tl fo rmulas ψ with h h 1 i i , h h p i i as path quan tifiers, s | = ψ iff s ′ | = ψ } ; ≈ A TL = { ( s, s ′ ) ∈ S × S | for all A tl fo rmulas ψ , s | = ψ iff s ′ | = ψ } ; where TS is the short form for transitio n systems. In the relation ≈ Game , nondeterministic and prob abilistic c hoice r epresen t the tw o pla ye rs of a game. In the relati on ≈ TS , nond e- terminism and probabilit y alw a ys co op erate as a single pla yer. Finally , the relat ion ≈ A TL arises f r om the full log ic A tl , where n ondeterminism and probabilit y can b e e ither an tago- nistic or co op erativ e. The relatio ns ≈ TS , ≈ Game , an d ≈ A TL can b e computed in p olynomial time via wel l-kno wn p artition-refinemen t algorithms [Mil90, AHKV98]. Figure 2 (resp. Figure 3) sum marizes the relat ionships b et ween d ifferen t equiv alence relations on alternating MDPs (resp. general MDPs) that w e will sho w in th is pap er. An arro w from relation A to relatio n B indicates t hat A imp lies B , i .e., that A is finer than B . 3. Mo del Chec king Qrctl In ord er to c h aracterize the equiv alence relations for Qrctl , it is useful to present first the al gorithms for Qrctl mo del c h ec king. The algorithms are b ased on the results of [dA97a, d A97b, d AH00]; see also [CdAH04]. As usual, w e presen t only the alg orithms for form ulas con taining one p ath quan tifier, as n ested form ulas can b e mo d el-c hec k ed b y QUALIT A TIVE LOGICS AND EQUIV ALENCES FOR PROBABILISTIC SYSTEMS 11 ≈ A TL ≈  ,> 0 ≈ > 0 ≈ Game ≈ > 0 ∗ finite branc hing ≈ TS finite branc hing ≈ al l Figure 3: Relationship b et ween equiv alence relations for MDPs. recursiv ely iterating the al gorithms. As a consequence of dualit ies (2.1), w e need to p ro vide algorithms only for th e operators ∃  , ∃ U , and ∃W , and for the mo dalities al l , 1 , > 0, and some . The algorithms use the foll o wing predecessor operators, for X, Y ⊆ S : Pr e ( X ) = { s ∈ S | ∃ a ∈ Γ ( s ) . D est ( s, a ) ∩ X 6 = ∅} Cpr e ( X ) = { s ∈ S | ∃ a ∈ Γ( s ) . Dest ( s, a ) ⊆ X } Apr e ( Y , X ) = { s ∈ S | ∃ a ∈ Γ( s ) . Dest ( s, a ) ⊆ Y ∧ Dest ( s, a ) ∩ X 6 = ∅} . The op erators Pr e and Cpr e are cla ssical; the op erator Apr e is from [dAHK98]. W e write the al gorithms in µ -calculus notation [Koz83]. Giv en an MDP G = ( S, A , Γ , δ, [ · ]), the in terpretation [ [ ψ ] ] of a µ -ca lculus formula ψ is a subset of stat es. In particular, for a prop ositional symb ol q ∈ AP , we ha ve [ [ q ] ] = { s ∈ S | q ∈ [ s ] } an d [ [ ¬ q ] ] = { s ∈ S | q 6∈ [ s ] } . The op erators ∪ , ∩ , and the ab ov e predecessor op erators are in terpreted as the corresp onding op erations on sets of states, and µ and ν indicate the least and greatest fixp oint, resp ectiv ely . The follo wing resu lt directly lea ds to mo d el-c hecki ng alg orithms for Qrctl . Theorem 3.1. F or atomic pr op ositions q and r , and for al l MDPs, the fol lowing e qualities hold: [ [ ∃ 1  q ] ] = [ [ ∃ al l  q ] ] = Cpr e ([ [ q ] ] ) (3.1) [ [ ∃ > 0  q ] ] = [ [ ∃ some  q ] ] = Pr e ([ [ q ] ]) (3.2) [ [ ∃ al l q U r ] ] = µX . ([ [ r ] ] ∪ ([ [ q ] ] ∩ Cpr e ( X ))) (3.3) [ [ ∃ > 0 q U r ] ] = [ [ ∃ some q U r ] ] = µX . ([ [ r ] ] ∪ ([ [ q ] ] ∩ Pr e ( X ))) (3.4) [ [ ∃ al l q W r ] ] = [ [ ∃ 1 q W r ] ] = ν Y . ([ [ r ] ] ∪ ([ [ q ] ] ∩ Cpr e ( Y ))) (3.5) [ [ ∃ some q W r ] ] = ν Y . ([ [ r ] ] ∪ ([ [ q ] ] ∩ Pr e ( Y ))) (3.6) If the MD P is finite, the fol lowing e qualities also hold: [ [ ∃ 1 q U r ] ] = ν Y .µX. ([ [ r ] ] ∪ ([ [ q ] ] ∩ Apr e ( Y , X ))) (3.7) [ [ ∃ > 0 q W r ] ] = [ [ ∃ > 0 q U (( r ∧ q ) ∨ ∃ al l ✷ q )] ] . (3.8) 12 K. CHA TTERJEE, L. ˜ DE ALF ARO, M. F AELLA, AND A. LEGA Y Pr o of. The form ulas inv olving the al l and some mo d alities (i.e., statemen ts (3.1) to (3.6)) are deriv ed by the corresp onding classical game algo rithms, thanks to Lemm a 2.2 and Lemma 2.3. F ormula (3.7) is from [dAHK98]. F ormula (3.8) ca n b e understo o d as follo ws. A close d c omp onent is a subset of sta tes T ⊆ S su c h that, for all s ∈ T , there is at least one a ∈ Γ( s ) suc h that Dest ( s, a ) ⊆ T . Using the relatio n q W r ≡ ( q U ( r ∧ q )) ∨ ✷ q [MP91], we ha v e for s ∈ S that s | = ∃ > 0 q W r iff (i) s | = ∃ > 0 q U ( q ∧ r ), or (ii) there is a closed comp onent T composed only of q -states, and a path s 0 , s 1 , . . . , s n in ( S, E ) composed of q -states, with s 0 = s and s n ∈ T (see, e.g. , [dA97 a]). F ormula (3.8 ) encodes the disjunction of (i) and (ii) . Note that, ev en though (3.8) is not a µ -calculus formula, it can b e readily translate d into the µ -cal culus via (3.4) and (3.5 ). Also observ e the µ -ca lculus form ulas corresp onding to Qrctl are either alternation free or co nt ain one quan tifier alternation b et we en the µ and ν op erator. T hus, from the complexit y of e v aluating µ -calculus formulas w e obtain the follo wing result. Theorem 3.2. Given a finite MDP G = ( S, A , Γ , δ, [ · ]) and a Qr c tl formula ψ , the set [ [ ψ ] ] G c an b e c ompute d in O ( | S | · | δ | · ℓ ) time, w her e | δ | = P s ∈ S P a ∈ Γ( s ) | Dest ( s, a ) | and ℓ denotes the length of ψ . Pr o of. W e first consider the computatio n of Pr e ( X ), Cpr e ( X ), and Apr e ( Y , X ) for X , Y ⊆ S . T o decide w hether s ∈ Pr e ( X ) we c hec k if there exists a ∈ Γ( s ) su ch that Dest ( s, a ) ∩ X 6 = ∅ . Similarly , to decide w hether s ∈ Cpr e ( X ) (resp. Apr e ( Y , X )) we chec k if there exists a ∈ Γ( s ) suc h that Dest ( s, a ) ⊆ X (resp. D est ( s, a ) ⊆ Y and Dest ( s, a ) ∩ X 6 = ∅ ). It follo ws that giv en sets X and Y , the sets Pr e ( X ), Cpr e ( X ), and Apr e ( Y , X ) can b e computed in time O ( P s ∈ S P a ∈ A | Dest ( s, a ) | ). Giv en a formula ψ in Qrctl , w ith all of its sub - form ulas already ev aluated, it follo ws from Th eorem 3.1 that the computation of [ [ ψ ] ] can b e obtained b y computing a µ -calculus formula of constan t length with at most one quan tifier alternatio n of µ and ν . Using the monotonici t y pr op ert y of Pr e , Cpr e and Apr e , and the computation of Pr e , Cpr e and Apr e , it follo w s that eac h inner iteration of the µ -calculus form ula can b e compu ted in time O ( P s ∈ S P a ∈ A | Dest ( s, a ) | ). S ince the outer iteration of the µ -calculus form u la con verge s in | S | iterations, it follo ws that [ [ ψ ] ] can b e computed in time O ( | S | · P s ∈ S P a ∈ A | Dest ( s, a ) | ). By a b ottom-up algorithm that ev aluates sub-formulas of a formula first, w e obtain the d esired b oun d for the algorit hm. 4. R elationship b et w een Qrctl an d A tl Equiv alences In th is section, w e compare the relatio ns indu ced b y Qrctl and A t l . These comparisons will then b e used in Section 5 to deriv e alg orithms to compute ≈ al l and ≈ > 0 . W e first compare ≈ al l with the relatio ns ind uced by A tl . As a first result, w e sho w that the relat ions induced by A tl coincide on alternating MDPs (AMD Ps). This result follo ws from the fact that the turn is visible to the logic. Prop osition 4.1. On AM DPs, we have ≈ Game = ≈ TS . Pr o of. Since th e turn is observ able (via the truth-v alue o f the predicate turn ), b oth ≈ Game and ≈ TS can relate only states where the same pla y er (1 or p ) can c ho ose the next mo v e. Based on this observ ation, the equalit y of the relations can b e prov ed straigh tforw ardly by induction. QUALIT A TIVE LOGICS AND EQUIV ALENCES FOR PROBABILISTIC SYSTEMS 13 s ′ s q 0 2 1 2 1 16 1 4 1 1 8 1 8 1 4 1 2 1 2 1 2 Figure 4: An infin ite Mark ov c hain in wh ic h state s s and s ′ cannot b e distinguished b y Qrctl > 0 , but are d istinguished by the A tl form ula h h p i i ✷ ¬ q . Corollary 4.2. On AMDPs, we have ≈ A TL = ≈ Game = ≈ TS . An immediat e consequence of Lemma 2.2 is that ≈ al l and ≈ A TL coincide. This en ables the computation of ≈ al l via the algorithms for alt ernating bisimulati on [AHKV98]. Prop osition 4.3. F or al l MDP s, ≈ al l = ≈ A TL . Next, we examine the relationship b et w een ≈ > 0 and ≈ A TL . On finitely-branc h ing MDPs, ≈ > 0 is finer than ≈ A TL ; the result cannot b e extended to infin itely-branc hin g MDPs . Theorem 4.4. The fol lowing assertions hold: (1) On finitely-br anching MDPs we have ≈ > 0 ⊆ ≈ A TL . (2) Ther e is an infinitely-br anching AMDP on which ≈ > 0 6⊆ ≈ A TL . Pr o of. Assertion 1. F or n > 0, w e consider the n -step appro ximation ≈ n A TL of ≈ A TL . In finite MDPs, we ha v e ≈ A TL = ≈ n A TL for n = | S | ; in finitely-branc hing MDPs, we ha v e ≈ A TL = ∩ ∞ n =0 ≈ n A TL , an d this do es not extend to MDPs that a re n ot fi n itely-branc hing. W e define a sequence Ψ 0 , Ψ 1 , Ψ 2 , . . . of sets of formulas suc h that, for al l s, t ∈ S , we ha v e s ≈ n A TL t iff s and t satisfy the same formulas in Ψ n . T o this end, giv en a fin ite set Ψ of form ulas, w e d enote b y BoolC(Ψ) the set of all form ulas that consist in disjunctions o f conjunctions of formulas in { ψ , ¬ ψ | ψ ∈ Ψ } . W e assum e th at ea c h conjun ction (resp. d isjunction) in Bo olC(Ψ) do es not con tain rep eated elemen ts, so that from the finiteness of Ψ foll o ws the one of Bo olC(Ψ). W e let Ψ 0 = BoolC ( AP ) and, f or k ≥ 0, w e let Ψ k +1 = BoolC (Ψ k ∪ {∃ > 0  ψ , ∃ al l  ψ | ψ ∈ Ψ k } ). The form u las in BoolC (Ψ 0 ) , BoolC (Ψ 1 ) , . . . , Bo olC(Ψ n ) pro vide w itnesses that ≈ > 0 ⊆ ≈ n A TL . Thus f or all n , we ha v e ≈ > 0 ⊆ ≈ n A TL , and it follo ws that ≈ > 0 ⊆ ≈ A TL . Assertion 2. Consider a Marko v chain, depicted in Figure 4, with state space S = N ∪ { s, s ′ } , with only one predicate s y mb ol q , suc h that [0] = { q } , and [ t ] = ∅ for all t ∈ S \ { 0 } . There is a transition from s to every i ∈ N with probabilit y 1 / 2 i +1 . There is a transition from s ′ to s ′ with probabilit y 1 / 2, and f r om s ′ to ev ery i ∈ N w ith probabilit y 1 / 2 i +2 . T h ere is a transition from i ∈ N w ith i > 0 to ev ery state in { j ∈ N | j < i } , with uniform probabilit y . There is a deterministic transition from 0 to itself. Since this is a Mark o v c hain, the t w o path quant ifiers ∃ and ∀ are equiv alen t, and we need only consider form ulas of the form ∃ > 0 and ∃ 1 . By induction on the length of a Qrctl formula ϕ , we can then sho w that ϕ cannot distinguish b et w een stat es in the se t { i ∈ N | i > | ϕ |} ∪ { s, s ′ } . Hence, s ≈ > 0 s ′ . On the other hand, we ha v e s 6≈ A TL s ′ , since s 6| = h h p i i ✷ ¬ q and s ′ | = h h p i i ✷ ¬ q . 14 K. CHA TTERJEE, L. ˜ DE ALF ARO, M. F AELLA, AND A. LEGA Y T o obtain a partial conv erse of this theorem, we need to translate all Qrctl formulas in to A tl . F or finite MDPs, Lemmas 2.2 and 2.3 enable u s to translate al l Qr ctl form u las, except for formulas of the t yp e ∃ 1 U and ∃ > 0 W . F or the latte r type, from (3.8) together with Lemmas 2.2 and 2.3, w e obtai n the follo wing result. Lemma 4.5. F or finite M DPs, and for al l atomic pr op ositions q , r , we have [ [ ∃ > 0 q W r ] ] = [ [ h h 1 , p i i  q U (( q ∧ r ) ∨ h h 1 i i ✷ q )  ] ] . Regarding formulas of the t yp e ∃ 1 U , they can b e mo del-c hec k ed using the µ -calculus ex- pression (3.7). T o obtain a translation int o A tl , whic h will b e giv en in pro of of T heorem 4.7, w e first translate into A tl the op erator Apr e . T o this end , for A t l form ulas ϕ , ψ , define F Apr e ( ϕ, ψ ) = ( h h 1 i i  ( ϕ ∧ ψ )) ∨  h h∅i i  ϕ ∧ h h p i i  ψ  . Lemma 4.6. F or AMD Ps, and for al l A tl formulas ϕ , ψ , we have [ [ F Apr e ( ϕ, ψ )] ] = Apr e ([ [ ϕ ] ] , [ [ ψ ] ] ) . Pr o of. W e consider the follo wing c haracterizat ion of the Apr e op erator, v alid f or AMD Ps: for sets X and Y , and a state s we ha v e s ∈ Apr e ( Y , X ) iff the follo wing conditions hold: (a) if s ∈ S 1 , then there exists a ∈ Γ( s ) suc h that δ ( s, a ) ∈ X ∩ Y ; and (b) if s ∈ S p , then for the unique action a ∈ Γ ( s ), we h a ve Dest ( s, a ) ⊆ Y and De st ( s, a ) ∩ X 6 = ∅ . The definition of F Apr e captures the ab ov e t wo conditions. The result follo w s. Note that the lemma holds only for alternating MDPs: indeed, w e will sho w that, on non-alternating MDPs, the op erator Apr e is n ot translatable in to A t l . Using these le mmas, w e can sho w that on finite AMDPs, we hav e ≈ A TL ⊆ ≈ > 0 . This result is tigh t: we cannot relax the assumption that the MDP is finite, nor t he assumption that it is alternating. Theorem 4.7. The fol lowing assertions hold: (1) On finite AMDPs, we have ≈ A TL ⊆ ≈ > 0 . (2) Ther e is a finite M D P on which ≈ A TL 6⊆ ≈ > 0 . (3) Ther e is an infinite, but finitely-br anching, AMDP on which ≈ A TL 6⊆ ≈ > 0 . Pr o of. Assertion 1. W e pro v e that on a fi nite, alternating MD P , the coun terp ositiv e holds: if s 6≈ > 0 t , then s 6≈ A TL t . Let s and t b e t wo state s such that s 6≈ > 0 t . Then, there m ust b e a formula ϕ in Qr c tl > 0 that distinguishes s from t . F rom this formula, we deriv e a form ula f ( ϕ ) in A tl that distinguish es s from t . W e pro ceed b y structural induction on ϕ , starting from the inner part of the formula and replacing successiv e parts that are in the scop e of a path quantifier by their A tl ve rsion. The cases wh ere ϕ is an atomic prop osition, or a b o olean combinati on of formulas are trivial. Using (2.1), w e reduce Qrctl > 0 -form ulas that in volv e a ∀ op erator to f orm ulas that only in v olv e the ∃ op erator. Lemma 2.3 pro vid es translations for all suc h form u las, exc ept those of type ∃ 1 ( ϕ U ψ ). F or instance, (2.3) lea ds to f ( ∃ > 0 ϕ U ψ ) = h h 1 , p i i f ( ϕ ) U f ( ψ ). In order to translate a form ula of the form γ = ∃ 1 ( ϕ U ψ ), we translate the ev aluation of the nested µ -calculus form ula (3.7) into the ev aluation of a nested A tl formula , as follo ws. Define the set of form ulas { α i,j | 0 ≤ i, j ≤ n } , where n = | S | is the num b er of state s of the AMD P , QUALIT A TIVE LOGICS AND EQUIV ALENCES FOR PROBABILISTIC SYSTEMS 15 b c b q a q a c s t Figure 5: States s and t cannot b e distinguished by A tl , but are d istinguished b y ∃ 1 ✸ q . q q q q q q q q 1 − x 1 1 − x 2 1 − x 3 t s y 1 y 2 y 3 1 − y 1 1 − y 2 1 − y 3 x 1 x 2 x 3 Figure 6: An infin ite Marko v chain on whic h ≈ A TL 6⊆ ≈ > 0 , where x i ’s and y i ’s represent the probabilities that the co rresp onding edge is tak en. via the follo wing clauses: ∀ i ∈ [0 .. n ] : α i, 0 = ff ∀ j ∈ [1 ..n ] : α 0 ,j = tt ∀ i ∈ [1 .. n ] . ∀ j ∈ [0 ..n − 1] : α i,j +1 = f ( ψ ) ∨  f ( ϕ ) ∧ F Apr e ( α i − 1 ,n , α i,j )  . F rom Lemma 4.6, the abov e set of form u las encod es the iterativ e ev aluatio n of the nested fixp oint (3.7), so that w e hav e [ [ α n,n ] ] = [ [ γ ] ] , and we can define f ( γ ) = α n,n . This c oncludes the translation. Assertion 2. Consider the MDP sho wn in Figure 5 . Th e states s and t are suc h that ( s, t ) ∈ ≈ A TL . Ho w ev er, s | = ∃ 1 ( ✸ q ) (co nsider the strategy that pla ys alw a ys a ), whereas t 6| = ∃ 1 ( ✸ q ). Assertion 3. Consid er the infinite AMDP shown in Figure 6. All state s are probabilistic states, i.e . S 1 = ∅ . F or all i > 0, we set x i = 1 2 and y i = 2 − 1 2 i , so that Q i> 0 x i = 0 and Q i> 0 y i = 1 2 . It is easy to see that s ≈ A TL t . Ho wev er, s | = ∃ > 0 ( ✷ q ) and t 6| = ∃ > 0 ( ✷ q ). The example in Figure 5 also sho ws that on non-alternating MDPs, unlike on alternating ones (see Lemma 4.6), the Apr e op erator cannot b e enco ded in A tl . If we were able to enco de Apr e in A tl , b y pro ceeding as in the pro of of the first assertion, giv en t wo states s , t with s 6≈ > 0 t , we could construct an A tl f ormula distinguishing s from t . As a corollary to T heorems 4.4 and 4.7, we ha v e that on finite, alternating MDPs, the equiv alences induced by A tl and Qrctl co incide. Thus the d iscrete graph theoreti c algorithms to compute equiv alences for A tl can b e used to compute the Qrctl equiv alences for finite AMDPs. Corollary 4.8. F or finite AMDPs, we ha ve ≈ > 0 = ≈ A TL . 16 K. CHA TTERJEE, L. ˜ DE ALF ARO, M. F AELLA, AND A. LEGA Y a b c b a r q u s s ′ t (a) A non-alternating MDP . h s ′ , c i r q s ′ s h s, b i h s ′ , b i h s ′ , a i h s, a i u t (b) An alternating MDP . Figure 7: MDPs illustrating ho w separating nondeterministic and pr obabilistic choice does not help to compu te ≈ > 0 . 5. C omputing Qrctl Equiv alences In this section, we tak e adv an tage of the results obtained in Section 4 to deriv e algorithms to co mpute ≈ > 0 and ≈ al l for AMD Ps. W e also pro vide an algorithm to compute those relations on non-alternating MDPs. 5.1. Alternat ing MDPs Corollary 4.8 immediately p ro vides an algorithm for t he computation of t he Qrctl equiv a- lences on AMD Ps, via the computation of the A tl equiv alences (inte rpreting nondetermin- ism and probabilit y as the t wo p la yers). In particular, the partition-refinemen t algorithms presen ted in [A HK02] can b e direct ly applied t o the problem. This yie lds the foll o wing result. Theorem 5.1. The two pr oblems of c omputing ≈ > 0 and ≈ al l on finite AMDPs ar e P TIM E- c omplete. Pr o of. Consider a tu r n-based game and consider the AMDP obtained from the game as- signing u niform transitio n probabilities to all out-going edges from a pla y er 2 state. Th en the 2-pla yer game in terpretation of the AMDP coincides with the original turn-based game. The r esult then follo ws from Corolla ry 4.8, and from the PTIME-completeness of A TL mo del c hec k in g and computing ≈ A TL [AHK02]. 5.2. Non-Alternat ing MDPs F or the general case of non-alter nating MDPs, on the other hand , the situatio n is not nearly as simple. First, let us disp el the b elief that, in order to compute ≈ > 0 on a non- alternating MDP , w e can con vert the MDP in to an alternating one, compute ≈ > 0 via ≈ A TL (using Coroll ary 4.8) on the alternating one, a nd then some ho w obtain ≈ > 0 on the original non-alternating MDP . Th e follo wing example sho ws that this, in general, is not p ossible. Example 5.2. Consid er the MDP d epicted in Figure 7(a), where the set of predicates is AP = { q , r } . W e ha v e s ≈ > 0 s ′ . Indeed, the only difference b et ween s and s ′ is that at state s ′ the action c is a v ailable: since c is a con v ex combinatio n of a and b , s and s ′ are probabilistically b isimilar in the sense of [SL94], and thus also related by ≈ > 0 . W e tran s f orm this MDP in to an alternati ng one by adding, for eac h state s and eac h a ∈ Γ( s ), a state QUALIT A TIVE LOGICS AND EQUIV ALENCES FOR PROBABILISTIC SYSTEMS 17 h s, a i whic h represen ts the d ecision of c ho osing a at s ; the result i s depicted i n Figure 7 (b). In this AMDP , how ev er, the stat e h s ′ , c i has no equiv alen t, as it satisfies b oth ∃ > 0  q and ∃ > 0  r . Therefore, on th is AMDP we ha v e s 6≈ > 0 s ′ , as witnessed by the form u la ∃ al l  (( ∃ > 0  q ) ∧ ( ∃ > 0  r )). As the example illustrates, t he problem is that once n ondeterminism and p robabilit y are separated into different states, the distinguishing p o w er of ≈ > 0 increases, so that computing ≈ A TL on the resulting al ternating MDP do es not help t o compute ≈ > 0 on the original non- alternating one. F ailure of lo cal partition refinemen t. Sim ulation and bisim ulation relations can b e com- puted via partition refinemen t algo rithms that consider, at eac h step, the 1-neighb ourho o d of eac h state: that is, the set of states reac hable from a giv en state in one step [Mil90]. W e call suc h algorithms 1-neig hb ourho o d p artition r efinements. Here, w e sho w a general re- sult: no 1-neigh b ourho o d partition refinement al gorithm ex ists for ≈ > 0 on non-alternating MDPs. W e mak e this notion precise as follo ws. Consid er an MD P G = ( S, A , Γ , δ , [ · ]), together with an equiv alence relation ∼ on S . In tuitiv ely , tw o states are 1-neigh b ourho o d isomorphic up to ∼ if their 1-step future looks identica l, up to the equiv alence ∼ . F ormally , w e sa y that t w o states s, t ∈ S are 1-neighb ourho o d isom orphic up to ∼ , written s 1 ∼ t , iff s ∼ t , and if there is a bijectio n R b et wee n E ( s ) and E ( t ), and a bijection ˆ R b et w een Γ( s ) and Γ( t ), whic h pr eserv e ∼ and the trans ition probabilities. Preci sely , w e require that: • if s ′ ∈ E ( s ) and t ′ ∈ E ( t ) with s ′ R t ′ , then s ′ ∼ t ′ ; • if a ∈ Γ( s ) and b ∈ Γ( t ) with a ˆ R b , then for all s ′ ∈ E ( s ) and t ′ ∈ E ( t ) with s ′ R t ′ , we ha v e δ ( s, a )( s ′ ) = δ ( t, b )( t ′ ). Let P art S b e the set of equiv alence relations on S . A p artition r efinement op er ator f : Part S 7→ Part S is an op erator suc h that, for all ∼ ∈ Part S , w e hav e f ( ∼ ) is finer than ∼ . W e say that a partition op erator c omputes a relation ≈ if w e ha ve ≈ = lim n →∞ f n ( ∼ pr e d ), where f n denotes n rep eated applications of f and s ∼ pr e d t iff [ s ] = [ t ]. W e sa y that a partition r efi n emen t op erator f is 1-neighb ourho o d if i t refines an equiv a- lence relation ∼ on the basis of the 1-neigh b ourho o d of the states, treating in the same fash- ion state s whose 1-neigh b ourho o ds are isomorphic u p to ∼ . Precisely , f is 1-neighb ourho o d if, f or all ∼ ∈ Part S and for all s, s ′ , t, t ′ ∈ S with s 1 ∼ s ′ , t 1 ∼ t ′ , w e ha v e either ( s, t ) , ( s ′ , t ′ ) ∈ f ( ∼ ), or ( s, t ) , ( s ′ , t ′ ) 6∈ f ( ∼ ). W e can no w stat e the n on-existence of 1- neigh b ourho o d refinemen t op erators for ≈ > 0 as follo ws. q r s 4 s 3 s 2 s 1 Figure 8: MDP sho wing the lac k of 1- neigh b ourh o o d r efinemen t op erators. 18 K. CHA TTERJEE, L. ˜ DE ALF ARO, M. F AELLA, AND A. LEGA Y Theorem 5.3. Ther e is no 1-neighb ourho o d p artitio n r e finement op er ator which c omputes ≈ > 0 on al l MDPs. Pr o of. Consider the states s 1 , s 2 , s 3 , s 4 of the MDP depicted in Figure 8, and tak e ∼ = ∼ pr e d . Let f b e an y 1-neigh b ourho o d partition refinement op erator. F rom s 1 ∼ s 2 ∼ s 3 ∼ s 4 , we can s ee that s 2 1 ∼ s 3 1 ∼ s 4 . Let ∼ ′ = f ( ∼ ). Considerin g the pairs ( s 1 , s 2 ), ( s 1 , s 3 ), and ( s 1 , s 4 ) in the definition o f 1-neig h b our ho o d partitio n refinement op erator, w e ha v e that ∼ ′ satisfies one of the follo wing tw o cases: (1) s 1 6∼ ′ s 2 and s 1 6∼ ′ s 3 and s 1 6∼ ′ s 4 , (2) s 1 ∼ ′ s 2 and s 1 ∼ ′ s 3 and s 1 ∼ ′ s 4 . In the fi rst case , the partition refinemen t terminates with a relation ∼ ′′ suc h that s 1 6∼ ′′ s 2 . This is incorrect, since w e can pro ve b y ind uction on the le ngth of Qrctl > 0 form ulas that no suc h formula distinguishes s 1 from s 2 , so that s 1 ≈ > 0 s 2 . In the second case, the partition refinemen t terminates with a relatio n ∼ ′′ suc h that s 1 ∼ ′′ s 3 . This is also incorrect, since the f orm ula ∃ 1 ♦ r is a witness to s 1 6≈ > 0 s 3 . W e conclude that a 1-neigh b ourho o d p artition refinemen t op erator cannot compute ≈ > 0 . T o giv e an algorithm for the computation of ≈ > 0 , giv en t w o sets of states C 1 and C 2 , let: U ( C 1 , C 2 ) = { ~ ω = h s 0 , s 1 , . . . i | ∃ j ≥ 0 . s j ∈ C 2 and ∀ 0 ≤ i < j . s i ∈ C 1 } E U 1 ( C 1 , C 2 ) = { s ∈ S | ∃ σ ∈ Σ . Pr σ s ( U ( C 1 , C 2 )) = 1 } . In tuitiv ely , if C 1 = [ [ ϕ 1 ] ] and C 2 = [ [ ϕ 2 ] ] for t wo Qrctl formula s ϕ 1 and ϕ 2 , then E U 1 ( C 1 , C 2 ) is [ [ ∃ 1 ( ϕ 1 U ϕ 2 )] ]. W e sa y that an equiv alence relation ≃ is 1 , p, E U -stable if, for all unions C 1 , C 2 of equiv alence classes with resp ect to ≃ , and for al l s, t ∈ S with s ≃ t , w e hav e: (1) s ∈ Pr e ( C 1 ) iff t ∈ Pr e ( C 1 ); (2) s ∈ Cpr e ( C 1 ) iff t ∈ Cpr e ( C 1 ); (3) s ∈ E U 1 ( C 1 , C 2 ) iff t ∈ E U 1 ( C 1 , C 2 ). Let ≈ EU A TL b e the coarsest equiv alence r elatio n that is 1 , p, E U -stable. W e sho w that ≈ EU A TL coincides with ≈ > 0 . Theorem 5.4. F or al l finite MDPs, we ha ve ≈ EU A TL = ≈ > 0 . Pr o of. W e pro ve con tainmen t in the t w o directi ons. ≈ EU A TL ⊆ ≈ > 0 . T h is stat emen t is equ iv alen t to saying that for all f orm ulas ϕ in Qr ctl > 0 , [ [ ϕ ] ] is the union of classes in S/ ≈ EU A TL . L et s and t b e t w o states suc h that s 6≈ > 0 t , and let ϕ b e a f orm ula from Qr ctl > 0 suc h that s | = ϕ and t 6| = ϕ . W e sho w b y structural ind uction on ϕ that s 6≈ EU A TL t . T he cases where ϕ is a prop osition, or th e b o olean com bination of form ulas are trivia l. All other cases follo w as in the pro of of the fi rst part of Theorem 4.7, except for the case ϕ = ∃ 1 ( ϕ 1 U ϕ 2 ). F or ϕ = ∃ 1 ( ϕ 1 U ϕ 2 ), we ha v e s ∈ E U 1 ([ [ ϕ 1 ] ] , [ [ ϕ 2 ] ]), while t 6∈ E U 1 ([ [ ϕ 1 ] ] , [ [ ϕ 2 ] ]). By inductiv e h yp othesis, w e can assume that [ [ ϕ 1 ] ] and [ [ ϕ 2 ] ] are unions of classes in S / ≈ EU A TL . So, ( s, t ) 6∈ ≈ EU A TL . ≈ > 0 ⊆ ≈ EU A TL . The pro of follo w s the same idea of t he pro of of the first part of Theorem 4.4. The only mo dification needed is in the inductive definition of the set of form ulas: w e tak e here Ψ k +1 = BoolC (Ψ k ∪ {∃ > 0  ψ , ∃ al l  ψ , ∃ 1 ψ U ψ ′ | ψ, ψ ′ ∈ Ψ k } ). QUALIT A TIVE LOGICS AND EQUIV ALENCES FOR PROBABILISTIC SYSTEMS 19 The follo wing theorem pro vides an upp er b oun d for the comple xit y of computing ≈ > 0 on MD Ps. The PTIME-completeness o f ordinary sim ulation [ABGS91] pro vides a lo w er b ound , but no tigh t lo wer b oun d is kno wn. Theorem 5.5. Th e pr oblem of de ciding whether s ≈ > 0 t for two sta tes s and t of an MDP is in c o-NP. Pr o of. W e sho w that the problem of d eciding s 6≈ > 0 t is in NP . T o this end, w e ha v e to sho w that there is a certificate for s 6≈ > 0 t that has p olynomial size, and is p olynomially c h ec k able. Consider the u sual partition-refinemen t metho d for computing ≈ A TL [Mil90, AHKV98]. T he metho d starts with an equiv alence relatio n ≃ that refl ects prop ositional equiv alence. Then, ≃ is refined at most m = | S | times. At eac h r efinemen t ste p, some state-pairs are remo v ed from ≃ . A certificate for the remo v al of a pair from ≃ is simply a Cpr e or Pr e or E U 1 op erator, along with a union of equiv alence classes; it is th u s of size p olynomial in m . S ince a t most m 2 pairs can b e remo v ed from ≃ , the total size of these state -pair remo v al certi ficates is p olynomial in m . Th is yields a p olynomial-size and p olynomially-c heck able certificate for s 6≈ > 0 t . 6. T he R oles of Un til and W ait-F or In this section we study the roles of the unti l and the wa it-for op erator, and the relationship b et w een the equiv alences in d uced by Qr ctl and Qrctl ∗ . It is w ell kn o wn that in t he s tand ard b ranc hing logics Ctl and Ctl ∗ , as w ell as in A TL, the next-time op erator  is the only temp oral op erator needed for charac terizing bisim ulation. F or Qrctl , this is not the case: the op erators U and W can increase the distinguishing p ow er of the log ics, as the follo win g theorem indicates. Theorem 6.1. The fol lowing assertions hold: (1) On finitely-br anching MDPs, we have ≈ > 0 ,  = ≈ A TL . (2) F or al l MDPs, we have ≈ > 0 ⊆ ≈ > 0 ,  . (3) F or finite AM D Ps, we have ≈ > 0 ,  = ≈ > 0 . (4) Ther e is a finitely-br anching, infinite A MDP on which ≈ > 0 ,  6⊆ ≈ > 0 . (5) Ther e is a finite, (non-alternating) MDP on which ≈ > 0 ,  6⊆ ≈ > 0 . Pr o of. Assertion 1. Th e inclusion ≈ > 0 ,  ⊆ ≈ A TL follo ws from the fact that f ormulas used in the first part of th e pro of of T heorem 4.4 mak e use only of the  temp oral op erator, and from ≈ A TL = ≈  A TL . T o pro v e the inclusion ≈ A TL ⊆ ≈ > 0 ,  , consider t w o states s, t ∈ S suc h that s 6≈ > 0 ,  t . Then, there is a Qrctl > 0 form ula ϕ that distinguishes them. F rom this form ula w e deriv e an A tl form ula f ( ϕ ) that also d istinguishes them. W e p r o ceed b y structural ind u ction. The r esu lt is ob vious for b o olean operators and atomic prop ositions. The cases ϕ = ∃ 1  ϕ 1 and ϕ = ∃ > 0  ϕ 1 are an easy consequence of Lemma 2.3. Assertion 2. Immediate, as the set of Qrctl > 0 form ulas without U and W is a subs et of the set of all Q rctl > 0 form ulas. Assertion 3. Th e result is derived as follo ws: ≈ > 0 ,  ⊆ ≈ A TL = ≈ > 0 . Th e inclusion ≈ > 0 ,  ⊆ ≈ A TL is a c onsequence o f Asse rtion 1 of this th eorem. The equalit y ≈ Game = ≈ > 0 follo ws b y com bin in g Assertion 1 of Theorem 4.4 and Assertion 1 of T heorem 4.7. 20 K. CHA TTERJEE, L. ˜ DE ALF ARO, M. F AELLA, AND A. LEGA Y Assertion 4. The result follo ws by considering agai n the infinite AMDP of Fi gure 6. Rea- soning as in the p ro of of Theorem 4.7, it holds ( s, t ) ∈ ≈ > 0 ,  , but ( s, t ) 6∈ ≈ > 0 : ind eed, note that s | = ∃ > 0 ( ✷ q ) and t 6| = ∃ > 0 ( ✷ q ). Assertion 5. The result is a consequence of T heorem 4.7, Assertion 2, and of the present theorem, Assertio n 1: the same MDP used to show ≈ A TL 6⊆ ≈ > 0 , d epicted in Fi gure 5, also sho ws ≈ > 0 ,  6⊆ ≈ > 0 . 7. Linear Tim e N esting The logics Ctl and Ctl ∗ induce the same equiv alence, namely , bisimula tion. Similarly , A TL and A TL* b oth induce alternating bisim u lation. W e sho w here that Qrctl and Qrctl ∗ induce th e same equiv alences on finite, alternating MDPs, but w e sho w that for infinite, or non-alternati ng, MDPs, Qrctl ∗ induces finer relations than Qrctl . These results are sum m arized b y the follo wing theorem. Theorem 7.1. The fol lowing assertions hold: (1) F or al l MDPs, we have ≈ > 0 ∗ ⊆ ≈ > 0 . (2) F or al l finite AMDPs, we have ≈ > 0 ∗ = ≈ > 0 . (3) Ther e is a finitely-br anching, infinite A MDP, on which ≈ > 0 6⊆ ≈ > 0 ∗ . (4) Ther e is a finite M D P on which ≈ > 0 6⊆ ≈ > 0 ∗ . Before presen ting the p ro of of this resu lt, it is useful to recall some facts ab out Rabin automata, Mark o v decision pro cesses, and probabilistic v erification. Rabin automata and temp oral logic. An infinite-wor d automat on over AP is a tuple A = ( L, L init , p · q , ∆ ), where L is a finite set of locations, L init ⊆ L is the set of initial lo cations, p · q : L 7→ 2 AP is a lab eling fu nction that asso ciates with eac h location l ∈ L the set p l q ⊆ AP of predicates that are true at l , and ∆ : L 7→ 2 L is the transition relatio n. The automaton A is d eterministic if the follo wing co nditions hold: • for all η ⊆ AP , there is a unique l ∈ L init with p l q = η ; • for all l ∈ L and all η ⊆ AP , there is l ′ ∈ ∆( l ) with p l ′ q = η ; • for all l , l ′ , l ′′ ∈ L , w e ha v e that l ′ , l ′′ ∈ ∆( l ) and l ′ 6 = l ′′ implies p l ′ q 6 = p l ′′ q . The set of paths of A is Paths ( A ) = { l 0 , l 1 , l 2 , . . . | l 0 ∈ L init ∧ ∀ k ≥ 0 . l k +1 ∈ ∆( l k ) } . A R abin ac c eptanc e c ondition o v er a set L is a set of pairs F = { ( P 1 , R 1 ) , ( P 2 , R 2 ) , . . . , ( P m , R m ) } where, for 1 ≤ i ≤ m , we hav e P i , R i ⊆ L . T h e acceptance condition F defines a set of paths o ver L . F or a p ath τ = s 0 , s 1 , s 2 , . . . ∈ L ω , w e defin e I nf ( τ ) to b e the s et of locations that o ccur infinitely often al ong τ . W e define P aths ( F ) = { τ ∈ L ω | ∃ i ∈ [1 .. m ] . (Inf ( τ ) ∩ P i = ∅ ∧ I nf ( τ ) ∩ R i 6 = ∅ ) } . A R abin automato n ( A, F ) is an infi nite-w ord automato n A with set of lo cations L , toge ther with a Rabin acceptance conditio n F on L ; w e asso ciate wit h it the set of paths Paths ( A, F ) = Paths ( A ) ∩ Paths ( F ). Giv en a set of predicates AP , a tr ac e ρ ∈ (2 AP ) ω over AP is an infinite sequence of interpretat ions of AP ; we indicate with T r ac es ( AP ) = (2 AP ) ω the set of all traces o ver AP . A Rabin au tomaton ( A, F ) with A = ( L, L init , p · q , ∆ ) induces the set of traces T r ac es ( A, F ) = { p l 0 q , p l 1 q , p l 2 q , . . . | l 0 , l 1 , l 2 , . . . ∈ Paths ( A, F ) } . An L tl formula ϕ o v er the set of prop ositions AP induces the set of traces T r ac es ( ϕ ) ⊆ T r ac es ( AP ), defined as QUALIT A TIVE LOGICS AND EQUIV ALENCES FOR PROBABILISTIC SYSTEMS 21 usual (see, e.g., [MP9 1]). F rom [VW 86] it is known that for an L tl formula ϕ w e can construct a d eterministic Rab in automa ton ( A, F ) suc h that T r ac es ( A, F ) = T r ac es ( ϕ ). W e can n ow pro ceed to pr o ve Theorem 7.1. Pro of of Theorem 7.1. Pr o of. The first assertion is obvio us. F or the other assertions, w e pro ceed as follo ws. Assertion 2. Let G = ( S, A , Γ , δ , [ · ]) b e a finite, alternating MDP . Since Qrctl is a frag- men t of Qrctl ∗ , it follo ws that ≈ > 0 ∗ ⊆ ≈ > 0 . T o p ro v e ≈ > 0 ⊆≈ > 0 ∗ , w e sho w that if there exists a Qrctl ∗ form ula that distinguishes tw o stat es s and t , then there also exists a Qrctl form ula t hat distinguishes s and t . W e fo cus on form ulas of the typ e ∃ > 0 ϕ and ∃ 1 ϕ , wh ere ϕ is an L tl form ula. The generalization to the complete logic follo ws b y structural ind u ction and dualit y . T h us, assume that there are t w o state s s ∗ , t ∗ ∈ S and α ∈ { 1 , > 0 } suc h that s ∗ | = ∃ α ϕ and t ∗ 6| = ∃ α ϕ . Let ( A, F ) b e a deterministic Rabin automaton s uc h that T r ac es ( A, F ) = T r ac e s ( ϕ ), and assum e that A = ( L, L init , p · q , ∆ ) and F = { ( P 1 , R 1 ) , . . . , ( P m , R m ) } . Let G ′ = G × A = ( S ′ , A , Γ ′ , δ ′ , [ · ] ′ ) b e the MDP resulting from forming the usual sync hronous prod uct of G and A . In detail, we ha ve: • S ′ = { ( s, l ) ∈ S × L | [ s ] = p l q } ; • Γ ′ ( s, l ) = Γ( s ) for all ( s, l ) ∈ S ′ ; • for all ( s 1 , l 1 ) , ( s 2 , l 2 ) ∈ S ′ and a ∈ A , we ha v e δ ′ (( s 1 , l 1 ) , a )( s 2 , l 2 ) = δ ( s 1 , a )( s 2 ) if l 2 ∈ ∆( l 1 ), and δ ′ (( s 1 , l 1 ) , a )( s 2 , l 2 ) = 0 otherwise; • [( s, l )] = p l q , for all ( s, l ) ∈ S ′ . Let F ′ b e the Rabin accepta nce condition of G ′ , defined b y F ′ = { ( P ′ 1 , R ′ 1 ) , . . . , ( P ′ m , R ′ m ) } , where eac h P ′ i , R ′ i ⊆ S ′ is defi n ed as follo ws: P ′ i = { ( s, l ) ∈ S ′ | l ∈ P i } and R ′ i = { ( s, l ) ∈ S ′ | l ∈ R i } . F or ev ery s ∈ S , denote with l init ( s ) the u nique l ∈ L init suc h that [ s ] = p l q . Using the results of [dA97a, d AHK98, CdAH04] on the mo del-c h ec king of MDPs with resp ect to probabilistic te mp oral-log ic p rop erties, w e ca n construct µ -calculus form ulas to distinguish ( s ∗ , l init ( s ∗ )) and ( t ∗ , l init ( t ∗ )). Define, first of all, the follo wing abbreviati ons: ˆ ψ al l = m [ i =1 ν Y . µX. h P ′ i ∩  Cpr e ( X ) ∪ ( R ′ i ∩ Cpr e ( Y ))  i ˆ ψ 1 = m [ i =1 ν Y .µX. h P ′ i ∩  Apr e ( Y , X ) ∪ ( R ′ i ∩ Cpr e ( Y ))  i ˆ ψ some = m [ i =1 ν Y . µX. h P ′ i ∩  Pr e ( X ) ∪ ( R ′ i ∩ Pr e ( Y ))  i . On the basis of the abov e form u las, define: ψ al l = µW .  ˆ ψ al l ∪ Cpr e ( W )  ψ 1 = ν Z . µW .  Apr e ( Z, W ) ∪ ˆ ψ 1  ψ > 0 = µW .  ˆ ψ 1 ∪ Pr e ( W )  ψ some = µW .  ˆ ψ some ∪ Pr e ( W )  . F or α ∈ { al l , 1 , > 0 , some } and s ∈ S , we ha ve: ( s, l init ( s )) ∈ [ [ ψ α ] ] G ′ iff s | = ∃ α ϕ 22 K. CHA TTERJEE, L. ˜ DE ALF ARO, M. F AELLA, AND A. LEGA Y so that, in particular, ( s ∗ , l init ( s ∗ )) ∈ [ [ ψ α ] ] G ′ and ( t ∗ , l init ( t ∗ )) 6∈ [ [ ψ α ] ] G ′ . Hence, the form ula ψ α is a µ -cal culus witness, on G ′ , of the distinction b et w een s ∗ and t ∗ . W e n o w sh o w h ow to transform ψ α , first in to a µ -c alculus formula to b e ev aluated on G , and then in to a Qrctl form ula to b e ev aluated on G . This will sho w th at s ∗ 6≈ > 0 t ∗ , as r equired. T o obtain a µ -calculus formula on G , f rom ψ α w e construct a µ -calculus formula γ α with the follo wing prop ert y: for all s ∈ S , w e h a ve s ∈ [ [ γ α ] ] G iff ( s, l init ( s )) ∈ [ [ ψ α ] ] G ′ . The idea, tak en from [dAHM01], is as follo ws. First, ψ α can b e rewritten in e quational form [B C96], as a sequence of b lo c ks B ′ 1 , . . . , B ′ k , where B ′ 1 is the innermost blo c k and B ′ k the outermost blo c k. Eac h blo ck B ′ j , for 1 ≤ j ≤ k , has the form v j = λe j , wh ere λ ∈ { µ, ν } , and where e j is an expression not cont aining µ , ν , in whic h all the occurrences of the v ariables v 1 , . . . , v k ha v e p ositiv e p olarit y [BC96]; the output v ariable is v k . F rom this formula , we obtain another formula γ α , also in equational form, with sets of v ariables { v l i | 1 ≤ i ≤ k ∧ l ∈ L } ∪ { v k +1 } . F orm u la γ α sim ulates on G the ev aluation of ψ α on G ′ : f or eac h v ariable v i , w ith 1 ≤ i ≤ k , form ula γ α con tains the set of v ariables { v l i | l ∈ L } , where the v alue of v i at lo cation l ∈ L is encoded as the v alue of v l i at s . The form ula ψ consists of the b lo c ks B 1 , . . . , B k , p lus an additio nal b lo c k B k +1 . F or 1 ≤ i ≤ k , the block B i con tains the equations for the v ariables { v l i | l ∈ L } . The equatio n for v l i is obtained from the equation f or v i as follo w s: • replace eac h v ariable v i on the left-hand side w ith the v ariable v l i ; • replace P j (resp. R j ), for 1 ≤ j ≤ m , with S if l ∈ P j (resp. l ∈ R j ), and with ∅ if l 6∈ P j (resp. l 6∈ R j ); • replace Cpr e ( v h ), for v ariable 1 ≤ h ≤ k , with Cpr e ( S l ′ ∈ ∆( l ) v l ′ h ); • int ersect the r igh t-hand side with T q ∈ p l q q ∩ T q ∈ AP \ p l q ¬ q . The block B k +1 consists of only one equation v k +1 = S l ∈ L init v l k , and can b e either a µ or a ν -blo c k. The output v ariable is v k +1 . The r esu lt of the ab o v e transf ormation is a µ -calc ulus form ula γ α on G con taining only the op erators Cpr e and Apr e . By (3.1) and Lemma 4.6, b oth op erators can b e enco ded in Qrctl . Th en, pro ceeding as in the first part of the pro of of Theorem 4.7, w e can “unroll” the computation of the fixp oin ts of the µ -calculus formulas, since we know that eac h fixp oin t con verges in a t most | S | it erations. The result of these t wo transformations is a Qr ctl form ula λ α , such that s ∗ | = λ α and t ∗ 6| = λ α , as required. Assertion 3. Consider the AMDP G with state sp ace S = ( { 1 , 2 , 3 } × N ) ∪ { 0 } . Th e only successor of 0 is 0 itself. States of the t yp e h i, 2 n i , for i ∈ { 1 , 2 , 3 } (i.e., even stat es ) b elong to pla y er 1, while od d states b elong to pla y er p . F or all n ≥ 0 w e hav e: Γ( h 1 , 2 n i ) = Γ( h 3 , 2 n i ) = { a, b } and Γ( h 2 , 2 n i ) = { a, b, c } , wh ere, for all i ∈ { 1 , 2 , 3 } : Dest ( h i, 2 n i , a ) = {h i, 2 n i} Dest ( h i, 2 n i , b ) = {h i, 2 n + 1 i} Dest ( h 2 , 2 n i , c ) = {h 3 , 2 n + 1 i} . Pla yer p states starting with 1 or 2 lead to the next state in their c h ain and to the sink state 0 with equal probabilit y . F ormally , Γ( h i, 2 n + 1 i ) = { x } and δ ( h 1 , 2 n + 1 i , x )( h 1 , 2 n + 2 i ) = δ ( h 1 , 2 n + 1 i , x )(0) = δ ( h 2 , 2 n + 1 i , x )( h 2 , 2 n + 2 i ) = δ ( h 2 , 2 n + 1 i , x )(0) = 1 / 2 . QUALIT A TIVE LOGICS AND EQUIV ALENCES FOR PROBABILISTIC SYSTEMS 23 s t u q s ′ t ′ u ′ q Figure 9: An MDP wh ere s ≈ > 0 s ′ and s 6≈ > 0 ∗ s ′ . Finally , states starti ng w ith 3 ob ey the follo wing distribution. δ ( h 3 , 2 n + 1 i , x )( h 2 , 2 n + 2 i ) = exp( − 1 / 2 n ) δ ( h 3 , 2 n + 1 i , x )(0) = 1 − exp( − 1 / 2 n ) . Observe that G is a finitely-branc hing, infinite AMDP . W e tak e AP = { q } , and we ask that the p redicate q b e true at all od d states. Then, b y induction on the stru ctur e of a Qrctl form ula, it is not hard to see that h 1 , 0 i ≈ > 0 h 2 , 0 i . On the other h an d , w e ha v e h 2 , 0 i | = ∃ > 0 ✷ ♦ q and h 1 , 0 i 6| = ∃ > 0 ✷ ♦ q . Assertion 4. Consid er the MDP depicted in Figure 9. By induction on the structure of a Qrctl f ormula, it is not hard to see that s ≈ > 0 s ′ . On the other h and , for ϕ = ∃ 1 ( ♦ q ∧ ✷ ∃ > 0  q ) we ha ve s | = ϕ , s ′ 6| = ϕ . W e do not provide an alg orithm f or computing ≈ > 0 ∗ on non-alternating MDPs. Iden ti- fying suc h an alg orithm is an op en problem. Ac kno wledgemen ts This researc h w as su pp orted in part by the NSF gran ts CCR-0132780 , CNS-072 0884 and CNS-083481 2, and b y a BAEF grant. References [ABGS91] C. ´ Alv arez, J. L. Ba lc´ azar, J. Gabarr´ o, and M. S´ antha. P arallel complexit y in the design and analysis of concurrent systems. In P ARLE ’91: Pr o c. on Par al l el ar chite ctur es and l anguages Eur op e . Springer-V erlag, 1991. [AHK02] R. Alur, T.A. Henzinger, and O. Kupferman. Alternating time temp oral logic. J. A CM , 49:672– 713, 2002. [AHKV98] R. Alur, T.A. Henzinger, O. Ku pferman, and M.Y. V ardi. Alternating refinement relations. In CONCUR 98: Concurr ency The ory. 9t h Int. Conf. , volume 1466 of L e ct. Notes in Comp. Sci. , pages 163–178 . Springer-V erlag, 1998. [ASB + 95] A. Aziz, V. Singhal, F. Balarin, R.K. Bra y t on, and A.L. Sangio v anni-Vincentelli . It u sually w orks: The temp oral logic of stochastic systems. In Comput er Aide d V erific ation , vo lume 939 of L e ct. Not es in Comp. Sc i. Springer-V erlag, 1995. [BC96] G. Bhat and R. Cl eav eland. Efficient mo del chec k ing via t he equational µ -calculus. In Pr o c. 11th IEEE Symp. L o gic in Comp. Sci. , pages 304–312, 1996. [BdA95] A. Bi anco and L. de Alfaro. Model chec king of probabilistic and n ondeterministic systems. In F ound. of Softwar e T e ch. and The or. Comp. Sci. , v olume 1026 of L e ct. Notes in Comp. Sci. , pages 499–513 . Springer-V erlag, 1995. [Ber95] D.P . Bertsek as. Dynamic Pr o gr amm i ng an d Optimal Contr ol . Athena Scientific, 1995. V olumes I and I I. 24 K. CHA TTERJEE, L. ˜ DE ALF ARO, M. F AELLA, AND A. LEGA Y [BS01] E. Bandini and R. Segala. Axiomatizations for probabilistic bisimulatio n. In Pr o c. 28th Int. Col lo q. Aut. L ang. Pr o g. , v olume 2076 of L e ct. Notes in Comp. Sci. , p ages 3 70–381. Springer- V erlag, 2001 . [CdAH04] K. Chatterjee, L. de Alfaro, and T.A. Henzinger. T rading memory for randomness. I n QEST 04 . IEEE Computer Society Press, 2004. [CE81] E.M. Clarke and E.A. Emerson. Design and synthesis of synchroniza tion ske letons u sing branch- ing time temp oral logic. In Pr o c. Workshop on L o gic of Pr o gr ams , volume 131 of L e ct. Notes in Comp. Sci. , pages 52–7 1. Springer-V erlag, 1981. [CY95] C. Courcoub etis and M. Y annak akis. The complexit y of p robabilistic verificatio n. J. A C M , 42(4):857 –907, 19 95. [dA97a] L. de A lfaro. F ormal V erific ation of Pr ob abilistic Systems . PhD thesis, St anford Universit y , 1997. T ec hnical Rep ort ST AN- CS-TR-98-1601. [dA97b] L. d e Alfaro. T emp oral lo gics for the specification of p erformance and reliabilit y . In Pr o c. of Symp. on The or. Asp. of Comp. Sci. , v olume 1200 of L e ct. Notes in Comp. Sci. , pages 165–176. Springer-V erlag, 1997. [dAFMR05] L. de Alfaro, M. F aella, R . Ma jumdar, and V. Raman. Code-aw are resource management. In EMSOFT 05: ACM Confer enc e on Emb e dde d Softwar e , Lect. Notes in Comp. Sci. Springer- V erlag, 2005 . [dAH00] L. de Alfaro and T.A. Henzinger. Concurrent omega-regular games. In Pr o c. 15t h IEEE Sy mp. L o gic i n Comp. Sci. , pages 141 –154, 2000. [dAHK98] L. de Alfaro, T.A. H enzinger, and O. Kupferman. Concurrent reachabilit y games. In Pr o c. 39th IEEE Symp. F ound. of Comp. Sci. , pages 564–5 75. IEEE Computer So ciety Press, 1998. [dAHM01] L. d e Alfaro, T.A. Henzinger, and R. Majumdar. F rom verification to con trol: Dynamic pro- grams for omega-regula r ob jectives . In Pr o c. 16t h I EEE Symp. L o gic in Comp. Sci. , pages 279–29 0. IEEE Press, 2001. [dAKN + 00] L. de Alfa ro, M. Kwia tko wsk a, G. Norman, D . P arker, and R. Segala . Symbolic model c heck- ing of concurrent probabilistic pro cesses using MT BDDs and the Kronec ker representation. In T A CAS: T o ols and Algorithms for the Construction and A nalysis of Systems , volume 1785 of L e ct. Notes in Comp. Sci. , pages 395 –410. S pringer-V erlag, 2000. [Der70] C. Derman. Finite St ate Marko vian De cision Pr o c esses . Academic Press, 1970. [DGJP99] J. Desharnais, V . Gupta, R. Jagadeesan, and P . P anan gaden. Metrics for la b elled mark ov systems. In CONCUR’99: Concurr ency The ory. 10th I nt. Conf. , volume 1664 of L e ct. Notes i n Comp. Sci. , pages 258– 273. Springer, 1999. [HJ94] H. Hansson and B. Jonsson. A logi c for reasoning a b out ti me and probabilit y . F ormal Asp e cts of Computing , 6(5):512–535, 1994. [KNP00] M. K wiatk o wsk a, G. Norman, and D. P arker. V erifying randomized d istributed algorithms with prism. In Workshop on A dvanc es in V erific ation (W A VE’00) , 2000. [Koz83] D. Kozen. Results on the p ropositional µ -calculus. The or etic al Computer Scienc e , 27(3):3 33– 354, 1983. [Mil90 ] R. Milner. Op erational and alg ebraic semantics of concurrent processes. I n J. v an Leeuw en, editor, Handb o ok of The or etic al Computer Scienc e , volume B, pages 1202–12 42. Elsevier Science Publishers (North-Holland), Amsterdam, 1990. [MP91] Z. Manna and A . Pn ueli. The T emp or al L o gic of R e active and Concurr ent Systems: Sp e cific ation . Springer-V erlag, N ew Y ork, 1991. [PSL00] A. Pog osya nts, R . Segala, and N . Lynch. V erification of t h e randomized consensus algor ithm of Aspnes and Herlihy: a case study . Distribute d Computing , 13(3):1 55–186, July 2 000. [Seg95] R. Segala. Mo deling and V erific ation of R andomize d Di stribute d Re al -Time Systems . PhD thesis, MIT, 1995. T echnical Report MIT/LCS/TR-676. [SL94] R. Segala and N.A. Ly nch. Probabilistic simulations for probabilistic pro cesses. In CONC UR’94: Concurr ency The ory. 5th Int. Conf. , volume 836 o f L e ct. Notes in Comp. Sci. , p ages 481–496. Springer-V erlag, 1994. [ST05] R. Segal a and A. T urrini. Co mparative anal ysis of bisimulatio n relations on alternating and non-alternating probabilistic models. I n QEST 05 . I EEE, 2005 . [Sto02] M.I.A. Sto elinga. F un with FireWire: Exp eriments with verifying the IEEE1394 ro ot conten tion protocol. In F ormal Asp e cts of Computing , 2002. QUALIT A TIVE LOGICS AND EQUIV ALENCES FOR PROBABILISTIC SYSTEMS 25 [VW86] M.Y. V ardi and P . W olp er. Au t omata theoretic techniques for modal logics of programs. J. Comp. Sys. Sci. , 32:1 83–221, 1 986.