Equilibria in Quitting Games - Basics

ε -Equilibria in Quitting Games – Basics Katharina Fisc her ∗ No v em b er 5, 2018 Abstract Quitting games are one of the simplest sto c hastic games in which at any stage each play er has only tw o po ssible actions, c ontinue and quit . The ga me ends as soon a s at least one play er choos es to q uit. The play ers then re c e iv e a pay off, whic h depends o n the set o f play ers that did choos e to quit. If the g ame never ends, the pay off to each player is z ero. Examples of quitting games w ere studied first b y Flesh, Th uijsman and V rieze in 1997 ([1]). Sola n 199 9 ([6]) pro ved that all three- pla yer quitting ga mes have approximate equilibria. In the pa per Quitt ing Games Solan and Vieille 2001 ([7]) proved the existence of subga me-perfect a ppro ximate equilibria under so me restrictio ns on the payoff funct ion. F urthermo re Solan and Vieille studied in [8] a four- pla yer quitting game example in which the simplest equilibrium s tr ategy is p erio dic with p erio d tw o. In The structur e of non- zer o-sum sto chastic games ([5]) Simon show ed under which properties quitting games hav e approximate equilibria among other things by genera lization of the solution-idea from Solan an Vieille. This paper giv es a shor t introductio n in to the topic quitting games a nd tries to illus - trate sev eral prop erties with examples. First the mathematical mo del o f a quitting game is present ed. After the definition of the strategy and strategy profile the cor respo nding probability space and the underlying sto chastic pro cess are stated. This leads to the exp ected payoff and the definition of some equilibria. F or a better ana lysis of quitting games the so calle d one-step game is in tro duced in the second part of this pap er. Imp ortant prop erties of strateg y profiles in one-step games are posted and proved. In the third sec tion an imp ortant theorem from Solan and Vieille (cf. [7]) is cited, in whic h the exis tence of a ppr o ximate eq uilibria, under so me assumptions to the pa yoff function, is p ostulated. It’s pro of is divided in to three parts, how ever this paper concen- trates only on the fir st one. In the referred literature only a few s teps of the pro of are denoted. It is the aim to show the pro of at length under usage of the then known results. 1 The mo del A quitting game is a sequen tial N -pla y er ( N ∈ N ) game and play ed as follo ws. In e very game turn eac h p la y er has only t wo p ossible actions c ontinue and quit . The game end s as so on as at least one of the N -pla y ers c ho oses to quit. W e d enote S (quitting coal ition) as the subset of ∗ This article app eared un der th e former family name Heimann in the conference pro ceedings of the ”Inter- national Symp osium on Dynamic Games and App lica tions”, W ro la w 2008. 1 the play ers who c ho ose to quit. If S = ∅ the pla yers r ecei ve no pa y off and the ga me con tin ues to the next stage. If S 6 = ∅ eac h pla y er n ∈ { 1 , . . . , N } receiv es th e pa yo ff r n S ∈ R and the game terminates. Definition 1.1 (Q uitting Game) A quitting game is a tuple G = ( N , ( r S ) ∅⊆ S ⊆N ) (1) wher e – N = { 1 , . . . , N } ⊂ N is a finite set of players, N ∈ N , – S ∈ P ( N ) denotes the quitting c o alition and – ( r S ) S ∈P ( N ) ∈ R N is a se quenc e of p ayoff-ve ctors to the players under the quitting c o ali- tion S with r ∅ = 0 ( 0 := (0 , . . . , 0) T ∈ R N ) and r S = ( r 1 S , . . . , r N S ) T . Remark 1.2 A quitting game is a sp e cial c ase of a (sto c ha stic) game, wher e tr ansition pr ob- abilities ar e even deterministic. F or c omp arison (cf. e.g. [4]): – The state sp ac e is give n by Z := { S | ∅ ⊆ S ⊆ N } = P ( N ) . – The action sp ac e is given by A := { 0 , 1 } N , wher e 0 stands for con tin ue and 1 for quit . We denote a S = ( a 1 S , . . . , a N S ) T as element of A with a n S := ( 0 for n ∈ N \ S 1 for n ∈ S ∀ n ∈ N , ∅ ⊆ S ⊆ N . (2) – The tr ansition law t : Z × A × Z → [0 , 1] is given by t ( z |∅ , a S ) := ( 1 for z = S 0 otherwise t ( z | ˜ S , a S ) := ( 1 for z = ˜ S 0 oth erwise (3) wher e z , S, ˜ S ∈ Z , ˜ S 6 = ∅ and a S ∈ A . – The p ayoff function is given by ˜ r : A → R N , a S 7→ ˜ r ( a S ) := r S . – Ther e is no disc ounting in this mo del. Example 1: A t yp ical wa y to describ e t w o- or three-pla y er quitting games is in a matrix. F or example let tw o p la ye rs b e given. Pla yer one is the so calle d ro w play er and pla yer t wo the column pla yer. Pla y er 2 con tin ue quit Pla y er 1 con tin ue quit  ( 1 , − 1 ) ( 1 , 1 ) ( − 2 , − 2 ) 2 Where  means, that th e pla y ers do es not r ece ive any pa y off and the game con tin ues to the next roun d. In this case the quitting game is giv en b y G =  { 1 , 2 } ,  r ∅ =  0 0  , r { 1 } =  1 − 1  , r { 2 } =  1 1  , r N =  − 2 − 2  . F or fu r ther analysis the term “strategy profile” is needed. Definition 1.3 (strat egy profile, strategy) L et G = ( N , ( r S ) ∅⊆ S ⊆N ) b e a given qu itting game. A se qu enc e of pr ob ability ve ctors π := ( p i ) i ∈ N with p i = ( p 1 i , . . . , p N i ) T ∈ [0 , 1] N is c al le d str ate gy pr ofile in the quitting game G for the players 1 , . . . , N . p n i stands for the pr ob ability that player n wil l play the action quit at stage i . The se quenc e π n := ( p n i ) i ∈ N is c al le d str ate gy for player n , n ∈ N . L et Π b e the set of al l str ate gy pr ofiles for the given q uitting game. Definition 1.4 (subgame profile) L et G = ( N , ( r S ) ∅⊆ S ⊆N ) b e a given quitting game and π = ( p i ) i ∈ N a str ate gy pr ofile i n G . F or e ach j ∈ N , π j := ( p i ) j ≤ i ∈ N denotes the sub game pr ofile induc e d by π in the qui tting g ame starting at time j . Definition 1.5 (pure, cyclic, stationary) L e t π = ( p i ) i ∈ N b e a str ate gy pr ofile in a q u it- ting game G . A str ate gy π n = ( p n i ) i ∈ N for player n is c al le d – pur e, if p n i ∈ { 0 , 1 } f or al l i ∈ N . – cyclic, if a k 0 ∈ N exists such tha t p n k = p n k + k 0 for every k ∈ N . – stationary, if p n k = p n 1 for al l k ∈ N . A str ate gy pr ofile π is c al le d pur e , if al l str ate gies π n , n ∈ N , ar e pur e. It is cyclic, if al l str ate gies ar e cyclic, and stationary, if al l str ate gies ar e stationary. Notation 1.6 L et π = ( p i ) i ∈ N b e a str ate gy pr ofile and ˜ π n = ( ˜ p n i ) i ∈ N an alterna tive str ate gy for pla yer n , n ∈ N . We d enote by π − n the str ate gy pr ofile for the play ers j ∈ N \ { n } and by ( π − n , ˜ π n ) an alternative str ate gy pr ofile for player n i n which the players j ∈ N \ { n } c arry on playing π − n , that me ans π − n :=           p 1 1 p 1 2 . . . . . . . . . p n − 1 1 p n − 1 2 . . . p n +1 1 p n +1 2 . . . . . . . . . p N 1 p N 2 . . .           and ( π − n , ˜ π n ) :=             p 1 1 p 1 2 . . . . . . . . . p n − 1 1 p n − 1 2 . . . ˜ p n 1 ˜ p n 2 . . . p n +1 1 p n +1 2 . . . . . . . . . p N 1 p N 2 . . .             . 3 1.1 The underlying sto cha stic pro cess Let G = ( N , ( r S ) ∅⊆ S ⊆N ) b e th e giv en quitting game, Z = { S | ∅ ⊆ S ⊆ N } the corresp onding state space and A = { 0 , 1 } N the corresp onding action sp ace (cf. remark 1. 2 ). S et Ω := ( Z × A ) ∞ and A := P ( Z ) ⊗ P ( A ) ⊗ P ( Z ) ⊗ P ( A ) ⊗ . . . . F urthermore and with ou t loss of generalit y let z = ∅ b e the initial stat e. If a strategy profile π = ( p i ) i ∈ N ∈ Π is giv en, a un ique probabilit y measure P π on (Ω , A ) and a stoc hastic pro cess ( X k , Y k ) k ∈ N with v alues in ( Z × A ) exist, where – X k ( ω ) = X k  ( z 1 , a 1 , z 2 , a 2 , . . . )  := z k ( X k denotes the ran d om state of the system at time k , k ∈ N , ω ∈ Ω), – Y k ( ω ) = Y k  ( z 1 , a 1 , z 2 , a 2 , . . . )  := a k ( Y k denotes the random actio n tak en at time k , k ∈ N , ω ∈ Ω), – H k := ( X 1 , Y 1 , . . . , X k ), that means H k ( ω ) = H k  ( z 1 , a 1 , z 2 , a 2 , . . . )  = ( z 1 , a 1 , z 2 , a 2 , . . . , z k ) ( H k describ es the random history at time k , k ∈ N , ω ∈ Ω) hold and P π is defined b y – P π ( X 1 = ∅ ) := 1, – P π ( X k +1 = z | H k = ( z 1 , a 1 , . . . , z k ) , Y k = a ) := t ( z | z k , a ) if P π ( H k = ( z 1 , a 1 , . . . , z k ) , Y k = a ) > 0 and – P π ( Y k = a | H k = ( z 1 , a 1 , . . . , z k )) := Q { n ∈N : a n =1 } p n k Q { m ∈N : a m =0 } (1 − p m k ) if P π ( H k = ( z 1 , a 1 , . . . , z k )) > 0 with z , z i ∈ Z for all i = 1 . . . , k and a, a i ∈ A for all i = 1 , . . . , k − 1. Equiv alent ly , P π can a lso b e d escrib ed as the unique probability measure on (Ω , A ) for w h ic h – P π ( X 1 = ∅ ) := 1 and – P π ( H k = ( z 1 , a 1 , z 2 , a 2 , . . . , z k )) := P π ( X 1 = z 1 ) k − 1 Q i =1 t ( z i +1 | z i , a i ) ·  Q { n ∈N : a n i =1 } p n i Q { m ∈N : a m i =0 } (1 − p m i )  , where z i ∈ Z for all i = 1 , . . . , k and a i ∈ A for all i = 1 , . . . , k − 1. 4 1.2 Exp ected pay offs and equilibria In this section the exp ected pa y off for a giv en quitting game will b e defined and additionally the terms N ash-e q u ilibria , ε -e quilibria and appr oximate e quilibria . In [7] the exp ected pay off for a quitting game G = ( N , ( r S ) ∅⊆ S ⊆N ) and the strategy p rofile π ∈ Π is defined with a stopping time τ : Ω → N ∪ { + ∞} wh ere τ ( ω ) := inf  k ∈ N : Y k ( ω ) ∈ A \ { (0 , . . . , 0) T }  concerning the fi ltrati on ( A k ) k ∈N with A k := σ { Y i : 1 ≤ i ≤ k } . Definition 1.7 (Exp ected pa y off ) L et G = ( N , ( r S ) ∅⊆ S ⊆N ) b e a qu itting game and π ∈ Π the chosen str ate gy pr ofile. The exp e cte d p ayoff of the game is given by γ ( π ) := E π ( ˜ r ( Y τ ) 1 { τ < ∞} ) with ˜ r fr om r emark 1.2, γ ( π ) = ( γ 1 ( π ) , . . . , γ N ( π )) T and E π as exp e cte d value with r esp e ct to the pr ob ability me asur e P π . With use of the definition of P π and ˜ r (0 ) = r ∅ = 0 one obtains γ ( π ) = E π ( ˜ r ( Y τ ) 1 τ < ∞ ) = X k ∈ N X a k ∈ A P π ( τ = k , Y k = a k ) · ˜ r ( a k ) = X k ∈ N  X a k ∈ A P π  H k − 1 = ( ∅ , 0 , ∅ , 0 , . . . , ∅ ) , Y k = a k  · ˜ r ( a k )  = X k ∈ N  k − 1 Y i =1 Y n ∈N (1 − p n i ) · X a k ∈ A ˜ r ( a k ) · Y { n ∈N : a n k =1 } p n k Y { m ∈N : a m k =0 } (1 − p m k )  and with r emark 1.2 follo ws γ ( π ) = X k ∈ N  k − 1 Y i =1 Y n ∈N (1 − p n i ) · X S ∈P ( N ) r S · Y n ∈ S p n k Y m ∈N \ S (1 − p m k )  . Definition 1.8 ( ε -equilibrium, Nash-equilibrium, approxim ate equilibria) L et G = ( N , ( r S ) ∅⊆ S ⊆N ) b e a quitting game. A str ate gy pr ofile π = ( p i ) i ∈ N is c al le d ε -e quilibrium ( ε ≥ 0 ) if f or every player n ∈ N and every str ate gy ˜ π n of player n γ n ( π ) ≥ γ n (( π − n , ˜ π n )) − ε (4) holds. The str ate gy pr ofile π = ( p i ) i ∈ N is c al le d Nash-e quilibrium or (0 − ) e quilibrium if π is an ε -e quilib riu m for ε = 0 . A game has got appr oximate e quilibria, if for al l ε > 0 an ε -e quilibrium exists. Definition 1.9 (subgame ε -equilibrium) L et G = ( N , ( r S ) ∅⊆ S ⊆N ) b e a quitting game. A str ate gy p r ofile π = ( p i ) i ∈ N is c al le d sub game ε -e quilibriu m ( ε ≥ 0 ) if for al l j ∈ N the sub game pr ofile π j is also an ε -e quilibrium in G . 5 2 One-step game The consideration of so calle d one-step games is an instrumen t for analyzing quitting games. These games are also k n o wn as one-stage ga mes ([5], p . 15) or as one-shot game ([7], p. 269). Definition 2.1 (O ne-step game) L et G = ( N , ( r S ) ∅⊆ S ⊆N ) b e a given quitting game. F or every v ∈ R N the tuple Γ v := ( G, v ) =  ( N , ( r S ) ∅⊆ S ⊆N  , v ) denotes the one-step game c orr esp onding to the quitting game G , wher e the players r e c e ive the p ayoff v if S = ∅ and r S otherwise ( ∅ 6 = S ⊆ N ). A one-step game has only one stage. Th e tr ansition law, the state and action sp ace are the same as the transition la w, the state and action space of the quitting game (cf. remark 1.2). F or the pa y off fun ctio n ˜ r v : A → R N a 7→ ˜ r v ( a ) := ( v if a = 0 r { n ∈N | a n =1 } otherwise. Definition 2.2 (Stra tegy profile, strat e gy in the one-step game) L et Γ v = ( G, v ) b e a give n one-step game. A ve ctor p = ( p 1 , . . . , p N ) T ∈ [0 , 1] N is c al le d str ate gy pr ofile for the one-step g ame Γ v , wher e p n stands for the pr ob ability that player n wil l play the action qu it . p n denotes the str ate gy for player n , n ∈ N , in the one-step g ame Γ v . Notation 2.3 L et p ∈ [0 , 1] N b e a str ate gy pr ofile for a one-step game Γ v and ˜ p n a str ate gy for player n . Similar to nota tion 1.6, p − n denotes the str ate gy pr ofile for the players j ∈ N \ { n } and ( p − n , ˜ p n ) an alternative str ate gy pr ofile for player n in which the players j ∈ N \ { n } c arry on playing p − n , that me ans p − n :=           p 1 . . . p n − 1 p n +1 . . . p N           and ( p − n , ˜ p n ) :=             p 1 . . . p n − 1 ˜ p n p n +1 . . . p N             . Let Γ v = ( G, v ) and p ∈ [0 , 1] N b e giv en. Without loss of generalit y the game starts in the state z = ∅ . Th e corresp onding probabilit y space ( ¯ Ω , ¯ A , P p ) and the sto c hastic pro cess ( ¯ X 1 , ¯ Y 1 , ¯ X 2 ) are defin ed b y – ¯ Ω := Z × A × Z , – ¯ A := P ( Z ) ⊗ P ( A ) ⊗ P ( Z ), – ¯ X i ( ω ) = ¯ X i (( z 1 , a, z 2 )) := z i , i = 1 , 2, 6 – ¯ Y 1 ( ω ) = ¯ Y 1 (( z 1 , a, z 2 )) := a , – P p ( ¯ X 1 = ∅ ) := 1, P p ( { ω } ) = P p (( z 1 , a, z 2 )) := P p ( ¯ X 1 = z 1 ) · t ( z 2 | z 1 , a ) Q { n ∈N : a n =1 } p n Q { m ∈N : a m =0 } (1 − p m ), with ω ∈ ¯ Ω, z 1 , z 2 ∈ Z , a ∈ A . The exp ected pa y off γ v for the one-step game Γ v under the strategy profile p ∈ [0 , 1] N is giv en b y γ v ( p ) := E p  ˜ r v ( Y 1 )  = X a S ∈ A P p ( Y 1 = a S ) · ˜ r v ( a S ) = P p ( Y 1 = a ∅ ) · v + X S ∈P ( N ) P p ( Y 1 = a S ) · r S where γ v ( p ) = ( γ 1 v ( p ) , . . . , γ N v ( p )) T and E p is the exp ected v alue with resp ect to the probabilit y measure P p . γ n v ( p ) is the exp ect ed pa yoff for p la ye r n ( n ∈ N ) in the one-step game Γ v under the strategy p rofile p . Notation 2.4 The function  : [0 , 1] N × P ( N ) → [0 , 1] , ( p, S ) 7→  ( p, S ) := Y n ∈ S p n Y m ∈N \ S (1 − p m ) , with p = ( p 1 , . . . , p N ) T , denotes the pr ob ability that a quitting c o alition S or – e quivalent to that – an action a S ∈ A is chosen under the ve ctor p . With this n ota tion for the exp ected pay off γ v under the strateg y profile p ∈ [0 , 1] N follo w s γ v ( p ) =  ( p, ∅ ) · v + X S ∈P ( N )  ( p, S ) · r S . Prop osition 2.5 L et Γ v b e a give n one-step game and p ∈ [0 , 1] N a str ate gy pr ofile in Γ v . Then for the exp e cte d p ayoff γ v ( p ) γ n v ( p ) ∈ [ − δ v , δ v ] holds for al l n ∈ N , wher e δ v := max  max n ∈N | v n | , max  | r n S |   S ∈ P ( N )  . (5) Pro of: F or all p ∈ [0 , 1] N and all n ∈ N γ n v ( p ) =  ( p, ∅ ) · v n + X S ∈P ( N ) \{∅}  ( p, S ) · r n S ≤  ( p, ∅ ) · | v n | + X S ∈P ( N ) \{∅}  ( p, S ) · | r n S | 7 holds. With δ v lik e in (5 ) γ n v ( p ) ≤  ( p, ∅ ) · δ v + X S ∈P ( N ) \{∅}  ( p, S ) · δ v = δ v ·   ( p, ∅ ) + X S ∈P ( N ) \{∅}  ( p, S )  follo w s on th e one hand and on the other γ n v ( p ) ≥ −  ( p, ∅ ) · | v n | − X S ∈P ( N ) \{∅}  ( p, S ) · | r n S | ≥ − δ v ·   ( p, ∅ ) + X S ∈P ( N ) \{∅}  ( p, S )  . With  ( p, ∅ ) + X S ∈P ( N ) \{∅}  ( p, S ) = 1 , γ n v ( p ) ∈ [ − δ v , δ v ] holds for all p ∈ [0 , 1] N and n ∈ N .  In order to sho w that the expected p a yo ff γ v ( p ) is linear in the strateg y p n of pla yer n for all n ∈ N the follo wing p rop ositio n is needed. Prop osition 2.6 F or al l p ∈ [0 , 1] N , al l S ∈ P ( N ) and al l i ∈ N  ( p, S ) = p i ·   ( p − i , 1) , S  + (1 − p i ) ·   ( p − i , 0) , S  holds. Pro of: Case 1: i ∈ S Because  ( p, S ) = Y n ∈ S p n Y n ∈N \ S (1 − p n ) = p i · Y n ∈ S \{ i } p n Y n ∈N \ S (1 − p n ) , with ( p − i , 1) i = 1 and p n = ( p − i , 1) n for all n ∈ N \ { i } , where ( p − i , 1) n denotes the n -th comp onen t of the alternativ e strategy pr ofi le f or pla yer i ,  ( p, S ) = p i · ( p − i , 1) i · Y n ∈ S \{ i } ( p − i , 1) n Y n ∈N \ S  1 − ( p − i , 1) n  = p i · Y n ∈ S ( p − i , 1) n Y n ∈N \ S  1 − ( p − i , 1) n  = p i ·   ( p − i , 1) , S  follo w s. Case 2: i ∈ N \ S 8 Because  ( p, S ) = (1 − p i ) · Y n ∈ S p n Y n ∈N \ ( S ∪{ i } ) (1 − p n ) , with ( p − i , 0) i = 0 and p n = ( p − i , 0) n for all n ∈ N \ { i }  ( p, S ) = (1 − p i ) ·  1 − ( p − i , 0) i  · Y n ∈ S ( p − i , 0) n Y n ∈N \ ( S ∪{ i } )  1 − ( p − i , 0) n  = (1 − p i ) · Y n ∈ S ( p − i , 0) n Y n ∈N \ S  1 − ( p − i , 0) n  = (1 − p i ) ·   ( p − i , 0) , S  follo w s. Because of   ( p − i , 1) , S  = 0 for i ∈ N \ S and   ( p − i , 0) , S  = 0 for i ∈ S , case 1 and case 2 imply the prop osition.  Prop osition 2.7 L et Γ v b e a given one-step game. Then for al l p ∈ [0 , 1] N and al l n ∈ N γ v ( p ) = γ v (( p − n , 0)) + p n ·  γ v (( p − n , 1)) − γ v (( p − n , 0))  holds, that me ans the e xp e cte d p ayoff γ v ( p ) is line ar in the str ate gy p n of pla yer n for al l p ∈ [0 , 1] N and al l n ∈ N . Pro of: γ v ( p ) =  ( p, ∅ ) · v + X S ∈P ( N )  ( p, S ) · r S With pr op osition 2.6 and   ( p − n , 1) , ∅  = 0 one obtains for all n ∈ N γ v ( p ) =  p n ·   ( p − n , 1) , ∅  + (1 − p n ) ·   ( p − n , 0) , ∅   · v + X S ∈P ( N )  p n ·   ( p − n , 1) , S  + (1 − p n ) ·   ( p − n , 0) , S   · r S = (1 − p n ) ·   ( p − n , 0) , ∅  · v + (1 − p n ) X S ∈P ( N )   ( p − n , 0) , S  · r S + p n X S ∈P ( N )   ( p − n , 1) , S  · r S F urthermore γ v (( p − n , 0)) =   ( p − n , 0) , ∅  · v + X S ∈P ( N )  (( p − n , 0) , S ) · r S 9 and γ v (( p − n , 1)) = X S ∈P ( N )  (( p − n , 1) , S ) · r S . That implies γ v ( p ) = (1 − p n ) · γ v (( p − n , 0)) + p n · γ v (( p − n , 1)) .  Conclusion 2.8 L et p ∈ [0 , 1] N b e a str ate gy pr ofile in the one-step game Γ v . The fol lowing e quations hold: 1. γ v (( p − n , 1)) = p i γ v  ( p − n , 1) − i , 1  + (1 − p i ) γ v  ( p − n , 1) − i , 0  2. γ v (( p − n , 0)) = p i γ v  ( p − n , 0) − i , 1  + (1 − p i ) γ v  ( p − n , 0) − i , 0  for al l i, n ∈ N , i 6 = n . Definition 2.9 ( ε -e quilibrium, Nash-e quilibrium, appr oximate e quilibria of the one-step game) L et Γ v b e a one-step game c orr esp onding to a quitting game G . The str ate g y pr ofile p ∈ [0 , 1] N is c al le d an ε -e quilibrium for ε ≥ 0 if ∀ n ∈ N ∀ ˜ p n ∈ [0 , 1] : γ n v ( p ) ≥ γ n v  ( p − n , ˜ p n )  − ε. If p is an ε -e quilibrium with ε = 0 , p i s also c al le d (Nash-)e quilibrium. A one-step game Γ v has got appr oximate e quilibria, if for al l ε > 0 an ε -e qu ilibrium in Γ v exists. Because of the linearit y of the exp ected pa yoff γ v ( p ) in the strategies p n ( n ∈ N ) it is s ufficien t to consider th e exp ected pa yoff only for pure strateg ies in order to fi nd out whether a giv en strategy profile in a one-step game is an equilibr ium or not, since the extreme v alues of γ v ( p ) is f or eac h single pla ye r attained in a b order p oint . T he follo wing example illustrates this fact. Example 2: Consider example 1 again. The corresp onding one-step game Γ v for a v ector v = ( v 1 , v 2 ) T ∈ R 2 is giv en b y Pla y er 2 con tin ue quit Pla y er 1 con tin ue quit ( v 1 , v 2 ) ( 1 , − 1 ) ( 1 , 1 ) ( − 2 , − 2 ) Consider four d ifferen t giv en v ’s: 10 1. v 1 =  2 2  : The strategy profile p =  0 . 1 0  is a 0.1-equilibrium with the exp ected p a yoff γ v 1 ( p ) = 0 . 9 ·  2 2  + 0 . 1 ·  1 − 1  =  1 . 9 1 . 7  . Because if pla ye r 1 c ho oses to pla y c ontinue while pla ye r 2 k eeps on p laying c ontinue he has got an exp ected pa yo ff of γ 1 v 1 ((0 , 0) T ) = 2, whic h is 0.1 b etter than his exp ected pa y off under p . If he p la ys quit his exp ected pay off w ould b e γ 1 v 1 ((1 , 0) T ) = 1. Otherwise if pla y er 2 c ho oses to pla y quit w hile pla y er 1 ke eps on pla ying quit with a probabilit y of 0.1, pla y er 2 gains a pa yoff of γ 2 v 1 ((0 . 1 , 1) T ) = 0 . 9 · 1 + 0 . 1 · ( − 2) = 0 . 7 . Pla y er 2 w ould ev en c h ange for the worse. 2. v 2 =  0 2  : The strategy pr ofile p =  1 0  is a Nash-equilibrium with the exp ected pa yoff γ v 2 ( p ) =  1 − 1  . Because if pla y er 1 c ho oses to p la y c ontinue while pla y er 2 k eeps on p la ying c ontinue he has got an exp ecte d pay off of γ 1 v 2 ((0 , 0) T ) = 0 and if play er 2 chooses to p lay quit w hile pla yer 1 k eeps on pla ying quit , pla yer 2 gains a pa yo ff of γ 2 v 2 ((1 , 1) T ) = − 2. Pla y er 2 w ould ev en c h ange for the worse, to o. 3. v 3 =  2 0  : Analogously to case 2, the strategy profile p =  0 1  is a Nash-equilibrium with the exp ected pa yo ff γ v 3 ( p ) =  1 1  . 4. v 4 =  0 0  : The strategy profile p 1 =  1 0  is a Nash-equilibriu m with the exp ected p a yoff γ v 4 ( p 1 ) =  1 − 1  . Analogously p 2 =  0 1  is also a Nash-equilibrium with the exp ected pa yo ff γ v 4 ( p 2 ) =  1 1  . Ob viously the c hoice of v is imp ortan t. This leads to the question which v ’s are exp edien t referring to finding an ( ε -)equilibrium in the corresp onding quitting game 1 . F or example: It 1 Simon th erefo re in tro duced in [5] the term fe asible for a v ector v ∈ R N : A vector v ∈ R N is feasible if it is in the conv ex hull of { r S |∅ 6 = S ⊂ N } ∪ { 0 } . 11 do es n ot mak e sense to c ho ose v like in the first case, b ecause in the corresp ond ing quitting game the exp ected pa yoffs are limited by one for eac h pla ye r. F urthermore p r op osition 2.7 m oti v ates the d efinition of the b est r eply , but b efore stating the definition it is necessary to introd uce th e mapping supp . supp : [0 , 1] → P ( { 0 , 1 } ) denotes the actions that are pla y ed with p ositiv e p robabilit y under ˜ p , that means supp ( ˜ p ) :=      { 0 } for ˜ p = 0 { 0 , 1 } for ˜ p ∈ (0 , 1) { 1 } for ˜ p = 1 . Definition 2.10 (b est reply , p erfect) L et Γ v b e a given one-step game and p ∈ [0 , 1] N a str ate gy pr ofile in Γ v . An action b ∈ { 0 , 1 } of player n is an ε -b est r eply for p − n if γ n v (( p − n , b )) ≥ max ˜ b ∈{ 0 , 1 } γ n v (( p − n , ˜ b )) − ε n ∈ N . A str ate g y pr ofile p ∈ [0 , 1] N in Γ v is c al le d ε -p erfe ct 2 , if for every player n ∈ N , every action b ∈ supp ( p n ) is an ε -b est r eply for p − n . Remark 2.11 L et Γ v b e the giv en one-step ga me and ε ≥ 0 . The se c ond p art o f the definition ab ove is e qui v alent to the fol lowing: The str ate gy pr ofile p for the one-step game Γ v is ε -p erfe ct, if ∀ n ∈ N :      γ n v (( p − n , 1)) − γ n v (( p − n , 0)) ≤ ε for p n = 0 γ n v (( p − n , 1)) − γ n v (( p − n , 0)) ∈ [ − ε, ε ] for p n ∈ (0 , 1) γ n v (( p − n , 1)) − γ n v (( p − n , 0)) ≥ − ε for p n = 1 . No w lo ok at Example 1 again: Example 3: Consider the one-step game Γ v with v =  0 2  . 1. p =  1 0  : p is a Nash-equilibrum in Γ v , but is p also (0-)p erfect? It holds that p 1 = 1 : γ 1 v (( p − 1 , 1)) − γ 1 v (( p − 1 , 0)) = 1 − 0 = 1 ≥ 0 and p 2 = 0 : γ 2 v (( p − 2 , 1)) − γ 2 v (( p − 2 , 0)) = − 2 − ( − 1) = − 1 ≤ 0 . So with remark 2.11 p is p erf ect . 2 Solan an Vieille used in [7] in stead of the t erm “ ε -p erfect” the term “p erfect ε -equilibrium”. This form u- lation is confusing with regard t o t heorem 2.12 . The here used phrase is more accurate. 12 2. p =  1 0 . 1  : p is a 0 . 1 -equilibrium, b ecause: γ v ( p ) = 0 . 1 ·  − 2 − 2  + 0 . 9 ·  1 − 1  =  0 . 7 − 1 . 1  . If pla yer one c h o oses to p la y c ontinue , while pla yer t w o keeps on pla ying p 2 , he gains a pa y off of 0.1, which is less than b efore. If pla yer t wo c ho oses to pla y c ontinue with certa int y , while pla yer o ne k eeps on pla ying quit , h e an ticipates a pa yo ff of − 1, wh ich is 0 . 1 more than b efore. But is p also 0 . 1-p erfect? The answer is no, b ecause p 2 ∈ (0 . 1) : γ 2 v (( p − 2 , 1)) − γ 2 v (( p − 2 , 0)) = − 2 − ( − 1) = − 1 / ∈ [ − 0 . 1 , 0 . 1] . Whic h relation exists b et wee n ( ε -)equilibria strategy profiles and ( ε -) p erfect strategy profiles ( ε ≥ 0)? Theorem 2.12 L et Γ v b e a given one-step game and ε ≥ 0 . Then the fol lowing pr op ositions hold: 1. p ∈ [0 , 1] N is ε -p erfe ct for Γ v = ⇒ p is an ε -e quilibrium in Γ v ; 2. p ∈ [0 , 1] N is an ε -e quilibrium in Γ v = ⇒ p is εξ p -p erfe ct for Γ v , wher e ξ p := max n ∈N ξ n p and ξ n p := ( max( 1 p n , 1 1 − p n ) for p n ∈ (0 , 1) 1 for p n ∈ { 0 , 1 } . Pro of: 1.: Let p ∈ [0 , 1] N b e ε -p erfect for Γ v . It is to s ho w that p is also a n ε -equilibrium in Γ v , that means γ n v ( p ) ≥ max ˜ p ∈ [0 , 1] γ n v  ( p − n , ˜ p )  − ε for all n ∈ N . Because of the linearit y of γ n v ( p ) with r esp ect to p n (cf. prop osition 2.7) it is sufficient to sho w that γ n v ( p ) ≥ max ˜ p ∈{ 0 , 1 } γ n v  ( p − n , ˜ p )  − ε (6) for all n ∈ N . Since p is ε -p er f ect in Γ v , the inequ alit y (6 ) follo w s immediately f or p n = 0 and p n = 1. F or p n ∈ (0 , 1) it h olds either 13 (a) γ n v  ( p − n , 1)  ≥ γ n v ( p ) ≥ γ n v  ( p − n , 0)  or (b) γ n v  ( p − n , 0)  > γ n v ( p ) > γ n v  ( p − n , 1)  . Case (a): With p ε -p erfect in Γ v and remark 2.11 , it holds γ n v ( p ) ≥ γ n v  ( p − n , 0)  ≥ γ n v  ( p − n , 1)  − ε. Because γ n v  ( p − n , 1)  = max ˜ p ∈ [0 , 1] γ n v  ( p − n , ˜ p )  γ n v ( p ) ≥ max ˜ p ∈ [0 , 1] γ n v  ( p − n , ˜ p )  − ε follo w s. Case (b): Analogo usly to case (a) with p ε -p erfect in Γ v and remark 2.11 γ n v ( p ) > γ n v  ( p − n , 1)  ≥ γ n v  ( p − n , 0)  − ε = max ˜ p ∈ [0 , 1] γ n v  ( p − n , ˜ p )  − ε follo w s. So for b oth cases (6) holds. 2.: Let p b e an ε -equilibrium, that means for all n ∈ N and for all ˜ p ∈ [0 , 1] γ n v ( p ) ≥ γ n v  ( p − n , ˜ p )  − ε (7) holds. That imp lies γ n v ( p ) ≥ γ n v  ( p − n , 1)  − ε (8) for p n = 0 and γ n v ( p ) ≥ γ n v  ( p − n , 0)  − ε (9) for p n = 1. Consider now p n ∈ (0 , 1). F or all ˜ p ∈ [0 , 1] γ n v ( p ) = p n · γ n v  ( p − n , 1)  + (1 − p n ) · γ n v  ( p − n , 0)  ≥ γ n v  ( p − n , ˜ p )  − ε. (10) holds (c.f. prop osition 2.7). F or ˜ p = 1, with (10) (1 − p n ) · γ n v  ( p − n , 0)  − (1 − p n ) · γ n v  ( p − n , 1)  ≥ − ε follo w s and consequently γ n v  ( p − n , 1)  − γ n v  ( p − n , 0)  ≤ ε 1 − p n . F or ˜ p = 0, with (10) p n · γ n v  ( p − n , 1)  − p n · γ n v  ( p − n , 0)  ≥ − ε 14 follo w s and therefore γ n v  ( p − n , 1)  − γ n v  ( p − n , 0)  ≥ − ε p n . That implies γ n v  ( p − n , 1)  − γ n v  ( p − n , 0)  ∈  − ε p n , ε 1 − p n  ∈  − εξ n p , εξ n p  (11) where ξ n p := max  1 1 − p n , 1 p n  . Denote M ( p ) := { n ∈ N | p n ∈ (0 , 1) } and ξ p :=    max n ∈ M ( p ) ξ n p if M ( p ) 6 = ∅ 1 otherwise . With (8), (9) and (11) ∀ n ∈ N :      γ n v (( p − n , 1)) − γ n v (( p − n , 0)) ≤ ε for p n = 0 γ n v (( p − n , 1)) − γ n v (( p − n , 0)) ∈ [ − εξ p , εξ p ] for p n ∈ (0 , 1) γ n v (( p − n , 1)) − γ n v (( p − n , 0)) ≥ − ε for p n = 1 follo w s. With remark 2.1 1 and ξ p ≥ 1, p is εξ p -p erfect in Γ v .  Remark 2.13 With (11) even ∀ n ∈ N :        γ n v (( p − n , 1)) − γ n v (( p − n , 0)) ≤ ε for p n = 0 γ n v (( p − n , 1)) − γ n v (( p − n , 0)) ∈ h − ε p n , ε 1 − p n i for p n ∈ (0 , 1) γ n v (( p − n , 1)) − γ n v (( p − n , 0)) ≥ − ε for p n = 1 holds. Conclusion 2.14 L et Γ v b e a given one-step game. p ∈ [0 , 1] N is (0 - ) p erfe ct f or Γ v ⇐ ⇒ p ∈ [0 , 1] N is a Nash-e quilibrium in Γ v Conclusion 2.15 L et Γ v b e a given one-step game. p ∈ { 0 , 1 } N is ε -p erfe ct for Γ v ⇐ ⇒ p ∈ { 0 , 1 } N is an ε -e quilibrium in Γ v 3 Equilibria in Quitting Games This section presents an imp orted result referring to equilibr ia in qu itting games. It was pro ve d b y Solan and Vieille in [7], they show ed th at a cyclic ε -e quilibr ium ( ε > 0) un der some assumptions on th e pa yoff fun ctio n exists. 15 3.1 Preview This section studies the influ ence of a v ariation in one comp onen t of the strateg y profile p ∈ [0 , 1] N for a give n quitting game Γ v . Th erefore define ˆ p ∈ [0 , 1] N as follo ws ˆ p := ˆ p m,λ ( p ) :=  p − m , (1 − λ ) p m + λ  , (12) where p ∈ [0 , 1] N , λ ∈ [0 , 1] and m ∈ N . That means, ˆ p m is a con v ex com bin ation of p m and t he pure strate gy 1, whic h acc ords to the action quit . F or λ = 0 one obtains ˆ p = p and for λ = 1 ˆ p = ( p − m , 1). Theorem 3.1 L et Γ v b e a given one-step game, λ ∈ [0 , 1] , p ∈ [0 , 1] N and m ∈ N an arbitr ary but fixe d chosen player. Then the fol lowing hold: 1.  ( ˆ p, ∅ ) = (1 − λ )  ( p, ∅ ) That me ans, the pr ob ability that al l players play con tinue under ˆ p is for the λ -fold smal ler of the contin u e -pr ob ability under p . 2. γ v ( ˆ p ) = (1 − λ ) · γ v ( p ) + λ · γ v  ( p − m , 1)  3. k γ v ( ˆ p ) − γ v ( p ) k ≤ λ · ( r max + δ v ) wher e r max := max {| r n S |   n ∈ N , S ∈ P ( S ) } and δ v = max { max n ∈N | v n | , r max } 3 4. If p ∈ [0 , 1] N is η -p erfe ct in Γ v ( η ≥ 0 ) and if p m ∈ (0 , 1] for the given player m ∈ N holds, then ˆ p = ˆ p m,λ is ˜ η -p e rfe ct in Γ v , with ˜ η := max(2 λr max + (1 − λ ) η, η ) . Pro of: T o 1.: The defin ition of ˆ p (c.f. (12) ) implies  ( ˆ p, ∅ ) = Y n ∈N (1 − ˆ p n ) =  1 − (1 − λ ) p m − λ  · Y n ∈N \{ m } (1 − p n ) = (1 − λ ) · Y n ∈N (1 − p n ) = (1 − λ ) ·  ( p, ∅ ) . T o 2.: With pr oposition 2.7 an d the definition of ˆ p γ v ( ˆ p ) = ˆ p m · γ v  ( ˆ p − m , 1)  + (1 − ˆ p m ) · γ v  ( ˆ p − m , 0)  =  (1 − λ ) p m + λ  · γ v  ( p − m , 1)  + (1 − λ )(1 − p m ) · γ v  ( p − m , 0)  = (1 − λ )  p m · γ v  ( p − m , 1)  + (1 − p m ) · γ v  ( p − m , 0)   + λ · γ v  ( p − m , 1)  = (1 − λ ) · γ v ( p ) + λ · γ v  ( p − m , 1)  (13) holds. T o 3.: Under use of (13) one obtains 3 k · k denotes th e maximum norm, that means k y k := max i ∈ N | y i | for all y = ( y 1 , . . . , y N ) T ∈ R N . 16 k γ v ( ˆ p ) − γ v ( p ) k =   (1 − λ ) · γ v ( p ) + λ · γ v  ( p − m , 1)  − γ v ( p )   =   λ · γ v  ( p − m , 1)  − λγ v ( p )   = λ ·   γ v  ( p − m , 1)  − γ v ( p )   ≤ λ ·    γ v  ( p − m , 1)    +   γ v ( p )    Because play er m pla ys qu it with certai nt y in the alte rn ati ve strategy profile ( p − m , 1) γ n v  ( p − m , 1)  = X S ∈P ( N )   ( p − m , 1) , S  · r n S ∈ [ − r max , r max ] follo w s for all n ∈ N with r max = max {| r n S |   n ∈ N , S ∈ P ( S ) } . = ⇒ k γ v ( ˆ p ) − γ v ( p ) k ≤ λ ·  r max + δ v  where δ v = max { max n ∈N | v n | , r max } . T o 4.: F or λ = 0 and as w ell as for p m = 1, ˆ p = p follo ws and therefore ˆ p is η -p erfect in Γ v in that case. F or λ ∈ (0 , 1] and p m ∈ (0 , 1) it is to sho w, that ∀ n ∈ N :      γ n v (( ˆ p − n , 1)) − γ n v (( ˆ p − n , 0)) ≤ ˜ η for ˆ p n = 0 γ n v (( ˆ p − n , 1)) − γ n v (( ˆ p − n , 0)) ∈ [ − ˜ η , ˜ η ] f or ˆ p n ∈ (0 , 1) γ n v (( ˆ p − n , 1)) − γ n v (( ˆ p − n , 0)) ≥ − ˜ η for ˆ p n = 1 (14) holds with ˜ η = max(2 λr max + (1 − λ ) η , η ). Case 1: Consider play er m . With p η -p erfect and p m ∈ (0 , 1) γ v  ( ˆ p − m , 1)  − γ v  ( ˆ p − m , 0)  = γ v  ( p − m , 1)  − γ v  ( p − m , 0)  ∈ [ − η , η ] follo w s immediately and therefore the second inequalit y from (14) for ˆ p m ∈ (0 , 1) r esp ectiv ely the last in equalit y of (14) for ˆ p = 1. Case 2: Consider play er n ∈ N \ { m } . With the defi nition of ˆ p for all i ∈ N and b ∈ [0 , 1] ( ˆ p − n , b ) i = ( (1 − λ ) · ( p − n , b ) i + λ for i = m ( p − n , b ) i for i ∈ N \ { m } follo w s. Under use of this and equation (13) one obtains γ n v  ( ˆ p − n , b )  = (1 − λ ) · γ n v  ( p − n , b )  + λ · γ n v  ( p − n , b ) − m , 1  . This implies γ n v  ( ˆ p − n , 1)  − γ n v  ( ˆ p − n , 0)  = (1 − λ ) · γ n v  ( p − n , 1)  + λ · γ n v  ( p − n , 1) − m , 1  − (1 − λ ) · γ n v  ( p − n , 0)  − λ · γ n v  ( p − n , 0) − m , 1  = (1 − λ ) ·  γ n v  ( p − n , 1)  − γ n v  ( p − n , 0)   + λ  γ n v  ( p − n , 1) − m , 1  − γ n v  ( p − n , 0) − m , 1   . (15) 17 (a) Consider pla ye r n ∈ N \ { m } with p n = 0. p n = 0 ∧ p η -p erfect = ⇒ γ n v  ( p − n , 1)  − γ n v  ( p − n , 0)  ≤ η (16) With use of (15) and (16) γ n v  ( ˆ p − n , 1)  − γ n v  ( ˆ p − n , 0)  ≤ (1 − λ ) · η + λ ·  γ n v  ( p − n , 1) − m , 1  − γ n v  ( p − n , 0) − m , 1   ≤ (1 − λ ) · η + λ ·    γ n v  ( p − n , 1) − m , 1    +   γ n v  ( p − n , 0) − m , 1     ≤ (1 − λ ) · η + λ ( r max + r max ) = 2 λr max + (1 − λ ) · η follo w s. (b) Consider pla y er n ∈ N \ { m } with p n ∈ (0 , 1). p n ∈ (0 , 1) ∧ p η -p erfect = ⇒ γ n v  ( p − n , 1)  − γ n v  ( p − n , 0)  ∈ [ − η , η ] (17) With this analogously to case 2(a) it follo ws immediately th at γ n v  ( ˆ p − n , 1)  − γ n v  ( ˆ p − n , 0)  ≤ 2 λr max + (1 − λ ) · η . Under us age of (17) and equation (15) one obtains γ n v  ( ˆ p − n , 1)  − γ n v  ( ˆ p − n , 0)  ≥ − (1 − λ ) · η − λ ·    γ n v  ( p − n , 1) − m , 1    +   γ n v  ( p − n , 0) − m , 1     ≥ − 2 λr max − (1 − λ ) · η (1 8) and therefore γ n v  ( ˆ p − n , 1)  − γ n v  ( ˆ p − n , 0)  ∈ h −  2 λr max + (1 − λ ) · η  , 2 λr max + (1 − λ ) · η i . (c) Consider pla yer n ∈ N \ { m } w ith p n = 1. p n = 1 ∧ p η -p erfect = ⇒ γ n v  ( p − n , 1)  − γ n v  ( p − n , 0)  ≥ − η (19) With equation (19), (15) and inequalit y (18) γ n v  ( ˆ p − n , 1)  − γ n v  ( ˆ p − n , 0)  ≥ − (2 λr max + (1 − λ ) · η ) follo w s.  18 Remark 3.2 (T o th e or em 3.1 3.) 1. L et Γ v b e a gi ven one-step g ame with v ∈ [ − 2 r max , 2 r max ] , p ∈ [0 , 1] N , wher e p m ∈ (0 , 1) N for at le ast one player m ∈ N , a str ate gy pr ofile in Γ v and λ ∈ (0 , 1) . Solan and Vieil le state in [7] the fol lowing estimation: k γ v ( ˆ p ) − γ v ( p ) k ≤ 2 λr max , (20) with ˆ p and r max like b efor e. Counter-example: Consider the fol lowing one-step game Γ v with v =  1 2  . Player 2 c ontinue quit Player 1 c ontinue quit ( 1 , 2) ( 1 , − 1 ) ( 0 , 1 ) ( − 1 , − 0 . 5 ) = ⇒ r max = max {| r n S |   n ∈ N , S ∈ P ( N ) } = 1 and m ax n ∈N | v n | = 2 = ⇒ δ v = 2 Obviously p =  0 0  is one (and the only) e quilibrium in Γ v with the exp e cte d p ayoff γ v ( p ) = v . L et λ = 0 . 1 b e g i ven. It holds that  ( p, ∅ ) = 1 > 1 − λ = 0 . 9 . F urthermor e let ˆ p λ, 1 b e define d like b efor e , that me ans ˆ p λ, 1 = ˆ p =  p 1 + λ (1 − p 1 ) p 2  =  0 + 0 . 1 · 1 0  =  0 . 1 0  . It holds tha t γ v ( ˆ p ) = 0 . 9 ·  1 2  + 0 . 1  1 − 1  =  1 1 . 7  . F r om this k γ v ( ˆ p ) − γ v ( p ) k = k  1 1 . 7  −  1 2  k = 0 . 3 = λ ( r max + δ v ) > 2 λr max = 2 · 0 . 1 · 1 = 0 . 2 fol lows. So estimation (20) do es not hold. F urthermor e the c ounter-example shows that the estimation in the or em 3.1 3. is even the b est estimation. 2. Interpr etation: L et Γ v b e a given one-step game, λ ∈ [0 , 1] and p ∈ [0 , 1] N with p m ∈ (0 , 1) for at le ast one player m ∈ N . If the cont inue pr ob ability of one player m ( m ∈ N ) is de cr e ase d by the λ -fold, then the exp e cte d p ayoff of the players changes maximal at λ ( r max + δ v ) for a c omp onent. 19 Remark 3.3 (T o theorem 3.1 4.) Assuming tha t p m = 0 in the or em 3.1 4., ˆ p m = λ ∈ (0 , 1) fol lows. In or der to pr ove that ˆ p is ˜ η - p erfe ct it is to show that γ m v (( ˆ p − m , 1)) − γ m v (( ˆ p − m , 0)) ∈ [ − ˜ η , ˜ η ] But with p η -p erfe ct only γ v  ( ˆ p − m , 1)  − γ v  ( ˆ p − m , 0)  = γ v  ( p − m , 1)  − γ v  ( p − m , 0)  ≤ η ≤ ˜ η fol lows. F or the other dir e ction hold s γ v  ( ˆ p − m , 1)  − γ v  ( ˆ p − m , 0)  = γ v  ( p − m , 1)  − γ v  ( p − m , 0)  ≥ −   γ v  ( p − m , 1)    −   γ v  ( p − m , 0)    ≥ − r max − δ v , however this hold s f or al l p ∈ [0 , 1] N in Γ v . But the pr o of that ∀ n ∈ N \ { m } :      γ n v (( ˆ p − n , 1)) − γ n v (( ˆ p − n , 0)) ≤ ˜ η for ˆ p n = 0 γ n v (( ˆ p − n , 1)) − γ n v (( ˆ p − n , 0)) ∈ [ − ˜ η , ˜ η ] for ˆ p n ∈ (0 , 1) γ n v (( ˆ p − n , 1)) − γ n v (( ˆ p − n , 0)) ≥ − ˜ η for ˆ p n = 1 wil l r emain u naffe cte d fr om this c ase. Conclusion 3.4 With the or e m 3.1 4. i t fol lows imme diately that if p ∈ [0 , 1] N with p m ∈ (0 , 1) , for at le ast one m ∈ N , is (0 − ) p erfe ct in Γ v (and ther efor e an e quilibrium in Γ v ), ˆ p is 2 λr max -p erfe ct in Γ v . The follo w ing example shows, that the estimation in theorem 3.1 4. is ev en th e b est app ro x - imation. Example 4: Consider the follo w ing one-step game Γ v with v = ( 9 10 , 10 9 ) T ∈ R 2 giv en b y Pla y er 2 con tin ue quit Pla y er 1 con tin ue quit  9 / 10 , 10 / 9   1 , − 1   1 / 2 , 1   − 1 , 1  = ⇒ r max = 1 Let p =  0 . 1 0  b e the giv en strategy profile in Γ v . It h olds γ 1 v  ( p − 1 , 1)  − γ 1 v  ( p − 1 , 0)  = 1 − v 1 = 1 − 9 10 = 0 . 1 ∈ [ − 0 . 1 , 0 . 1] and γ 2 v  ( p − 2 , 1)  − γ 2 v  ( p − 2 , 0)  = 0 . 1 · 1 + 0 . 9 · 1 − 0 . 1 · ( − 1) − 0 . 9 · 10 9 = 0 . 1 ≤ 0 . 1 . Therefore p is η -p erfect in Γ v with η = 0 . 1. 20 Let λ = 0 . 2 b e giv en. = ⇒ ˆ p λ, 1 = ˆ p =  (1 − λ ) · p 1 + λ p 2  =  0 . 8 · 0 . 1 + 0 . 2 0  =  0 . 28 0  F or ˆ p the follo wing hold γ 1 v  ( ˆ p − 1 , 1)  − γ 1 v  ( ˆ p − 1 , 0)  = 1 − 9 10 = 0 . 1 ∈ [ − 0 . 1 , 0 . 1] and γ 2 v  ( ˆ p − 2 , 1)  − γ 2 v  ( ˆ p − 2 , 0)  = 0 . 28 · 1 + 0 . 72 · 1 − 0 . 28 · ( − 1) − 0 . 72 · 10 9 = 0 . 48 = (1 − λ ) η + 2 λr max = 0 . 8 · 0 . 1 + 2 · 0 . 2 · 1 = 0 . 48 . So ˆ p is only 0.48- p erfect in Γ v and the estimat ion in theorem 3.1 4. holds. 3.2 Equilibria under some assumptions on the pa y off function This sec tion shows wh ic h imp ortance on e-step games ha v e, referring to the detection of equi- libria in quitting games. Firstly an imp ortant theorem from Solan and Vieille, stat ed in [7] is quoted. The pro of of this theorem is divid ed in to three p arts, r epresen ted b y the prop ositions 3.6, 3.8 and 3.9. Secondly the prop ositio n 3.6 is p ro ved at length b y using th e n o w kno wn results ab out one-step games and their strategy profiles. Theorem 3.5 L et b e ε > 0 . Every q uitting game G that satisfies the fol lowing has a c yclic sub game ε -e q uilibrium. 1. r n { n } = 1 for every n ∈ N ; 2. r n S ≤ 1 for every n ∈ N and every S such tha t n ∈ S . Before quoting th e ab o ve men tioned p rop ositio ns another notation is n eeded. Let ˜ V b e a subset of R N and ε ∈ (0 , 1) b e giv en. ψ ε denotes a corresp ondence 4 from ˜ V int o ˜ V , w h ere ψ ε ( v ) := ψ ε, ˜ V ( v ) :=  γ v ( p )   γ v ( p ) ∈ ˜ V , p ∈ [0 , 1] N , p 2 εr max -p erfect ,  ( p, ∅ ) ≤ 1 − ε  . Prop osition 3.6 L et ε ∈ (0 , 1) b e given. Define V :=  ˜ v ∈ [ − 2 r max , 2 r max ] N   ∃ n ∈ N : ˜ v n ≤ 1  . Assume that 1. r n { n } = 1 for every n ∈ N 2. for every v ∈ V an e quilibrium p in Γ v exists, such tha t ei ther (a) p = (0 , 0 , . . . , 0) T (that me ans al l players cho ose con tinue ) or 4 Let K and L b e sets. A correspond ence J : K ։ L is a subset J of K × L and one defines for all k ∈ K : J ( k ) := { l | ( k , l ) ∈ J } . It is not assumed a priori th at J ( k ) 6 = ∅ for all or any p articular k ∈ K . 21 (b) p 6 = (0 , 0 , . . . , 0) T and γ n v ( p ) ≤ 1 hold for some n ∈ N with p n > 0 . Then ψ ε ( v ) 6 = ∅ for al l v ∈ V . Remark 3.7 1. L et G b e a quitting game, which satisfies the assumptions of the or e m 3.5, and Γ v a c orr esp onding one-step game to G with v ∈ V . Then a str ate gy pr ofile p ∈ [0 , 1] N exists, which satisfies the assump tion 2 of pr op osition 3.6. Pr o of: That every one-step game has got an e qui librium was shown in [2]. The pr o of uses Kakutani’s fixe d p oint the or em. F urthermor e either p = (0 , . . . , 0) T or p 6 = (0 , . . . , 0) T holds. F or the se c ond c ase it is to show, that ther e a player m with p m > 0 and γ m v ( p ) ≤ 1 exists. Be c ause of p 6 = (0 , . . . , 0) T at le ast one player m ∈ N exists with p m > 0 . Consider the c ase that p m = 1 . Then with the assumpt ion 2 of the or em 3.5 γ m v ( p ) = X S ∈P ( N )  ( p, S ) r m S = X ∅6 = S ⊆N \{ m }  ( p, S ∪ { m } ) r m S ∪{ m } ≤ X ∅6 = S ⊆N \{ m }  ( p, S ∪ { m } ) · 1 ≤ 1 fol lows. Consider the c ase p m ∈ (0 , 1) . Be c ause p is an e quilib riu m in Γ v , p is also (0-)p erfe ct in Γ v . This implies γ m v ( p − m , 1) − γ m v ( p − m , 0) = 0 . Analo gously to the c ase ab ove it fol lows that γ m v ( p − m , 1) ≤ 1 and with use of the line arity of γ v ( p ) in p m one obtains γ m v ( p ) = γ m v ( p − m , 1) ≤ 1 . (21)  2. The assumption s of pr op osition 3.6 ar e b asic al ly ther e to al low the in r emark 3.3 men- tione d estimation b elow for p = (0 , . . . , 0) r esp e ctively to ensur e that p m ∈ (0 , 1] in the c ase p 6 = (0 , . . . , 0) T . Prop osition 3.8 L et ε ∈ (0 , 1) b e gi ven. If a c omp act set V exists such that ψ ε ( v ) 6 = ∅ for al l v ∈ V , then a cyclic pr ofile π = ( p i ) i ∈ N in G exists, such that for every i ∈ N : 1. π i = ( p j ) i ≤ j ∈ N is terminating 5 and 2. p i is (2 r max + 2) ε -p erfe c t in γ γ ( π i +1 ) . 5 Let G = ( N , ( r S ) ∅⊆ S ⊆N ) b e a given quitting game and π ∈ Π th e chos en strategy profile. If P π ( τ < + ∞ ) = 1 the game G is called terminating. 22 Prop osition 3.9 L et π = ( p i ) i ∈ N b e a str ate gy pr ofile in G . Assume that the fol lowing pr op erties hold for ev e ry i ∈ N : 1. π i = ( p j ) i ≤ j ∈ N is terminating and 2. p i is ε -p erfe ct in γ γ ( π i +1 ) . Then either π is a sub game ε 1 6 -e qu i librium, or ther e is a stationary ε 1 6 -e qu i librium. 3.2.1 Pro of of Prop osition 3.6 Pro of: Let v ∈ V and ε ∈ (0 , 1) b e arb itrary b ut fi x. The aim is to construct a ˆ p ∈ [0 , 1] N with γ v ( ˆ p ) ∈ ψ ε ( v ). It holds that ψ ε ( v ) 6 = ∅ , if a strateg y profile p ∈ [0 , 1] N in Γ v exists, such that (i) γ v ( p ) ∈ V =  ˜ v ∈ [ − 2 r max , 2 r max ] N   ∃ n ∈ N : ˜ v n ≤ 1  (ii) p is 2 εr max -p erfect in Γ v (iii)  ( p, ∅ ) ≤ 1 − ε . No w let p b e an equilibrium in Γ v that satisfies the assumptions of the prop ositio n 6 . T o (i): If p = (0 , . . . , 0) T , then γ v ( p ) = v ∈ V h olds . In the other case ( p 6 = (0 , . . . , 0) T ) the prop osition p ostulated that a p la ye r m ∈ N exists such that γ m v ( p ) ≤ 1. F ur thermore with the definition of V , p rop ositio n 2.5 and δ v = max  max n ∈N | v n | , r max  ≤ max  max v ∈ V ,n ∈N | v n | , r max  = 2 r max γ v ( p ) ∈ [ − 2 r max , 2 r max ] N holds. That imp lies γ v ( p ) ∈ V . T o (ii): Conclusion 2.14 implies that p is ev en 0-p erfect in Γ v . T o (iii): F or p = (0 , . . . , 0) T ,  ( p, ∅ ) = 1  1 − ε holds and for p 6 = (0 , . . . , 0) T ,  ( p, ∅ ) < 1 but not necessarily  ( p, ∅ ) ≤ 1 − ε . So γ v ( p ) is n ot necessarily an element of ψ ε ( v ). Based on th e giv en str ate gy profile p , a n ew p r ofile ˆ p ∈ [0 , 1] N lik e in section 3.1 for the one-step game Γ v will b e constructed such that  ( ˆ p, ∅ ) ≤ 1 − ε holds. Afterwa rd s it will b e sho wn that this profile ˆ p satisfies th e conditions (i) and (ii), stated at the b eginning of this pro of, as w ell. First to the construction of ˆ p : Fix a play er m w ith v m = 1 if p = (0 , . . . , 0) T or with p m > 0 and γ m v ( p ) ≤ 1 otherwise 7 and set ˆ p lik e in (12), that m eans ˆ p n = ( (1 − ε ) · p n + ε for n = m p n for n 6 = m . 6 Such a probability p ∈ [0 , 1] exists, c.f. remark 3.7. 7 Let p = (0 , . . . , 0) T b e the given equilibrium in Γ v . Since v ∈ V , a play er m ∈ N with v m ≤ 1 ex ists. Because p is an equilibrium in Γ v , v m = 1 fo llow s. Assume that v m < 1, then play er m could change for t he b etter, if he chooses to play quit w ith certain ty , hence r m { m } = 1 (c.f . assumption 1. of prop osi tion 3.6). This is a contra diction to p is an equilibrium. 23 Theorem 3.1 1. implies  ( ˆ p, ∅ ) = (1 − ε ) ·  ( p, ∅ ) ≤ 1 − ε. T o (i): It will b e sh o wn that γ v ( ˆ p ) ∈ V =  ˜ v ∈ [ − 2 r max , 2 r max ] N   ∃ n ∈ N : ˜ v n ≤ 1  . With prop osition 2.5 and δ v ≤ 2 r max , γ v ( ˆ p ) ∈ [ − 2 r max , 2 r max ] N follo w s. Cons id er the c hosen pla y er m ∈ N . Because p is an equilibrium in Γ v γ m v ( p ) ≥ γ m v  ( p − m , ˆ p m )  = γ m v ( ˆ p ) holds and with the sp ecial c hoice of pla y er m 1 ≥ γ m v ( p ) ≥ γ m v ( ˆ p ) (22) follo w s. So γ v ( ˆ p ) ∈ V . T o (ii): It is to sho w that ˆ p is 2 εr max -p erfect in Γ v . Case 1: p m ∈ (0 , 1] With theorem 3.1 4. and p (0 − )p erfect in Γ v it follo w s immediately , that ˆ p is 2 εr max -p erfect in Γ v . Case 2: p m = 0 (a) C onsider pla yer m . Because p is an equilibrium in Γ v , p is also (0- )p erfect in Γ v . With this and p m = 0 γ m v  ( ˆ p − m , 1)  − γ m v  ( ˆ p − m , 0)  = γ m v  ( p − m , 1)  − γ m v  ( p − m , 0)  = r m { m } − v m = 1 − 1 = 0 ∈ [ − 0 , +0] follo w s. (b) Consider pla ye r n ∈ N . With theorem 3.1 4. and remark 3.3 ∀ n ∈ N \ { m } :      γ n v (( ˆ p − n , 1)) − γ n v (( ˆ p − n , 0)) ≤ 2 εr max for ˆ p n = 0 γ n v (( ˆ p − n , 1)) − γ n v (( ˆ p − n , 0)) ∈ [ − 2 εr max , 2 εr max ] for ˆ p n ∈ (0 , 1) γ n v (( ˆ p − n , 1)) − γ n v (( ˆ p − n , 0)) ≥ − 2 εr max for ˆ p n = 1 holds. T ogether with Case 2(a) f oll o ws that ˆ p with p m = 0 is 2 εr max -p erfect in the on e-step game Γ v . So ˆ p ∈ ψ ε ( v ) 6 = ∅ for all v ∈ V .  24 References [1] Flesh, J ., Thuijsman, F. and V rieze, O.J.: Cyclic Markov Equilib ria in a Cubi c Game , In ternational Journal of Game Theory , 26, pp. 303-314, 1997 [2] Holle r, M., Illing, G.: Einf ¨ uhrung in die Spielthe orie , Springer V erlag, Be rlin, 2003 [3] M¨ uller, P . H. and Nollau V.: Steuerung sto chastischer Pr ozesse , Ak ademie-V erlag, Berlin, 1984 [4] Piskuric, M.: V e ctor-value d Markov Games , Dissertation, TU-Dresden, F akult¨ at Mathe- matik und Naturw issensc h aften, 2001 [5] Simon, R.S.: The structur e on non-zer o-sum sto c ha stic games , Adv ances in Applied Mathematics 38, pp 1-26 , 2007 [6] Solan, E.: Thr e e-Player A bsorbing Games , Ma thematics of Op erations Rese arc h, 24, pp. 669-6 98, 1999 [7] Solan, E. and Vieil le, N.: Quitting Games , Mathematics of Op erations Researc h , 2 6, pp . 265-2 85, 2001 [8] Solan, E. and Vieille, N.: Quitting Games - An Example , In ternational Journal of Game Theory , 31, pp. 365 -381, 2002 25