A partial proof of Nashs Theorem via exchangeable equilibria

1 Error in Nash existence pro of Noah D. Stein ∗ Septem b er 27, 2018 The goal of this note is to explain a problem with the pro of of Nash’s theorem in [5], whic h is the main document attached b elo w. W e will use the notation of that pap er through- out without further commen t, with the exception that eac h do cumen t has its o wn section n um b ers and list of references. W e are indebted to Sergiu Hart and Eran Shmay a for their careful study of [5] whic h led to their sim ultaneous discov ery of this error. So far the error has not b een fixed, but many of the results and techniques of the pap er remain v alid, so we will contin ue to make it a v ailable online. 1 The problem The error is in our pro of of Lemma 4 . 6, which is the main driv er b ehind our assertion that order m exc hangeable equilibria exist (Theorem 4 . 7). Of course they do exist; this is the main b enefit of setting out to prov e something you already know to b e true. T o our kno wledge ev erything b efore and after this step remains unaffected, but this is an imp ortan t step in the middle of the proof and logically it m ust be patched if w e are to reac h the desired conclusion. The particular problem is our implicit use of the “fact” b elow. In the incorrect pro of of Lemma 4 . 6 of [5], this “fact” w as applied with Ξ m Γ in place of Γ and appropriate c hoices of σ ∈ ∆ Π G × S m (Ξ m Γ) and τ ∈ ∆ Π G × S m (Π m Γ). “F act” 1. L et σ, τ ∈ ∆ Π (Γ) b e two str ate gy pr ofiles such that σ i is p ayoff e quivalent to τ i in Γ for al l i . Then σ and τ ar e p ayoff e quivalent when viewe d as str ate gies for the maximizer in the asso ciate d zer o-sum game Γ 0 . This may sound reasonable at first because it seems that the difference b et w een pay off- equiv alen t strategies should not b e game-theoretically significant. In fact it is false! And not just for o dd counterexamples: it is false generically . Example 2 . T o see ho w this can fail, let Γ b e the following symmetric bimatrix game: A = B T =   1 1 1 1 1 1 0 0 2   . ∗ Departmen t of Electrical Engineering, Massac h usetts Institute of T echnology: Cambridge, MA 02139. nstein@mit.edu . Let pla yer i ’s strategy set b e C i = { a i , b i , c i } . The first tw o columns of A are equal, as are the first t wo ro ws, so a 1 is pa yoff equiv alent to b 1 . Symmetrically , a 2 is pa yoff equiv alent to b 2 . In the asso ciated zero-sum game Γ 0 w e ha ve u 0 M (( a 1 , a 2 ) , ( a 1 , c 1 )) = 1 6 = 0 = u 0 M (( b 1 , b 2 ) , ( a 1 , c 1 )) , so ( a 1 , a 2 ) and ( b 1 , b 2 ) are not pay off equiv alen t in Γ 0 . Wh y do es this happen? The reason is that when w e consider correlated strategies, w e are in terested not only in what utility our recommendations give us, but also in what information they give. While we ma y not care from a pay off p ersp ectiv e whether strategy a i or b i has b een suggested, the tw o p ossibilities ma y allo w us to make completely differen t inferences ab out the b eha vior of our opp onen ts. It is this information whic h may b e pay off-relev an t. T o see this in action, consider the follo wing t w o distributions o ver outcomes of Γ: W 1 = 1 4   1 0 1 0 0 0 1 0 1   and W 2 = 1 4   1 0 0 0 0 1 0 1 1   . One can v erify that W 1 is a correlated equilibrium, and W 2 is obtained from W 1 b y mo v- ing some mass from ( a 1 , c 2 ) and ( c 1 , a 2 ) to ( b 1 , c 2 ) and ( c 1 , b 2 ), whic h are profiles of pay off equiv alen t strategies. Nonetheless, W 2 is not a correlated equilibrium. If pla y er 1 receives the recommendation b 1 then he knows with probability one that play er 2 has receiv ed the recommendation c 2 , hence it is in the in terest of pla y er 1 to deviate to c 1 . 2 A digression on con tin uous games In this section w e highligh t the complications discussed in Section 1 b y lo oking at how they manifest themselves in the geometry of Nash and correlated equilibria of con tin uous games 1 . Existence of b oth types of equilibria in contin uous games is generally prov en in four steps: 1) discretize the strategy spaces to obtain a finite game, 2) prov e the corresp onding existence theorem for finite games, 3) observ e that an exact equilibrium of the finite game is an approximate equilibrium of the con tin uous game, and 4) sho w that a limit of such appro ximate equilibria is an exact equilibrium of the con tin uous game. Steps 1 and 3 are more or less conten t-free. Step 2 can b e accomplished for Nash equilibria b y fixed-p oint metho ds or for correlated equilibria b y Hart and Schmeidler’s clever minimax argumen t. There is an in teresting div ergence at step 4: in the case of Nash equilibria it is trivial and in the case of correlated equilibria it seems to require a hairy measure-theoretic argumen t (see [1] or [6]). The difference is easiest to understand in terms of an auxiliary map  : ∆(Γ) → R ≥ 0 . This is defined by the condition that  ( µ ) is the smallest  such that µ is an  -correlated equilibrium (no measurable deviation yields an exp ected gain of more than  ). Let  Π b e 1 F or concreteness’ sake, define these to b e games with finitely man y pla y ers, compact metric strategy spaces, and con tin uous utilit y functions. 2 the restriction of  to ∆ Π (Γ). Then  Π ( µ ) is also the smallest  such that µ is an  -Nash equilibrium in the usual sense. W e can study the map  via t wo differen t top ologies on its domain ∆(Γ): the weak and strong topologies. Tw o measures are near eac h other in the strong top ology if y ou can get from one to the other without mo ving m uc h mass. They are near eac h other in the weak top ology if you can get from one to the other without moving muc h mass very far . It is easy to sho w that  is contin uous with resp ect to the strong top ology , but this is not v ery useful b ecause ∆(Γ) is not compact with resp ect to this top ology (at least when Γ is not a finite game) so the limiting arguments do not go through. It is compact with resp ect to the w eak top ology and viewing ∆ Π (Γ) as a space of tuples it is easy to show that  Π is con tin uous with resp ect to the weak top ology , hence we get the necessary limiting argument for the existence of Nash equilibria. F or correlated equilibria the situation is different:  is not weakly con tin uous! The argumen t is essentially the same as wh y W 1 w as a correlated equilibrium in Example 2 but W 2 w as not. W e can mo v e some mass a little bit (thinking now of strategically equiv alent strategies as b eing quite “close” to eac h other) and change the information con vey ed by the correlating device drastically . This changes an exact correlated equilibrium to a distribution for whic h  is b ounded aw ay from zero, no matter ho w small the distance we mov ed the mass. In particular this sho ws that  is not w eakly upper semicon tin uous. In fact this do esn’t matter; all we really need for the existence pro of is weak low er semicon tinuit y . It turns out  has this prop erty , but proving it is a delicate technical matter (again, see [1] or [6]). The fact that Nash equilibria b eha ve nicely with resp ect to strategic equiv alence is what allo ws us to turn problems regarding mixed Nash equilibria of p olynomial games into finite- dimensional problems in momen t spaces: see for example [2] and [3]. The failure of correlated equilibria to b eha ve nicely with resp ect to strategic equiv alence is captured b y the fact that the corresp onding problems for correlated equilibria are inherently infinite-dimensional [4]. 3 A more sp ecific problem The erroneous pro of of Lemma 4 . 6 claims to construct, for any symmetric mixed strategy y of the minimizer in the game (Ξ m Γ) 0 , a mixed strategy π y ∈ ∆ G × S m (Π m Γ) for the maximizer suc h that u 0 M ( π y , y ) = 0. F urthermore, the construction implicitly claims that such a π y can b e found which do es not dep end on the utilities of Γ (as is the case in [1]) and whic h is rational in y . That is to sa y , the elements of π y are in the field generated by the elements of y o v er Q . F or simplicity we will write π y ∈ Q [ y ]. Let us see ho w rationality of π y follo ws from the pro of of Lemma 4 . 6. First note that a finite zero-sum game has a maximin strategy whic h is rational in the utilities, b ecause the set of maximin strategies is a p olytop e defined b y inequalities with co efficien ts linear in the utilities. Therefore Lemmas 2 . 15 and 3 . 8 show that the certificates π y constructed in Hart and Sc hmeidler’s pro of of the existence of correlated equilibria and our proof of the existence of excha ngeable equilibria are in Q [ y ]. But Lemma 4 . 6 claims to construct the certificate 3 π y for order m exchangeable equilibria by taking the one from Lemma 3 . 8 for exchangeable equilibria and replacing it by the pro duct of its marginals. That is to say , we are taking the image of a tuple of elements of Q [ y ] under a p olynomial map, and so we end up with another tuple π y ∈ Q [ y ]. Example 3 . In fact such a π y need not exist for order m exc hangeable equilibria, as the follo wing example due to Sergiu Hart shows. Consider the symmetric bimatrix game Γ and its con tracted second p o wer Ξ 2 Γ sho wn in T able 1. Supp ose the minimizer pla ys the mixed strategy y α,β whic h assigns probability α to 00 → 01 and 00 → 10 and probability β to 11 → 01 and 11 → 10 for each pla y er, so w e ha ve 4( α + β ) = 1. T o hav e π y α,β ∈ ∆ Z 2 × S 2 (Π 2 Γ) means that π y α,β is a pro duct of four i.i.d. Bernoulli( p ) random v ariables for some p ∈ [0 , 1]. F or notational simplicit y we will let q = 1 − p . Then w e can compute the exp ected utilit y of the maximizer in (Ξ 2 Γ) 0 as: u 0 M ( π y α,β , y α,β ) = 4( αq 2 − β p 2 )( q r + ps ) . F or this expression to b e zero for all r and s we must ha v e αq 2 = β p 2 , so p = √ α √ α + √ β and q = √ β √ α + √ β , and these expressions are clearly not rational in α and β . But p and q are marginals of π y α,β and so can be written as sums of its elemen ts. Th us the en tries of π y α,β cannot all b e rational in α and β either. References [1] S. Hart and D. Sc hmeidler. Existence of correlated equilibria. Mathematics of Op er ations R ese ar ch , 14(1), F ebruary 1989. [2] S. Karlin. Mathematic al Metho ds and The ory in Games, Pr o gr amming, and Ec onomics , v olume 2: Theory of Infinite Games. Addison-W esley , Reading, MA, 1959. [3] P . A. P arrilo. Polynomial games and sum of squares optimization. In Pr o c e e dings of the 45th IEEE Confer enc e on De cision and Contr ol (CDC) , 2006. [4] N. D. Stein, A. Ozdaglar, and P . A. P arrilo. Structure of extreme correlated equilibria. In preparation. [5] N. D. Stein, P . A. P arrilo, and A. Ozdaglar. A new pro of of Nash’s Theorem via ex- c hangeable equilibria. In preparation. [6] N. D. Stein, P . A. P arrilo, and A. Ozdaglar. Correlated equilibria in con tinuous games: Characterization and computation. Games and Ec onomic Behavior , 2010. 4 Γ ( u 1 , u 2 ) 0 1 0 ( r , r ) ( s, 0) 1 (0 , s ) (0 , 0) (Ξ 2 Γ) 0 Minimizer: Deviations for pla yer 1 Deviations for pla yer 2 z }| { z }| { 00 00 00 01 01 01 10 10 10 11 11 11 00 00 00 01 01 01 10 10 10 11 11 11 Maximizer: ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ s 2 2 s 1 2 s 2 1 s 1 1 01 10 11 00 10 11 00 01 11 00 01 10 01 10 11 00 10 11 00 01 11 00 01 10 0 0 0 0 r r 2 r 0 0 0 0 0 0 0 0 0 r r 2 r 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 − r 0 r 0 0 0 0 0 0 s r r + s 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 − r 0 r 0 0 0 r s r + s 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 − 2 r − r − r s s 2 s 0 0 0 0 0 0 0 0 0 0 1 0 0 s r r + s 0 0 0 0 0 0 0 0 0 0 0 0 − r 0 r 0 0 0 0 0 0 0 1 0 1 0 0 0 − s r − s r 0 0 0 0 0 0 0 0 0 − s r − s r 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 − r s − r s 0 0 0 0 0 0 − r s − r s 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 − r − s − r − s 0 0 0 − s 0 s 0 0 0 0 0 0 1 0 0 0 r s r + s 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 − r 0 r 0 0 0 1 0 0 1 0 0 0 − r s − r s 0 0 0 0 0 0 0 0 0 0 0 0 − r s − r s 0 0 0 1 0 1 0 0 0 0 0 0 0 − s r − s r 0 0 0 0 0 0 0 0 0 − s r − s r 0 0 0 1 0 1 1 0 0 0 0 0 0 0 0 0 − r − s − s − r 0 0 0 0 0 0 − s 0 s 0 0 0 1 1 0 0 s s 2 s 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 − 2 r − r − r 1 1 0 1 0 0 0 − s 0 s 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 − r − s − r − s 1 1 1 0 0 0 0 0 0 0 − s 0 s 0 0 0 0 0 0 0 0 0 0 0 0 − r − s − s − r 1 1 1 1 0 0 0 0 0 0 0 0 0 − 2 s − s − s 0 0 0 0 0 0 0 0 0 − 2 s − s − s T able 1: Ab o v e: Utilities for Γ. Pla y er 1 c ho oses ro ws and pla y er 2 chooses columns. Below: Utilities for the maximizer in (Ξ 2 Γ) 0 . The maximizer c ho oses rows and the minimizer chooses columns. Columns asso ciated with a deviation from a giv en strategy to itself would ha ve all zeros and hav e b een omitted. 5 A new pro of of Nash’s Theorem via exc hangeable equilibria Noah D. Stein, P ablo A. P arrilo, and Asuman Ozdaglar ∗ Septem b er 27, 2018 Abstract W e giv e a nov el pro of of the existence of Nash equilibria in all finite games without using fixed p oin t theorems or path following arguments. Our approach relies on a new notion in termediate betw een Nash and correlated equilibria called exchangeable equilibria, whic h are correlated equilibria with certain symmetry and factorization prop erties. W e prov e these exist by a duality argument, using Hart and Sc hmeidler’s pro of of correlated equilibrium existence as a first step. In an appropriate limit exc hangeable equilibria con verge to the con v ex hull of Nash equilibria, proving that these exist as well. Exc hangeable equilibria are defined in terms of symmetries of the game, so this metho d automatically prov es the stronger statemen t that a symmetric game has a symmetric Nash equilibrium. The case without symmetries follows by a symmetrization argumen t. 1 In tro duction Nash’s Theorem is one of the most fundamen tal results in game theory and states that an y finite game has a Nash equilibrium in mixed strategies. Despite its imp ortance, the authors of the present pap er kno w of only three essentially differen t pro ofs. The first and most common w a y to pro ve Nash’s Theorem is by applying a fixed p oint theorem, usually Brou w er’s or Kakutani’s, to a map whose fixed p oin ts are easily sho wn to b e Nash equilibria. The fixed point theorem is usually prov en com binatorially , say by Sp erner’s Lemma [15] or Gale’s argument using the game of hex [5], or with (co-)homology theory , a suite of p ow erful but less elementary to ols from algebraic top ology [8]. The second pro of of Nash’s Theorem (historically) is algorithmic and consists of showing that the Lemk e-Ho wson path-following algorithm terminates at a Nash equilibrium [11]. In fact this is not so differen t from the fixed p oint pro of, b ecause Sp erner’s Lemma is also pro v en by a path-following argument. ∗ Departmen t of Electrical Engineering, Massac h usetts Institute of T echnology: Cambridge, MA 02139. nstein@mit.edu , parrilo@mit.edu , and asuman@mit.edu . The third pro of of Nash’s theorem is due to Kohlb erg and Mertens and is top ological [10]. The idea is to sim ultaneously consider the set of all games of a given size and the set of all (game , equilibrium) pair s. Under appropriate compactifications b oth of these sets become spheres of the same dimension. One then sho ws that the mapping sending an equilibrium to the corresp onding game is homotopic to the identit y map on the sphere. A (co-)homological or degree-theoretic argumen t sho ws that such a map must b e onto [8]. T ec hnically sp eaking this step is almost iden tical to the pro of of Brouw er’s Fixed Poin t Theorem, so the third pro of is closely related to the first. All three pro ofs ha v e provided unique insigh ts into the structure of Nash equilibria and it is our hop e that a differen t pro of, whic h uses neither fixed point theorems, nor path-following argumen ts, nor any algebro-top ological to ols, will provide further insigh ts. Hart and Schmeidler [7] ha ve prov en the weak er result that correlated equilibria exist b y a clev er application of the Minimax Theorem, summarized in Section 2.2. F or games endo wed with a group action, a simple a veraging argumen t then prov es that a symmetric correlated equilibrium exists (Prop osition 2.26). W e sho w that for suc h games Hart and Schmeidler’s pro of can b e strengthened to pro duce correlated equilibria with additional symmetry and factorization prop erties, which w e call exchange able e quilibria (Theorem 3.9). T o illustrate this idea, consider the case of k × k symmetric bimatrix games (t wo-pla yer games fixed under the op eration of swapping the play ers). Let X = { xx T | x ∈ R k × 1 ≥ 0 } . Then w e ha ve Nash C E ∩ X ⊆ conv ex hull of Nash con v( C E ∩ X ) ⊆ exchangeable C E ∩ conv( X ) ⊆ correlated C E , where each t yp e of (symmetric) equilibrium is defined by the set written b elo w it. This definition sho ws that the exchangeable equilibria are a natural mathematical ob ject. F or examples and game-theoretic in terpretations of exc hangeable equilibria, see the companion pap er [16]. In Section 3 we extend the definition of exchangeable equilibria games to n -play er games with arbitrary symmetry groups. The preceding discussion sho ws that the set of exc hange- able equilibria is conv ex, compact, contained in the set of symmetric correlated equilibria, and con tains the con vex h ull of the set of symmetric Nash equilibria. One can show that these con tainmen ts can all b e strict [16], so pro ving existence of exchangeable equilibria is a step in the right direction, but do es not immediately pro ve existence of Nash equilibria. Ho w ever, w e can use the same tec hniques to pro v e existence of exc hangeable equilibria with successively stronger symmetry prop erties as follo ws. F or a fixed n -play er game Γ and n um b er m ∈ N , we define t wo new games Π m Γ and Ξ m Γ, which w e call m th p owers of Γ. These are larger games in which m copies of Γ are play ed sim ultaneously . The difference b et w een the tw o p ow ers is that Π m Γ has a differen t group of play ers for each cop y , so mn pla y ers total, whereas Ξ m Γ has just one group of n pla yers pla ying all m copies, but p erhaps c ho osing different actions in eac h cop y (Figure 1). There is a natural marginalization map sending exchangeable equilibria of either of these p o w ers to exchangeable equilibria of Γ. In fact, an y exchangeable equilibrium of Γ can b e lifted to an exchangeable equilibrium of either p o wer, and for a symmetric Nash equilibrium w e can take these tw o lifts to b e the same. How ever, for a general exchangeable equilibrium 2 the t wo lifts need not b e compatible, so it is natural to consider the intersection XE m (Γ) := XE(Π m Γ) ∩ XE(Ξ m Γ) of the sets of exchangeable equilibria of the t w o p ow ers. W e call these or der m exchange able e quilibria of Γ and prov e they exist using a similar Minimax argument (Theorem 4.7). Under appropriate iden tifications these sets turn out to b e nested as m gro ws and, b eing con v ex, they all con tain the conv ex hull of the symmetric Nash equilibria: NE(Γ) ⊆ con v(NE(Γ)) ⊆ . . . ⊆ XE 3 (Γ) ⊆ XE 2 (Γ) ⊆ XE 1 (Γ) = XE(Γ) ⊆ CE(Γ) . Assuming Γ has a rich enough symmetry group (e.g. if it is a symmetric bimatrix game or, more generally , if it satisfies a condition w e call player tr ansitivity ), then as m go es to infinity these conv erge to symmetric correlated equilibria in which the outcome of the correlating device is common knowledge; such correlated equilibria are known to b e mixtures of sym- metric Nash equilibria (Theorems 4.9 and 5.1). In particular, this prov es that symmetric Nash equilibria exist in games with ric h enough symmetry groups. Note that symmetry is fundamen tal in this argument. F or example, if we had b egun with a general bimatrix game and let X = { xy T | x, y ∈ R k × 1 ≥ 0 } we w ould hav e had conv( X ) = R k × k ≥ 0 , so the exc hangeable equilibria (even the order m exc hangeable equilibria) would ha v e b een exactly the correlated equilibria and we would not ha v e strengthened the equilibrium notion at all. Ho w ever, there are sev eral w a ys of turning general games into symmetric games [6] and applying such a pro cedure pro v es existence of Nash equilibria in all games (Section 5.2). Up to the step of taking m to infinit y , all the steps of our proof are computationally effec- tiv e. P apadimitriou has shown ho w to apply the ellipsoid algorithm to Hart and Sc hmeidler’s pro of to efficiently compute a correlated equilibrium of a large game [13]. The same tec h- nique applied to our pro of allows one to compute an exchangeable equilibrium (or an order m exc hangeable equilibrium for fixed m ) in p olynomial time, even though the set of these is not p olyhedral. Computing these is interesting in its own right [16] and ma y b e useful for computing approximate Nash equilibria. How ever, computation is not the fo cus of this pap er and we lea ve a detailed inv estigation of these ideas for future work. The remainder of the pap er is organized as follows. W e b egin with background material in Section 2. W e cov er the definitions of games and equilibria, give an ov erview of Hart and Schmeidler’s pro of of the existence of correlated equilibria so w e can mo dify it later, and introduce group actions. In Section 3 we in tro duce exchangeable equilibria and pro ve existence of these for games under arbitrary group actions. W e do the same for order m ex- c hangeable equilibria in Section 4, introducing p o w ers of games along the w ay . W e complete the argument in Section 5 b y showing that the order m exchangeable equilibria conv erge to mixtures of symmetric Nash equilibria in games with a pla y er transitiv e symmetry group, and then sho wing that w e can symmetrize any game to mak e this condition hold. Section 6 concludes and gives directions for future work. 3 2 Bac kground This section is divided in to three parts. In the first part w e lay out the basic definitions of finite games as w ell as Nash and correlated equilibria to fix notation. W e assume the reader is familiar with these concepts and do not attempt to motiv ate them. The second part reviews Hart and Sc hmeidler’s pro of of the existence of correlated equilibria [7], preparing for similar arguments in this pap er. The third part co vers symmetries of games. The concept of a symmetry of a game extends back at least to Nash’s pap er [12]. Sym- metries are fundamental to the presen t pap er, so w e s pend more time on these and giv e some examples. Although w e use the language of group theory to discuss symmetries, it is w orth noting that we do not use an y but the most basic theorems from group theory (e.g., the fact that for any h in a group G , the maps g 7→ g h and g 7→ hg are bijections from G to G ). Everything in this section is standard except for the notions of a go o d r eply , a go o d set , a player-trivial symmetry gr oup , a player-tr ansitive symmetry gr oup , and the remarks follo wing the statement of Nash’s Theorem. 2.1 Games and equilibria Definition 2.1. A (finite) game has a finite set I of n ≥ 2 pla yers, eac h with a finite set C i of at least t wo strategies (also called pure strategies ) and a utilit y function u i : C → R , where C = Q C i . A game is zero-sum if it has t wo pla y ers, called the maximizer (denoted M ) and the minimizer (denoted m ), and satisfies u M + u m ≡ 0. F or elemen ts of C i w e use Roman letters subscripted with the pla yer’s identit y , such as s i and t i . W e will t ypically use the unsubscripted letter s to denote a strategy profile (a c hoice of strategy for eac h play er). F or a choice of a strategy for all play ers except i we use the symbol s − i . T o denote the set of Borel probability distributions on a space X w e write ∆( X ). F or muc h of the pap er X will b e finite so w e can view ∆( X ) as a simplex in the finite-dimensional vector space R X of real-v alued functions on X . F or x ∈ X the probability distribution which assigns unit mass to x will b e written δ x ∈ ∆( X ). Definition 2.2. A mixed strategy for play er i in a game Γ is a probability distribution o v er his pure strategy set C i , and the set of mixed strategies for play er i is ∆( C i ). The set of mixed strategy profiles (also called indep enden t or pro duct distributions ) will b e denoted ∆ Π (Γ) := Q i ∆( C i ). F or indep enden t distributions it is imp ortan t that we write ∆ Π (Γ) rather than ∆ Π ( C ), b ecause Γ sp ecifies how C is to b e though t of as a pro duct. F or example, the set S × S × S could b e view ed as a product of three copies of S , or a product of S with S × S , and these lead to different notions of an indep enden t distribution – one is a pro duct of three terms and one is a pro duct of tw o terms. This distinction will b e particularly imp ortan t when w e define p ow ers of games in Section 4. T o mak e the notation fit together w e will write ∆(Γ) for ∆( C ). W e ma y then view ∆ Π (Γ) as the (nonconv ex) subset of ∆(Γ) consisting of pro duct distributions or as a conv ex subset 4 of R t i C i . The former view will b e natural when we define exc hangeable equilibria, whic h liv e in ∆(Γ), as conv ex combinations of pro duct distributions. The latter will b e useful when lo oking for product distributions which are fixed b y a group action (see the pro of of Lemma 3.8); such fixed distributions are easy to find with a con v ex setup (Prop osition 2.18). Whic h of these views we are using will b e clear from con text if not explicitly sp ecified. As usual w e extend the domain of u i from C to ∆(Γ) by linearit y , defining u i ( π ) = P s ∈ C u i ( s ) π ( s ). Ha ving done so we can define equilibria. Definition 2.3. A Nash equilibrium is an n -tuple ( ρ 1 , . . . , ρ n ) ∈ ∆ Π (Γ) = Q i ∆( C i ) of mixed strategies, one for each play er, suc h that u i ( s i , ρ − i ) ≤ u i ( ρ i , ρ − i ) for all strategies s i ∈ C i and all play ers i . The set of Nash equilibria of a game Γ is denoted NE(Γ). Definition 2.4. A correlated equilibrium is a joint distribution π ∈ ∆(Γ) suc h that P s − i ∈ C − i [ u i ( t i , s − i ) − u i ( s )] π ( s ) ≤ 0 for all strategies s i , t i ∈ C i and all pla yers i . The set of correlated equilibria of a game Γ is denoted CE(Γ). The follo wing alternativ e c haracterization of correlated equilibria is standard and we omit its proof. Supp ose ( X 1 , . . . , X n ) is a random vector taking v alues in C . W e think of X i as a (random) strategy recommended to play er i . Given this information, pla y er i can form his conditional b eliefs P ( X − i | X i ) ab out the recommendations to the other pla y ers given his own recommendation. That is to sa y , P ( X − i | X i ) is a random v ariable taking v alues in ∆( C − i ) which is a function of X i . One can then define the ev en t { the pure strategy X i is a b est resp onse to the distribution P ( X − i | X i ) for all i } . The distribution of ( X 1 , . . . , X n ) is a correlated equilibrium if and only if this ev ent happ ens almost surely . More succinctly: Prop osition 2.5. L et ( X 1 , . . . , X n ) b e a r andom ve ctor taking values in C distribute d ac- c or ding to π ∈ ∆(Γ) . Then π is a c orr elate d e quilibrium if and only if X i is a b est r esp onse to P ( X − i | X i ) almost sur ely for al l i . Nash equilibria corresp ond exactly to the correlated equilibria which are product distri- butions, so viewing ∆ Π (Γ) as a subset of ∆(Γ) we can write NE(Γ) = CE(Γ) ∩ ∆ Π (Γ). W e in tro duce the existence theorems for correlated and Nash equilibria in Sections 2.2 and 2.3. W e need the Minimax Theorem at this p oint to define the v alue of a zero-sum game. It also pla ys an important role in our pro of of Nash’s Theorem. An elemen tary proof can b e giv en using the separating hyperplane theorem [2]. Minimax Theorem. L et U and V b e finite-dimensional ve ctor sp ac es with c omp act c onvex subsets K ⊂ U and L ⊂ V . L et Φ : U × V → R b e a biline ar map. Then sup x ∈ K inf y ∈ L Φ( x, y ) = inf y ∈ L sup x ∈ K Φ( x, y ) , and the optima ar e attaine d. 5 Definition 2.6. Given a zero-sum game Γ, we can apply this theorem with K = ∆( C M ), L = ∆( C m ), and Φ = u M . The common v alue of these t wo optimization problems is called the v alue of the game and denoted v (Γ). Maximi zers on the left hand side are called maximin strategies and the set of such is denoted Mm(Γ) ⊆ ∆( C M ). W e no w introduce the notion of a go o d r eply in a zero-sum game. This is not a standard definition, but it will simplify the statemen ts of several arguments b elo w. The name is mean t to be ev o cativ e of the term b est r eply : while a best reply is one whic h maximizes one’s pa yoff, a go o d reply is merely one which returns a “go od” pay off: at least the v alue of the game 1 . Definition 2.7. In a zero-sum game Γ, w e sa y that a strategy σ ∈ ∆( C M ) for the maximizer is a go o d reply to θ ∈ ∆( C m ) if u M ( σ, θ ) ≥ v (Γ). W e say that a set Σ ⊆ ∆( C M ) of strategies is go o d against the set Θ ⊆ ∆( C m ) if for all θ ∈ Θ there is a σ ∈ Σ which is a go o d reply to θ . If Σ is go o d against ∆( C m ) we say that Σ is go o d . The main result ab out go o d sets is: Prop osition 2.8. If Γ is a zer o-sum game and Σ ⊆ ∆( C M ) is go o d, then Γ has a maximin str ate gy in con v(Σ) , i.e., conv(Σ) ∩ Mm(Γ) 6 = ∅ . Pr o of. Apply the Minimax Theorem with K = conv(Σ) and L = ∆( C m ). It is w orth noting that in general a go o d set need not include a maximin strategy . F or example, in an y zero-sum game the set C M ( ∆( C M ) is a go o d set, but some zero-sum games such as matching p ennies only ha v e mixed maximin strategies, i.e. C M ∩ Mm(Γ) = ∅ . The notion of pay off equiv alence is a standard wa y to turn structural information ab out a game into structural information ab out equilibria. The pro ofs of b oth prop ositions b elow are immediate. Definition 2.9. Tw o mixed strategies σ i , τ i ∈ ∆( C i ) are said to b e pa y off equiv alen t if u j ( σ i , s − i ) = u j ( τ i , s − i ) for all s − i ∈ C − i and all play ers j . Prop osition 2.10. If Γ is a zer o-sum game, Σ is go o d, and e ach σ ∈ Σ is p ayoff e quivalent to some σ 0 ∈ Σ 0 , then Σ 0 is go o d. Prop osition 2.11. If σ i is p ayoff e quivalent to τ i for al l i , then ( σ 1 , . . . , σ n ) is a Nash e quilibrium if and only if ( τ 1 , . . . , τ n ) is a Nash e quilibrium. 2.2 Hart and Sc hmeidler’s pro of In this section we recall the structure of Hart and Sc hmeidler’s pro of of the existence of correlated equilibria based on the Minimax Theorem [7]. The goal of this is to frame their argumen t in a wa y which will allo w us to extend it, redoing as little as p ossible of the work they hav e done. W e will use similar arguments to prov e Theorems 3.9 and 4.7. 1 Go o d sets are similar in spirit to V oorneveld’s pr ep sets [18], but tailored to zero-sum games. 6 Hart and Schmeidler’s argument b egins by asso ciating with a game Γ a new zero-sum game Γ 0 and interpreting correlated equilibria of Γ as maximin strategies of this new game. In Γ 0 the maximizer plays the roles of all the play ers in Γ sim ultaneously and the minimizer tries to find a profitable unilateral deviation from the strategy profile selected b y the maximizer. Definition 2.12. Giv en any game Γ, define a tw o-play er zero-sum game Γ 0 with C 0 M := C , C 0 m := F i C i × C i , and utilities u 0 M ( s, ( r i , t i )) = − u 0 m ( s, ( r i , t i )) := ( u i ( s ) − u i ( t i , s − i ) if r i = s i , 0 otherwise . Prop osition 2.13. L et Γ b e any game. F or any player i in Γ , r i ∈ C i , and s ∈ C we have u 0 M ( s, ( r i , r i )) = 0 , so we c an b ound the value of Γ 0 by v (Γ 0 ) ≤ 0 . A mixe d str ate gy σ ∈ ∆( C 0 M ) = ∆( C ) for the maximizer in Γ 0 satisfies u 0 M ( σ, ( r i , t i )) ≥ 0 for al l ( r i , t i ) ∈ C 0 m if and only if σ ∈ CE(Γ) . Ther efor e, if v (Γ 0 ) = 0 then Mm(Γ 0 ) = CE(Γ) . Pr o of. Immediate from the definitions. T o prov e v (Γ 0 ) = 0, and hence the existence of correlated equilibria (Theorem 2.16), w e must show that for an y y ∈ ∆( C 0 m ) there is a π ∈ ∆( C 0 M ) suc h that u M ( π , y ) ≥ 0. Hart and Schmeidler actually sho w that there exists suc h a π with some extra structure, whic h we summarize in Lemma 2.15. W e will exploit this extra structure b elow to prov e stronger statements in a similar spirit: Lemmas 3.8 and 4.6. These in turn allo w us to prov e the existence of exc hangeable equilibria (Theorem 3.9) and order m exchangeable equilibria (Theorem 4.7). Giv en a y = ( y 1 , . . . , y n ) ∈ ∆( C 0 m ), y i ∈ R C i × C i , a go o d reply π can b e constructed in terms of certain auxiliary games γ ( y i ). F or the purp oses of the presen t pap er, it is more imp ortan t to understand the statement of Lemma 2.15 than to remem b er the details of this construction. Besides this lemma the only property of γ ( y i ) w e will need is that its definition is indep endent of ho w elemen ts of C i are lab eled (Prop osition 3.7). Definition 2.14. F or an y pla y er i in Γ and an y nonnegativ e y i ∈ R C i × C i , define the zero-sum game γ ( y i ) with strategy sets C M = C m := C i and utilities u M ( s i , t i ) = − u m ( s i , t i ) := ( P r i 6 = t i y s i ,r i i if s i = t i , − y s i ,t i i otherwise . Lemma 2.15 ([7]) . Fix a game Γ and c onsider Γ 0 . If y ∈ ∆( C 0 m ) , then any str ate gy π ∈ Mm( γ ( y 1 )) × · · · × Mm( γ ( y n )) ⊂ ∆( C 0 M ) satisfies u M ( π , y ) = 0 . In p articular v (Γ 0 ) = 0 , π is go o d against y , and ∆ Π (Γ) is go o d. Pr o of. Omitted. See [7] for a pro of using the Minimax Theorem. Theorem 2.16 ([7]) . F or any game Γ , the value v (Γ 0 ) = 0 , so Mm(Γ 0 ) = CE(Γ) and a c orr elate d e quilibrium of Γ exists. 7 Pr o of. Combining Lemma 2.15 and Prop osition 2.13, w e get Mm(Γ 0 ) = CE(Γ). F or exis- tence, apply Prop osition 2.8 to Γ 0 with Σ = ∆ Π (Γ). This pro of merits t w o remarks. First of all, since con v(∆ Π (Γ)) = ∆(Γ), Prop osition 2.8 do es not yield any b enefit in this case ov er directly applying the Minimax Theorem to Γ 0 . Rather, w e hav e used Prop osition 2.8 to illustrate our pro of strategy for Theorems 3.9 and 4.7, in which w e use stronger versions of Lemma 2.15 to choose Σ with conv(Σ) ( ∆(Γ). Second, note that in this case we kno w that there is a maximin strategy of Γ 0 in the go o d set ∆ Π (Γ): this is just the statement of Nash’s Theorem. Ho w ever, we cannot conclude this directly from the fact that ∆ Π (Γ) is a go o d set b ecause of the remark after Prop osition 2.8. 2.3 Groups acting on games In this section we recall the notion of a group acting on a game, as defined by Nash [12]. All groups will b e finite throughout. In an y group e will denote the identit y elemen t. The subgroup generated by group elements g 1 , . . . , g n will b e denoted h g 1 , . . . , g n i . F or n ∈ N w e will write Z n for the additive group of integers mo d n and S n for the symmetric group on n letters. W e will use cycle notation to express p ermutations. F or example σ = (1 2 3)(4 5)(6) is shorthand for σ (1) = 2 , σ (2) = 3 , σ (3) = 1 , σ (4) = 5 , σ (5) = 4 , and σ (6) = 6 . Definition 2.17. A left action of the group G on the set X is a map · : G × X → X written with infix notation which satisfies the iden tity condition e · x = x and the asso ciativity condition g · ( h · x ) = ( g h ) · x . A righ t action of G on X is a map · : X × G → X such that x · e = x and ( x · g ) · h = x · ( g h ). W e sa y that an action is linear if it extends to an action on an ambien t vector space V con taining X and the map x 7→ x · g on V is linear for all g ∈ G . An x ∈ X is G -in v arian t if x · g = x for all g ∈ G . The set of G -inv ariant elements is denoted X G . Prop osition 2.18. If G acts line arly on the c onvex set X then ther e is a map a v e G : X → X G given by a ve G ( x ) = 1 | G | P g ∈ G x · g . In p articular if X is nonempty then X G is nonempty. Pr o of. F or an y x ∈ X , av e G ( x ) is a conv ex combination of elemen ts x · g ∈ X , hence a v e G ( x ) ∈ X . F or an y h ∈ G we ha ve a v e G ( x ) · h = " 1 | G | X g ∈ G x · g # · h = 1 | G | X g ∈ G ( x · g ) · h = 1 | G | X g ∈ G x · ( g h ) = 1 | G | X g ∈ G x · g = av e G ( x ) , where we hav e used linearity , the definition of a group action, and bijectivity of g 7→ g h . 8 A left action of G on X induces righ t actions on many function spaces defined on X . F or example R X is the space of functions X → R . F or y ∈ R X w e can define y · g ∈ R X b y ( y · g )( x ) = y ( g · x ). The condition that this is a right action of G on R X follo ws immediately from the fact that we b egan with a left action of G on X . F or finite X (the case of most in terest to us), the same argument sho ws that G acts on ∆( X ) on the right. Definition 2.19. W e sa y that a group G acts on the game Γ if the follo wing conditions hold. The group G acts on the left on the set of play ers I and F i C i , making g · s i ∈ C g · i for s i ∈ C i . Suc h actions automatically induce a left action of G on C = Q i C i defined b y ( g · s ) g · i = g · s i . W e require that the utilities b e inv ariant under the induced action on the righ t: u g · i · g = u i , i.e., u g · i ( g · s ) = u i ( s ) for all i ∈ I , s ∈ C , and g ∈ G . W e sa y that G is a symmetry group of Γ and call elements of G symmetries of Γ. Note that an action of G on a game can b e fully sp ecified b y its action on F i C i or on C . One wa y to do this is to choose G to be a subgroup of the symmetric group on F i C i or C satisfying the ab ov e prop erties. Definition 2.20. The stabilizer subgroup of pla y er i is G i := { g ∈ G | g · i = i } , and acts on C i on the left. W e sa y that the action of G is pla y er trivial if G i = G for all i , or in other words if g · i = i for all g and i . W e say that the action of G is play er transitiv e if for all i, j ∈ I there exists g ∈ G such that g · i = j . W e illustrate the notion of group actions on a game using four examples. Example 2.21 . Let Γ b e any game and G any group. Define g · s = s for all g ∈ G and s ∈ C . This defines a play er-trivial action of G on Γ called the trivial action. Example 2.22 . A t w o-play er finite game is often called a bimatrix game b ecause it can b e describ ed by tw o matrices A and B , such that if play er one pla ys strategy i and play er tw o pla ys strategy j then their pay offs are A ij and B ij , resp ectiv ely . If these matrices are square and B = A T then w e call the game a symmetric bimatrix game . One example is the game of chic k en, which has A = [ 4 1 5 0 ] = B T . T o put this in the context of group actions defined abov e, let each play er’s strategy set b e C 1 = C 2 = { 1 , . . . , m } indexing the ro ws and columns of A and B . Define g · ( i, j ) = ( j , i ) for ( i, j ) ∈ C , so g · ( g · ( i, j )) = ( i, j ). The assumption B = A T is exactly the utilit y compatibilit y condition saying that this sp ecifies an action of G = { e, g } ∼ = Z 2 on this game. Of course, dep ending on the structure of A and B there may b e other non trivial symmetries as well. The element g sw aps the play ers, so the action of G is play er transitive. Example 2.23 . Note that the condition that a bimatrix game b e symmetric is not that A = A T and B = B T . Indeed, such a game need not ha ve any non trivial symmetries. F or example, consider the game defined b y A = [ 0 2 2 1 ] and B = [ 3 0 0 1 ]. The unique Nash equilibrium of this game is for pla y er 1 to pla y the mixed strategy p = [ 1 4 3 4 ] and play er 2 to play q = [ 1 3 2 3 ]. Since the equilibrium is unique, any symmetry of the game m ust induce a corresp onding symmetry of the equilibrium by Nash’s Theorem. But the four entries of p and q are all distinct, so the only symmetry of this game is the trivial one. 9 ( u 1 , u 2 ) H 2 T 2 H 1 (1 , − 1) ( − 1 , 1) T 1 ( − 1 , 1) (1 , − 1) T able 1: Matching p ennies. Play er 1 chooses rows and play er 2 chooses columns. Example 2.24 . Consider the game of matching p ennies, whose utilities are sho wn in T able 1. The lab els H and T stand for heads and tails, resp ectiv ely , and the subscripts indicate the iden tities of the pla y ers for notational purp oses. This a bimatrix game, but it is not a symmetric bimatrix game in the sense of Example 2.22. Nonetheless this game do es hav e symmetries. The easiest to see is the map σ which in terc hanges the roles of heads and tails. Letting g b e the p ermutation of F i C i giv en in cycle notation as g = ( H 1 T 1 )( H 2 T 2 ), w e define g · s i = g ( s i ). Another symmetry is the p erm utation h = ( H 1 H 2 T 1 T 2 ). These satisfy g 2 = e and h 2 = g , so G = h h i ∼ = Z 4 . Note that g acts on I as the iden tity whereas h sw aps the pla yers, so G acts play er transitively , whereas h g i ∼ = Z 2 acts play er trivially . Example 2.25 . Now w e consider an example of an n -play er game with symmetries. Through- out this example all arithmetic will be done mod n . F or simplicit y in this example we will index the play ers using the members of Z n instead of the set { 1 , . . . , n } . Each play er’s strategy space will b e C i = Z n as well. Define u i ( s 1 , . . . , s n ) = ( 1 , when s i = s i − 1 + 1 0 , otherwise . Then w e can define a symmetry g by g ( s i ) = s i + 1, which increments each pla yer’s strategy by one mo d n , but fixes the identities of the play ers. Clearly g is a p erm utation of order n . W e can define another symmetry h which maps a strategy for pla y er i to the same n um b ered strategy for pla yer i + 1. That is to sa y , h acts on C by cyclically p erm uting its argumen ts. Again, h is a p ermutation of order n . Note that g and h comm ute, so together they generate a symmetry group G ∼ = Z n × Z n . Both h h i ∼ = Z n and G act play er transitively , whereas h g i ∼ = Z n acts pla y er trivially . If n is comp osite and factors as n = k l for k , l > 1 then h h k i ∼ = Z l acts on Γ but neither pla yer transitiv ely nor pla y er trivially . The left actions in the definition of a group action on a game induce linear righ t actions on function spaces such as ∆(Γ) ( R C and ∆ Π (Γ) ( R t i C i . The inclusion map R t i C i → R C is G -equiv ariant (commutes with the action of G ), so with regard to this action it do es not matter whether we choose to view ∆ Π (Γ) as a subset of R t i C i or of R C . Because of the utilit y compatibility conditions of a group action on a game, the actions on ∆(Γ) and ∆ Π (Γ) restrict to actions on the sets CE(Γ) and NE(Γ), resp ectively . This allo ws us to define the G -in v ariant subsets ∆ G (Γ), ∆ Π G (Γ), CE G (Γ), and NE G (Γ). The action of the stabilizer subgroup G i on C i allo ws us to define the G -inv ariant subset ∆ G i ( C i ). 10 The main theorem w e set out to prov e is the follo wing. This theorem is most often applied in the case where G is the trivial group, but Nash prov ed the general case in [12] and so shall we. Nash’s Theorem. A game with symmetry gr oup G has a G -invariant Nash e quilibrium. T o prov e this w e will use Hart and Sc hmeidler’s tec hniques in a new wa y . W e will sho w that certain classes of symmetric games ha ve correlated equilibria with a m uch higher degree of symmetry than migh t b e exp ected without kno wledge of Nash’s Theorem. T o illustrate what we mean, consider the follo wing trivial impro v ement on Theorem 2.16. Prop osition 2.26. A game with symmetry gr oup G has a G -invariant c orr elate d e quilibrium. Pr o of. Apply Prop osition 2.18 to a correlated equilibrium, whic h exists by Theorem 2.16. A priori w e migh t not expect correlated equilibria with a greater degree of symmetry than predicted b y Proposition 2.26 to exist. But viewing G -in v ariant Nash equilibria as correlated equilibria, we see that we can often guaran tee m uch more. Supp ose we hav e an n -play er game whic h has iden tical strategy sets for all play ers and whic h is symmetric under cyclic p erm utations of the pla yers, suc h as the game in Example 2.25. Then Prop osition 2.26 yields a correlated equilibrium π whic h is in v ariant under cyclic p erm utations of the play ers, but need not b e in v ariant under other p erm utations. On the other hand the Nash equilibrium ρ = ( ρ 1 , . . . , ρ n ) given by Nash’s Theorem satisfies ρ 1 = · · · = ρ n so the corresp onding pro duct distribution π ( s 1 , . . . , s n ) = ρ 1 ( s 1 ) · · · ρ 1 ( s n ) is a correlated equilibrium which is in v ariant under arbitrary p ermutations of the pla y ers. 3 Exc hangeable equilibria In this section w e pro v e the existence of correlated equilibria with this higher degree of symmetry , as well as a useful factorization prop erty , without app ealing to Nash’s Theorem. 3.1 Exchangeable distributions First we need the notion of an exchangeable probability distribution. Our usage of the term “exc hangeable” is slightly nonstandard but is closely related to the usual notion in the case when G acts play er transitively . Definition 3.1. Viewing ∆ Π G (Γ) as a noncon v ex subset of the conv ex set ∆ G (Γ), we define the set of G -exchangeable probabilit y distributions ∆ X G (Γ) := conv ∆ Π G (Γ) ⊆ ∆ G (Γ) . W e use the term “exchangeable” b ecause of the imp ortan t case where the C i are all equal and the group G acts pla yer transitiv ely (e.g. in Example 2.25). Then distributions in ∆ X G (Γ) are inv ariant under arbitrary p ermutations of the play ers. F urthermore, by De 11 Finetti’s Theorem 2 these are exactly the distributions which can b e extended to exc hangeable distributions on infinitely many copies of C 1 , i.e., distributions inv ariant under permutations of finitely many indices. De Finetti’s Theorem will not pla y a direct role in our analysis; here it merely serves to motiv ate Definition 3.1. T o get a feel for these sets, w e will lo ok at them in the context of some examples. Example 2.21 (cont’d) . Since G acts trivially w e can ignore it entirely . Not all distributions are indep endent so ∆ Π G (Γ) ( ∆ G (Γ) = ∆(Γ), but ∆ X G (Γ) = ∆ G (Γ). As we ha v e seen, one inclusion is automatic. T o prov e the reverse note that for an y s ∈ C , δ s = δ s 1 · · · δ s n ∈ ∆ Π (Γ) = ∆ Π G (Γ). But for any π ∈ ∆(Γ) w e can write π = P s ∈ C π ( s ) δ s , and such a con vex com bination of the δ s is in ∆ X G (Γ) by definition. Example 2.22 (con t’d) . F or a symmetric bimatrix game Γ with m strategies p er play er, w e can view probability distributions ov er C as m × m nonnegativ e matrices with en tries summing to unity . The nontrivial symmetry g ∈ G acts by sw apping the play ers. F rom the definitions w e see that ∆ G (Γ) consists of symmetric matrices and ∆ Π G (Γ) of matrices which are outer pro ducts xx T for nonnegative column v ectors x ∈ R m . The elemen ts of ∆ X G (Γ) = conv ∆ Π G (Γ) are exactly the (normalized) completely p ositive matrices studied in [1]. Clearly all such matrices are symmetric, element wise nonnegativ e, and p ositiv e semidefinite; it turns out the con v erse holds if and only if m ≤ 4 [3]. Example 2.24 (con t’d) . The map on C induced by h is the p ermutation (( H 1 , H 2 ) ( T 1 , H 2 ) ( T 1 , T 2 ) ( H 1 , T 2 )) . In particular, a G -in v ariant probability distribution m ust assign equal probability to all four outcomes in C 1 × C 2 . There is only one suc h distribution and it is indep enden t, so ∆ Π G (Γ) = ∆ X G (Γ) = ∆ G (Γ). Example 2.25 (con t’d) . Recall that in this game there are n play ers and the C i are the same for all i . The group G p ermutes the play ers cyclically . Therefore the elements of ∆ Π G (Γ) are in v ariant under arbitrary permutations of the pla yers, hence so are the elemen ts of ∆ X G (Γ). (The con v erse statement is false; that is to sa y , there are probability distributions ov er C whic h are inv ariant under arbitrary p ermutations of the play ers but are not in ∆ X G (Γ). This is analogous to the presence in Example 2.22 of symmetric elemen twise nonnegativ e matrices whic h are not p ositive semidefinite, hence not completely p ositiv e.) On the other hand, an elemen t of ∆ G (Γ) need only b e in v ariant under cyclic p erm utations of the play ers. W e close the section on exc hangeable distributions with a characterization of ∆ X G (Γ) whic h w e will not need until Section 4.3 but which logically b elongs here. The characterization is a corollary of a more general con vexit y lemma whic h we state first. Lemma 3.2. L et f : K → V b e a c ontinuous map fr om a c omp act set K to a finite- dimensional r e al ve ctor sp ac e V . Extending f by line arity yields a we akly c ontinuous map 2 De Finetti’s theorem states that the distribution of an infinite sequence of random v ariables X 1 , X 2 , . . . is inv ariant under p erm utations of finitely many of the random v ariables ( exchangeable ) if and only if it is a mixture of i.i.d. distributions [14]. 12 ∆( K ) → V given by π 7→ R f dπ which we also c al l f . The image of this map is f (∆( K )) = con v( f ( K )) . Pr o of. The extension f is weakly con tinuous by definition. Clearly f (∆( K )) is con vex and con tains f ( K ), so one containmen t is immediate. By linearity of integration, any linear inequalit y v alid on f ( K ) must b e v alid on f (∆( K )), so the reverse containmen t follo ws by a separating hyperplane argumen t (see Theorem 3 . 1 . 1 of [9] for details or Theorem 2 . 8 of [17] for an alternative top ological argumen t). Corollary 3.3. The line ar extension of the inclusion map ∆ Π G (Γ) → ∆ G (Γ) is we akly c on- tinuous and maps ∆(∆ Π G (Γ)) onto ∆ X G (Γ) . 3.2 Exchangeable equilibria W e are now ready to define exc hangeable equilibria. The pro ofs of the propositions in this section are direct algebraic manipulations and some are omitted. Definition 3.4. The set of G -exc hangeable equilibria of a game Γ is XE G (Γ) := CE(Γ) ∩ ∆ X G (Γ) . When G can b e inferred from context we simply refer to exchangeable equilibria . It is immediate from the definitions that conv(NE G (Γ)) ⊆ XE G (Γ) ⊆ CE G (Γ). There are examples in which all of these inclusions are strict [16], so pro ving non-emptiness of XE G (Γ) do es not immediately prov e non-emptiness of NE G (Γ). Nonetheless, this is an imp ortan t step and the main result of this section. The pro of that a G -exc hangeable equilibrium exists pro ceeds along the same lines as the correlated equilibrium existence pro of in Section 2.2. W e again consider the zero-sum game Γ 0 and prov e that a certain set is go o d in this game (Lemma 3.8). The difference is that the action of G yields a smaller go o d set, ∆ Π G (Γ). T o pro ve this lemma w e need the follo wing three symmetry results, which ha ve straigh tforw ard pro ofs. Prop osition 3.5. If G acts on Γ then G acts player trivial ly on Γ 0 by g · ( s, ( r i , t i )) = ( g · s, ( g · r i , g · t i )) . Prop osition 3.6. If G acts player trivial ly on a zer o-sum game, then a set Σ ⊆ ∆ G ( C M ) is go o d if and only if it is go o d against ∆ G ( C m ) . Pr o of. F or all g ∈ G , σ ∈ ∆ G ( C M ), and θ ∈ ∆( C m ) we hav e u M ( σ, θ · g ) = u M ( σ · g , θ · g ) = u M ( σ, θ ), so u M ( σ, θ ) = u M ( σ, av e G ( θ )). Prop osition 3.7. The map Mm( γ ( · )) is natur al in the sense that if σ : C i → C j is a bije ction and y i = y j ◦ ( σ, σ ) , then c omp osition with σ maps Mm( γ ( y j )) to Mm( γ ( y i )) . Lemma 3.8. If G acts on the game Γ then the set ∆ Π G (Γ) is go o d in the zer o-sum game Γ 0 of Definition 2.12. 13 Pr o of. By Prop osition 3.5 and Prop osition 3.6, it suffices to consider only y ∈ ∆ G ( C 0 m ), and sho w that there is a π ∈ ∆ Π G (Γ) whic h is go o d against y . Lemma 2.15 states that an y π ∈ S ( y ) := Mm( γ ( y 1 )) × · · · × Mm( γ ( y n )) ⊂ ∆ Π (Γ) is go o d against y . By Prop osition 3.7 the action of G on ∆ Π (Γ) restricts to a linear action of G on S ( y ) since y ∈ ∆ G ( C 0 m ). Viewing S ( y ) as a con v ex subset of R t i C i , Prop osition 2.18 sho ws the G -in v ariant subset S G ( y ) ⊆ ∆ Π G (Γ) is nonempty , so ∆ Π G (Γ) is go o d. Theorem 3.9. A game with symmetry gr oup G has a G -exchange able e quilibrium. Pr o of. By Theorem 2.16, Mm(Γ 0 ) = CE(Γ). Lemma 3.8 shows w e can apply Prop osition 2.8 to Γ 0 with Σ = ∆ Π G (Γ), proving that Mm(Γ 0 ) ∩ ∆ X G (Γ) = XE G (Γ) is nonempty . It is worth contrasting the pro of that symmetric correlated equilibria exist (Prop osi- tion 2.26) with this proof. Both in volv e a veraging arguments to pro duce symmetric solutions. The difference is that in the pro of of Prop osition 2.26 the a v eraging o ccurs within the set ∆(Γ), whereas in the case of Theorem 3.9 (in particular Lemma 3.8), the a v eraging occurs within ∆ Π (Γ), view ed as a conv ex subset of R t i C i . By a v eraging within this smaller set, w e guaran tee that the resulting correlated equilibrium will ha v e the additional symmetries discussed at the end of Section 2.3. The latter a v eraging argumen t requires a bit more care. In particular, Prop osition 2.26 is an immediate corollary of Theorem 2.16 on the existence of correlated equilibria. On the other hand, to prov e Theorem 3.9 we hav e to “lift the ho o d” on Theorem 2.16 and use Lemma 2.15 on go o d sets. By doing so w e exhibit a correlated equilibrium whic h we can pro v e lies in ∆ X G (Γ) instead of just ∆ G (Γ). 4 Higher order exc hangeable equilibria In this section w e b egin with a game Γ and artificially add symmetries to pro duce t wo families of games Π m Γ and Ξ m Γ with larger symmetry groups for each m ∈ N . Having constructed these games, we can exploit our kno wledge of their structure to improv e Theorem 3.9 and sho w that there are distributions which are sim ultaneously exc hangeable equilibria of b oth Π m Γ and Ξ m Γ. W e call such distributions order m exchangeable equilibria. W e then use a compactness argumen t to exhibit a distribution whic h is sim ultaneously an order m exchangeable equilibrium for all m ∈ N , called an order ∞ exchangeable equilibrium. W e will see in the next section that for pla y er-transitive symmetry groups, an order ∞ exc hangeable equilibrium is just a mixture of symmetric Nash equilibria. Most of the w ork in this section consists of making the prop er definitions. Once that is done, the pro ofs are rather short. 14 4.1 Po w ers of games T o define order m G -exchangeable equilibria w e will n eed t w o notions of a p ow er of a game Γ. These are larger games in whic h m ultiple copies of Γ are play ed sim ultaneously 3 . Throughout this section we will tak e as fixed a game Γ with symmetry group G and a num b er m ∈ N . Definition 4.1. F or m ∈ N , the m th p o w er of Γ, denoted Π m Γ, is a game in which m indep enden t copies of Γ are play ed simultaneously . More sp ecifically , Π m Γ has mn pla yers lab eled by pairs i, j , 1 ≤ i ≤ n , 1 ≤ j ≤ m , strategy spaces Π m C ij := C i for all i, j with generic element s j i , and utilities Π m u ij ( s 1 1 , . . . , s m n ) := u i ( s j 1 , s j 2 , . . . , s j n ). The con tracted m th p o w er of Γ, denoted Ξ m Γ, is a game in whic h m copies of Γ are play ed sim ultaneously , but all by the same set of pla y ers. Sp ecifically , Ξ m Γ has n pla y- ers, strategy spaces Ξ m C i := C m i with generic element ( s 1 i , . . . , s m i ) for all i , and utilities Ξ m u i ( s 1 1 , . . . , s m n ) := P j u i ( s j 1 , s j 2 , . . . , s j n ). Γ Π m Γ Ξ m Γ s 2 s 1 s 1 2 s 2 2 · · · s m 2 s 1 1 s 2 1 · · · s m 1 s 1 2 s 2 2 · · · s m 2 s 1 1 s 2 1 · · · s m 1 Figure 1: Representing a 2-pla y er game Γ with pla y ers c ho osing strategies s 1 and s 2 as dra wn on the left, the p ow ers Π m Γ and Ξ m Γ are formed as s ho wn. A shaded b ox represen ts actions con trolled b y a single play er, whose utility is giv en b y the sum ov er all interactions. Prop osition 4.2. L et Γ b e a game with symmetry gr oup G and fix m ∈ N . Then b oth p owers Π m Γ and Ξ m Γ ar e games with symmetry gr oup G × S m and they satisfy: • ∆ G × S m (Π m Γ) = ∆ G × S m (Ξ m Γ) • ∆ Π G × S m (Π m Γ) ( ∆ Π G × S m (Ξ m Γ) • ∆ X G × S m (Π m Γ) ⊆ ∆ X G × S m (Ξ m Γ) . Pr o of. Both p o wers are inv ariant under arbitrary p erm utations of the copies and under symmetries in G applied to all of the copies sim ultaneously . In fact in the case of Π m Γ w e can apply a differen t symmetry in G to each cop y indep enden tly so that Π m Γ is inv ariant 3 These definitions can b e generalized to define tw o p ossible pro ducts of games, so that the p o wers we define reduce to m -fold pro ducts of a game with itself. If we remov e all mention of symmetry groups (so in particular exchangeable distributions and exchangeable equilibria are no longer defined) all the statements w e make ab out Nash and correlated equilibria of these p ow ers extend to corresponding statements ab out pro ducts with identical pro ofs. W e will not need this level of generalit y , how ever, so to av oid complicating notation we fo cus on p o wers. 15 under the larger group G o S m (the wreath pro duct of G and S m ), but we will not need this fact. Since G × S m acts on Π m C and Ξ m C in the same w a y , we get the first equalit y . The game Π m Γ has more play ers than Ξ m Γ, so ∆ Π G × S m (Π m Γ) has stronger indep endence conditions than ∆ Π G × S m (Ξ m Γ), yielding the strict containmen t. T aking con v ex h ulls giv es the final con tainmen t. Since ∆ G × S m (Π m Γ) = ∆ G × S m (Ξ m Γ), w e can compare the conditions for a distribution π to b e a correlated equilibrium of Π m Γ or Ξ m Γ. W e use the notation and terminology in tro duced for Prop osition 2.5 to do so. Prop osition 4.3. L et ( X 1 1 , . . . , X m n ) b e a r andom ve ctor taking values in C m distribute d ac c or ding to π ∈ ∆ G × S m (Π m Γ) = ∆ G × S m (Ξ m Γ) . Then • π is a c orr elate d e quilibrium of Π m Γ if and only if X j i is a b est r esp onse (in Γ ) to P ( X j − i | X j i ) almost sur ely for al l i and j , and • π is a c orr elate d e quilibrium of Ξ m Γ if and only if X j i is a b est r esp onse (in Γ ) to P ( X j − i | X 1 i , . . . , X m i ) almost sur ely for al l i and j . Pr o of. By Prop osition 2.5, π is a correlated equilibrium of Π m Γ if and only if X j i is a b est resp onse in Π m Γ to P ( X 1 , . . . , X j − 1 , X j − i , X j +1 , . . . , X m | X j i ) almost surely for all i and j . But the utility of play er ij in Π m Γ is u i ( X j 1 , . . . , X j n ), so play er ij can ignore X l k for all l 6 = j . No w w e consider when π is a correlated equilibrium of Ξ m Γ. Again b y Proposition 2.5, this happ ens if and only if ( X 1 i , . . . , X m i ) is a b est resp onse in Ξ m Γ to P ( X 1 − i , . . . , X m − i | X 1 i , . . . , X m i ) almost surely for all i . The utility of play er i in Ξ m Γ is P j u i ( X j 1 , X j 2 , . . . , X j n ) and no X j i app ears in more than one term of this sum. Thus the sum is maximized when eac h term is maximized indep enden tly . That is to say π ∈ CE G × S m (Ξ m Γ) if and only if X j i is a b est resp onse in Γ to P ( X j − i | X 1 i , . . . , X m i ) almost surely for all i and j . This c haracterization allows us to prov e the follo wing containmen ts b etw een equilibrium sets. One can construct examples sho wing that in general none of the inclusions in this prop osition can b e rev ersed. In particular, no containmen t holds b et w een XE G × S m (Π m Γ) and XE G × S m (Ξ m Γ) in either direction. This is connected to the fact that the inclusion b et w een the sets of correlated equilibria of Π m Γ and Ξ m Γ go es in the opp osite direction from the inclusion b etw een the sets of Nash equilibria. Prop osition 4.4. The e quilibria of Π m Γ and Ξ m Γ satisfy NE G × S m (Π m Γ) ⊆ NE G × S m (Ξ m Γ) ⊆ CE G × S m (Ξ m Γ) ⊆ CE G × S m (Π m Γ) . Pr o of. W e use Proposition 4.3. If X j i are distributed according to π ∈ ∆ Π G × S m (Π m Γ) then the X j i are all indep enden t, so the conditional distributions in Prop osition 4.3 are equal to the corresp onding unconditional distributions and b oth conditions are equiv alen t. This prov es the first containmen t. 16 The second con tainment follo ws b ecause Nash equilibria are alw ays correlated equilibria. F or the third con tainmen t, supp ose X j i is a b est resp onse to P ( X j − i | X 1 i , . . . , X m i ) almost surely . Summing o v er p ossible v alues of X − j i w e get that X j i is a b est resp onse to P ( X j − i | X j i ) almost surely . F or any 1 ≤ p < m we can define a pro jection map pro j : ∆ G × S m (Π m Γ) → ∆ G × S p (Π p Γ) whic h marginalizes out v ariables X p +1 , . . . , X m . This map respects the structure of all the sets of distributions mentioned in Prop osition 4.2, i.e., it restricts to maps ∆ Π G × S m (Π m Γ) → ∆ Π G × S p (Π p Γ), ∆ X G × S m (Ξ m Γ) → ∆ X G × S p (Ξ p Γ), etc. By Prop osition 4.3 it also respects the equi- librium structure of these games in the sense that XE G × S m (Π m Γ) maps into XE G × S p (Π p Γ), NE G × S m (Ξ m Γ) maps into NE G × S p (Ξ p Γ), etc. One can sho w that if p = 1 then all of these maps are on to. W e use this fact only to motiv ate what follo ws and not in an y of the argumen ts b elo w, so w e omit its pro of. In particular, this sho ws that pro j(XE G × S m (Π m Γ)) = XE G (Γ) = pro j(XE G × S m (Ξ m Γ)). One can give an example in whic h con v(NE G (Γ)) ( XE G (Γ) [16], so neither of these pro jected sets of exc hangeable equilibria approac hes the con v ex hull of the Nash equilibria of Γ as m gets large. W e will see that taking the intersection XE G × S m (Π m Γ) ∩ XE G × S m (Ξ m Γ) fixes this problem. 4.2 Order m G -exc hangeable equilibria By Prop ositions 4.2 and 4.4 w e ha v e NE G × S m (Π m Γ) ⊆ XE G × S m (Π m Γ) ∩ XE G × S m (Ξ m Γ). Th us w e exp ect the following definition not to b e v acuous. Definition 4.5. The set of order m G -exc hangeable equilibria of Γ is XE m G (Γ) := XE G × S m (Π m Γ) ∩ XE G × S m (Ξ m Γ) , or equiv alen tly b y Prop ositions 4.2 and 4.4, XE m G (Γ) := ∆ X G × S m (Π m Γ) ∩ CE G × S m (Ξ m Γ) . The relationship b etw een XE m G (Γ) and the sets of equilibria of the m th p o w ers is summa- rized in Figure 2. W e no w pro ve order m G -exchangeable equilibria exist. Lemma 4.6. If G acts on the game Γ then the set ∆ Π G × S m (Π m Γ) is go o d in the zer o-sum game (Ξ m Γ) 0 of Definition 2.12. Pr o of. By Lemma 3.8, Σ := ∆ Π G × S m (Ξ m Γ) is go o d. The utilities in Ξ m Γ are additiv ely separable, so an y mixed strategy σ ∈ Σ is pay off equiv alen t for the maximizer in (Ξ m Γ) 0 to a mixed strategy σ 0 ∈ Σ 0 := ∆ Π G × S m (Π m Γ) giv en b y the pro duct of the marginals of σ . W e can apply Prop osition 2.10 to Σ and Σ 0 . Theorem 4.7. A game with symmetry gr oup G has an or der m G -exchange able e quilibrium for al l m ∈ N . 17 NE G × S m (Π m Γ) NE G × S m (Ξ m Γ) XE m G (Γ) XE G × S m (Π m Γ) XE G × S m (Ξ m Γ) CE G × S m (Π m Γ) CE G × S m (Ξ m Γ) pro j NE G (Γ) pro j(XE m G (Γ)) XE G (Γ) CE G (Γ) Figure 2: At the left is a summary of the containmen ts b etw een equilibrium sets of the p o w ers Π m Γ and Ξ m Γ prov en in Section 4.1. An arrow A  → B indicates A ⊆ B . Under the marginalization map pro j : ∆ G × S m (Π m Γ) → ∆ G (Γ) each of these sets maps onto the set at the same height on the righ t. Pr o of. By Theorem 2.16, Mm((Ξ m Γ) 0 ) = CE(Ξ m Γ). Lemma 4.6 shows we can apply Prop o- sition 2.8 to (Ξ m Γ) 0 with Σ = ∆ Π G × S m (Π m Γ), so Mm((Ξ m Γ) 0 ) ∩ ∆ X G × S m (Π m Γ) = XE m G (Γ) is nonempt y . F or 1 ≤ p < m , the marginalization map sends XE G × S m (Π m Γ) in to XE G × S p (Π p Γ) and XE G × S m (Ξ m Γ) in to XE G × S p (Ξ p Γ). Therefore it sends XE m G (Γ) in to XE p G (Γ). Pro jecting the order m exchangeable equilibria in to ∆ G (Γ) for all m ∈ N we obtain NE G (Γ) ⊆ conv(NE G (Γ)) ⊆ · · · ⊆ pro j(XE 3 G (Γ)) ⊆ pro j(XE 2 G (Γ)) ⊆ XE G (Γ) ⊆ CE G (Γ) . This raises tw o natural questions: can w e prov e directly that T ∞ m =1 pro j(XE m G (Γ)) is non- empt y? and do es this intersection equal con v(NE G (Γ))? W e will tak e up these tw o questions, resp ectiv ely , in the following t wo sections. 4.3 Order ∞ G -exc hangeable equilibria Next we use a compactness argumen t to prov e existence of an order ∞ G -exchangeable equilibrium, a distribution whic h is in some sense an order m G -exc hangeable equilibrium for all finite m . As w e ha ve defined them the XE m G (Γ) are distributions ov er differen t num b ers of copies of C , so they are not directly comparable and w e can’t just take their intersection. W e could pro ject them all into ∆ G (Γ) and take the in tersection there as men tioned ab o v e, but this w ould destroy some structure. Analytically it will b e more conv enien t to view these sets within a larger space. 18 T o tak e the intersection properly , we will transport all the XE m G (Γ) in to ∆(∆ Π G (Γ)). Since S m acts transitiv ely on the copies of the game in Π m Γ, an element ρ ∈ ∆ Π G × S m (Π m Γ) satisfies ρ j i = ρ k i for all i , j , and k . Therefore the diagonal map ∆ Π G (Γ) → ∆ Π G × S m (Π m Γ) which sends a distribution to the pro duct of m indep enden t copies of itself is a homeomorphism. This extends to a homeomorphism ∆(∆ Π G (Γ)) → ∆(∆ Π G × S m (Π m Γ)) of the corresp onding spaces of distributions. If we comp ose this with the surjective map ∆(∆ Π G × S m (Π m Γ)) → ∆ X G × S m (Π m Γ) giv en b y Corollary 3.3, we get a surjectiv e map µ m : ∆(∆ G (Γ)) → ∆ X G × S m (Π m Γ). Define the in verse image sets A m = µ − 1 m (XE m G (Γ)). Elements of A m are represen tations of order m G -exchangeable equilibria as mixtures of indep endent G -in v ariant distributions. Definition 4.8. The set of order ∞ G -exchangeable equilibria is XE ∞ G (Γ) := T ∞ m =1 A m . Theorem 4.9. A game with symmetry gr oup G has an or der ∞ G -exchange able e quilibrium. Pr o of. Each set XE m G (Γ) is conv ex and closed b y definition and nonempty by Theorem 4.7. The map µ m is linear, w eakly con tinuous, has a compact Hausdorff domain, and is surjectiv e. Therefore each A m is conv ex, compact, and nonempty . F or 1 ≤ p < m , the comp osition µ p ◦ µ − 1 m is a well-defined map ∆ X G × S m (Π m Γ) → ∆ X G × S p (Π p Γ) whic h coincides with the marginalization map discussed ab o ve. This map sends XE m G (Γ) into XE p G (Γ) as p er the discussion at the end of Section 4.2, so the A m are nested A 1 ⊇ A 2 ⊇ A 3 ⊇ . . . . Thus they ha v e nonempty , con v ex, compact in tersection XE ∞ G (Γ). 5 Nash’s Theorem 5.1 The pla y er-transitive case Theorem 5.1. If G acts player tr ansitively on Γ , then XE ∞ G (Γ) = ∆(NE G (Γ)) . Pr o of. If σ ∈ NE G (Γ) then µ m ( δ σ ) ∈ NE G × S m (Π m Γ) ⊆ XE m G (Γ), so δ σ ∈ A m for all m and δ σ ∈ XE ∞ G (Γ). But XE ∞ G (Γ) is conv ex and weakly closed, so ∆(NE G (Γ)) = con v { δ σ | σ ∈ NE G (Γ) } ⊆ XE ∞ G (Γ). F or the conv erse let R b e a rand om v ariable taking v alues in ∆ Π G (Γ) distributed according to π ∈ XE ∞ G (Γ). Let X j i , 1 ≤ i ≤ n , 1 ≤ j < ∞ , b e random v ariables taking v alues in C i with distribution R i whic h are conditionally indep enden t giv en R . W e m ust show that R i is a b est resp onse to R − i almost surely . W e will do this by appro ximating R i and R − i in terms of the X j i . F or each k ∈ N the finite collection of random v ariables X j i with j ≤ k is distributed according to µ k ( π ) b y construction; here w e implicitly use the fact that µ k ( π ) is an order ∞ G -exc hangeable equilibrium, so µ k ( π ) ∈ ∆ X G × S m (Π m Γ). F urthermore we hav e µ k ( π ) ∈ CE G × S k (Ξ k Γ), so Prop osition 4.3 states that for an y 1 ≤ j ≤ k the strategy X j i is a b est resp onse to the random conditional distribution P ( X j − i | X 1 i , . . . , X k i ) almost surely . Since µ k ( π ) is symmetric, P ( X j − i | X 1 i , . . . , X k i ) ≡ P ( X 1 − i | X 1 i , . . . , X k i ) for all i , j , and k . W e define P k i to b e this common random conditional distribution. Let Y j i b e the random 19 v ariable taking v alues in ∆( C i ) which is the empirical distribution of X 1 i , . . . , X j i . Then Y j i is a b est resp onse to P k i whenev er j ≤ k . W e will sho w that Y j i and P k i con v erge to R i and R − i , resp ectively , as j and k go to infinit y . Let Σ i b e the completion of the σ -algebra generated b y X 1 i , X 2 i , . . . and define P ∞ i := P ( X 1 − i | Σ i ). Then P k i → P ∞ i almost surely as k go es to infinity (Theorem 10 . 5 . 1 in [4]). Therefore Y j i is a b est resp onse to P ∞ i for all j . By the strong law of large n umbers, Y j i con v erges almost surely to R i as j go es to infinity , so R i is a b est resp onse to P ∞ i . F urthermore, R i is measurable with resp ect to Σ i b ecause the Y j i are. The X j i are conditionally indep enden t given R , so we ha v e P ∞ i = E ( P ( X 1 − i | R ) | Σ i ). Since G acts play er transitiv ely , for an y play er j we ha v e R j = R i · g for some g ∈ G , hence R j is measurable with resp ect to Σ i and so is R . In particular P ( X 1 − i | R ) is measurable with resp ect to Σ i and we obtain P ∞ i = E ( P ( X 1 − i | R ) | Σ i ) = P ( X 1 − i | R ) = R − i . This shows that R i is a b est resp onse to R − i almost surely for all i , so R ∈ NE G (Γ) almost surely and π ∈ ∆(NE G (Γ)). If G is the trivial group one can sho w that µ 1 (XE ∞ G (Γ)) = CE(Γ) and µ 1 (∆(NE G (Γ))) = con v(NE(Γ)). These sets are differen t for some games (e.g., c hic k en), so the abov e theorem can fail without the play er-transitivity assumption. Nash’s Theorem (play er-transitive case) . A game with player-tr ansitive symmetry gr oup G has a G -invariant Nash e quilibrium. Pr o of. Combine Theorems 4.9 and 5.1, noting that ∆( ∅ ) = ∅ . 5.2 Arbitrary symmetry groups In this section we sho w how to embed an arbitrary game Γ with symmetry group G in a game Γ Sym with a play er-transitiv e symmetry group, preserving the existence of G -in v ariant Nash equilibria. This allo ws us to drop the play er-transitivit y assumption from the previous section, proving Nash’s Theorem in full generality . There are a v ariety of w a ys to symmetrize games. The one we ha ve c hosen is a natural n -pla y er generalization of von Neumann’s tensor-sum symmetrization discussed in [6]. The idea is that eac h of the n play ers in Γ Sym pla ys all the roles of the pla y ers in Γ sim ultaneously . The play ers in Γ Sym pla y n ! copies of Γ, one for eac h assignment of pla yers in Γ Sym to roles in Γ. A play er’s utility in Γ Sym is the sum of his utilities o ver the copies. Definition 5.2. Giv en an n -pla y er game Γ with strategy sets C i and utilities u i w e define its symmetrization Γ Sym to b e the n -pla yer game with strategy sets C Sym i := C (with typical strategy s i = ( s i 1 , . . . , s i n )) and utilities u Sym i ( s ) := X τ ∈ S n u τ ( i ) ( d ( τ  s )) , 20 where s = ( s 1 , . . . , s n ) ∈ C Sym = C n ,  : S n × C Sym → C Sym is defined by ( τ  s ) k := s τ − 1 ( k ) , and d : C Sym → C is defined by [ d ( s )] k := s k k . W e no w show that Γ Sym is a game with play er-transitive symmetry group. W e will use  to denote the action on Γ Sym to distinguish it from the action · on Γ. Prop osition 5.3. If Γ is a game with symmetry gr oup G then Γ Sym is a game with player- tr ansitive symmetry gr oup G × S n , wher e σ ∈ S n acts by  as define d ab ove and g ∈ G acts by g  ( s 1 , . . . , s n ) 7→ ( g · s 1 , . . . , g · s n ) . Pr o of. Note that  defines an action of G on C Sym . Also, for σ, τ ∈ S n w e ha ve ( τ  ( σ  s )) k = ( σ  s ) τ − 1 ( k ) = s σ − 1 ( τ − 1 ( k )) = s ( τ σ ) − 1 ( k ) = (( τ σ )  s ) k , so  is an action of S n on C Sym as well. These actions comm ute, so together they define an action  of G × S n on C Sym . Note that the induced actions on play ers are g  i = i and σ  i = σ ( i ). T o show that this is an action of G × S n on Γ Sym it suffices to sho w that the utilities of Γ Sym are inv ariant under the action of an y σ ∈ S n and any g ∈ G . T o see the former, let σ ∈ S n . Then we hav e u Sym σ ?i ( σ  s ) = X τ ∈ S n u τ ( σ ( i )) ( d ( τ  ( σ  s ))) = X τ ∈ S n u ( τ σ )( i ) ( d (( τ σ )  s ) = X τ ∈ S n u τ ( i ) ( d ( τ  s )) = u Sym i ( s ) , where we ha ve used in the p en ultimate equation the fact that S n is a group, so the map τ 7→ τ σ is a bijection. T o see the latter, let g ∈ G and let γ ∈ S n b e the permutation induced b y g on the set of pla yers in Γ. Then we hav e d ( g  s ) = g · d ( γ − 1  s ), so u Sym g ?i ( g  s ) = X τ ∈ S n u τ ( i ) ( d ( τ  ( g  s ))) = X τ ∈ S n u τ ( i ) ( d ( g  ( τ  s ))) = X τ ∈ S n u τ ( i )  g · d ( γ − 1  ( τ  s ))  = X τ ∈ S n u ( γ − 1 τ )( i )  d (( γ − 1 τ )  s )  = X τ ∈ S n u τ ( i ) ( d ( τ  s )) = u Sym i ( s ) , where the fourth equation follows b ecause g is a symmetry of Γ. Clearly S n acts transitively on the set of play ers. Nash’s Theorem. A game with symmetry gr oup G has a G -invariant Nash e quilibrium. Pr o of. Let Γ b e a game with symmetry group G . Then Γ Sym is a game with pla y er-transitiv e symmetry group G × S n b y Prop osition 5.3, so it has a ( G × S n )-symmetric Nash equilibrium b y the play er-transitive version of Nash’s Theorem. By definition of the action of G × S n 21 on Γ Sym , this Nash equilibrium is of the form ( ρ, . . . , ρ ), with ρ ∈ ∆ G (Γ). Notice that for each pla y er i , each utilit y u Sym k ( s 1 , . . . , s n ) is a sum of functions which only dep end on s i j for a single v alue of j . Th us ρ is pay off equiv alen t to the pro duct of its marginals ρ 1 × · · · × ρ n ∈ ∆ Π G (Γ). Therefore we can take the Nash equilibrium ( ρ, . . . , ρ ) to b e such that ρ ∈ ∆ Π G (Γ) by Prop osition 2.11. It remains to verify that ρ ∈ NE G (Γ). F or any s i ∈ C w e can compute u Sym i ( ρ, . . . , ρ, s i , ρ, . . . , ρ ) = X τ ∈ S n u τ ( i )  ρ 1 , . . . , ρ τ ( i ) − 1 , s i τ ( i ) , ρ τ ( i )+1 , . . . , ρ n  = ( n − 1)! n X j =1 u j ( ρ 1 , . . . , ρ j − 1 , s i j , ρ j +1 , . . . , ρ n ) . F or each v alue of j we can v ary the s i j comp onen t of s i indep enden tly and it is a best resp onse for play er i to play ρ in Γ Sym if the rest of the pla yers pla y ρ , so we must ha ve u j ( ρ 1 , . . . , ρ j − 1 , s j , ρ j +1 , . . . , ρ n ) ≤ u j ( ρ ) for all play ers j and all s j ∈ C j , i.e., ρ ∈ NE G (Γ). 6 Conclusion W e hav e sho wn that by studying group actions on games and introducing the notion of exc hangeable equilibrium, we can prov e Nash’s Theorem. T o the authors’ kno wledge, this is the first pro of of this theorem whic h uses conv exity-based metho ds (i.e., the minimax theorem). Previous pro ofs use path-follo wing arguments or fixed-p oin t theorems, whic h are essen tially equiv alent to path-following arguments b y Sp erner’s Lemma. This new pro of in vites new approaches for computing or appro ximating Nash equilibria. One can rewrite the existence pro of ab o ve for (order m ) exc hangeable equilibria in terms of linear programs and separation arguments instead of the Minimax Theorem and apply the ellipsoid algorithm, just as Papadimitriou has done for Hart and Sc hmeidler’s pro of of the existence of correlated equilibria [13]. This sho ws that exc hangeable equilibria can b e computed in p olynomial time, at least under some assumptions on the parameters. F or example, order m exc hangeable equilibria of symmetric bimatrix games can b e computed in p olynomial time for fixed m . W e hav e seen that in the play er-transitiv e case order m exc hangeable equilibria conv erge to conv ex com binations of Nash equilibria as m go es to infinit y . There are a v ariety of w a ys one could imagine “rounding” exchangeable equilibria to try to pro duce approximate Nash equilibria. W e lea ve the analysis of such pro cedures, along with the question of whic h assumptions on G allow computation of exchangeable equilibria in p olynomial time, for future work. The p o wer of these metho ds suggests that exchangeable equilibria should not merely b e viewed as a step on the w ay to Nash equilibria. Rather, they deserve further study in their o wn right. W e consider sev eral interpretations of exchangeable equilibria and the applications they suggest in [16]. 22 References [1] A. Berman and N. Shaked-Monderer. Completely p ositive matric es . W orld Scientific Publishing Co. Pte. Ltd., River Edge, NJ, 2003. [2] D. P . Bertsek as, A. Nedi ´ c, and A. E. Ozdaglar. Convex A nalysis and Optimization . A thena Scien tific, Belmon t, MA, 2003. [3] P . H. Diananda. On non-negativ e forms in real v ariables some or all of whic h are non- negativ e. Mathematic al Pr o c e e dings of the Cambridge Philosophic al So ciety , 58(1):17 – 25, January 1962. [4] R. M. Dudley . R e al Analysis and Pr ob ability . Cambridge Universit y Press, New Y ork, 2002. [5] D. Gale. The game of Hex and the Brou wer fixed-p oin t theorem. The Americ an Math- ematic al Monthly , 86(10), December 1979. [6] D. Gale, H. W. Kuhn, and A. W. T uc k er. On symmetric games. In Contributions to the The ory of Games , v olume 1, pages 81 – 87. Princeton Universit y Press, Princeton, NJ, 1950. [7] S. Hart and D. Sc hmeidler. Existence of correlated equilibria. Mathematics of Op er ations R ese ar ch , 14(1), F ebruary 1989. [8] A. Hatcher. Algebr aic T op olo gy . Cambridge Universit y Press, New Y ork, NY, 2002. [9] S. Karlin. Mathematic al Metho ds and The ory in Games, Pr o gr amming, and Ec onomics , v olume 2: Theory of Infinite Games. Addison-W esley , Reading, MA, 1959. [10] Elon Kohlb erg and Jean-F rancois Mertens. On the strategic stability of equilibria. Ec onometric a , 54(5):1003 – 1037, Septem b er 1986. [11] C. E. Lemke and J. T. Ho wson, Jr. Equilibrium p oints in bimatrix games. SIAM Journal on Applie d Math , 12:413 – 423, 1964. [12] J. F. Nash. Non-co op erative games. Annals of Mathematics , 54(2):286 – 295, September 1951. [13] C. H. P apadimitriou. Computing correlated equilibria in multi-pla yer games. In Pr o- c e e dings of the 37th A nnual ACM Symp osium on The ory of Computing (STOC) , New Y ork, NY, 2005. ACM Press. [14] M. J. Schervish. The ory of Statistics . Springer-V erlag, New Y ork, 1995. [15] E. Sp erner. Neuer Bew eis f ¨ ur die In v arianz der Dimensionszahl und des Gebietes. A b- hand lungen aus dem Mathematischen Seminar der Universit¨ at Hambur g , 6(1):265 – 272, Decem b er 1928. 23 [16] N. D. Stein, A. Ozdaglar, and P . A. P arrilo. Exc hangeable equilibria, part I: the sym- metric bimatrix case. In preparation. [17] N. D. Stein, A. Ozdaglar, and P . A. P arrilo. Separable and low-rank contin uous games. International Journal of Game The ory , 37(4):475 – 504, Decem b er 2008. [18] Mark V o orneveld. Preparation. Games and Ec onomic Behavior , 48:403 – 414, 2004. 24