Semidefinite Programming for Min-Max Problems and Games

SEMIDEFINITE PR OGRAMMING F OR MIN-MAX PR OBLEMS AND GAMES R. LARAKI AND J.B. LASSERRE Abstract. W e consider t w o min-max problems: (1) minimizing the supre- mum of finitely man y rational f unctions o ve r a compact basic semi- algebraic set and (2) solving a 2-play er zero-sum p olynomial game in randomized strate- gies with compact basic semi-algebraic s ets of pure strategies. In b oth problems the optimal v alue can be appro ximated by solving a hierarch y of semidefinite relaxations, in the spir it of the momen t approac h developed in Lasserre [24, 26]. This provides a unified approach and a class of algorithms to compute Nash equilibria and min-max strategies of several static and dynamic games. Eac h semidefinite r el axation can b e solv ed in time which is p ol ynomial in its input size and practice on a sample of experim en ts rev eals that few relaxations are needed for a go o d appro ximation (and sometimes ev en for finite con ve rgence), a behavior simi lar to what was observ ed in polynomial optimization. 1. In troduction Initially , this pap er was motiv ated by dev eloping a unified metho dolo gy for solv- ing several types o f (neither necessarily finite nor zero -sum) N -play er ga mes. But it is a lso o f self-interest in optimizatio n for minimizing the maximum of finitely many r ational functions (whence min-max) on a co mpact ba s ic semi-alg ebraic set. Briefly , the moment-s.o.s. appro a ch developed in [24, 26] is extended to t wo large classes of min-max problems. This allows to obtain a new n umerical scheme ba sed on semidefinite progr amming to co mpute approximately and sometimes exactly (1) Nash equilibria a nd the min-max of any finite game 1 and (2) the v a lue and the optimal stra teg ies of any poly nomial tw o-play er zero -sum game 2 . In par ticular, the approach can b e applied to the so-called Lo omis g ames defined in [32] and to some dynamic ga mes describ ed in Kolhber g [21]. Bac kground. Nas h equilibrium [33] is a central co ncept in non-c o o per ative game theory . It is a pr ofile of mixed strategie s (a strateg y for each player) such that each play er is b est-resp onding to the strategies of the opp onents. An important problem is to compute numerically a Nash equilibr ium, a n approximate Nash e q uilibrium for a given pr ecision or all Nash equilibria in mixed s trategies of a finite game. Key wor ds and phr ases. N -play er games; Nash equlibria; m in-max optimization problems; semidefinite programming. W e would like to thank Bernhard von Steng el and the referees f or their commen ts. The w ork of J.B. Lasserre w as supp orted by the (F rench) ANR under grant NT05 − 3 − 41612. 1 Games with finitely many pla y ers where the set of pure actions of each pla y er is finite. 2 Games with tw o pla y ers where the set of pure actions of eac h pla ye r is a compact basic semi-algebraic-set and the pa yoff f unction is polynomial. 1 2 R. LARAKI AND J.B. LASSERRE It is well known that an y tw o-play er zer o-sum finite ga me (in mixed strategie s ) is reducible to a linear prog ram (Dant zig [5]) and hence equilibria could be computed in p olynomial time (K hachiy an [20]). Lemke and Howson [29] pr ovided a famous algo rithm that co mputes a Nash- equilibrium (in mixed strateg ies) of any 2- play er non- z er o-sum finite g ame. The algorithm ha s b een extended to N -play er finite games b y Rosenm ¨ uller [39], Wilso n [50] and Govindan and Wilson [12]. An alternative to the Lemke-Howson algo rithm for 2- play er games is provided in v an den Elz e n and T alman [9] and has b een extended to n -play er g ames by Herings and v an den E lzen [16]. As s hown in the recent survey of Herings a nd Peeters [17], all these a lgorithms (including the Lemke-Howson) are homotopy-based and conv erge (only) when the ga me is non-degenera te. Recently , Sav ani and von Stengel [41] prov ed that the Lemk e-Howson algor ithm for 2-play er games may be ex po ne ntial. One may exp ect that this result extends to all known homo top y metho ds. Dask alakis, Goldb erg and Papadimitriou [4] prov e d that solving numerically 3 -play er finite games is hard 3 . The result ha s been e x - tended to 2-player finite games b y Chen and Deng [6]. Hence, c o mputing a Nash equilibrium is as hard as finding a Bro u wer-fixed p oint. F or a r ecent and deep sur- vey on the complexity of Nash equilibr ia see Papadimitriou [35]. F or the co mplexit y of computing equilibria on game theory in g eneral, see [34] and [40]. A different approach to s olve the problem is to vie w the set o f Nash equilibria a s the set o f real no nneg ative solutio ns to a s ystem of poly nomial equations. Metho ds of computational alg e bra (e.g. using Gr¨ obner bases) c a n b e applied as s uggested and studied in e.g . Datta [8], Lipton [30] and Sturmfels [47]. How ev er, in this approach, one fir st computes al l complex solutions to sort out all real no nneg ative solutions afterwards. In terestingly , po lynomial equations ca n also b e solved via homotopy-based metho ds (see e.g . V erschelde [48]). Another imp ortant co ncept in game theory is the min-max pay off of some play er i , v i . This is the level at which the team o f play ers (other than i ) ca n punish play er i . The concept plays an impo r tant r ole in rep eated games and the famous folk theor em of Aumann and Sha pley [2]. T o our k nowledge, no algor ithm in the litera ture deals with this problem. How ev er, it has b een prov ed recently that computing the min-max for 3 or more player g ames is NP-hard [3]. The alg orithms describ ed ab ov e concer n finite games. In the c lass of p olyno mial games introduced by Dres her, Kar lin and Shapley (1950 ), the set o f pure s tr ategies S i of player i is a pr o duct of r eal interv als and the pay off function of each play er is po lynomial. When the game is zer o-sum a nd S i = [0 , 1] for each player i = 1 , 2, Parrilo [37] show ed that finding an optimal solution is eq uiv alent to solving a sing le semidefinite pr ogram. In the sa me framework but with several play ers, Stein, Parrilo and Ozdag lar [4 6] propo se sev eral algorithms to c ompute correlated equilibria, one among them using SDP rela x ations. Shah and Parrilo [44] extended the metho dology in [3 7] to disc o un ted zer o-sum sto chastic games in which the transition is controlled by o ne of the tw o pla yers. Finally , it is worth no ticing r ecent algorithms designed to solve some sp ecific classe s of infinite g ames (no t nec essarily po lynomial). F or instance, G ¨ u rk an and Pang [13]. 3 More precisely , it is complete in the PP AD class of all searc h problems that are guarant eed to exist b y means of a direct graph argument . This class was introduced b y Papadimitriou in [36] and is betw een P and N P . SEMIDEFINITE PROGRAMMING FOR MIN-MAX PROBLEMS AND GAMES 3 Con tribution. In the fir s t part w e consider wha t we call the MRF proble m which consists of minimizing the s upr em um of finitely many r ational functions over a com- pact ba sic semi-alg ebraic set. In the spirit o f the moment appro a ch develop ed in Lasserr e [24, 26] for po lynomial o ptimization, we define a hierarch y of semidefi- nite r elaxations (in shor t SDP r e la xations). E ach SDP relax ation is a semidefinite progra m which, up to arbitrary (but fixed) precisio n, can be so lv ed in po lynomial time and the monotone sequence of optimal v alues ass o ciated with the hierar chy conv erges to the optimal v a lue of MRF . So metimes the conv ergence is finite a nd a sufficient co nditio n p ermits to detect whether a certa in relaxa tion in the hierar ch y is exact (i.e. provides the optimal v a lue), a nd to extr act optimal solutions. It is shown that computing the min- ma x o r a Nash equilibrium in mixed stra tegies for static finite g ames or dynamic finite absorbing g ames reduces to solv ing an MRF problem. F or zero- sum finite games in mixed strategies the hierarch y of SDP re- laxations for the asso ciated MRF r educes to the firs t one of the hier arch y , which in turn reduces to a linear pro gram. This is in supp ort of the claim tha t the above MRF formulation is a natural extensio n to the non linear ca se of the well-known LP-appro ach [5] as it reduces to the latter for finite zero- s um ga mes . In addition, if the SDP solver uses prima l- dual interior p o in ts metho ds and if the conv ergence is finite then the algorithm returns al l Nash equilibria (if of cour se there ar e finitely many). T o compute all Nash equilibria, homotopy algor ithms are developed in K ostrev a and K inard [2 2] a nd Herings and Peeters [18]. They apply numerical techniques to obtain all solutions o f a system of po lynomial equa tions. In the second pa r t, we co nsider general 2-play er zero- sum p olynomial games in mixed strategies (whose action sets are basic co mpa ct semi-algebra ic sets of R n and pay off functions are p olynomials). W e show that the co mmon v alue of max-min and min-max problems can be a pproximated as closely as desired, again by solving a cer tain hier arch y of SDP r elaxations. Moreov er, if a certain rank condition is satisfied at an optimal solution o f so me relaxa tion of the hierarch y , then this rela xation is exact and one may ex tract optimal str a tegies. Interestingly and not surpr isingly , as this hierarch y is derived from a min-max problem ov er sets of measures , it is a subtle combination of moment and sums o f squares (s.o.s.) constraints wherea s the hierarch y for p olyno mial o ptimiza tion is based on either moments (prima l for m ulation) o r s.o.s . (dual form ulation) but not b o th. This is a m ultiv ar iate extension of Parrilo’s [37] result for the univ ariate case wher e o ne needs to solve a single semidefinite pr ogram (as opp osed to a hier a rch y). The a pproach may b e extended to dynamic abs o rbing games 4 with discounted r ew ards, and in the univ ariate case, one can construct a p oly no mial time alg orithm that combines a binar y sear c h with a semidefinite program. Hence the first main contribution is to formulate s everal ga me problems as a par- ticular instance of the MRF problem while the seco nd main contribution extends the momen t-s.o.s. appro ach o f [24, 2 6] in tw o directions. The first extension ( the MRF pro blem) is e s sent ially a non trivial adaptation of Lasse rre’s appro ach [24] to the pr oblem of minimizing the sup o f finitely man y rationa l functions. Notice that the sup of finitely many ra tional functions is not a rational function. How ever one reduces the initial problem to that of minimizing a single rationa l function 4 In dynamic absorbing games, transitions are con trolled b y b oth play ers, in con trast with Pa rril o and Shah [44] where only one play er con trols the transitions. 4 R. LARAKI AND J.B. LASSERRE (but now of n + 1 v aria bles) on an appropria te set o f R n +1 . As such, this ca n a ls o be v iewed as an extensio n of Jib etean and De Kler k’s result [19] for minimizing a single rational function. The second extension generalizes (to the m ultiv ariate case) Parrilo’s appro ach [37] for the univ ariate case, and provides a hiera rch y of mixed mo men t-s.o.s. SDP relaxations. The pro of of conv ergence is delica te a s one has to consider simultaneously momen t constraints a s well as s.o.s.-r epresentation of pos itiv e p o lynomials. (In particular , and in cont rast to po lynomial optimization, the conv erging sequence o f optimal v alues asso cia ted with the hierarch y of SDP relaxatio ns is not mono tone anymore.) T o conclude, within the game theor y co mm unity the rather negative compu- tational complexity re sults ([3 ], [4], [6], [40], [41]) have reinforced the idea that solving a game is computationally hard. On a more p ositive tone our contribution provides a unified semidefinite progra mming appro ach to man y ga me pr oblems. It shows that optimal v alue and o ptimal strategies can b e approximated as closely as desired (and so metimes obtained exa ctly) by solving a hierarch y of s e midefinite relaxatio ns, very muc h in the spir it of the moment approach describ ed in [2 4] for solving p oly no mial minimization problems (a par ticular instance of the Generalized Problem of Moments [26]). Moreover, the metho dology is consistent with pr evious results of [5] a nd [37] as it re duce s to a linea r pr ogram for finite zero-sum games and to a sing le semidefinite program for univ ariate polyno mial zero -sum games . Finally , even if practice fr o m problems in p olynomial optimization s eems to reveal that this approa c h is efficien t, o f cours e the size of the semidefinite relaxatio ns grows rapidly w ith the initial problem s ize. Therefore, in view of the present status of public SDP solvers av ailable, its application is limited to s ma ll and medium size problems so far. Quoting Papadimitriou [35]: “ The PP AD-c ompleteness of Nash suggests that any appr o ach to finding N ash e quilibria that aspir es to b e efficient [...] should ex plicitly take advantage of c omputational ly b eneficial sp e cial pr op erties of the game in hand ”. Hence to make our a lgorithm efficient for larger s ize pro blems, one could exploit po ssible sparsity and regularities often present in the data (which will b e the case if the game is the norma l for m of an extensive form ga me). Indeed sp ecific SDP relaxatio ns for minimization pr o blems that exploit sparsity efficiently hav e b een provided in Ko jim a et al. [49] and their con vergence has be en prov ed in [25] under some co ndition on the sparsity pattern. 2. N o t a tion and preliminar y resul ts 2.1. Notation and definitions. Let R [ x ] b e the r ing o f real p olynomials in the v ar ia bles x = ( x 1 , . . . , x n ) and let ( X α ) α ∈ N be its canonical basis of monomials. Denote by Σ[ x ] ⊂ R [ x ] the subset (co ne) of po lynomials that are sums of squar es (s.o.s.), and by R [ x ] d the space o f p olyno mia ls of degree at mos t d . Finally let k x k denote the Euclidean no rm of x ∈ R n . With y =: ( y α ) ⊂ R being a sequence indexed in the canonical monomial basis ( X α ), let L y : R [ x ] → R be the linear functional f (= X α ∈ N n f α x α ) 7− → X α ∈ N n f α y α , f ∈ R [ x ] . Moment matrix. Given y = ( y α ) ⊂ R , the moment matrix M d ( y ) of order d asso ciated with y , has its rows and co lumns indexed by ( x α ) and its ( α, β )-entry is SEMIDEFINITE PROGRAMMING FOR MIN-MAX PROBLEMS AND GAMES 5 defined by: M d ( y )( α, β ) := L y ( x α + β ) = y α + β , | α | , | β | ≤ d. Lo calizing matrix. Similarly , given y = ( y α ) ⊂ R and θ ∈ R [ x ] (= P γ θ γ x γ ), the lo c alizing matrix M d ( θ, y ) o f order d asso ciated with y and θ , has its rows and columns indexed by ( x α ) a nd its ( α, β )-en try is defined by: M d ( θ, y )( α, β ) := L y ( x α + β θ ( x )) = X γ θ γ y γ + α + β , | α | , | β | ≤ d. One s ays that y = ( y α ) ⊂ R ha s a r epr esent ing measure supp orted on K if there is some finite Borel meas ure µ on K suc h tha t y α = Z K x α dµ ( x ) , ∀ α ∈ N n . F o r later use, write M d ( y ) = X α ∈ N n y α B α (2.1) M d ( θ, y ) = X α ∈ N n y α B θ α , (2.2) for r eal symmetric matrices ( B α , B θ α ) of appropr iate dimensions. No te that the ab ov e tw o s ummations contain only finitely many terms. Definition 2 . 1 (P utina r ’s prop erty) . L et ( g j ) m j =1 ⊂ R [ x ] . A b asic close d semi algebr aic set K := { x ∈ R n : g j ( x ) ≥ 0 , : j = 1 , . . . , m } satisfies Put inar’s pr op erty if ther e exists u ∈ R [ x ] such that { x : u ( x ) ≥ 0 } is c omp act and (2.3) u = σ 0 + m X j =1 σ j g j for some s.o.s. p olynomi als ( σ j ) m j =0 ⊂ Σ[ x ] . Equivalently, for some M > 0 the quadr atic p olynomial x 7→ M − k x k 2 has Putinar’s r epr esent ation (2.3). Obviously Putinar ’s pr op erty implies co mpactness of K . How ever, notice tha t Putinar’s prop erty is no t geometric but alg ebraic a s it is related to the representa- tion o f K by the defining p olynomials ( g j )’s. Putinar’s prop erty holds if e.g. the level set { x : g j ( x ) ≥ 0 } is compact for some j , or if a ll g j are affine and K is com- pact (in whic h ca se it is a p olyto pe). In case it is not satisfied and if for some known M > 0, k x k 2 ≤ M whenever x ∈ K , then it suffices to add the redunda nt quadratic constraint g m +1 ( x ) := M − k x k 2 ≥ 0 to the definition of K . The impor tance o f Putinar’s pro per t y stems from the following result: Theorem 2.2 (Putinar [38]) . L et ( g j ) m j =1 ⊂ R [ x ] and assume that K := { x ∈ R n : g j ( x ) ≥ 0 , j = 1 , . . . , m } satisfies Putinar’s pr op erty. (a) L et f ∈ R [ x ] b e strictly p ositive on K . Then f c an b e written as u in (2.3). (b) L et y = ( y α ) . Then y has a r epr esent ing me asure on K if and only if (2.4) M d ( y )  0 , M d ( g j , y )  0 , j = 1 , . . . , m ; d = 0 , 1 , . . . W e also ha ve: 6 R. LARAKI AND J.B. LASSERRE Lemma 2.3 . L et K ⊂ R n b e c omp act and let p, q c ontinuous with q > 0 on K . L et M ( K ) b e the set of finite Bor el me asure s on K and let P ( K ) ⊂ M ( K ) b e its subset of pr ob ability me asur es on K . Then min µ ∈ P ( K ) R K p dµ R K q dµ = min ϕ ∈ M ( K ) { Z K p dϕ : Z K q dϕ = 1 } (2.5) = min µ ∈ P ( K ) Z K p q dµ = min x ∈ K p ( x ) q ( x ) (2.6) Pr o of. Le t ρ ∗ := min x { p ( x ) /q ( x ) : x ∈ K } . As q > 0 on K , R K p dµ R K q dµ = R K ( p/q ) q dµ R K q dµ ≥ ρ ∗ . Hence if µ ∈ P ( K ) then R K ( p/q ) dµ ≥ ρ ∗ R K dµ = ρ ∗ . On the other hand, with x ∗ ∈ K a global minimizer of p/ q on K , let µ := δ x ∗ ∈ P ( K ) be the Dirac meas ur e at x = x ∗ . The n R K pdµ/ R K q dµ = p ( x ∗ ) /q ( x ∗ ) = R K ( p/q ) dµ = ρ ∗ , a nd therefore min µ ∈ P ( K ) R K pdµ R K q dµ = min µ ∈ P ( K ) Z K p q dµ = min x ∈ K : p ( x ) q ( x ) = ρ ∗ . Next, for every ϕ ∈ M ( K ) with R K q dϕ = 1, R K p dϕ ≥ R K ρ ∗ q dϕ = ρ ∗ , and s o min ϕ ∈ M ( K ) { R K p dϕ : R K q dϕ = 1 } ≥ ρ ∗ . Finally ta k ing ϕ := q ( x ∗ ) − 1 δ x ∗ yields R K q dϕ = 1 a nd R K p dϕ = p ( x ∗ ) /q ( x ∗ ) = ρ ∗ . Another w ay to s ee why this is tr ue is throughout the following arg ument. The function µ → R K p dµ R K q dµ is q uasi-concave (and also quas i- conv ex) so that the optimal v alue of the minimization pro blem is achiev ed o n the bo undary .  3. Minimizing th e max of finitel y many ra tional functions Let K ⊂ R n be the basic semi-a lgebraic set (3.1) K := { x ∈ R n : g j ( x ) ≥ 0 , j = 1 , . . . , p } for some p olynomia ls ( g j ) ⊂ R [ x ], and let f i = p i /q i be ratio na l functions, i = 0 , 1 , . . . , m , with p i , q i ∈ R [ x ]. W e assume that: • K satisfies Putinar’s prop erty (see Definition 2.1) and, • q i > 0 on K for every i = 0 , . . . , m . Consider the following problem denoted by MRF : (3.2) MRF : ρ := min x { f 0 ( x ) + max i =1 ,...,m f i ( x ) : x ∈ K } , or, equiv alen tly , (3.3) MRF : ρ = min x,z { f 0 ( x ) + z : z ≥ f i ( x ) , i = 1 , . . . , m ; x ∈ K } . With K ⊂ R n as in (3.1), let b K ⊂ R n +1 be the basic semi algebra ic set (3.4) b K := { ( x, z ) ∈ R n × R : x ∈ K ; z q i ( x ) − p i ( x ) ≥ 0 , i = 1 , . . . , m } and conside r the ne w infinite-dimensional optimization pro blem (3.5) P : ˆ ρ := min µ { Z K ( p 0 + z q 0 ) dµ : Z K q 0 dµ = 1 , µ ∈ M ( b K ) } SEMIDEFINITE PROGRAMMING FOR MIN-MAX PROBLEMS AND GAMES 7 where M ( b K ) is the s et of finite Borel mea s ures on b K . Pr oblem (3.5) is a particular instance of the Gener alize d Pr oblem of Moments for which a gener al methodolo gy (based o n a hierarch y o f semidefinite relaxatio ns) has b een describ ed in [26]. T o make the pap er s elf-contained w e expla in b elow how to apply this methodolo gy in the a bove sp ecific context. Prop osition 3.1. L et K ⊂ R n b e as in (3.1). Then ρ = ˆ ρ . Pr o of. The following upper and lo wer b ounds z := min i =1 ,...,m min x ∈ K f i ( x ); z := max i =1 ,...,m max x ∈ K f i ( x ) , are b oth well-defined since K is compact a nd q i > 0 on K for every i = 1 , . . . , m . Including the additional constraint z ≤ z ≤ z in the definition of b K makes it compact without changing the v alue of ρ . Next observe that (3.6) ρ = min ( x,z )  p 0 ( x ) + z q 0 ( x ) q 0 ( x ) : ( x, z ) ∈ b K  . Applying Lemma 2.3 with b K in lieu of K , and with ( x, z ) 7→ p ( x, z ) := p 0 ( x )+ z q 0 ( x ) and ( x, z ) 7→ q ( x, z ) := q 0 ( x ), yields the desired result.  Remark 3.2. (a) When m = 0, that is whe n one wishes to minimize the rationa l function f 0 on K , then P reads min µ { R K f 0 dµ : R K q 0 dµ = 1; µ ∈ M ( K ) } . Using the dual pro blem P ∗ : max { z : p 0 ( x ) − z q 0 ( x ) ≥ 0 ∀ x ∈ K } , J ibetea n and DeKlerk [19] pr op o sed to approximate the optimal v alue by solving a hierarch y of semidefinite r elaxations. The case m ≥ 2 is a nontrivial extensio n of the ca s e m = 0 bec ause one now wishes to minimize on K a function which is not a rational function. How ev er, b y adding an extra v ariable z , one obtains the momen t problem (3.5), which indeed is the same as minimizing the r ational function ( x, z ) 7→ p 0 ( x )+ z q 0 ( x ) q 0 ( x ) on a domain b K ⊂ R n +1 . And the dual of P now reads max { ρ : p 0 ( x ) + z q 0 ( x ) − ρq 0 ( x ) ≥ 0 ∀ ( x, z ) ∈ b K } . Hence if b K is compact and satisfies Putinar’s pro per t y one may use the hierarchy of semidefinite rela xations defined in [19] a nd ada pted to this sp e c ific co n text; see also [2 6]. (b) The case m = 1 (i.e., when one wan ts to minimize the sum of r a tional functions f 0 + f 1 on K ) is als o in teresting. One wa y is to r educe to the same denominator q 0 × q 1 and minimize the rational function ( p 0 q 1 + p 1 q 0 ) /q 0 q 1 . But then the first SDP relaxation o f [26, 19] would have to c o nsider p olynomials of degree at least d := ma x[deg p 0 + deg q 1 , deg p 1 + deg q 0 , deg q 0 + deg q 1 ], whic h may be very p enalizing when q 0 and q 1 hav e large degree (and s o metimes it may even b e impo ssible!). Indeed this SDP r elaxation has as many as O ( n d ) v ariables and a linear matr ix inequa lit y of size O ( n ⌈ d/ 2 ⌉ ). In contrast, by pro cee ding as ab ov e in introducing the additional v a riable z , one now minimizes the rational function ( p 0 + z q 0 ) /q 0 which may b e hig hly prefera ble since the firs t r e la xation only considers p oly no mials of degree b ounded by ma x[deg p 0 , 1 + deg q 0 ] (but now in n + 1 v a riables). F or instance, if deg p i = deg q i = v , i = 1 , 2, then one has O ( n v +1 ) v ariables instea d of O ( n 2 v ) v ariables in the former appro ach. W e next descr ib e how to solve the MRF problem via a hier arch y o f semidefinite relaxatio ns. 8 R. LARAKI AND J.B. LASSERRE SDP relaxations for solving the MRF proble m. As K is compac t and q i > 0 on K , for a ll i , le t (3.7) M 1 := ma x i =1 ,...,m  max {| p i ( x ) | , x ∈ K } min { q i ( x ) , x ∈ K }  , and (3.8) M 2 := min i =1 ,...,m  min { p i ( x ) , x ∈ K } max { q i ( x ) , x ∈ K }  . Redefine the set b K to be (3.9) b K := { ( x, z ) ∈ R n × R : h j ( x, z ) ≥ 0 , j = 1 , . . . p + m + 1 } with (3.10)    ( x, z ) 7→ h j ( x, z ) := g j ( x ) j = 1 , . . . , p ( x, z ) 7→ h j ( x, z ) := z q j ( x ) − p j ( x ) j = p + 1 , . . . , p + m ( x, z ) 7→ h j ( x, z ) := ( M 1 − z )( z − M 2 ) j = m + p + 1 . Lemma 3. 3. L et K ⊂ R n satisfy Putinar’s pr op erty. Th en the set b K ⊂ R n +1 define d in (3.9) satisfies Putinar’s pr op erty. Pr o of. Since K satisfies Putinar ’s proper t y , equiv alently , the q uadratic p olynomial x 7→ u ( x ) := M − k x k 2 can b e written in the for m (2.3), i.e., u ( x ) = σ 0 ( x ) + P p j =1 σ j ( x ) g j ( x ) for some s.o.s. polyno mia ls ( σ j ) ⊂ Σ[ x ]. Next, co ns ider the quadratic p olynomia l ( x, z ) 7→ w ( x, z ) = M − k x k 2 + ( M 1 − z )( z − M 2 ) . Obviously , its level set { x : w ( x, z ) ≥ 0 } ⊂ R n +1 is compact and moreover, w can be written in the form w ( x, z ) = σ 0 ( x ) + p X j =1 σ j ( x ) g j ( x ) + ( M 1 − z )( z − M 2 ) = σ ′ 0 ( x, z ) + m + p +1 X j =1 σ ′ j ( x, z ) h j ( x, z ) for appropria te s.o .s. p olynomials ( σ ′ j ) ⊂ Σ[ x, z ]. Ther efore b K satisfies Putinar ’s prop erty in Definition 2 .1, the desired r e s ult.  W e are now in p osition to define the hier a rch y of semidefinite rela xations for so lv- ing the M RF pr oblem. Let y = ( y α ) be a real sequence indexed in the monomial basis ( x β z k ) o f R [ x, z ] (with α = ( β , k ) ∈ N n × N ). Let h 0 ( x, z ) := p 0 ( x ) + z q 0 ( x ), and let v j := ⌈ (deg h j ) / 2 ⌉ for every j = 0 , . . . , m + p + 1 . F or r ≥ r 0 := max j =0 ,...,p + m +1 v j , intro duce the hier arch y of semidefinite pro- grams: (3.11) Q r :        min y L y ( h 0 ) s . t . M r ( y )  0 M r − v j ( h j , y )  0 , j = 1 , . . . , m + p + 1 L y ( q 0 ) = 1 , with o ptima l v alue denoted inf Q r (and min Q r if the infim um is attained). SEMIDEFINITE PROGRAMMING FOR MIN-MAX PROBLEMS AND GAMES 9 Theorem 3.4. L et K ⊂ R n (c omp act) b e as in (3.1 ). L et Q r b e t he semidefinite pr o gr am (3.11) with ( h j ) ⊂ R [ x, z ] and M 1 , M 2 define d in (3.10) and (3.7)-(3.8) r esp e ct ively. Then: (a) inf Q r ↑ ρ as r → ∞ . (b) L et y r b e an optimal solution of the S DP r elaxation Q r in (3.11). If (3.12) rank M r ( y r ) = rank M r − r 0 ( y r ) = t then min Q r = ρ and one may extr act t p oints ( x ∗ ( k )) t k =1 ⊂ K , al l glob al minimizers of the MRF pr oblem. (c) L et y r b e a ne arly optimal solution of the SDP r elaxation (3.11) (with say inf Q r ≤ L y r ≤ inf Q r + 1 / r . If (3.6) has a unique glob al minimizer x ∗ ∈ K then the ve ctor of fir s t -or der moments ( L y r ( x 1 ) , . . . , L y r ( x n )) c onver ges to x ∗ as r → ∞ . Pr o of. As already mentioned in Remark 3 .2, co n vergence o f the dual of the semidef- inite relaxa tions (3.11) was first proved in Jib etean a nd de K lerk [19] for minimizing a r ational function on a basic compact semi-algebr a ic set (in our context, for mini- mizing the rational function ( x, z ) 7→ ( p 0 ( x ) + z q 0 ( x )) /q 0 ( x ) on the set b K ⊂ R n +1 ). See also [26, § 4.1] and [2 8, Theor. 3.2 ]. In par ticular to get (b) se e [28, Theo r . 3.4]. The pro of of (c) is easily adapted fro m Sch w eighofer [43].  Remark 3.5. Hence, by Theorem 3.4(b), when finite conv ergence o ccurs o ne may extract t := rank M r ( y ) global minimizers. On the o ther hand, a generic MR F problem has a unique global minimizer x ∗ ∈ K and in this case, even when the conv ergence is only asymptotic, one ma y still o btain an approximation of x ∗ (as closely a s des ired) fro m the vector of first- o rder moments ( L y r ( x 1 ) , . . . , L y r ( x n )). F o r instance, o ne wa y to have a unique globa l minimizer is to ǫ -per turb the ob jec- tive function of the M RF problem by so me randomly genera ted p olynomia l of a sufficiently large degr e e , or to sligh tly per turb the co efficients of the data ( h i , g j ) of the MRF pro blem. T o so lve (3.11) one may use e.g. the Matlab based public so ft ware GloptiPoly 3 [15] dedicated to solve the g e neralized problem of moments descr ibed in [2 6]. It is an e xtension of Glo ptiPoly [14] previously dedicated to solve po lynomial opti- mization pr oblems. A pro cedure for extracting optimal solutions is implemented in Gloptip oly when the r ank condition (3 .12) is sa tisfied 5 . F or mo re details the int erested reader is r eferred to [15] a nd ww w.laas .fr/ ∼ h enrion/software/ . Remark 3 . 6. If g j is affine for every j = 1 , . . . , p and if p j is affine and q j ≡ 1 for every j = 0 , . . . , m , then h j is affine for every j = 0 , . . . , m . One may also replace the single quadratic co nstraint h m + p +1 ( x, z ) = ( M 1 − z )( z − M 2 ) ≥ 0 with the t wo e quiv alent linear constra in ts h m + p +1 ( x, z ) = M 1 − z ≥ 0 and h m + p +2 ( x, z ) = z − M 2 ≥ 0 . In this case, it suffices to solve the sing le se midefinite relaxatio n Q 1 , 5 In fact GloptiPoly 3 extracts al l solutions b ecause most SDP solvers that one may call in GloptiP oly 3 (e.g. SeDuMi ) use primal- dual interior p oints metho ds wi th the self -dual embedding tec hnique whi c h find an optimal solution in the relative int erior of the set of optimal solutions; see [27, § 4.4.1, p. 663]. In the presen t cont ext of (3.11) this m eans that at an optimal solution y ∗ , the momen t m atrix M r ( y ∗ ) has m axim um r ank and its rank corresp onds to the num ber of solutions. 10 R. LARAKI AND J.B. LASSERRE which is in fact a linear progr am. Indeed, fo r r = 1, y = ( y 0 , ( x, z ) , Y ) and M 1 ( y ) =     y 0 | ( x z ) − −  x z  | Y     . Then (3.11) reads Q 1 :        min y h 0 ( x ) s . t . M 1 ( y )  0 h j ( x, z ) ≥ 0 , j = 1 , . . . , m + p + 2 y 0 = 1 . . As v j = 1 for every j , M 1 − 1 ( h j , y )  0 ⇔ M 0 ( h j , y ) = L y ( h j ) = h j ( x, z ) ≥ 0, a linear co nstraint. Hence the constra in t M 1 ( y )  0 can b e discarded a s g iv en any ( x, z ) one may always find Y such that M 1 ( y )  0. Therefore , (3.11) is a linear progra m. This is fortunate for finite zer o-sum g a mes applications since co mputing the v alue is equiv alent to minimizing a maximum of finitely many linear functions (and it is a lready known that it can be solved by Linear Progra mming). 4. A pplica tions to finite games In this section we show that sev eral solution concepts of static and dynamic finite games reduce to so lving the MR F problem (3.2). Those are just exa mples and one exp ects that suc h a reduction also holds in a m uc h larger class of games (when they are descr ib ed by finitely many scalars ). 4.1. Standard static games. A finite game is a tuple ( N ,  S i  i =1 ,...,N ,  g i  i =1 ,...,N ) where N ∈ N is the set of pla yers, S i is the finite set of pure strategies of player i and g i : S → R is the payoff function o f player i , where S := S 1 × ... × S N . The set ∆ i = (  p i ( s i )  s i ∈ S i : p i ( s i ) ≥ 0 , X s i ∈ S i p i ( s i ) = 1 ) of proba bilit y distributio ns ov er S i is called the set of mixed strategies o f player i . Notice that ∆ i is a compac t basic semi-algebra ic s e t. If each player j choos es the mixed s trategy p j ( · ) , the vector denoted p =  p 1 , ..., p N  ∈ ∆ : = ∆ 1 × ... × ∆ N is called a pr ofile of mixe d strateg ie s and the exp ected payoff of a player i is g i ( p ) = X s =( s 1 ,...,s N ) ∈ S p 1 ( s 1 ) × ... × p i ( s i ) × ... × p N ( s N ) g i ( s ) . This is nothing but the m ulti-linear extension o f g i . F o r a play er i , and a profile p, let p − i be the pro file of the other play ers except i : that is p − i = ( p 1 , ..., p i − 1 , p i +1 , ..., p N ) . Let S − i = S 1 × ... × S i − 1 × S i +1 × ... × S N and define g i ( s i , p − i ) = X s − i ∈ S − i p 1 ( s 1 ) × ... × p i − 1 ( s i − 1 ) × p i +1 ( s i +1 ) × ... × p N ( s N ) g i ( s ) , where s − i := ( s 1 , ..., s i − 1 , s i +1 , ..., s N ) ∈ S − i . SEMIDEFINITE PROGRAMMING FOR MIN-MAX PROBLEMS AND GAMES 11 A profile p 0 is a Nash [33] equilibrium if and only for a ll i = 1 , ..., N and all s i ∈ S i , g i ( p 0 ) ≥ g i ( s i , p − i 0 ) o r equiv alent ly if: (4.1) p 0 ∈ arg min p ∈ ∆  max i =1 ,...,N max s i ∈ S i  g i ( s i , p − i 0 ) − g i ( p 0 )   . Since each finite game admits at least one Nash equilibrium [33], the optimal v alue of the min-max pr oblem (4.1) is zero. Notice that (4.1) is a particular instance of the MRF problem (3.2) (with a set K = ∆ that satisfies Putinar’s pr op e r t y and with q i = 1 for every i = 0 , . . . , m ), and so Theorem 3 .4 applies. Finally , observe that the num b er m of p olynomials in the inner do uble ” ma x” of (4.1) (or , equiv alently , m in (3.2 )) is just m = P n i =1 | S i | , i.e., m is just the tota l num b er of all p ossible actions. Hence b y solving the hierarch y of SDP relaxations (3.11), one ca n a pproximate the v alue of the min-max problem as c losely as desired. In a ddition, if (3.12) is satisfied at some r elaxation Q r , then one ma y extr a ct all the Nash e q uilibria of the game. If there is a unique eq uilibrium p ∗ then by The o rem 3.4(c), one may o btain a so- lution arbitra ry clo se to p ∗ and so obtain an ǫ -equilibrium in finite time. Since game problems are not gener ic MRF pro blems, they hav e potentially s e v eral equilibria which ar e all global minimizers of the a sso ciated MRF problem. Also , p erturbing the data of a finite game s till leads to a non g eneric asso ciated MRF pro blem with p o ssibly mult iple solutions. How ever, as in Remark 3.5, one co uld perturb the MRF pro blem as so ciated with the or iginal ga me pro blem to obtain (gener ically) an ǫ - p ertur bed MRF problem with a unique optimal solution. Notice that the ǫ -p erturb ed M RF is no t nec e ssarily co ming fro m a finite ga me. Doing so, b y The- orem 3.4(c), one obtains a sequence that conv erges as y mptotically (a nd sometimes in finitely man y steps) to an ǫ -eq uilibrium of the game pro blem. Recently , Lipton, Mark a kis and Meh ta [31] pr ovided a n algorithm that computes an ǫ -equilibr ium in less than exp onential time but still not po lynomial (namely n logn ǫ 2 where n is the tota l num ber of str a tegies). This promising result yields Papadimitriou [3 5] to argue that “ fin ding a mixe d Nash e quilibrium is PP AD-c omplete r aises some inter- esting quest ions r e gar ding the usefulness of Nash e quilibrium, and helps fo cus ou r inter est in alternative notions (most inter esting among them the appr oximate Nash e quilibrium) ” . Example 4.1. Consider the simple illustrative example of a 2 × 2 game with data s 2 1 s 1 2 s 1 1 ( a, c ) (0 , 0) s 1 2 (0 , 0) ( b, d ) for some scalars ( a , b, c, d ). Denote x 1 ∈ [0 , 1] the probabilit y for play er 1 of playing s 1 1 and x 2 ∈ [0 , 1] the p rob ab ility for pla yer 2 of playing s 2 1 . Then one must solve min x 1 ,x 2 ∈ [0 , 1] max 8 > > < > > : ax 1 − ax 1 x 2 − b (1 − x 1 )(1 − x 2 ) b (1 − x 2 ) − ax 1 x 2 − b (1 − x 1 )(1 − x 2 ) cx 1 − cx 1 x 2 − d (1 − x 1 )(1 − x 2 ) d (1 − x 1 ) − cx 1 x 2 − d (1 − x 1 )(1 − x 2 ) . 12 R. LARAKI AND J.B. LASSERRE W e hav e solved the hierarch y of semidefin ite programs (3.11) with GloptiP oly 3 [15 ]. F or instance, t he moment m atrix M 1 ( y ) of the first SDP relaxation Q 1 reads M 1 ( y ) = 2 6 6 4 y 0 y 100 y 010 y 001 y 100 y 200 y 010 y 001 y 010 y 110 y 020 y 011 y 001 y 101 y 011 y 002 3 7 7 5 , and Q 1 reads Q 1 : 8 > > > > > > > > > > > < > > > > > > > > > > > : min y y 001 s . t . M 1 ( y )  0 y 001 − ay 100 + ay 110 + b ( y 0 − y 100 − y 010 + y 110 ) ≥ 0 y 001 − by 0 + by 010 + ay 110 + b ( y 0 − y 100 − y 010 + y 110 ) ≥ 0 y 001 − cy 100 + cy 110 + d ( y 0 − y 100 − y 010 + y 110 ) ≥ 0 y 00 − dy 0 + dy 100 + cy 110 + d ( y 0 − y 100 − y 010 + y 110 ) ≥ 0 y 100 − y 200 ≥ 0; y 010 − y 020 ≥ 0; 9 − y 002 ≥ 0 y 0 = 1 . With ( a, b, c, d ) = (0 . 05 , 0 . 82 , 0 . 56 , 0 . 76), solving Q 3 yields th e optimal v alue 3 . 93 . 10 − 11 and the three optimal solutions (0 , 0), (1 , 1) and (0 . 57575 , 0 . 94253 ). With randomly gen- erated a, b, c, d ∈ [0 , 1] we also ob t ained a very go o d app roximation of th e global optim um 0 and 3 optimal solutions in most cases with r = 3 (i.e. with moments or order 6 only) and sometimes r = 4. W e hav e also solved 2-pla yer non-zero-sum p × q games with randomly generated rewa rd matrices A, B ∈ R p × q and p, q ≤ 5. W e could solve (5 , 2) and (4 , q ) (with q ≤ 3) games exactly with the 4th (sometimes 3rd) SDP relaxation, i.e. inf Q 4 = O (10 − 10 ) ≈ 0 and one extracts an optimal solution 6 . How ever , the size is relativ ely large and one is close to th e limit of p resen t pub lic SDP solv ers like SeDuMi. I ndeed, for a 2-playe r (5 , 2) or (4 , 3) game, Q 3 has 923 vari ables and M 3 ( y ) ∈ R 84 × 84 , whereas Q 4 has 3002 v ariables and M 4 ( y ) ∈ R 210 × 21 0 . F or a (4 , 4) game Q 3 has 1715 v ariables and M 3 ( y ) ∈ R 120 × 12 0 and Q 3 is still solv able, whereas Q 4 has 6434 vari ables and M 4 ( y ) ∈ R 330 × 33 0 . Finally we have also solved randomly generated instances of 3-pla yer non-zero sum games with (2 , 2 , 2) actions and (3 , 3 , 2) actions. In all (2 , 2 , 2) cases the 4th relaxation Q 4 provided the optimal v alue and the rank-test (3.12) was passed (h ence allow ing to extract global minimizers). F or the (3 , 3 , 2) games, the th ird relaxation Q 3 w as enough in 30% of cases and th e fourth relaxation Q 4 in 80% of cases. Another imp ortant co ncept in game theory is the min-ma x pa yoff v i which plays an imp ortant role in rep eated games (Aumann and Shapley [2]): v i = min p − i ∈ ∆ − i max s i ∈ S i g i ( s i , p − i ) where ∆ − i = ∆ 1 × ... × ∆ i − 1 × ∆ i +1 × ... × ∆ N . This is again a particular instance of the MRF problem (3.2). Hence, it seems more difficult to compute the appr oximate min-max strategies compared to approximate Nash equilibrium strategies be c a use we do not know in adv ance the v alue of v i while we kno w that the min-max v alue asso ciated to the Nas h problem is a lw ays zero. This is not surpris ing: in theory , computing a Nash-equilibrim is PP AD-complete [35] while computing the min-max 6 In general, it is not kno wn which relaxation suffices to solve the mi n-max problem. Also, as already ment ioned, GloptiPoly 3 extract s al l solutions because most SDP solv ers that one may call in GloptiPo ly 3 (e.g. SeDuMi) use primal- dual inte rior points metho ds with the self- dual em b edding tec hnique which find an optimal solution in the relative inte rior of the set of optimal solutions. Thi s is explained in [27, § 4.4.1, p. 663] SEMIDEFINITE PROGRAMMING FOR MIN-MAX PROBLEMS AND GAMES 13 is NP-hard [3]. In the case of tw o players, the function g i ( s i , p − i ) is linear in p − i . By rema rk 3.6 it suffices to so lve the fir st rela x ation Q 1 , a linear pr ogram. Remark 4 . 2. The Nash equilibrium problem may be reduced to solv ing a sys tem of polyno mial equations (see e.g . [8]). In the same s pirit, an alternative for the Nash-equilibrium pr oblem (but not for the MRF problem in g eneral) is to apply the momen t approach descr ibed in Lasser re et al. [2 7] for finding r e al ro ots of poly- nomial equations. If there a re finitely many Nash equilibria then its c onv ergence is finite and in co ntrast with the algebr aic metho ds [8, 3 0, 47] men tioned ab ov e, it provides al l rea l solutions without co mputing all complex ro ots. 4.2. Lo om is games. Lo omis [32] extended the min-max theorem to ze r o-sum games with a ra tional fraction. His mo del may be ex tended to N - player g ames as follows. Our extension is justified by the next sectio n. Asso ciated with each play er i ∈ N are tw o functions g i : S → R and f i : S → R where f i > 0 and S := S 1 × ... × S N . With same notation as in the last sec tion, let their mult i-linear extension to ∆ still b e denoted by g i and f i . Tha t is, for p ∈ ∆ , let: g i ( p ) = X s =( s 1 ,...,s N ) ∈ S p 1 ( s 1 ) × ... × p i ( s i ) × ... × p N ( s N ) g i ( s ) . and similar ly for f i . Definition 4.3. A L o omis game is define d as fol lows. The st r ate gy set of player i is ∆ i and if the pr ofile p ∈ ∆ is chosen, his p ayoff function is h i ( p ) = g i ( p ) f i ( p ) . One can show the following new lemma 7 . Lemma 4.4. A L o omis game admits a N ash e quilibrium 8 . Pr o of. No te that each pay off function is quasi-co ncav e in p i (and als o quasi-c onv ex so that it is a quas i-linear function). Actually , if h i ( p i 1 , p − i ) ≥ α and h i ( p i 2 , p − i ) ≥ α then for any δ ∈ [0 , 1], g i ( δ p i 1 + (1 − δ ) p i 2 , p − i ) ≥ f i ( δ p i 1 + (1 − δ ) p i 2 , p − i ) α, so that h i ( δ p i 1 + (1 − δ ) p i 2 , p − i ) ≥ α . O ne can now apply Glicksberg’s [11] theorem bec ause the strateg y sets are compa ct, convex, and the pay off functions are qua si- concav e a nd co ntin uous.  Corollary 4. 5 . A pr ofile p 0 ∈ ∆ is a Nash e quilibrium of a L o omis game if and only if (4.2) p 0 ∈ arg min p ∈ ∆  max i =1 ,...,N max s i ∈ S i  h i ( s i , p − i ) − h i ( p )   . 7 As far as we know, non-zero sum Loomis games are not considered i n the li terature. This model could be of in terest i n situations where there are populations wi th many pla ye rs. A mixed strategy for a p opulation describ es the pr oportion of play ers in the population that uses some pure action. g i ( p ) is the non-normali zed pay off of population i and f i ( p ) ma y be interpreted as the v alue of m oney for population i so that h i ( p ) = g i ( p ) f i ( p ) is the normalized pay off of population i . 8 Clearly , the lemma and its pro of still hold in infinite games where the s ets S i are con v ex- compact-metric and the functions f i and g i are con tin uous. The summation in the multi-linear extension should b e replaced b y an inte gral. 14 R. LARAKI AND J.B. LASSERRE Pr o of. Cle a rly , p 0 ∈ ∆ is an equilibrium of the Loo mis game if and only if p 0 ∈ arg min p ∈ ∆  max i =1 ,...,N max e p i ∈ ∆ i  g ( e p i , p − i ) f i ( e p i , p − i ) − g i ( p ) f i ( p )  . Using the quasi-linear it y of the pay offs or Lemma 2.3, o ne deduces that: max e p i ∈ ∆ i g i ( e p i , p − i ) f i ( e p i , p − i ) = max s i ∈ S i g i ( s i , p − i ) f i ( s i , p − i ) which is the desired result.  Again, the min-max o ptimization pro blem (4.2) is a particular instance of the MRF pro blem (3.2) a nd so can b e solved via the hierar ch y o f semidefinite relax- ations (3.1 1). Notice that in (4 .2) o ne has to minimize the supr e m um o f ra tio nal functions (in contrast to the supremum of p olynomials in (4.1)). 4.3. Fini te absorbi ng games. This s ub clas s of sto chastic games has been in tro- duced by Ko hlber g [2 1]. The following formulas are established in [2 3]. It shows that a bsorbing games co uld b e reduced to Lo omis ga mes . An N - player finite ab- sorbing ga me is de fined as follo ws. As a b ove, there a re N finite sets ( S 1 , ..., S N ) . There are t wo functions g i : S → R and f i : S → R asso cia ted to each play er i ∈ { 1 , ..., N } a nd a pr obability transition function q : S → [0 , 1]. The game is play ed in discrete time as follows. A t each sta g e t = 1 , 2 , ... , if the game has not been abso rbe d b e fo re tha t day , eac h play er i choos e s (sim ultaneously) at ra ndom an a ction s i t ∈ S i . If the profile s t = ( s 1 t , ..., s N t ) is chosen, then: (i) the pay off of pla yer i is g i ( s t ) a t stage t . (ii) with pro babilit y 1 − q ( s t ) the game is terminated (abs orb ed) and each play er i gets at every stag e s > t the pay off f i ( s t ) , a nd (iii) with probability q ( s t ) the ga me contin ues (the situation is rep eated at stage t + 1). Consider the λ -discounted game G ( λ ) (0 < λ < 1). If the pay off of play er i a t stage t is r i ( t ) then its λ -disc o un ted pay off in the g ame is P ∞ t =1 λ (1 − λ ) t − 1 r i ( t ). Hence, a play er is optimizing his ex pected λ -discounted pay off. Let e g i = g i × q and e f i = f i × (1 − q ) and extend e g i , e f i and q mu ltilinearly to ∆ (as ab ove in Nash and Lo omis ga mes). A profile p ∈ ∆ is a sta tio nary equilibrium of the absorbing game if (1) ea ch play er i plays iid a t ea c h stage t the mixed stra tegy p i un til the game is absor b ed and (2) this is optimal fo r him in the discoun ted abso rbing game if the other play ers do not deviate. Lemma 4. 6 . A pr ofile p 0 ∈ ∆ is a stationary e quilibrium of t he absorbing game if and only if it is a Nash e quilibrium of the L o omis game with p ayoff functions p → λ e g i ( p )+(1 − λ ) e f i ( p ) λq ( p )+(1 − q ( p )) , i = 1 , ..., N . Pr o of. See Laraki [23].  As s hown in [23], the min-max of a disco un ted absorbing game sa tisfies: v i = min p − i ∈ ∆ − i max s i ∈ S i λ e g i ( s i , p − i ) + (1 − λ ) e f i ( s i , p − i ) λq ( s i , p − i ) + (1 − q ( s i , p − i )) . SEMIDEFINITE PROGRAMMING FOR MIN-MAX PROBLEMS AND GAMES 15 Hence solving a finite absorbing g ame is equiv a lent to s o lving a Lo omis g a me (hence a particular instance of the MR F problem (3.2)) which ag ain can be solved via the hier arch y of semidefinite r elaxations (3.11). Again one has to minimize the supremum o f rationa l functions. 5. Z ero-sum pol ynomial games Let K 1 ⊂ R n 1 and K 2 ⊂ R n 2 be t w o ba s ic a nd clos e d semi-algebr aic sets (not necessarily with same dimension): K 1 := { x ∈ R n 1 : g j ( x ) ≥ 0 , j = 1 , . . . , m 1 } (5.1) K 2 := { x ∈ R n 2 : h k ( x ) ≥ 0 , k = 1 , . . . , m 2 } (5.2) for so me p olynomials ( g j ) ⊂ R [ x 1 , . . . x n 1 ] and ( h k ) ⊂ R [ x 1 , . . . x n 2 ]. Let P ( K i ) b e the set of Bore l probability measures on K i , i = 1 , 2 , and c onsider the following min-max problem: (5.3) P : J ∗ = min µ ∈ P ( K 1 ) max ν ∈ P ( K 2 ) Z K 2 Z K 1 p ( x, z ) dµ ( x ) dν ( z ) for so me p olynomial p ∈ R [ x, z ]. If K 1 and K 2 are c ompact, it is well-known that J ∗ = min µ ∈ P ( K 1 ) max ν ∈ P ( K 2 ) Z K 2 Z K 1 p ( x, z ) dµ ( x ) dν ( z ) (5.4) = max ν ∈ P ( K 2 ) min µ ∈ P ( K 1 ) Z K 1 Z K 2 p ( x, z ) dν ( z ) dµ ( x ) , that is, there exist µ ∗ ∈ P ( K 1 ) a nd ν ∗ ∈ P ( K 2 ) such that: (5.5) J ∗ = Z K 2 Z K 1 p ( x, z ) dµ ∗ ( x ) dν ∗ ( z ) . The proba bilit y measur es µ ∗ and ν ∗ are the optimal strateg ies of players 1 and 2 resp ectively . F o r the univ ariate ca s e n = 1, Parrilo [37] show ed that J ∗ is the optimal v alue of a single semidefinite pr ogram, namely the semidefinite pr ogram (7) in [37, p. 2858], and mentioned how to extr act optimal strategies since there exis t optimal strategies ( µ ∗ , ν ∗ ) with finite supp ort. In [37] the author mentions that extensio n to the m ultiv ar iate case is p oss ible. W e provide b elow such an extensio n which, in view of the pr o of of its v alidity given below, is non trivial. The price to pay for this extension is to replace a single se midefinite program with a hier arch y of se midefinite progra ms of incr easing siz e . But contrary to the p olynomial optimization case in e.g. [24], proving convergence of this hierarchy is more delicate because one has (simu ltaneously in the same SDP) moment ma trices of increasing size and a n s.o.s.- representation o f some p olyno mial in P utinar’s form (2.3) with increasing degr ee bo unds for the s.o.s. weigh ts. In pa rticular, the co n vergence is not monotone anymore. When we do n = 1 in this extension, one re trieves the origina l result of Parrilo [37], i.e., the first semidefinite progra m in the hierarch y (5.9) r educes to (7) in [37, p. 28 58] and provides us with the exact desir ed v alue. 16 R. LARAKI AND J.B. LASSERRE Semidefini te relaxations for solving P. With p ∈ R [ x, z ] as in (3.2), write p ( x, z ) = X α ∈ N n 2 p α ( x ) z α with (5.6) p α ( x ) = X β ∈ N n 1 p αβ x β , | α | ≤ d z where d z is the total degr ee of p when seen as p olynomial in R [ z ]. So , let p αβ := 0 for every β ∈ N n 1 whenever | α | > d z . Let r j := ⌈ deg g j / 2 ⌉ , for every j = 1 , . . . , m 1 , and consider the following semi- definite pr ogram: (5.7)                                min y ,λ,Z k λ s . t . λ I α =0 − X β ∈ N n 1 p αβ y β = h Z 0 , B α i + m 2 X k =1 h Z k , B h k α i , | α | ≤ 2 d M d ( y )  0 M d − r j ( g j , y )  0 , j = 1 , . . . , m 1 y 0 = 1 Z k  0 , k = 0 , 1 , . . . m 2 , where y is a finite sequence indexed in the canonical basis ( x α ) of R [ x ] 2 d . Denote by λ ∗ d its optimal v alue. In fact, with h 0 ≡ 1 and p y ∈ R [ z ] defined by: (5.8) z 7→ p y ( z ) := X α ∈ N n 2   X β ∈ N n 1 p αβ y β   z α , the se midefinite pro g ram (5.7) has the equiv alent formulation: (5.9)                              min y ,λ,σ k λ s . t . λ − p y ( · ) = m 2 X k =0 σ k h k M d ( y )  0 M d − r j ( g j , y )  0 , j = 1 , . . . , m 1 y 0 = 1 σ k ∈ Σ[ z ]; : deg σ k + deg h k ≤ 2 d, k = 0 , 1 , . . . , m 2 , where the first co nstraint should be understo o d as an equality of po lynomials. Observe that for an y admiss ible solution ( y , λ ) and p y as in (5.8), (5.10) λ ≥ max z { p y ( z ) : z ∈ K 2 } . SEMIDEFINITE PROGRAMMING FOR MIN-MAX PROBLEMS AND GAMES 17 Similarly , with p as in (3.2), write p ( x, z ) = X α ∈ N n 1 ˆ p α ( z ) x α with (5.11) ˆ p α ( z ) = X β ∈ N n 2 ˆ p αβ z β , | α | ≤ d x where d x is the total degree of p when seen as p olynomial in R [ x ]. So, let ˆ p αβ := 0 for every β ∈ N n 2 whenever | α | > d x . Let l k := ⌈ deg h k / 2 ⌉ , fo r every k = 1 , . . . , m 2 , a nd with (5.12) x 7→ ˆ p y ( x ) := X α ∈ N n 1   X β ∈ N n 2 ˆ p αβ y β   x α , consider the following semidefinite pr ogram (with g 0 ≡ 1): (5.13)                                max y ,γ ,σ j γ s . t . ˆ p y ( · ) − γ = m 1 X j =0 σ j g j M d ( y )  0 M d − l k ( h k , y )  0 , k = 1 , . . . , m 2 y 0 = 1 σ j ∈ Σ[ x ]; deg σ j + deg g j ≤ 2 d, j = 0 , 1 , . . . , m 1 . where y is a finite sequence indexe d in the cano nical basis ( z α ) of R [ z ] 2 d . Denote by γ ∗ d its o ptimal v alue. In fact, (5.13) is the dual of the semidefinite progra m (5.7). Observe that for any a dmissible so lutio n ( y , γ ) and ˆ p y as in (5.12), (5.14) γ ≤ min x { ˆ p y ( x ) : x ∈ K 1 } . Theorem 5. 1 . L et P b e the min-max pr oblem define d in (3.2) and assum e that b oth K 1 and K 2 ar e c omp act and satisfy Put inar’s pr op erty (se e Definition 2.1). L et λ ∗ d and γ ∗ d b e the optimal values of the semidefinite pr o gr am (5.9) and (5.13) r esp e ct ively. Then λ ∗ d → J ∗ and γ ∗ d → J ∗ as d → ∞ . W e also ha ve a test to detect whether finite con vergence has o ccur red. Theorem 5.2. L et P b e the min-max pr oblem define d in (3.2). L et λ ∗ d b e the optimal value of the s emidefinite pr o gr am (5.9), and supp ose t hat with r := max j =1 ,...,m 1 r j , the c ondition (5.15) ra nk M d − r ( y ) = rank M d ( y ) (= : s 1 ) holds at an optimal solution ( y , λ, σ k ) of (5.9). L et γ ∗ t b e the optimal value of the semidefinite pr o gr am (5.13), and supp ose that with r := max k =1 ,..., m 2 l k , the c ondition (5.16) rank M t − r ( y ′ ) = rank M t ( y ′ ) (=: s 2 ) holds at an optimal solution ( y ′ , γ , σ j ) of (5.13). 18 R. LARAKI AND J.B. LASSERRE If λ ∗ d = γ ∗ t then λ ∗ g = γ ∗ t = J ∗ and an optimal stra te gy for player 1 (r esp. player 2) is a pr ob ability me asure µ ∗ (r esp. ν ∗ ) supp orte d on s 1 p oints of K 1 (r esp. s 2 p oints of K 2 ). F o r a pr o o f the reader is referred to § 8. Remark 5. 3. In the univ ar iate case, when K 1 , K 2 are (not necessar ily b ounded) int erv als of the real line, the optimal v alue J ∗ = λ ∗ d (resp. J ∗ = γ ∗ d ) is obtained by solving the single semidefinite progr am (5.9) (resp. (5.13)) with d = d 0 , which is equiv alent to (7 ) in P arrilo [37, p. 2 858]. 6. Z ero-sum pol y n o mial abso rbing games As in the previous s ection, consider two c o mpact basic semi-alge br aic sets K 1 ⊂ R n 1 , K 2 ⊂ R n 2 and p olynomials g , f and q : K 1 × K 2 → [0 , 1]. Recall that P ( K 1 ) (resp. P ( K 2 )) denotes the set o f proba bilit y measures on K 1 (resp. K 2 ). The absor bing ga me is play ed in discrete time as follows. At stage t = 1 , 2 , ... play er 1 choos es at rando m x t ∈ K 1 (using some mixed a ction µ t ∈ P ( K 1 )) and, simult aneously , Player 2 chooses a t random y t ∈ K 2 (using some mixed action ν t ∈ P ( K 2 )). (i) pla yer 1 receives g ( x t , y t ) a t stage t ; (ii) with pr obability 1 − q ( x t , y t ) the ga me is a bsorb ed and player 1 receives f ( x t , y t ) in all sta ges s > t ; and (iii) with probability q ( x t , y t ) the game c o n tinues (the situation is rep eated at step t + 1). If the strea m of pa yoffs is r ( t ), t = 1 , 2 , ..., the λ -discounted-pa yoff of the game is P ∞ t =1 λ (1 − λ ) t − 1 r ( t ). Let e g = g × q and e f = f × (1 − q ) and e x tend e g , e f and q m ultilinearly to P ( K 1 ) × P ( K 2 ). Play er 1 maximizes the expected discounted-pay o ff and pla yer 2 minimizes that pay off. Using a n extension of the Sha pley op era tor [45] one can deduce that the game has a v alue v ( λ ) that uniquely satisfies, v ( λ ) = max µ ∈ P ( K 1 ) min ν ∈ P ( K 2 ) Z Θ  λ e g + (1 − λ ) v ( λ ) p + (1 − λ ) e f  dµ ⊗ ν = min ν ∈ P ( K 2 ) max µ ∈ P ( K 1 ) Z Θ  λ e g + (1 − λ ) v ( λ ) p + (1 − λ ) e f  dµ ⊗ ν with Θ := K 1 × K 2 . As in the finite ca se, it may b e shown [23] that the problem may b e reduced to a zero -sum Lo omis game, that is: (6.1) v ( λ ) = max µ ∈ P ( K 1 ) min ν ∈ P ( K 2 ) R Θ P dµ ⊗ ν R Θ Q dµ ⊗ ν = min ν ∈ P ( K 2 ) max µ ∈ P ( K 1 ) R Θ P dµ ⊗ ν R Θ Q dµ ⊗ ν where ( x, y ) 7→ P ( x, y ) := λ e g ( x, y ) + (1 − λ ) e f ( x, y ) ( x, y ) 7→ Q ( x, y ) := λq ( x, y ) + 1 − q ( x, y ) SEMIDEFINITE PROGRAMMING FOR MIN-MAX PROBLEMS AND GAMES 19 Or equiv a len tly , as it was originally presented by Lo omis [32], v ( λ ) is the unique real t such that 0 = ma x µ ∈ P ( K 1 ) min ν ∈ P ( K 2 )  Z Θ ( P ( x, y ) − t Q ( x, y )) dµ ( x ) dν ( y )  = m in ν ∈ P ( K 2 ) min ν ∈ P ( K 1 )  Z Θ ( P ( x, y ) − t Q ( x, y )) dµ ( x ) dν ( y )  . Actually , the function s : R → R defined b y: t → s ( t ) := max µ ∈ P ( K 1 ) min ν ∈ P ( K 2 )  Z Θ ( P ( x, y ) − tQ ( x, y )) dµ ( x ) dν ( y )  is contin uous, str ictly decrea sing from + ∞ to −∞ as t increa ses in ( −∞ , + ∞ ). In the univ aria te case, if K 1 and K 2 are b oth real interv als (not necessarily compact), then ev aluating s ( t ) for some fixe d t can b e do ne by solving a single semidefinite progra m; see Remark 5.3. Therefore, in this cas e, one may approximate the optimal v alue s ∗ (= s ( t ∗ )) of the game by binary search o n t and s o, the problem can b e solved in a p olynomial time. This extends Shah a nd Parrilo [44]. 7. Concl u s io n W e have propo sed a common metho dology to appr oximate the optimal v a lue of games in tw o different contexts. The fir st alg o rithm, intended to compute (or approximate) Nash equilibria in mixed strategies for static finite games o r dynamic absorbing g ames, is based on a hierar chy of semidefinite pr o grams to approximate the supr e m um of finitely ma n y ra tional functions on a compact basic semi-alge br aic set. Actually this latter formulation is a lso of self-interest in optimizatio n. The second algo rithm, in tended to appr oximate the o ptimal v alue of p olyno mial games whose ac tio n sets are compact basic se mi-algebraic sets, is also base d on a hierarchy of semidefinite progr ams. Not surpr isingly , a s the latter alg orithm comes from a min-max pro blem ov er sets of measur es, it is a subtle c o m bination of moment and s.o.s. constraints whereas in p olynomial optimization it is en tirely formulated either with moments (primal form ulation) or with s.o .s. (dual formulation). Hence the ab ov e metho dology illustrates the p ow er of the combined moment-s.o.s. appro ach. A natural o pen question arises: how to adapt the second alg orithm to compute Nash equilibr ia of a non-zero -sum p olyno mial game? 8. A ppendix 8.1. Pro of of Theorem 5.1. W e first nee d the follo wing pa rtial result. Lemma 8.1. L et ( y d ) d b e a se quenc e of admissible solutions of the semidefinite pr o gr am (5.7). Then ther e ex ist s ˆ y ∈ R ∞ and a subse quenc e ( d i ) such that y d i → ˆ y p ointwise as i → ∞ , that is, (8.1) lim i →∞ y d i α = ˆ y α , ∀ α ∈ N n . The pro of is omitted b ecause it is exactly along the same lines as the pro o f of Theorem 3.4 as amo ng the constraints o f the feasible set, o ne has y d 0 = 1 , M d ( y d )  0 , M d ( g j , y d )  0 , j = 1 , . . . , m 1 . 20 R. LARAKI AND J.B. LASSERRE Pr o of of The or em 5.1. Let µ ∗ ∈ P ( K 1 ) , ν ∗ ∈ P ( K 2 ) b e optimal str ategies of play er 1 and play er 2 resp ectively , a nd let y ∗ = ( y ∗ α ) be the sequence o f momen ts of µ ∗ (w ell-defined b eca us e K 1 is compact). Then J ∗ = max ν ∈ P ( K 2 ) Z K 2  Z K 1 p ( x, z ) dµ ∗ ( x )  dν ( z ) = max ν ∈ P ( K 2 ) Z K 2 X α ∈ N n 2   X β ∈ N n 1 p αβ Z K 1 x β dµ ∗ ( x )   z α dν ( z ) = max ν ∈ P ( K 2 ) Z K 2 X α ∈ N n 2   X β ∈ N n 1 p αβ y ∗ αβ   z α dν ( z ) = max ν ∈ P ( K 2 ) Z K 2 p y ∗ ( z ) dν ( z ) = max z { p y ∗ ( z ) : z ∈ K 2 } = min λ,σ k { λ : λ − p y ∗ ( · ) = σ 0 + m 2 X k =1 σ k h k ; ( σ j ) m 2 j =0 ⊂ Σ[ z ] } with z 7→ p y ∗ ( z ) defined in (5.8). Therefore, with ǫ > 0 fixed arbitrar y , (8.2) J ∗ − p y ∗ ( · ) + ǫ = σ ǫ 0 + m 2 X k =1 σ ǫ k h k , for so me p olynomials ( σ ǫ k ) ⊂ Σ[ z ] of deg ree at most 2 d 1 ǫ . So ( y ∗ , J ∗ + ǫ, σ ǫ k ) is an admissible solutio n for the semidefinite pro gram (5 .9) whenever d ≥ max j r j and d ≥ d 1 ǫ + max k l k , b ecaus e (8.3) 2 d ≥ deg σ ǫ 0 ; 2 d ≥ deg σ ǫ k + deg h k , k = 1 , . . . , m 2 . Therefore, (8.4) λ ∗ d ≤ J ∗ + ǫ, ∀ d ≥ ˜ d 1 ǫ := max  max j r j , d 1 ǫ + max k l k  . Now, let ( y d , λ d ) b e a n a dmissible solution o f the s emidefinite progr am (5.9) with v alue λ d ≤ λ ∗ d + 1 /d . By Lemma 8 .1, there exists ˆ y ∈ R ∞ and a subsequence ( d i ) such that y d i → ˆ y p oint w is e, tha t is, (8.1) holds. But then, inv oking (8 .1) yields M d ( ˆ y )  0 a nd M d ( g j , ˆ y )  0 , ∀ j = 1 , . . . , m 1 ; d = 0 , 1 , . . . By Theor em 2.2, ther e exists ˆ µ ∈ P ( K 1 ) such that ˆ y α = Z K 1 x α d ˆ µ, ∀ α ∈ N n . On the other hand, J ∗ ≤ max ν ∈ P ( K 2 ) Z K 2  Z K 1 p ( x, z ) d ˆ µ ( x )  dν ( z ) = max z { p ˆ y ( z ) : z ∈ K 2 } = min { λ : λ − p ˆ y ( · ) = σ 0 + m 2 X k =1 σ k h k ; ( σ j ) m 2 j =0 ⊂ Σ[ z ] } SEMIDEFINITE PROGRAMMING FOR MIN-MAX PROBLEMS AND GAMES 21 with z 7→ p ˆ y ( z ) := X α ∈ N n   X β ∈ N n p αβ ˆ y β   z α . Next, let ρ := ma x z ∈ K 2 p ˆ y ( z ) (hence ρ ≥ J ∗ ), a nd cons ider the p olyno mial z 7→ p y d ( z ) := X α ∈ N n   X β ∈ N n p αβ y d β   z α . It ha s same deg ree as p ˆ y , and b y (8 .1 ), k p ˆ y ( · ) − p y d i ( · ) k → 0 as i → ∞ . Hence, ma x z ∈ K 2 p y d i ( z ) → ρ as i → ∞ , and by construction o f the semidefinite progra m (5.9), λ ∗ d i ≥ max z ∈ K 2 p y d i ( z ). Therefore, λ ∗ d i ≥ ρ − ǫ for all sufficiently large i (sa y d i ≥ d 2 ǫ ) and so, λ ∗ d i ≥ J ∗ − ǫ for a ll d i ≥ d 2 ǫ . This combined with λ ∗ d i ≤ J ∗ + ǫ for all d i ≥ ˜ d 1 ǫ , y ie lds the desired result that lim i →∞ λ ∗ d i = J ∗ bec ause ǫ > 0 fixed was arbitrar y; Finally , as the co n verging subsequence ( r i ) was a rbitrary , w e get that the en tire sequence ( λ ∗ d ) con verges to J ∗ . The co nvergence γ ∗ d → J ∗ is prov ed with similar arguments.  8.2. Pro of of Theorem 5.2. By the fla t extension theore m of Cur to and Fialkow (see e.g. [26]), y has a representing s 1 -atomic pro bability measur e µ ∗ suppo rted o n K 1 and s imilarly , y ′ has a representing s 2 -atomic probability mea sure ν ∗ suppo rted on K 2 . B ut then from the pro of of Theorem 5 .1, J ∗ ≤ max ψ ∈ P ( K 2 ) Z K 2  Z K 1 p ( x, z ) dµ ∗ ( x )  dψ ( z ) = max ψ ∈ P ( K 2 ) Z K 2 p y ( z ) dψ ( z ) = max z { p y ( z ) : z ∈ K 2 } ≤ λ ∗ d . J ∗ ≥ min ψ ∈ P ( K 1 ) Z K 1  Z K 2 p ( x, z ) dν ∗ ( z )  dψ ( x ) = min ψ ∈ P ( K 1 ) Z K 1 ˆ p y ′ ( x ) dψ ( x ) = min x { ˆ p y ′ ( x ) : x ∈ K 1 } ≥ γ ∗ t , and so as λ ∗ d = γ ∗ t one has J ∗ = λ ∗ d = γ ∗ t . This in turn implies that µ ∗ (resp. ν ∗ ) is an optimal str ategy for pla yer 1 (resp. play er 2).  Ackno wledgments This w ork was supp orted by french ANR-gra nt NT05 − 3 − 4161 2. References [1] Ash R. (1972). R ea l Analysis and Pr ob ability . Academic Press, San Diego. [2] Aumann R. J. and L. S. Shapley (1994) . Long-term comp etition—A game theo retic analysis, in Essays on Game The ory , N. Megiddo (Ed.), Springer-V erlag, New-Y ork, pp. 1-15. [3] Borgs, C. , J. Cha y es, N. Immorli ca, A. T. Kalai, V. Mir rokni, C. P apadimitriou (2009). THe Mith of the F olk Theorem. T o appear in Games and Ec onomic Behavior [4] Dask alakis C., P . W. Goldberg and C. H. Papadimitriou (2009). The Complexity of Comput- ing a Nash Equilibrium. T o appear in SIAM J. Comput. [5] Dant zig G. B. (1963). Linea r Pr o gr amming and Extenstions . Princeton Universit y Press. [6] Chen X. and X. Deng (2009). Settling the Complexity of Tw o-Play er Nash Equilibrium. T o appear i n J. ACM. 22 R. LARAKI AND J.B. LASSERRE [7] Dresher M. , S. Karlin and L. S. Shapley (1950). P olynomial Games, in Contribut ions to the The ory of Games , Annals of Mathemat ics Studies 24 , Princeton Uni v ersity Press, pp. 161-180. [8] Datta R. S. (2009). Finding all Nash Equilibria of a Fini te Game Using Polynomial Algebra. T o appear i n Eco nomic The ory . [9] Elzen A. H. v an den and A . J. J. T alman (1991). A Pr ocedure for Finding Nash Equil ibria in Bi-Matrix Games. ZOR - Met ho ds and Mo dels of Op er ations R ese ar ch , 35 , 27-43. [10] Fink A. M. (1964). Equilibri um in a Sto c hastic N -Person Game. J. Sci. Hir oshima Univ. 28 , 89-93. [11] Glicksberg I. (1952) . A F urther Generalization of the Kakutani Fixed Poin t Theorem with Applications to Nash Equilibrium Poin ts. Pr o c . Amer. Math. So c. 3 , 170-174. [12] Go vindan S. and R. Wilson (2003). A Global Newton Metho d to Compute Nash Equili bria. Journal of Ec onomic The ory , 110 , 65-86. [13] G¨ urk an G. and J. S. Pang (2009). Approximations of Nash Equilibria. Math. Pr o gr am. B, 117 , 223-253. [14] Henrion D. and J. B. Lasserre (2003). GloptiP oly : Global O ptimi zation ov er Po lynomials with Matlab and SeDuMi, ACM T r ans. Math. Soft. 29 , 165-194. [15] Henrion D., J. B. Lasserre and J. Lofb erg (2009). GloptiPoly 3: M oments, Optimization and Semidefinite Programming, Optim. Metho ds and Softwar e , 24 , 761–779. [16] Herings P . J.-J. and A. H . v an den Elzen (2002). Computat ion of the Nash Equilibrium Selected by the T racing Procedure in N-Person Games. Games and Ec onomic Behavior, 3 8 , 89-117. [17] Herings P . J.-J. and R. Peet ers (2009) . Homotop y M ethods to Compute Equili bria in Game Theory . T o appear in Ec onomic The ory . [18] Herings P . J.- J. and R. Peeters (2005). A Globally Conv ergen t A l gorithm to Compute all Nash Equilibria for n-Person Games. Annals of Op er ations R ese ar ch, 137 , 349-368. [19] Jib etean D. and E. De Klerk (2006) . Global optimization of rational f unctions: an SDP approac h. Math. Pr o g., , 10 6 , 103–1 09. [20] Khac hiyan L. (1979). A p olynomial algorithm in li near programming. Dokl. Akad. Nauk SSSR , 224 , 1093-1096. English translation in Soviet Math. Dokl., 20 , 191-194. [21] Kohlb erg E. (1974). Repeated Games with Absorbi ng States, A nn. Stat. 2 , 724-738. [22] Kostrev a M. M. and L. A. Kinard (1991). A Differen tial Homotop y Approac h f or Solving Po lynomial Optimization Pr oblems and Noncoop erative Games. Computers and Mathematics with Applic ations. [23] Laraki R., (2010) . Expli cit F ormulas for Rep eated Games with Absorbi ng Stat es. Interna- tional Journal of Game The ory , sp ecial issue in the memory of Micha el Masc hler, for thcoming (published online Decem ber 1, 2009). [24] Lasserre J. B. (2001). Gl obal Optimization with Polynomials and the Problem of Momen ts. SIAM J. Optim. 11 , 796-817. [25] Lasserre J.B. (2006) . Conv ergen t SDP-relaxations in p olynomial optimization wi th sparsity , SIAM J. Optim. 17 , 822–843. [26] Lasserre J. B. (2008) . A semidefinite programming approach to the Generalized Pr oblem of Momen ts. Math. Pr o gr am. B 112 , 65-92. [27] Lasserre J.B., M. Laurent, P . Rostalski ( 2008). Semidefinite c haracterization and c omputation of zero-dimensional real radical ideals. F ound. Comput. Math. , 8 , 607-647. [28] Lasserre J. B . (2009). Moments and sums of squares for polynomial optimization and r el ated problems. J. Glob al Optim. 45 , 39-61. [29] Lemk e C. E. and J. T. Howson (1964). Equilibrium Poin ts of Bimatrix Games. J. SIAM , 12 , 413-423. [30] Lipton R. and E. Mark akis (2004). Nash Equilibr ia via Polynomial Equations. Pr o c e e dings of the L atin Americ an Symp osium on The or etic al Informatics , Buenos A i res, Argen tina, Lecture Notes in Computer Sciences, Springer V erlag, 413-422. [31] Lipton, R. J., E. Mark akis and A. Meh ta (2003). P l a ying Large Games Us ing Simple Strate- gies. Pr o c ee dings of the 8th ACM c onfer enc e on Ele ctr onic c ommer ce , pp. 36-41. [32] Loomi s L. H. (1946 ). O n a Theorem of v on Neumann. Pr o c. Nat. Ac ad. Sc i. 32 , 213-215. [33] Nash J. F. (1950 ). Equilibr ium Poin ts in N -Person Games, Pr o c. Nat. A c ad. Sci. , 36 , 48-49. [34] N. Nisan, T. Roughgarden, E. T ardos, and V. V. V azirani (editors) (2008). Algorithmic Game The ory . Camb ridge Universit y Press (first published 2007). SEMIDEFINITE PROGRAMMING FOR MIN-MAX PROBLEMS AND GAMES 23 [35] Pap adimitriou C. H. (2008). On the Complexity of finding Nash Equilibria. Chapter 2 in Algo rithmic Game The ory. Edi ted by N . Nisan, T. Roughgarden, E. T ar dos, and V. V. V azirani, Cambridge Universit y Press (first published 2007). [36] Pap adimitriou C. H. (1994). On the Complexity of the Parit y Argument and Other Inefficien t Pro ofs of Existence. J. Compt. Syst . Sci. , 48-3 , 498-532. [37] Parrilo P . A. (2006). Polynomial Games and Sum of Squares Optimization. Pr o c ee dings of the 45 th IEEE Confer enc e on De cision and Contr ol , 2855-2860. [38] Putinar M . (1993). Positiv e Polynomials on Compact Semi-Algebraic Sets, Ind. Univ. Math. J. 42 , 969–984. [39] Rosenm¨ uller J. (1971). On a Generalization of the Lemke-Ho wson Algorithm to Non Coop- erativ e N-Person Games. SIAM J. Appl. Math. 21 , 73-79. [40] Roughgarden T. (2009). Computing Equilibria: a Computational Complexity Perspectiv e. T o appear i n Eco nomic The ory . [41] Sa v ani R. and B. v on Stengel (2006). Hard-to-Solv e Bimatrix Games. Ec onometric a , 74 , 397-429. [42] Sc hw eighofer, M. (2004). On the c omplexity of Schm¨ udgen ’s Positivstel lensatz . J. Complexit y , 20 , 529-543. [43] Sc hw eighofer, M. (2005). Optimization of p olynomials on c omp act semialgebr aic sets SIAM J. Optim. 15 , 805-825. [44] Shah P . and P . A. Parrilo (2007). Polynomial Stoc hastic Games via Sum of Squares Opti- mization. Pr o c e e dings of the 46 th IEEE Confer enc e on De cision and Contr ol , 745-750. [45] Shapley L. S. (1953). Stochastic Games. Pr o c. Nat. A c ad. Sci . 39 , 1095-1100. [46] N. Stein, P .A. Parrilo and A. Ozdaglar (2008). Correlated Equil ibria in Con tinuou s Games: Characterization and Computation. arXiv:0812.427 9v1 . [47] Sturmfels B. (2002) . Solving Syst ems of Polynomial Equations . American Mathemathical Society , Prov idence, Rho de Island. [48] V ersche lde V . (1999). PHCpac k: A general-purpose solve r for polynomial systems by homo- top y con tinu ation. ACM T r ansactions on Mathematic al Softwar e , 25 , 251-276. [49] H. W aki H., Ki m S. , Ko ji ma M ., and M . Muramatsu (2 006). Sums of Squa res and Semidefinite Programming Relaxations for Polynomial Optimi zation Pr oblems with Structured Sparsity , SIAM J. Optim. 17 , 218-242. [50] Wilson R. (1971). Computing Equilibria of N- Pe rson Games. SIAM J. Appl. Math. , 21 , 80-87. blabla E-mail addr ess : rida.larak i@polytechn ique.edu E-mail addr ess : lasserre@l aas.fr CNRS, Labora toire d’Econom ´ etrie de l ’Ecole Pol ytechnique, 91128 P alaiseau-Cedex, France LAAS-CNRS and Institute of Ma thema tics, University of Toulouse, LAAS, 7 a v enue du Colonel Roche, 31077 Toulouse C ´ edex 4, France