Extragradient methods for mean field games of controls and mean field type FBSDEs

EXTRA GRADIENT METHODS F OR MEAN FIELD GAMES OF CONTR OLS AND MEAN FIELD TYPE FBSDES MEYNARD CHARLES Abstract. In th is paper w e presen t a n umerical scheme to solve coupled mean ﬁeld forward- bac kwa rd stochastic diﬀe rential equations dri v en by monotone v ector ﬁelds. This is based on an adaptat ion of s o called extragradien t metho ds by c haracterizing solutions as zeros of m onotone v ari- ational inequalities in a Hilb er t space. W e ﬁrst in tro duce t he procedure in the context of mean ﬁeld games of controls and highligh t i ts connect ion to the ﬁctitious pla y . Under suﬃciently strong monotonicit y assumptions, we demonstrate that the sequence of approximat e sol utions conv erges ex- ponentially fast. Then we extend the method and main results to general forward bac kwa rd systems of stochastic diﬀerent ial equ ations tha t do not necessaril y stem from optimal con trol. 1. I ntr oduction 1.1. General in tro duction. In this article, w e present a n umerical metho d to approximate the so- lution of the coupled mean ﬁeld forward-backw ard sto chastic diﬀeren tial equation (FBSDE) (1.1)        X t = X 0 − Z t 0 F ( X s , U s , L ( X s , U s )) ds + √ 2 σ B t , U t = g ( X T , L ( X T )) + Z T t G ( X s , U s , L ( X s , U s )) ds − Z T t Z s dB s , for a given initial condition X 0 ∈ L 2 (Ω , R d ), mo notone co eﬃcient s ( F, G, g ), where L ( X , U ) indicates the joint law of the random v a riables X , U and ( Z t ) t ∈ [0 ,T ] is uniquely deﬁned in such a fashion that the pro cess ( U t ) t ∈ [0 ,T ] is progressively adapted with resp ect to the ﬁltra tion genera ted by the Brownian motion ( B t ) t ≥ 0 In recent years, there has b een considerable in terest in mea n ﬁeld for ward backw ard systems [19, 9]. Usually , such systems arise from the study of stochastic optimal co ntrol problems with an interacting po pulation, whether it be mean ﬁeld control [10], or mean ﬁeld ga mes [36, 34] for which t his pro babilistic formulation consists in an alternative to the study of the master eq uation [16]. In this ca se, the asso ciated problem ca n usually b e form ulated as fo llows (1.2)        X t = X 0 − Z t 0 ∇ p H ( X s , U s , L ( X s )) ds + √ 2 σ B t , U t = ∇ x u ( T , X T , L ( X T )) + Z T t ∇ x H ( X s , U s , L ( X s )) ds − Z T t Z s dB s , where ( x, p, µ ) 7→ H ( x, p, m ) is the Hamiltonian asso ciated to the control problem of play ers and u ( T , · ) the terminal v alue function for a giv en pla yer. There are ho wev er many adv antages to the added g e nr ality of considering coeﬃcients ( F, G, g ) that do not fall into this categoryn even for mea n ﬁeld g ames. Indeed the class of extended mean ﬁeld games [37] is con tained in (1.1), in particular this includes mean ﬁeld games of con trol [41]. Conceptually , this dependence of the co eﬃcients on the law of the solution makes the study of these systems diﬃcult b eyond short ho rizons of time [19, 14]. The existence and uniqueness of solutions to coupled mea n ﬁeld FBSDEs over arbitrarily lo ng interv als of time remains a challenging pro blem, and relies most often on monotonicit y ass umptions. In the particular case o f mean ﬁeld games, a theory of w ellpos edness has b een developed mostly th roug h arguments based on pa r tial diﬀerent ial equations (PDE s ) in the ﬂat monotone regime [16]. On the other hand, another notion of monotonicit y has been used to study for ward backw ard s ystems directly [36, 9, 39] through the Hilbertia n approach. In a ser ies of paper [4 3, 2 7, 4 0], this appro a ch w as extended to mean ﬁeld games under the notion o f displacement monotonicity . The metho d w e present in this article relies extensiv ely on this last notion of monotonicity we refer to as L 2 − monotonicity when co eﬃcients are not gradients. 1 2 MEYNARD CHARLES Part of this a rticle is also dedicated to the numerical approximation of mean ﬁeld FBSDEs with common noise and the developmen t of approximation sc heme for this problem. In mean ﬁeld games these mo dels ar is e from the a ddition of a noise impacting the dynamics of all play ers in the sa me fashio n [21, 36]. Mo dels in whic h such noise is pur ely additive ha ve led to master e q uations of s e c o nd order [16] and have b een studied extensively in the literature o n both MFGs [16, 13, 38, 17] and mean ﬁeld t yp e FBSDEs [20, 39]. On the other ha nd common noise can also come from a n additional sto chastic pro cess, as in [11, 12]. This leads us to consider systems of the form (1.3)                X t = X 0 − Z t 0 F ( X s , U s , L ( X s , U s |F 0 s )) ds + √ 2 σ B t , U t = g ( X T , L ( X T |F 0 T )) + Z T t G ( X s , U s , L  X s , U s |F 0 s  ) ds − Z T t Z s d ( B s , W s ) , p t = p 0 − Z t 0 b ( p s ) ds + √ 2 σ 0 W t where ( F 0 t ) t ≥ 0 represents the ﬁltra tion of the co mmo n noise and ( W s ) s ≥ 0 is a F 0 − Brownian motion. In particular it was prov en in [1 2] that this clas s of system includes the ca se of MF Gs with an additive common noise. In the s peciﬁc cas e of mean ﬁeld games, many numerical sc hemes hav e b een prop os ed to so lve the system (1.2). In the absence of c o mmon noise, s everal methods ba sed on an equiv alen t PDE formulation ha ve b een prop osed [3 , 1] and [2] for MF Gs of cont rols. Another approach is to rely on the ﬁctit ious pla y algorithm pres ent ed in [18]. On th e other ha nd, probabilistic sc hemes have also been developed to solv e FBSDEs o f the form (1.1). Ra nging from iterative metho ds based on P ica rd iterations [4, 24] to cost minimization with machine lea rning [2 2, 23]. Although the metho d we present in this article is also purely probabilistic, it relies instead on the monoto nicit y of co e ﬃcients and is inspired b y the family of extr a -gradient algo rithms [32, 44]. Let us insis t that there already exist an extensive literature on the application of these algorithms to monotone games, see [8, 2 9, 5, 30] a nd the references therein. In the ca se o f MFGs our work can b e seen as an extension to inﬁnite dimensiona l monotone games. 1.2. Main con tributions. In this article, we pr esent a probabilistic numerical method to solve mono- tone FBSDEs of the form (1.1). In the case of mean ﬁeld g ames (1.1), it can be seen as a na tural extension of prev iously developed algo rithm for mono tone games. F or this metho d, we give theoretical conv ergence r ates. In particular we show that under suﬃcientl y stro ng monotonicity as sumptions lin- ear convergence is achiev ed b y the a lgorithm including fo r mean ﬁeld games of controls. The numerical scheme we prop ose is not sens itive to the addition of common noise, in the sense that the conv ergence rate is unc hanged (although the sim ulation cost for each iteration is larg er from a pr a ctical viewp o int). 1.3. Organization o f the pap er. In Section 2 we start with some r eminders o n MF Gs of con trols. Then we exp o se the n umerical scheme prop osed in this particular context. W e give ex plicit conv ergence rates and in particular show that whenev er the game is displacement monotone, strongly in the controls, linear con vergence is ac hieved. Then, w e extend our results to the general case o f (1.1) by taking inspiration fro m the former c a se study . In section 3 we tre a t the case o f mean ﬁeld type FBSDEs with an indep endent common noise. F ormally the presence of a common noise do es not impact monotonicity , consequently the extension is straightforw ard. In a last short sectio n, we present some numerical results illustrating the con vergence b ounds obtained in the previous sections. 1.4. Notation. - let k ∈ N , k > 0, for the canonica l pro duct on R k we use the notation x · y = X i x i y i , and the follo wing notation for the induced norm | x | = √ x · x. - Let P ( R d ) b e the set o f (Bor e l) probabilit y measures o n R d , for q ≥ 0, we use the usual notation P q ( R d ) =  µ ∈ P ( R d ) , Z R d | x | q µ ( dx ) < + ∞  , for the s et of all probability mea sures with a ﬁnite q th momen t. EXTRA GRADIENT METHODS FOR MEAN FIELD GAME S OF CONTROLS AND M E AN FIELD TYPE FBSDES 3 - F or tw o measures µ, ν ∈ P ( R d ) w e deﬁne Γ( µ, ν ) to be the set of all pr o bability measure s γ ∈ P ( R 2 d ) satisfying γ ( A × R d ) = µ ( A ) γ ( R d × A ) = ν ( A ) , for all Borel set A on R d . - The W as s erstein distance b etw een tw o measures b elo nging to P q ( R d ) is deﬁned as W q ( µ, ν ) =  inf γ ∈ Γ( µ,ν ) Z R 2 d | x − y | q γ ( dx, dy )  1 q . In what follows P q ( R d ) is a lways endow ed with the a sso ciated W a sserstein distance, ( P q , W q ) being a complete metric space. - W e sa y that a function U : P q ( R d ) → R d is Lipschitz if ∃ C ≥ 0 , ∀ ( µ, ν ) ∈  P q ( R d )  2 , | U ( µ ) − U ( ν ) | ≤ C W q ( µ, ν ) . - Co nsider (Ω , F , P ) a pr obability space , - W e deﬁne L q (Ω , R d ) =  X : Ω → R d , E [ | X | q ] < + ∞  . - Whenever a random v ariable X : Ω → R d is distributed alo ng µ ∈ P ( R d ) w e use equiv a- lent ly the notations L ( X ) = µ or X ∼ µ . - M d × n ( R ) is the set of a ll matrices of s ize d × n with reals co eﬃcients, with the notation M n ( R ) ≡ M n × n ( R ). - C b ( R d , R k ) is the set of all contin uous bounded functions from R d to R k . 1.5. Prelim inary results. In this ar ticle we ﬁx an horizon T > 0 and co ns ider a complete pro babilit y space (Ω , F , P ) endo wed with a ( d + d 0 ) − dimensional Brownian motion ( B t , W t ) 0 ≤ t ≤ T . In particular we remind that the space L 2 (Ω , R ) endow ed with the inner pro duct h· , ·i : ( X , Y ) 7→ E [ X Y ] , is a Hilb ert space. W e use the notation H = ( L 2 (Ω , R ) , h· , ·i ) and the induced norm is deno ted b y ∀ X ∈ H , k X k = p h X X i . W e a lso use the notation H T for the Hilb ert s pace of square in tegrable F − adapted random pro cesses on [0 , T ] ( ( X s ) s ∈ [0 ,T ] , E " E Z T 0 | X s | 2 ds # < + ∞ and ( X s ) s ∈ [0 ,T ] is F − ada pted ) , endow ed with the inner product h X, Y i T = E " Z T 0 X t Y t dt # . Deﬁnition 1 .1. F or a function F : R d × P 2 ( R d ) → R d , we deﬁne its lift on H d , ˜ F to b e ˜ F :  H d → H d , X 7→ F ( X , L ( X )) . W e remind the follo wing adaptation o f Lemma 2.3 o f [43] Prop osition 1 .2. L et f :  R 2 d − → R ( x, y ) 7→ f ( x, y ) b e a c ontinuous function, such that ∀ ( x, y ) ∈ R 2 d , f ( x, y ) = f ( y , x ) , f ( x, x ) = 0 . Supp ose t hat for s ome µ ∈ P 2 ( R d ) with ful l supp ort on R d the fol lowing holds ∀ ( X, Y ) ∈ H 2 d , X ∼ µ, Y ∼ µ, E [ f ( X , Y )] ≥ 0 , then the ine quality is satisﬁe d p ointwise ∀ ( x, y ) ∈ R d , f ( x, y ) ≥ 0 . In par ticula r this direct cor ollary 4 MEYNARD CHARLES Corollary 1.3. L et F : R d × P 2 ( R d ) → R d b e a c ontinuous function. Supp ose that ther e exists a c onstant C F such that ∀ ( X, Y ) ∈ H 2 d , k ˜ F ( X ) − ˜ F ( Y ) k ≤ C F k X − Y k . Then for the same c onstant C F ∀ ( µ, x, y ) ∈ P 2 ( R d ) ×  R d  2 , | F ( x, µ ) − F ( y , µ ) | ≤ C F | x − y | . Pr o of. It suﬃces to apply the above pro po sition to f : ( x, y ) 7→ C 2 F | x − y | 2 − | F ( x, µ ) − F ( y , µ ) | 2 , for ﬁxed µ with full supp ort in R d . F or mea s ure that do not hav e full supp ort, the r esult then follows b y density and the con tinuit y of F in P 2 ( R d ) as in [43].  1.5.1. A primer on monotone variational ine qualities in Hilb ert sp ac es. Let ( K , h· , ·i K ) indicate a Hilber t space. Consider v : K → K a contin uous function and suppo se we are interested in the following problem (1.4) ﬁnd x ∗ ∈ K , ∀ x ∈ K , h v ( x ∗ ) , x − x ∗ i K ≥ 0 . Since K is reﬂexive, this is clea rly equiv a len t to ﬁnding a x ∗ such that v ( x ∗ ) = 0 K . If we further assume that v is mono to ne non-degenerate, i.e. ∀ x, y ∈ K , x 6 = y , h v ( x ) − v ( y ) , x − y i K > 0 , then there can exists at mo st one x ∗ solving (1.4). In particular if we assume that v is strong ly monotone, that is ∃ β > 0 , ∀ x, y ∈ K , h v ( x ) − v ( y ) , x − y i K ≥ β k x − y k 2 K , then v is a n inv ertible opera tor and it follows that there e x ists a x ∗ solution to (1.4). A v ery natural question is how to compute x ∗ in this setting. T o that end let us ﬁx so me x 0 ∈ K , γ > 0 and in tro duce the following sequence ∀ n ≥ 1 , x n +1 = x n − γ v ( x n +1 ) . Letting v γ : x 7→ x + γ v ( x ) , this is equiv alent to ∀ n ≥ 1 , x n +1 = v − 1 γ ( x n ) . How ev er since v is β − str ongly monotone, it follo ws that ∀ x, y ∈ K , h v γ ( x ) − v γ ( y ) , x − y i ≥ (1 + γ β ) k x − y k 2 K . This implies that k v − 1 γ k Lip ≤ 1 1 + γ β . Since v − 1 γ is a contraction on K , we deduce from the deﬁnition of ( x n ) n ∈ N that it conv erges expo nent ialiy to the ﬁxed point of v − 1 γ which also happens to be x ∗ . By this method w e hav e constructed a sequence ( x n ) n ∈ N which conv erges to the unique solution x ∗ of (1.4). The ma in issue with the ab ov e co ns tr uction is that the deﬁnition o f x n +1 from x n is implicit. In fact since it ma y require a ﬁxed point proc edure, computing x n +1 at each iteration can be just as hard as computing x ∗ directly . This is where extra- gradient metho d come into play , the key idea is to approximate the sequence with x n +1 ≈ x n − γ v ( x n − γ v ( x n )) , at each step. The deﬁnition of x n +1 is now explicit in function of x n , and it turns out that this approximation yields go o d res ults both in theory and practise. In particular , w e remind this quite general inequality from [6] EXTRA GRADIENT METHODS FOR MEAN FIELD GAME S OF CONTROLS AND M E AN FIELD TYPE FBSDES 5 Lemma 1.4. L et ( K , h· , ·i K ) b e a Hilb ert sp ac e, and the se quenc e ( X n , X n + 1 2 , Y n ) n ∈ N ⊂ K 3 b e deﬁne d thr ough    X n + 1 2 = X n − γ n V n , Y n +1 = Y n − V n + 1 2 , X n +1 = γ n +1 Y n +1 , for ( γ n , V n , V n + 1 2 ) n ∈ N ⊂ H 2 d , ( γ n ) n ∈ N ⊂ R + a non incr e asing se quenc e of step size and Y 1 = 1 γ 1 X 1 . Then the fol lowing holds for any x ∈ K n X i =1 h V i + 1 2 , X i + 1 2 − x i K ≤ k x − X 1 k 2 K 2 γ 1 +  1 2 γ n +1 − 1 2 γ 1  k x k 2 K + 1 2 n X i =1  γ i k V n + 1 2 − V n k 2 K − 1 γ i k X n + 1 2 − X n k 2 K  Pr o of. Although the original pro of is conducted for a seq uence in R d , s ince it o nly uses cla s sical r esults on v ector spaces endow ed with an inner pro duct, there is no diﬃcult y in extending the pro of to a general Hilb ert spa ce.  Remark 1. 5. Whenever γ n ≡ γ is a constant sequence a nd V n = v ( X n ) for some function v , this is equiv a len t to ∀ n ≥ 1 , X n +1 = X n − γ v ( X n − γ v ( X n )) . The ab ove inequality allows to work in a v ery genera l framework compared to our v ery limited in tro- duction on e x tragradient metho ds. 2. Decoupling algorithm In this ﬁrst part, we present the main idea b ehind the algorithm we in tro duce to solve FBSDEs n umerically . W e prese nt it ﬁrst in the co ntext of mean ﬁeld games of control, see [42] fo r a pr esentation of mean ﬁeld games of controls in the displace men t monotone setting. In this con text, the propo s ed method is very natural. Let us emphasize once again that there is already a n extensive litterarure on extragradient metho ds for monoto ne games [7, 6], and the extension to mea n ﬁeld g ames is quite straightforw ard. In this section we also consider dy na mics without commo n noise. 2.1. A motiv ating example from mean ﬁeld g ames. W e start with some reminders o n mean ﬁeld games of controls in the displacement mo notone framework, from the formulation of the equilibrium condition to the a sso ciated FBSDE. In particular, w e adopt a strategy which is sligthly diﬀ erent from usual. Instead of int ro ducing an implicit ﬁxed p oint mapping to write the forward backw ard system, we show that the equilibrium can b e expressed explicitely in function of the co eﬃcients through the use of an inﬁnite dimensional in verse. Although this formulation is strictly eq uiv alent to more s tandard o nes for mea n ﬁeld g ames o f c ontrols, we b elieve this makes the exp os itio n clea rer and mor e stra ightf orward (and in par ticular totally ex plicit). 2.1.1. Un iqueness and char acterization of optimal c ontr ols. Let us ﬁrst reca ll exactly the formulation of the mean ﬁeld game problem in this se tting. F or a given ﬂow of meas ur es ( m t ) t ∈ [0 ,T ] ∈ P 2 ([0 , T ] , R 2 d ) we consider the cost J ( α, ( m t ) t ∈ [0 ,T ] ) =              E " g ( X α T , µ T ) + Z T 0 L ( X α t , α t , m t ) dt # , X α t = X 0 − Z t 0 α t dt + √ 2 σ B t , µ T = π d m T . , where π d m indicates the marg inal of m ov er the ﬁrs t d v ariables for any m ∈ P 2 ( R 2 d ). W e suppo s e that J ( α, ( m t ) t ∈ [0 ,T ] ) represents the cost paid b y a play er with cont rol α whenever the distribution of all other players and their controls is ( m t ) t ∈ [0 ,T ] . The mean ﬁeld g ame problem co nsists in ﬁnding an equilibrium, that is, a distribution ( m ∗ t ) t ∈ [0 ,T ] such tha t after solving inf α J  α, ( m ∗ t ) t ∈ [0 ,T ]  , 6 MEYNARD CHARLES the play ers are still b e distributed along ( m ∗ ) t ∈ [0 ,T ] . There exist several mathematically equiv alent rigoro us deﬁnitions of this pro blem. In this a rticle we fo cus on formulating it directly at the level of controls, which is na tur a l for displacemen t monotone MFGs. Namely , solving th e mean ﬁeld ga me problem is equiv alent to ﬁnding a con trol ( α ∗ t ) t ∈ [0 ,T ] ⊂ T such tha t (2.1) α ∗ = arginf α J  α,  L ( X α ∗ t , α ∗ t )  t ∈ [0 ,T ]  . Clearly , this deﬁnition results from a ﬁxed p oint of an optimisation pro blem c hara cteristic of mean ﬁeld games. At ﬁrst glance, the existence of such a control, let alo ne the uniqueness is not clear. L et us start with a standar d result on ﬁnite dimensional optimal control Lemma 2.1. Supp ose t hat ( x, α ) 7→ ( g ( x, µ ) , L ( x, α, m )) ∈ C 1 , 1 ( R 2 d , R 2 ) uniformly in m ∈ P 2 ( R 2 d ) - If ( α t ) t ∈ [0 ,T ] ⊂  H T  d satisﬁes (2.2) α ∈ arginf α ′ ∈H T J ( α ′ , ( m t ) t ∈ [0 ,T ] ) , the fol lowing hol ds (2.3) ∀ α ′ ∈  H T  d , h∇ x g ( X α T , µ T ) , X α ′ T − X α T i + Z T 0 h  ∇ x L ∇ α L  ( X α t , α t , m t ) ,  X α ′ t − X α t α ′ t − α t  i dt ≥ 0 , wher e µ T ( dx ) = Z y ∈ R d m T ( dx, dy ) , indic ates the mar ginal of m T over its ﬁrst d varia bles. - If, furthermor e, x 7→ g ( x, µ ) i s c onvex un iformly in µ ∈ P 2 ( R d ) , and ( x, α ) 7→ L ( x, α, m ) i s c onvex, stro ngly in α , un iformly in m ∈ P 2 ( R 2 d ) t hen for any ﬂow of me asur es ( m t ) t ∈ [0 ,T ] ⊂ P 2 ( R 2 d ) , ther e exists a un ique solution to t he optimal c ontr ol pr oblem inf α J ( α, ( m t ) t ∈ [0 ,T ] ) , and it is the unique c ont ro l satisfying (2.3) . Pr o of. Let us ass ume that there ex ists a control α ∈  H T  d satisfying (2.2), for a given co n trol α ′ ∈  H T  d , (2.3) is obtained by observing that for an y λ ∈ (0 , 1] 1 λ  J ( α + λ ( α ′ − α ) , ( m t ) t ∈ [0 ,T ] ) − J ( α, ( m t ) t ∈ [0 ,T ] )  ≥ 0 . The result follows naturally by taking the limit a s λ → 0. As fo r the second pro p o sition, under the additional conv exity a ssumptions α 7→ J ( α, ( m t ) t ∈ [0 ,T ] ) is strongly con vex on  H T  d and the existence and uniqueness of a minimizer follow from standard optimization theor y . Finally let us assume that there exist tw o controls α 1 , α 2 satisfying (2.3), then the conv exity o f co eﬃcients implies that k α 1 − α 2 k T ≤ 0 , yielding α 1 = α 2 in  H T  d .  W e now mak e an ass umption on the displacemen t monotonicity of L and g . Hyp othesis 2.2. ∃ c L > 0 , ∀ ( X, Y , α, α ′ ) ∈ H 4 d , h∇ x g ( X , L ( X )) − ∇ x g ( Y , L ( Y )) , X − Y i ≥ 0 , h  ∇ x L ∇ α L  ( X, α, L ( X , α )) −  ∇ x L ∇ α L  ( Y , α ′ , L ( Y , α ′ )) ,  X − Y α − α ′  i ≥ c L k α − α ′ k 2 Remark 2. 3. Hyp othesis 2 .2 implies the conv exit y of x 7→ g ( x, µ ) and the c L − strong conv exity of ( x, α ) 7→ L ( x, α, m ) in α unifor mly in m ∈ P 2 ( R 2 d ), as a direct conse q uence of Pro po sition 1.2. W e now show thatn under these ass umptions, there exists a unique so lution to a FBSDE involving the Hilb ertian in verse o f ∇ α L . EXTRA GRADIENT METHODS FOR MEAN FIELD GAME S OF CONTROLS AND M E AN FIELD TYPE FBSDES 7 Hyp othesis 2.4. t he derivatives ∇ x g , ∇ x L, ∇ α L a r e wel l de ﬁne d and c ontinuous in a l l their ar gu- ments. Mor e over ( x, a, m ) 7→ ( ∇ x g ( x, µ ) , ∇ x L ( x, a , m ) , is Lipschi tz and ( x, m ) 7→ ∇ α L ( x, a , m ) , is Lipschi tz uniformly in a ∈ R d . Lemma 2.5. Under Hyp otheses 2.2 and 2.4, the Hilb ertian mapp ing α 7→ ∇ α L ( X , α, L ( X, α )) , admits a c ontinuous inverse in α ∈ H d for a give n X ∈ H d denote d α 7→ ∇ α ˜ L − 1 ( X, α ) . Mor e over, ther e exists a Lipsch itz c ontinuous function L inv : R 2 d × P 2 ( R 2 d ) → R d such that ∀ ( α, X ) ∈ H 2 d , ∇ α ˜ L − 1 ( X, α ) = L inv ( X, α, L ( X , α )) , and for a ny initial c ondition X 0 ∈ H d ther e exists a unique str ong solution ( X t , U t , Z t ) t ∈ [0 ,T ] to the forwar d b ackwar d system (2.4)            X t = X 0 − Z t 0 ∇ α ˜ L − 1 ( X s , U s ) ds + √ 2 σ B t , U t = U T + Z T t ∇ x L ( X s , ∇ α ˜ L − 1 ( X s , U s ) , L  X t , ∇ α ˜ L − 1 ( X s , U s )  ) ds − Z T t Z s dB s , U T = ∇ x g ( X T , L ( X T )) . Pr o of. Let us ﬁrst observe that ∇ α ˜ L − 1 is w ell deﬁned and Lipsc hitz as a function from H 2 d in to H d . F o r a given X ∈ H d , the in vertibilit y follows from the strong monotonicity of α 7→ ∇ α L ( X , α ) on H d . The fact that the mapping ( X, U ) 7→ ∇ α ˜ L − 1 ( X, U ) is Lipsc hitz o n H 2 d follows from the following inequality ∀ ( X, Y , α, α ′ ) ∈ H 4 d , C 2 F 2 c L k X − Y k 2 + h∇ α ˜ L ( X, α ) − ∇ α ˜ L ( Y , α ′ ) , α − α ′ i ≥ c L 2 k α − α ′ k 2 , where C F indicates the Lipschitz nor m of X 7→ ∇ α ˜ L ( X, α ) whic h is bo unded indep enden tly of α ∈ H d b y a ssumption. The f act that ∇ α L − 1 can b e expressed as the lift of Lipschitz con tinuous function follows from [39] Lemma 3.10. In fac t, one ca n directly check that for a n y ( x, u ) ∈ R 2 d , ( X, α ) ∈ H 2 d , L inv is given b y (2.5) L inv ( x, u, L ( X , α )) = D p H ( x, u, L ( X , ∇ α ˜ L − 1 ( X, α ))) Consequently , (2.4) is a FBSDE with Lipschitz co e ﬃcients. It is standard [14, 20] that under this condition, w ellpo sedness holds o ver a suﬃcien tly shor t horizon of time and we now prov e that this system is monotone yielding wellposedness on any time in terv a l. Pro ceeding with F ( X, U ) = ∇ α ˜ L − 1 ( X, U ) , G ( X , U ) = ∇ x L ( X s , ∇ α ˜ L − 1 ( X s , U s ) , L  X t , ∇ α ˜ L − 1 ( X s , U s )  ) . By the deﬁnition of F = ˜ L − 1 and the displacemen t monotonicity of L ∀ ( X, Y , α, α ′ ) ∈ H 4 d , h  F G  ( X, U ) −  F G  ( Y , V ) ,  X − Y U − V  i ≥ c L k F ( X , U ) − F ( Y , V ) k 2 . Since g is a lso displacement monotone b y assumption, we b elieve the existence and uniqueness o f a solution to (2.4) for an y T > 0 follow from standard tec hniques. Howev er since we couldn’t ﬁnd this exact result in the literature, we brieﬂy explain how to obtain it in Le mma A.1 in the Appendix.  Remark 2.6. Since we are co nsidering the Hilbertia n in verse of α 7→ ∇ α ( X, α, L ( X , α )) and not the in verse in a α ∈ R d for a given measure m ∈ P 2 ( R 2 d ), it is imp orta nt to note that we do not hav e the equality ∇ α ˜ L − 1 = D p H, where ( x, p, m ) 7→ H ( x, p, m ) is the Hamiltonia n a sso ciated to L . In fact L inv is given exactly by (2.5). 8 MEYNARD CHARLES W e now pres e nt how the FBSDE (2.4) relates to equilibria of the mean ﬁeld game of control (2.1). Theorem 2.7 . U nder Hyp otheses 2.2 and 2.4, ther e exists a unique solut ion to the me an ﬁeld game of c ontr ol (2.1) . It is the unique c ontr ol satisfying ∀ α ′ ∈  H T  d , h∇ x g ( X α T , L ( X α T )) , X α ′ T − X α T i + Z T 0 h  ∇ x L ∇ α L  ( X α t , α t , L ( X α t , α t )) ,  X α ′ t − X α t α ′ t − α t  i dt ≥ 0 . (2.6) and is given by ∀ t ≤ T , α t = ∇ α ˜ L − 1 ( X t , U t ) , wher e ( X t , U t , Z t ) t ∈ [0 ,T ] is the unique solution t o (2.4) . Pr o of. The fact that a ny control solution to the mean ﬁeld game problem m ust satisfy (2.7) is a direct consequence of Lemma 2.1 in the displacement monotone setting. Un iqueness of a solution to the mean ﬁeld ga me problem then follows from the fa c t that, under our displacement mo notonicity assumption on L, g , there can b e a t most o ne control satisfying (2.7). Finally it remains to ch eck that (2.7) holds for ∀ t ≤ T , α t = ∇ α ˜ L − 1 ( X t , U t ) , which is a direct consequence of the deﬁnition of ( X t , U t , Z t ) t ∈ [0 ,T ] as a solution to the FBSDE (2.4).  Existence and uniqueness results of a solution to the mean ﬁeld ga me o f co nt rol under the standing assumptions is already well known, see for example the recent work [31 ]. The main novelt y here is that the forward backw ard system c a n b e made explicit by using the Hilbertian inv erse instead of relying on an implicit ﬁxed po in t map. In our opinion, this makes the exposition more straightforward and will also b e relev a n t later in the ar ticle. 2.1.2. A n algorithm fr om the the ory of monotone variational ine qualities. F or a given con trol ( α t ) t ∈ [0 ,T ] ∈  H T  d , we ﬁrst introduce the follo wing backward pro cess U α t = ∇ x g ( X α T , L ( X α T )) + Z T t ∇ x L ( X α s , α s , L ( X α s , α s )) ds − Z T t Z α s dB s . The existence and uniqueness of a pa ir ( U α , Z α ) t ∈ [0 ,T ] for a g iven control is classic in the theory of decoupled for ward bac kward systems, see [46] Theo rem 4 .3.1. Indeed since the pair ( α s , X α s ) s ∈ [0 ,T ] is known in this cas e , the ab ov e system ca n b e view ed as a standa r d backw ard Lipschitz SDE of the for m (2.7) Y t = ξ − Z T t f ( s, ω , Y s , Z s ) ds − Z T t Z s dB s , with f ( s, ω , Y s , Z s ) = −∇ x L ( X α s ( ω ) , α s ( ω ) , L ( X α s , α s )) . W e now in tro duce the following functional on  H T  d v :   H T  d − →  H T  d ( α t ) t ∈ [0 ,T ] 7→ ( ∇ α L ( X α t , α t , L ( X α t , α t )) − U α t ) t ∈ [0 ,T ] . Lemma 2.8. Under Hyp othesis 2.4, t her e exists a c onstant C v dep ending only on the Lipschitz c on- stants of ∇ x g , ∇ x L and T only such that ∀ ( α, α ′ ) ∈  H T  2 d , k v ( α ) − v ( α ′ ) k T ≤ C v k α − α ′ k T . F urthermor e, if Hyp othesis 2.2 h olds then v is c L -str ongly monotone on  H T  d and a c ontr ol α ∗ ∈  H T  d is a solution to the me an ﬁeld game of c ontr ol pr oblem (2.1) if and only if ∀ α ∈  H T  d h v ( α ∗ ) , α − α ∗ i ≥ 0 . EXTRA GRADIENT METHODS FOR MEAN FIELD GAME S OF CONTROLS AND M E AN FIELD TYPE FBSDES 9 Pr o of. Let us ﬁrst prove the statement on the Lipschitz r e g ularity of v . It is quite evident that for any t wo con trols ( α, α ′ ) ∈  H T  2 d , Z T 0 k X α t − X α ′ t k dt ≤ k α − α ′ k T . Then applying Ito’s Lemma for ﬁxed t ∈ [0 , T ] k U α T − U α ′ T k 2 = k U α t − U α ′ t k 2 − E " Z T t  ∇ x ˜ L ( X α s , α s ) − ∇ x ˜ L ( X α ′ s , α ′ s )  ·  U α s − U α ′ s  ds # + Z T t k Z α s − Z α ′ s k 2 ds which implies that ∀ t ∈ [0 , T ] , k U α t − U α ′ t k 2 ≤ ( k∇ x g k Lip + T k∇ x L k Lip ) k α − α ′ k 2 T + k∇ x L k Lip 4 Z T t k U α s − U α ′ s k 2 ds, and the conclusion follows from Gr onw all lemma. F or the second part of the lemma, by deﬁnition of ( U α t ) t ∈ [0 ,T ] , ∀ α, α ′ ∈  H T  d , h v ( α ) , α ′ − α i T = h∇ x g ( X α T , L ( X α T )) , X α ′ T − X α T i + Z T 0 h  ∇ x L ∇ α L  ( X α t , α t , L ( X α t , α t )) ,  X α ′ t − X α t α ′ t − α t  i dt, and the r esult follows from Hypo thesis 2 .2 and Theorem 2.7  With this, we hav e proved tha t the pr oblem of solving for the mean ﬁeld game equilibrium can b e reformulated as solving a v a riational inequa lit y o n the space of controls  H T  d . F or Lipschit z functions v on the space of controls  H T  d , we in tro duce the standard notation k v k Lip = sup α,α ′ ∈ ( H T ) d k v ( α ) − v ( α ′ ) k T k α − α ′ k T . T a king inspiration from the dual extrap olation a lgorithm introduced in [32], w e present an algorithm that conv erge s to the mean ﬁeld game optimal con trol s tarting from a n y initial co ndition Theorem 2. 9. T ake an initial c ondition α 1 ∈  H T  d , for n ≥ 1 we intr o duc e the se quenc es  α n + 1 2 = α n − γ v ( α n ) , α n +1 = α n − γ v ( α n + 1 2 ) . If γ ≤ 1 k v k Lip , then under Hyp otheses 2.4, 2.2, letting ¯ α n = 1 n n X i =1 α i + 1 2 , the fol lowing holds ∀ n ≥ 1 , k α ∗ − ¯ α n k 2 T ≤ k α ∗ − α 1 k 2 T 2 γ c L n . Pr o of. Let us ﬁrst observe b y an a pplication of Lemma 1.4,for any con trol α ∈  H T  d , ∀ n ≥ 1 , n X i =1 h v ( α i + 1 2 ) , α i + 1 2 − α i T ≤ k α − α 1 k 2 T 2 γ . On the o ther hand, intro ducing the no ta tion ¯ α n = 1 n n X i =1 α i + 1 2 , 10 MEYNARD CHARLES n X i =1 h v ( α i + 1 2 ) , α i + 1 2 − α i T = n X i =1 h v ( α i + 1 2 ) − v ( α ) , α i + 1 2 − α i T + h v ( α ) , α i + 1 2 − α i T ≥ n h v ( α ) , ¯ α n − α i T + c L n X i =1 k α − α i + 1 2 k 2 T ≥ n h v ( α ) , ¯ α n − α i T + nc L k ¯ α n − α k 2 T , where we use d the strong mono tonicit y of v on  H T  d . Putting together those tw o inequalities and ev a luating for α = α ∗ solution of the mean ﬁeld g ame problem y ields ∀ n ≥ 1 , k α ∗ − ¯ α n k 2 T ≤ k α ∗ − α 1 k 2 T 2 γ c L n .  Remark 2 .10. A corollar y of this proo f is that + ∞ X n =1 k α ∗ − α n + 1 2 k 2 T < + ∞ , which implies that even tually , the sequence of last iterates ( α n + 1 2 ) n ∈ N conv erges to α ∗ with ∀ C > 0 , ∃ n 0 , ∀ n ≥ n 0 , k α n + 1 2 − α ∗ k 2 T ≤ C n , but this do e s not g ive an explicit rate, hence the choice of w orking r a ther on ( ¯ α n ) n ∈ N The ab ov e iterative pro cedur e can be generalized to other monotonicity conditions, as we will see later. In the sp eciﬁc setting of strong monotonicity in the cont rol, we can s how that for asuﬃciently small step size, the conv ergence of the last iterate is exponential Theorem 2. 11. Under the same assumptions as The or em 2.9, If γ < min 1 2 k v k Lip , c L k v k 2 Lip ! , then ther e exists a c onstant λ ∈ (0 , 1 ) dep ending only on γ , k v k Lip and c L such that ∀ n ∈ N , k α n − α ∗ k T ≤ λ n k α 0 − α ∗ k T . Pr o of. In the constant step regime let us ﬁr st rema rk that the sequence ( α i ) i ∈ N is deﬁned through α i +1 = α i − γ v ( α i − γ v ( α i )) . Letting α ∗ be the control asso ciated to (2 .8), it follows that k α i +1 − α ∗ k 2 T = k α i − α ∗ − γ v ( α i − γ v ( α i )) k 2 T = k α i − α ∗ k 2 T + γ 2 k v ( α i − γ v ( α i )) k 2 T − 2 γ h α i − α ∗ , v ( α i − γ v ( α i ) i T . Using the fact that v ( α ∗ ) = 0 , we get k α i +1 − α ∗ k 2 T = k α i − α ∗ k 2 T + γ 2 k v ( α i − γ v ( α i )) k 2 T − 2 γ h α i − α ∗ , v ( α i − γ v ( α i )) − v ( α ∗ ) i T . Using the str ong monotonicity of v , k α i +1 − α ∗ k 2 T ≤ k α i − α ∗ k 2 T + γ 2 k v ( α i − γ v ( α i )) k 2 T − 2 γ c L k α i − γ v ( α i ) − α ∗ k 2 T − 2 γ 2 h v ( α i ) , v ( α i − γ v ( α i )) − v ( α ∗ ) i T Whic h implies that k α i +1 − α ∗ k 2 T ≤ (1 − 2 γ c L ) k α i − α ∗ k 2 T − γ 2 k v ( α i − γ v ( α i )) k 2 T − 2 γ 3 c L k v ( α i ) k 2 T + 4 γ 2 c L h α i − α ∗ , v ( α i ) i T − 2 γ 2 h v ( α i ) − v ( α i − γ v ( α i )) , v ( α i − γ v ( α i )) i T . EXTRA GRADIENT METHODS FOR MEAN FIELD GAME S OF CONTROLS AND M E AN FIELD TYPE FBSDES 11 Since v is Lipsc hitz clearly k α i +1 − α ∗ k 2 T ≤ (1 − 2 γ c L + 4 γ 2 c L k v k Lip ) k α i − α ∗ k 2 T − γ 2 k v ( α i − γ v ( α i )) k 2 T − 2 γ 3 c L k v ( α i ) k 2 T + 2 γ 3 k v k Lip k v ( α i ) k T k v ( α i − γ v ( α i )) k T , and we ﬁnally obtain k α i +1 − α ∗ k 2 T ≤ (1 − 2 γ c L + 4 γ 2 c L k v k Lip ) k α i − α ∗ k 2 T + γ 2 γ k v k 2 Lip c L − 1 ! k v ( α i − γ v ( α i )) k 2 T . In particular, letting γ ∗ = min 1 2 k v k Lip , c L k v k 2 Lip ! , for any γ < γ ∗ the sequence ( α i ) i ∈ N conv erges expo nen tially at the following rate ∀ n ∈ N k α n − α ∗ k 2 T ≤ (1 − 2 γ c L + 4 γ 2 c L k v k 2 Lip ) n k α 0 − α ∗ k 2 T .  Remark 2.12. It is well known that a nother r egime o f wellpos edness for the mean ﬁeld game problem 2.1 is a strong monotonicity condition on the forward v ariable X . Namely , if co eﬃcients ar e Lipschitz, ∇ α ˜ L has a well deﬁned, Lipschi tz inv erse a nd the following mono tonicit y condition is satisﬁed ∃ c L > 0 , ∀ ( X , Y , α, α ′ ) ∈ H 4 d , h∇ x g ( X , L ( X )) − ∇ x g ( Y , L ( Y )) , X − Y i ≥ c L k X − Y k 2 , h  ∇ x L ∇ α L  ( X, α, L ( X , α )) −  ∇ x L ∇ α L  ( Y , α ′ , L ( Y , α ′ )) ,  X − Y α − α ′  i ≥ c L k X − Y k 2 then there ex ists a unique strong solution to (2.4 )(see [12] for example). Although mo s t results pre- sented in this article extend easily to this setting (in particular the con vergence o f ( ¯ α n ) n ∈ N ), it is unclear to us whether this Theorem on the exp onential co nv ergence of last iterates remains true. This is why w e pre s ent ed ﬁrst a more genera l result on the conv ergence of a veraged con trols ( ¯ α n ) n ∈ N . 2.1.3. Link with the ﬁctitious play. The algorithm we present is clea rly diﬀerent from t he ﬁctiti ous play a lgorithm in mea n ﬁeld g a me. The most obvious diﬀerence b eing that we consider dynamic on controls rather than on the distribution of play ers, but the learning pro cedure is also quite diﬀerent. Nevertheless, there is a s tr ong link betw een the tw o algorithms, namely the ﬁctitious play ca n also be in terpreted a s solving a v ariationa l inequalit y . In this section letting ( µ t ) t ∈ [0 ,T ] ⊂ P 2 ( R d ) we consider the cost J ( α, µ t ) t ∈ [0 ,T ] ) =          E " g ( X α T , µ T ) + Z T 0 L ( X α t , α t , µ t ) dt # X α t = X 0 − Z t 0 α t dt + √ 2 σ B t with L ( x, α, µ ) = L ( x, α ) + f ( x, µ ) , and the mea n ﬁeld game pro blem of ﬁnding a ﬂow of measure ( µ ∗ ) t ∈ [0 ,T ] such tha t ( α ∗ = arginf α J ( α, ( µ ∗ ) t ∈ [0 ,T ] ) , ∀ t ∈ [0 , T ] , µ ∗ t = L ( X α ∗ t ) . Let P a = n ( m t ) t ∈ [0 ,T ] ⊂ P 2 ( R 2 d ) , ∃ α ∈  H T  d , m = ( L ( X α t , α t )) t ∈ [0 ,T ] o , the s et o f a ll po ssible distributions. Indicating the duality pro duct b etw een f ∈ C b ( R d ) a nd µ ∈ P ( R d ) b y ( f , µ ) = Z R d f ( x ) µ ( dx ) , 12 MEYNARD CHARLES we introduce the following duality product b etw een ﬂow o f measures ( m t ) t ∈ [0 ,T ] ∈ C ([0 , T ] , P 2 ( R 2 d )) and functions ( f , g ) ∈ C ([0 , T ] , C ( R 2 , R )) × C ( R 2 d , R ) (( f , g ) , ( m t ) t ∈ [0 ,T ] ) T = ( g , m T ) + Z T 0 ( f , m t ) dt. F o r a g iven ( m t ) t ∈ [0 ,T ] ∈ P a with ma rginal ( µ t ) t ∈ [0 ,T ] ov er the ﬁrst d v aria bles w e deﬁne v ( m ) = (( t, x, α ) 7→ L ( x, α, µ t ) , x 7→ g ( x, µ T )) . By deﬁnition o f P a , it follo ws that for an y m ′ ∈ P a there exists a control α m ′ ∈  H T  d such tha t ∀ t ∈ [0 , T ] m t = L ( X α m ′ t , α m ′ t ) , moreov er the following holds ( v ( m ) , m ′ ) T = J (( α m ′ t ) t ∈ [0 ,T ] , ( µ t ) t ∈ [0 ,T ] ) . In particular s olving the mean ﬁeld game problem is equiv alen t to ﬁnding m ∗ ∈ P a such tha t ∀ m ∈ P a , ( v ( m ∗ ) , m − m ∗ ) T ≥ 0 , Moreov er let us observe that for any m, n ∈ P a , with marginal over the ﬁrst d v aria bles b eing r esp ec- tiv ely µ m , µ n ( v ( m ) − v ( n ) , m − n ) T = ( g ( · , µ m T ) − g ( · , µ n T ) , µ m T − µ n T ) + Z T 0 ( f ( · , µ m t ) − f ( · , µ n t ) , µ m t − µ n t ) dt. Consequently , if b oth f and g ar e ﬂat monotone then v is a monotone functional from P a in to C ([0 , T ] , C ( R 2 , R )) × C ( R 2 d , R ) with the dualit y pro duct ( · , · ) T . 2.2. Beyond mean ﬁeld games: monotone FBSDE. In the case o f mean ﬁeld game, there is a canonical monotone functional ar ising from the optimal con trol problem behind th e game. W e are now interested in extending the alg o rithm presen ted to general monotone FBSDEs. W e consider the following sy stem (2.8)        X t = X 0 − Z t 0 F ( X s , U s , L ( X s , U s )) ds + √ 2 σ B t , U t = g ( X T , L ( X T )) + Z T t G ( X s , U s , L ( X s , U s )) ds − Z T t Z s dB s . Clearly , the forward backw ard system asso ciated to the mean ﬁeld game of control falls in to a particular case of ( 2.8). W e place ourse lves under the following assumptions Hyp othesis 2.13. F , G : R 2 d × P 2 ( R 2 d ) → R d ar e Lipschitz and g : R d × P 2 ( R d ) → R d is Lipschi tz W e also ass ume that co eﬃcients are L 2 − monotone, Hyp othesis 2.14. ∃ c F > 0 , ∀ ( X, Y , U , V ) ∈ H 4 d , h g ( X , L ( X )) − g ( Y , L ( Y )) , X − Y i ≥ 0 , h  F G  ( X, Y , L ( X , Y )) −  F G  ( Y , V , L ( Y , V )) ,  X − Y U − V  i ≥ c F k U − V k 2 Obviously , this assumption corresp onds to the displacemen t monotonicit y assumption on the La - grangia n L in the case o f MF Gs. It has alr e a dy been observed numerous times that the w ellp osedness of this system under L 2 − monotonicity do es not dep end o n whether o r not the co eﬃcients F , G a re gradients. First in Lio ns’ Lectures with the Hilbertian approach to master equa tions [36, 1 4], then in [9] with a purely probabilistic appr o ach, a nd more recently in [39] for w eak solutions. Theorem 2.15. Under Hyp otheses 2.13 and 2.14, for any X 0 ∈ H d ther e exists a unique str ong solution ( X t , U t , Z t ) t ∈ [0 ,T ] to the FBSDE (2.8) . EXTRA GRADIENT METHODS FOR MEAN FIELD GAME S OF CONTROLS AND M E AN FIELD TYPE FBSDES 13 Pr o of. The exis tence of a strong solution is a direct consequence of [39] Lemma 3.2 1 combined with Lemma 3.9. As for uniqueness it is a natural consequence o f the stro ng monotonicity of coe ﬃcien ts. Let us ﬁx a n initial condition X ∈ H d an a ssume that there exist tw o solutions ( X i t , U i t , Z i t ) t ∈ [0 ,T ] for i = 1 , 2 starting from X i 0 = X . Deﬁning ∀ t ∈ [0 , T ] , V t = ( U 1 t − U 2 t ) · ( X 1 t − X 2 t ) , using the mo noto city of ( F, G ) , g and a pplying Ito’s Lemma to ( V t ) t ∈ [0 ,T ] we get ∀ t ∈ [0 , T ] , 0 ≤ E [ V T ] ≤ E [ V t ] ≤ E [ V 0 ] = 0 . Then using this time the s trong monotonicity of ( F, G ) in U it follows that E " Z T 0 | U 1 t − U 2 t | 2 dt # = 0 , and by a direct applica tion of Gr onw all’s lemma we also hav e ∀ t ∈ [0 , T ] , k X 1 t − X 2 t k = 0 , hence the uniqueness of a stro ng solution.  A natural ques tio n at this p oint is whether we ca n asso ciate a monotone v ariationa l inequality to this system a s we did in the case of mean ﬁeld g ames 2.1. T o that end, we intro duce the following parametrizatio n ∀ α ∈  H T  d , X α t = X 0 − Z t 0 α s ds + √ 2 σ B t . In the case of MFGs, this comes from the formulation of the co ntrol problem. Here, it ca n b e seen as a lineariza tion of the FBSDE (2.8). In particular, ( X α ∗ t ) t ∈ [0 ,T ] H T = ( X t ) t ∈ [0 ,T ] if and only if (2.9) E " Z T 0 | α ∗ t − F ( X t , U t , L ( X t , U t )) | 2 dt # = 0 . Let us now remark the follo wing Lemma 2.16. under Hyp otheses 2.13 and 2.14, the Hilb ertian lift of F , ˜ F U 7→ ˜ F ( X , U ) , has a wel l deﬁne d inverse in U ∈ H d for any given X ∈ H d . Mor e over, denoting t his inverse by ˜ F − 1 u , ther e exists a c onstant C d ep ending only on k ˜ F k Lip and c F such that ∀ ( X, Y , α, α ′ ) ∈ H 4 d , k ˜ F − 1 u ( X, α ) − ˜ F − 1 u ( Y , α ′ ) k ≤ C ( k X − Y k + k α − α ′ k ) . Pr o of. Both the in vertibilit y and the reg ula rity o f the in verse follows natura lly from the s tr ong mono - tonicit y of ˜ F on H d . This is a straighforward a daptation of Lemma 2.5.  Since it has no w been established that ˜ F is inv ertible, (2.9) implies that a co nt rol α solv es the FBSDE (2.8) if and only if ( U t ) t ∈ [0 ,T ] = ( ˜ F − 1 u ( X α t , α t )) t ∈ [0 ,T ] . In light of this new representation, we int ro duce the follow ing linear form Lemma 2.17. F or a given α ∈  H T  d , let l ( α ) b e the line ar form deﬁne d by ∀ α ′ ∈  H T  d , l ( α )( α ′ ) = h g ( X α T ) , X α ′ T i + Z T 0 h ˜ F − 1 u ( X α t , α t ) , α ′ t i + h ˜ G ( X α t , ˜ F − 1 u ( X α t , α t )) , X α ′ t i dt. Under Hyp otheses 2.14 and 2.13 - ∀ ( α, α ′ ) ∈ H T ( l ( α ) − l ( α ′ ))( α − α ′ ) ≥ c F Z T 0 k ˜ F − 1 u ( X α t , α t ) − ˜ F − 1 u ( X α ′ t , α ′ t ) k 2 dt 14 MEYNARD CHARLES - L et ∀ t ∈ [0 , T ] α ∗ t = ˜ F ( X t , U t ) , for ( X t , U t , Z t ) t ∈ [0 ,T ] the unique solution to (2 .8) , t hen for any ( α ′ , α ′′ ) ∈ ( H T ) 2 d l ( α ∗ )( α ′ − α ′′ ) = 0 . Pr o of. This ﬁr st statement is a direct consequence o f the monotonicity of co eﬃcients. F o llowing Hypo thesis 2.14, w e already know that for all ( X, Y , U , V ) ∈ H 4 d , h ˜ F ( X, U ) − ˜ F ( Y , V ) , U − V i + h ˜ G ( X , U ) − ˜ G ( Y , V ) , X − Y i ≥ c F k U − V k 2 . By choosing U = ˜ F − 1 u ( X, α ) , V = ˜ F − 1 u ( Y , α ′ ) for s ome α, α ′ ∈ H d , we get h α − α ′ , ˜ F − 1 u ( X, α ) − ˜ F − 1 u ( Y , α ) i + h ˜ G ( X , ˜ F − 1 u ( X, α )) − ˜ G ( Y , ˜ F − 1 u ( Y , α )) , X − Y i ≥ c F k ˜ F − 1 u ( X, α ) − ˜ F − 1 u ( Y , α ) k 2 , and the ﬁrs t statetement follo ws natura lly . As for the second statement, it suﬃces to remark that Z T 0 k X α ∗ t − X t k 2 dt = 0 .  W e now hav e all the tools at hand to pr esent a mono tone funct ional asso ciated to the FBSDE (2.8) Lemma 2.18. Under Hyp otheses 2.14 and 2.13, for any α ∈  H T  d , X 0 ∈ H d the system (2.10)        X α t = X 0 − Z t 0 α s ds + √ 2 σ B t , U α t = ˜ g ( X α T ) + Z T t ˜ G ( X α s , ˜ F − 1 u ( X α s , α s )) ds − Z T t Z α s dB s , has a unique solution ( X α t , U α t , Z α t ) t ∈ [0 ,T ] . Mor e over, letting v : (  H T  d − →  H T  d ( α t ) t ∈ [0 ,T ] 7→  ˜ F − 1 u ( X α t , α t ) − U α t  t ∈ [0 ,T ] , - t her e exists a c onstant C dep ending only on T , k ( ˜ F , ˜ G ) k Lip , k ˜ g k Lip and c F such that ∀ ( α, α ′ ) ∈ ( H T ) 2 d , k v ( α ) − v ( α ′ ) k T ≤ C k α − α ′ k T . - ∀ ( α, α ′ , α ′′ ) ∈ ( H T ) 3 d h v ( α ) , α ′ − α ′′ i T = l ( α )( α ′ − α ′′ ) . Pr o of. Since the system (2.1 0) is a decoupled forward bac kward sy s tem with Lipsc hitz co eﬃcients the wellpo sedness follo ws once aga in from [4 6] Theo rem 4 .3.1. Then, the ﬁrst statement follows from an application of Gro n wall’s lemma. The seco nd s ta temen t is just a consequence of the deﬁnition of ( U α t ) t ∈ [0 ,T ] for a given α ∈  H T  d and Ito’s lemma.  W e may no w state the corresp onding theorem to Theorem 2.9 in the case of FBSDEs Theorem 2.19. L et v b e deﬁne d as in L emma 2.18, for a given i nitial c ondition α 1 ∈  H T  d and n ≥ 1 we intr o duc e the se quenc es  α n + 1 2 = α n − γ v ( α n ) , α n +1 = α n − γ v ( α n + 1 2 ) , If γ ≤ 1 k v k Lip , then under Hyp otheses 2.14, and 2.13, letting ∀ t ∈ [0 , T ] , ¯ U n t = 1 n n X i =1 ˜ F − 1 u ( X α i + 1 2 t , α i + 1 2 t ) , the fol lowing holds EXTRA GRADIENT METHODS FOR MEAN FIELD GAME S OF CONTROLS AND M E AN FIELD TYPE FBSDES 15 - ∀ n ≥ 1 , Z T 0 k U t − ¯ U n t k 2 dt ≤ 1 2 γ c L n Z T 0 k ˜ F ( X t , U t ) − α 1 t k 2 dt. - L etting ∀ t ∈ [0 , T ] , ¯ X n t = X 0 − Z t 0 ˜ F ( ¯ X n s , ¯ U n s ) ds + √ 2 σ B t , ther e exists a c onstant C dep ending only on T and k ˜ F k Lip such that Z T 0   ¯ X n t − X t   2 dt ≤ C 2 γ c L n Z T 0   F ( X t , U t ) − α 1 t   2 dt. Pr o of. W e follow the same idea as in Theorem 2.9 of using the monotone functional v com bined with Lemma 1.4. Fixing α ∈  H T  d this yields n h v ( α ) , ¯ α n − α i T + c F n X i =1 Z T 0     ˜ F − 1 u ( X α t , α t ) − ˜ F − 1 u ( X α i + 1 2 t , α i + 1 2 t )     2 dt ≤ n X i =1 h v ( α i + 1 2 ) , α i + 1 2 − α i T ≤ k α − α 1 k 2 T 2 γ , for ¯ α n = 1 n n X i =1 α i + 1 2 . Ev aluating this express ion for α ∗ =  ˜ F ( X t , U t )  t ∈ [0 ,T ] where ( X t , U t , Z t ) t ∈ [0 ,T ] is the unique strong solution to (2.8 ) w e ge t n Z T 0      U t − 1 n n X i =1 ˜ F − 1 u ( X α i + 1 2 t , α i + 1 2 t )      2 dt ≤ n X i =1 Z T 0     U t − ˜ F − 1 u ( X α i + 1 2 t , α i + 1 2 t )     2 dt ≤ 1 2 γ c F Z T 0   ˜ F ( X t , U t ) − α t   2 dt proving the ﬁrst claim. The second claim follows naturally from a n application of Gro nw all Lemma.  Remark 2.20. Clearly this pro cedure requires precise knowledge of F − 1 u to lead to a re a sonable algorithm. Although this was not a problem in mean ﬁeld g ames of control, as no in verse w as used in the deﬁnition of v , it app ear s that this algor ithm is not v ery suitable to co mpute the solution of F o rward backward sys tems where the forward driver F depends on the law of the bac kward pro cess U . In this case the Hilbertian inv erse is in gener a l not trivial, whereas when F do esn’t dep end on the law of the backw ard pro cess, ˜ F − 1 u is just ( X, U ) 7→ F − 1 u ( X, U , L ( X )) where u 7→ F − 1 u ( x, u, µ ) indicates the inv erse of u 7→ F ( x, u, µ ) in R d for a given pair ( x, µ ) ∈ R d × P 2 ( R d ). Remark 2 .21. Le t us observe that, if the follo wing monoto nicit y condition ∃ c F > 0 , ∀ ( X , Y , U, V ) ∈ H 4 d , h g ( X , L ( X )) − g ( Y , L ( Y )) , X − Y i ≥ 0 , h  F G  ( X, Y , L ( X , Y )) −  F G  ( Y , V , L ( Y , V )) ,  X − Y U − V  i ≥ c F k F ( X , U, L ( X, U )) − F ( Y , V , L ( Y , V )) k 2 , is satisﬁed instead. Then ∀ ( α, α ′ ) ∈  H T  2 d , h v ( α ) − v ( α ′ ) , α − α ′ i T ≥ c F k α − α ′ k 2 T , and an analog ue to Theo rem 2.11, on the ex p o nential conv ergence to the solution of the FBSDE (2.8) follows natura lly . the W ellp osedness of (2.8) under suc h monotonicity condition is proven in the app endix (A.1). In particular, this monotonicity condition is alw ays satisﬁed whenever co eﬃcients are Lipschit z and the pair ( F , G ) is strongly monotone in ( X , U ) 16 MEYNARD CHARLES In the case of mean ﬁeld games, there exists a canonica l monotone fu nctional c o ming from the optimal co nt rol repr esentation of the problem. In a sense this cor resp onds to a po ten tial reg ime compared to g eneral FBSDEs. In this latter case, it is unclear to us whether the para metrization we in tro duce (2.10) yields the b est results. Although it cer tainly feels like a na tural ex tensio n to MF Gs, depending on the pro blem there might exist pa rametrizations leading to muc h b etter conv ergence rates. Before e nding this section w e present a lemma that will pr ove useful to estimate the conv ergence rate of a sequence of a pproximate s o lutions Lemma 2.2 2. Under Hyp otheses 2.13 and 2.14 , letting ( X s , U s , Z s ) s ∈ [0 ,T ] b e the unique str ong solution to (2.8) , ther e exists a c onstant C dep ending only on c F , T and t he Lipschitz c onstants of F , G, g such that for any c ontr ol α ∈ H T ∀ t ∈ [0 , T ] , k U α t − U T k , k X α t − X t k ≤ C k v ( α ) k T . Pr o of. By deﬁnition of v ( α ) ∀ t ∈ [0 , T ] , α t = F ( X α t , U α t + v ( α ) t , L ( X α t , U α t + v ( α ) t )) . Naturally , this is equiv alent to the fac t that the pair ( X α t , U α t ) t ∈ [0 ,T ] solves (2.11)        X α t = X 0 − Z t 0 F ( X α s , U α s + v ( α ) s , L ( X α s , U α s + v ( α ) s )) ds + √ 2 σ B t , U α t = g ( X α T , L ( X α T )) + Z T t G ( X α s , U α s + v ( α ) s , L ( X α s , U α s + v ( α ) s )) ds − Z T t Z α s dB s . Let ( X t , U t ) t ∈ [0 ,T ] be the unique solution to (2.8) w e deﬁne ∀ t ∈ [0 , T ] , V t = ( U t − U α t ) · ( X t − X α t ) . By Ito’s Lemma, ∀ t ∈ [0 , T ] , V T = V t − Z T t (∆ F s · ( U s − U α s ) + ∆ G s · ( X s − X α s )) ds, with the no tation ∀ s ∈ [0 , T ] , ∆ F s = ( X s , U s , L ( X s , U s )) − ( X α s , U α s + v ( α ) s , L ( X α s , U α s + v ( α ) s )) , and an ana logue deﬁnition for (∆ G s ) s ∈ [0 ,T ] . By the mo notonicity of coeﬃcients a nd their Lipschitz regularity , there exists a constant C Lip depending only on the Lipsc hitz norms of F, G suc h tha t (2.12) ∀ t ∈ [0 , T ] , 0 ≤ E [ V T ] ≤ E [ V t ] + C Lip Z T 0 k v ( α ) s k ( k X s − X α s k + k U s − U α s k ) ds | {z } C α T . Using the str ong monotono city of ( F, G ) we deduce that ∀ t ∈ [0 , T ] , c F Z t 0 | U s − U α s | 2 ds + V t ≤ V 0 + C Lip Z t 0 k v ( α ) s k ( k X s − X α s k + k U s − U α s k ) ds. Using (2.1 2), it follows that (2.13) ∀ t ∈ [0 , T ] , c F 2 Z t 0 k U s − U α s k 2 ds ≤ E [ V 0 ] + C 2 Lip  2 c F + 1  k v ( α ) k 2 T + Z t 0 k X s − X α s k 2 ds + C α T . Since ∀ t ∈ [0 , t ] , k X α t − X t k 2 ≤ C Lip Z t 0  k X α s − X s k 2 + 2 k U α s − U s k 2 + 2 k v ( α ) s k 2  ds, estimating the term dep ending o n ( U α s − U s ) s ∈ [0 ,t ] with (2.13), we get by a n application o f Gronw all’s lemma that there exists a constant C depending o n c F , T a nd the Lipsch itz constants o f ( F, G ) such that (2.14) ∀ t ∈ [0 , T ] , k X α t − X t k 2 ≤ C  k v ( α ) k 2 T + C α T  . In tegra ting from 0 to T and us ing Cauch y-sch wartz ineq ua lit y we get Z T 0 k X s − X α s k 2 ds ≤ k v ( α ) k 2 T ( C + C 2 Lip C 2 ) + C Z T 0 k v ( α ) s kk U α s − U s k ds ! . EXTRA GRADIENT METHODS FOR MEAN FIELD GAME S OF CONTROLS AND M E AN FIELD TYPE FBSDES 17 In particular, by plugging ba ck this estimate in (2.14), ∀ t ∈ [0 , T ] , k X α t − X t k 2 ≤ C ′ k v ( α ) k 2 T + Z T 0 k v ( α ) s kk U α s − U s k ds ! , for a constant C ′ with the same dep endencies as C . Using this estimate, we can estimate R T 0 k U α s − U s k 2 ds in a similar fashion and we conclude to the existence of a consta n t C ′′ depending o n c F , T , k ( F , G, g ) k Lip only such that ∀ t ∈ [0 , T ] , k X t − X α t k , k U t − U α t k ≤ C ′′ k v ( α ) k T .  2.3. Decreasing s tep size. So far, we ha ve been working with a constant step size γ . This yields a co nv ergence of the forw ard ba ckw ard system in  H T  d with a ra te of O  1 √ n  . Altough choosing a decreasing s tep size leads to a worse conv erge rate, it allows to weak en some assumption. In particular, the step size can then be chosen indep endently of the L ipschi tz no rm o f v . This is pa rticularly in teresting fo r FBSDEs posed on lar g e time in terv als since w e ex p ect that k v k Lip grows linearly with the horizo n T . Corollary 2.23. L et v b e deﬁne d as in 2.18 , for a given initial c ondition α 1 ∈  H T  d and n ≥ 1 we intr o duc e the se quenc es    α n + 1 2 = α n − γ n v ( α n ) , Y n +1 = Y n − v ( α n + 1 2 ) , α n +1 = γ n +1 Y n +1 , , with Y 1 = 1 γ 1 α 1 . If ( γ n ) n ∈ N ⊂ R + is a se quenc e of de cr e asing step such that ther e exists C ∈ R + , δ ∈ (0 , 1) such that ∀ n ∈ N , 1 γ n ≤ C γ n δ , and ∃ n 0 , γ n 0 ≤ k v k Lip , then under Hyp otheses 2.14, 2.13, letting ¯ U n = 1 n n X i =1 F − 1 u ( X α i + 1 2 , α i + 1 2 ) , and ( X t , U t , Z t ) t ∈ [0 ,T ] b e the u nique str ong solut ion to (2.8) , the fol lowing holds - Ther e exists a c onstant C v dep ending on t he c o eﬃcients and ( γ n ) n ∈ N such that ∀ n ≥ 1 , Z T 0   U t − ¯ U n t   2 dt ≤ 1 2 γ 1 c f n Z T 0   ˜ F ( X t , U t ) − α 1 t   2 dt + C γ 2 n 1 − δ Z T 0 k U t k 2 dt + C v c F n . - L etting ∀ t ∈ [0 , T ] , ¯ X n t = X 0 − Z t 0 ˜ F ( ¯ X n s , ¯ U n s ) ds + √ 2 σ B t , ther e exists a c onstant C dep ending only on T and k ˜ F k Lip such that Z T 0 k ¯ X n t − X t k 2 dt ≤ C 1 2 γ 1 c f n Z T 0 k ˜ F ( X t , U t ) − α 1 t k 2 dt + C γ 2 n 1 − δ Z T 0 k U t k 2 dt + C v c F n ! . Pr o of. In spirit the pro of is quite similar to that of Theorem 2 .19: by a pplying Lemma 1.4 in  H T  d with x = α ∗ =  ˜ F ( X t , U t )  t ∈ [0 ,T ] , 18 MEYNARD CHARLES we ﬁnd that c F n Z T 0      U t − 1 n n X i =1 ˜ F − 1 u ( X α i + 1 2 t , α i + 1 2 t )      2 dt ≤ 1 2 γ 1 Z T 0 k ˜ F ( X t , U t ) − α 1 t k 2 dt + 1 2 γ n +1 Z T 0 k U t k 2 dt + 1 2 n X i =1  γ i k v ( α i + 1 2 ) − v ( α i ) k 2 T − 1 γ i k α i + 1 2 − α i k 2 T  | {z } E n . Letting C v = 1 2 n 0 X i =1  γ i k v ( α i + 1 2 ) − v ( α i ) k 2 T − 1 γ i k α i + 1 2 − α i k 2 T  , from our a ssumption on the sequence ( γ n ) n ∈ N ∀ n ∈ N , E n ≤ C v . It follows that Z T 0      U t − 1 n n X i =1 ˜ F − 1 u ( X α i + 1 2 t , α i + 1 2 t )      2 dt ≤ 1 2 γ 1 c f n Z T 0 k ˜ F ( X t , U t ) − α 1 k 2 dt + C γ 2 n 1 − δ Z T 0 k U t k 2 dt + C v c F n . The rest of the pro of is a straig htforward consequence of this result  Remark 2 .24. the condition ∃ n 0 , γ n 0 ≤ 1 k v k Lip , is alw ays true as so on as the sequence tends to 0. If the op erator v is only as s umed to b e η Holder for some η ∈ (0 , 1), then this result can a lso b e adapted b y requir ing that + ∞ X n =1 γ 1+ η 1 − η n < + ∞ , instead. By an applica tio n of Y oung’s inequalit y n X i =1 γ i k α i + 1 2 − α i k 2 η ≤ 1 1 − η n X i =1 γ 1+ η 1 − η i + 1 η n X i =1 1 γ i k α i + 1 2 − α i k 2 . F o r monotone FBSDEs with Holder co eﬃcients w e refer to [39] 2.4. Outsi de of the mean ﬁeld regim e. Let us ment ion that this metho d is still v alid outside of the mea n ﬁeld regime, na mely this a llows to co mpute the characteristics of system of PDEs o f the for m (2.15) ∂ t U + F ( x, U ) · ∇ x U = G ( x, U ) , in the monotone regime. F or mean ﬁ eld games, this c o rresp onds to the study of ﬁnit e state spa ce MF Gs [36]. Computing solutions to this system can be challenging as the regularity of this P DE stems from its monotonicit y . In genera l this means that either the interv a l o f time considered mu st be suﬃcient ly small, o r the approximation used must b e monotonicit y preserv ing . Finally ev en in the so called p otential regime, that is, when (2.15) arises from the gr adient o f a conv ex HJB equation [45] ∂ t u + H ( x, ∇ x u ) = 0 , the metho d we prop ose a ppea rs new. Although it can only b e applied to the framework of conv ex HJB equatio n co mpared to more general metho d suc h as sho oting algor ithms [15], it exhibits globa l conv ergence independently of the c hosen initial co nt rol. Another point which is relev an t in the ﬁnite dimensional setting is that v can b e monotone without requiring that a ll co eﬃcients are. Consider the following HJB equation for simplicit y (2.16)  ∂ t u + H ( x, ∇ x u ) − σ ∆ x u = 0 , ( t, x ) ∈ (0 , T ) × R d , u ( T , x ) = g ( x ) ∀ x ∈ R d . W e place ourselve under the following assumptions Hyp othesis 2.25. L et L b e the fenchel c onjugate of H i n (2.1 6) , EXTRA GRADIENT METHODS FOR MEAN FIELD GAME S OF CONTROLS AND M E AN FIELD TYPE FBSDES 19 - t her e exists t wo c onstant c α > 0 , c x ≥ 0 such that ∀ ( x, α ) ∈ R 2 d  D 2 x L D 2 x,α L D 2 α,x D 2 α L .  ( x, α ) ≥  − c x I d 0 0 c α I d .  , in the sense of distributions. - t her e exists c g ≥ 0 such that ∀ ( x, y ) ∈ R 2 d , ( g ( x ) − g ( y )) · ( x − y ) ≥ − c g | x − y | 2 . - ∇ x L, ∇ α L, ∇ x g ar e lipschitz. - σ > 0 . Those assumptions, especially o n ∇ α L are muc h stronger than what is needed for the wellposedness (in a classical sense) of (2.16), nevertheless they ar e quite standard in o ptimal cont rol and it is a conv enien t setting to s tate the follo wing: Lemma 2. 26. Under H yp othesis 2.25, ther e ex ists a unique solution u ∈ C ([0 , T ] × R d ) ∪ C 1 , 2 ((0 , T ) × R d ) to (2.16) . Mo r e over x 7→ ∇ x u ( t, x ) is Lipschitz u niformly in t ∈ [0 , T ] . In p articular for any initial c onditi on X ∈ H t her e exists a unique solution to the forwar d b ackwar d system (2.17)  dX t = − D p H ( X t , U t ) dt + √ 2 σ dB t , X 0 = X , dU t = D x H ( X t , U t ) − Z t dB t , U T = ∇ x g ( X T ) , , whose de c oupling ﬁeld is given by ( t, x ) 7→ ∇ x u ( t, x ) . Pr o of. In this setting the uniqueness of a solution to (2.16) follows from the theory of viscosity solutions [25]. The ex istence is a lso quite s ta ndard. F or ex a mple, it s uﬃces to use any shor t time existence result [14] combined with sc hauder type estimates [33]. The fact that ∇ x u is the deco upling ﬁeld asso ciated to the FBSDE (2.17) is then simply a conse q uence of Ito’s lemma.  Since the existence of a solution to (2.17) for a n y initial condition has b een established we now s how the conv ergence of the a lgorithm introduced in this section Lemma 2.27. Fix an initial c onditio n X 0 ∈ H d and let v b e deﬁne d as in 2.8 for L, g . Un der Hyp othesis 2.25, if c v = c α − T ( c x + c g ) > 0 , and γ ≤ min 1 2 k v k Lip , c v k v k 2 Lip ! , then ther e exists a λ ∈ (0 , 1 ) dep ending only on c v , γ , k v k Lip such that for any α 1 ∈ ( H T ) d letting  α n + 1 2 = α n − γ v ( α n ) , α n +1 = α n − γ v ( α n + 1 2 ) . the fol lowing holds ∀ n ∈ N , n ≥ 1 , k α n − α ∗ k ≤ λ n k α 1 − α ∗ k for α ∗ = D p H ( X t , U t ) the optimal c ontr ol asso ciate d to (2 .17) . Pr o of. Since co eﬃcients a re all L ips chitz, it is eviden t that v is Lipsc hitz. Now let us tak e tw o controls α, α ′ , using Hypothesis 2 .2 5, we get h v ( α ) − v ( α ′ ) , α − α ′ i T ≥ c α k α − α ′ k 2 T − c x Z T 0 k X α t − X α ′ t k 2 dt − c g k X α T − X α ′ T k 2 . By deﬁnition ∀ t ∈ [0 , T ] , k X α t − X α ′ t k 2 ≤ k α − α ′ k 2 T . As such h v ( α ) − v ( α ′ ) , α − α ′ i T ≥ c v k α − α ′ k 2 T . Since this is true for a ny controls α, α ′ , v is Lipschit z contin uous and strongly monotone, and we are in the conditions of applications o f Theorem (2.11).  20 MEYNARD CHARLES Let us remark that even in mean ﬁeld ga mes , v may b e monotone even when co eﬃcients a ren’t. Indeed the above result is a dapted eas ily to the mean ﬁeld setting on short interv als o f time. So lo ng as the monotonicity of v is conser ved, the existence and uniqueness o f a so lution ar e not particular ly hard to pr ov e, this phenomenon is sometimes refered to a s the semi- mo notone regime in shor t time [42]. 3. FBSDE s with common noise W e now turn to the pro blem of FBSDEs with common no ise (3.1)                X t = X 0 − Z t 0 F ( X s , p s , U s , L ( X s , U s |F 0 s )) ds + √ 2 σ B t , U t = g ( X T , p T , L ( X T |F 0 T )) + Z T t G ( X s , p s , U s , L  X s U s |F 0 s  ) ds − Z T t Z s d ( B s , W s ) , p t = p 0 − Z t 0 b ( p s ) ds + √ 2 σ 0 W t , for X 0 ∈ H d , q 0 ∈ H d 0 initial co nditions indep endent of eac h other and o f ( W s , B s ) s ≥ 0 . In this ca s e the ﬁltra tio n asso cia ted to common noise is ( F 0 t ) t ∈ [0 ,T ] which is the augmented ﬁltration a sso ciated to ( ˜ F 0 t ) t ∈ [0 ,T ] deﬁned by ˜ F 0 t = σ ( q 0 , ( W s ) s ≤ t ) . Remark 3.1. W e insist on the fact that this class o f FBSDEs includes the case of additiv e common noise often studied in mean ﬁeld games. Indeed, consider the fo llowing problem (3.2)        X t = X 0 − Z t 0 F ( X s , U s , L ( X s , U s |F 0 s )) ds + √ 2 σ B t + √ 2 σ 0 W t , U t = g ( X T , L ( X T |F 0 T )) + Z T t G ( X s , U s , L  X s , U s |F 0 s  ) ds − Z T t Z s d ( B s , W s ) , In tro ducing the t wo pr o cesses ∀ t ∈ [0 , T ] ,  p t = √ 2 σ 0 W t , Y t = X t − p t , F o rmally , the triple ( Y s , p s , Z s ) s ∈ [0 ,T ] is a solution to                Y t = X 0 − Z t 0 F ( Y s + p s , U s , T ( · , p s ) # L ( Y s , U s |F 0 s )) ds + √ 2 σ B t , p t = √ 2 σ 0 W t , U t = U T + Z T t G ( Y s + p s , U s , T ( · , p s ) # L ( Y s , U s |F 0 s )) ds − Z T t Z s d ( B s , W s ) , U T = g ( Y T + p T , ( id R d + p T ) # L ( X T |F 0 T )) , where f # µ indicates the pushforward of a measure µ b y the function f and ( id R d + p ) : x 7→ x + p, T ( · , p ) : ( x, y ) 7→ ( x + p, y ) . In particular if ( F, G, g ) are Lipschit z functions on R 2 d × P 2 ( R 2 d ) then the co eﬃcients ar e still Lipschitz in all arguments after this transfor mation. A rig orous equiv alence b etw een these t wo form ulations is established in [1 2] Lemma 5 .3 . W e w ork under the following assumption Hyp othesis 3.2. The c o eﬃcients ( g , F, G, b ) ar e such that - Hyp othesis 2.14 is satisﬁe d by ( x, u, m ) 7→ ( g ( x, p, π d m ) , F ( x, p, u, m ) , G ( x, p, u, m )) uniformly in p ∈ R d 0 . - ( F , G, g , b ) ar e Lipschitz in al l their ar guments for W 2 in the me asur e ar gument. It has alrea dy b een obs e r ved in [11], that the addition of suc h co mmon noise does not perturb the monotonicity of the system. Hence results of wellposedness for mo notone FBSDEs ar e easily a dapted to this particular sys tem where the common noise ( p t ) t ≥ 0 evolv es in dep endently o f the rest of the dynamics EXTRA GRADIENT METHODS FOR MEAN FIELD GAME S OF CONTROLS AND M E AN FIELD TYPE FBSDES 21 Lemma 3. 3 . Under Hyp othesis 3.2, for any initial c ondition X 0 ∈ H d , q 0 ∈ H d 0 , indep endent of e ach other and of ( W t , B t ) t ≥ 0 , ther e exists a u nique str ong solution to the FBSDE (3.1) Pr o of. F or a pro of of this result based on the existence of a Lipschit z decoupling ﬁeld to the FBSDE (3.1), we refer to [3 9] Theorem 3.3 3 a nd the subsection 3.4.1 of s a id work.  Let us remark that under the standing assumptions, the function U 7→ F ( X , p, U, L ( X , U )) , is in vertible on H d for any ( X , p ) ∈ H d × R d 0 b y the monotonicity of F , with this Hilb ertian inv erse being denoted by ˜ F − 1 u ( X, p , · ). In pa rticular following [39] Lemma 3.1 0 there ex ists a Lipschitz function F inv : R d × R d 0 × R d × P 2 ( R 2 d ) → R d such tha t ∀ ( X, α, p ) ∈ H 2 d × R d 0 , F inv ( X, p , α, L ( X , α )) = ˜ F − 1 u ( X, p , α ) F o llowing the previous section, for a given ( α t ) t ∈ [0 ,T ] ∈  H T  d , we in tro duce the dynamics (3.3)                X α t = X 0 − Z t 0 α s ds + √ 2 σ B t , p t = p 0 − Z t 0 b ( p s ) ds + √ 2 σ 0 W t , U α t = g ( X α T , p T , L ( X α T |F 0 T )) − Z T t G F ( X α s , p s , α s , L ( X α s , α s |F 0 s )) ds − Z s d ( B s , W s ) , where ∀ ( x, p, a, X , α ) ∈ R 2 d + d 0 × H 2 d , G F ( x, p, a, L ( X , α )) = G ( x, p, F inv ( x, p, a, L ( X , α )) , L ( X, F inv ( X, p , α, L ( X , α )))) . Corollary 3.4. Un der Hyp othesis 3.2, for any α ∈  H T  d , X 0 ∈ H d the system (3.3) has a unique solution ( X α t , U α t , Z α t ) t ∈ [0 ,T ] . Mor e over, letting (3.4) v : (  H T  d − →  H T  d ( α t ) t ∈ [0 ,T ] 7→  F inv ( X α s , p s , α s , L ( X α s , α s |F 0 s )) − U α t  t ∈ [0 ,T ] , v is Lipsch itz on  H T  d and ther e exists a c onstant C such t hat if γ ≤ 1 k v k Lip , for any α 0 ∈  H T  d , the pr o c e dur e      α n + 1 2 = α n − γ v ( α n ) , α n +1 = α n − γ v ( α n + 1 2 ) , ∀ s ∈ [0 , T ] , ¯ U n s = 1 n P n i =1 F inv ( X α n + 1 2 s , p s , α n + 1 2 s , L ( X α n + 1 2 s , α n + 1 2 s |F 0 s )) satiﬁes ∀ n ≥ 0 , C Z T 0 k U t − ¯ U n t k 2 dt ≤ 1 γ n Z T 0 k F inv ( X t , p t , U t , L ( X t , U t |F 0 t )) − α 0 t k 2 dt. Pr o of. This is almost a direct adaptation of Theorem 2.19. The core o f the pro o f remaining unchanged in the presence of an additional common nois e . In particular it is str a ightf orward to sho w that v is still Lipsc hitz in this setting. Instead let us fo cus on showing that the monotonicit y of v is unchanged 22 MEYNARD CHARLES in this new problem. Let α, α ′ ∈ H T , by deﬁnition of the forw ard backw ard system (3.1) h v ( α ) − v ( α ′ ) , α − α ′ i T = h g ( X α T , p T , L ( X α T |F 0 T )) − g ( X α ′ T , p T , L ( X α ′ T |F 0 T )) , X α T − X ( α ′ ) T i + E " Z T 0  F inv ( X α s , p s , α s , L ( X α s , α s |F 0 s )) − F inv ( X α ′ s , p s , α ′ s , L ( X α ′ s , α ′ s |F 0 s ))  · ( α s − α ′ s ) ds # + E " Z T 0  G F ( X α s , p s , α s , L ( X α s , α s |F 0 s )) − G F ( X α ′ s , p s , α ′ s , L ( X α ′ s , α ′ s |F 0 s ))  · ( X α s − X α ′ s ) ds # . F o r ﬁxed p ∈ R d 0 , the monoto nicit y of the pair ( X, α ) 7→ ( F inv ( X, α, p, L ( X , α )) , G F ( X, α, p, L ( X , α ))) , has been established in Lemma 2.16. Thus, using an equiv alent deﬁnition of L 2 − monotonicity via fo r probability mea sures (instead of random v a riables), it is straightforw ard that E " Z T 0  F inv ( X α s , p s , α s , L ( X α s , α s |F 0 s )) − F inv ( X α ′ s , p s , α ′ s , L ( X α ′ s , α ′ s |F 0 s )) G F ( X α s , p s , α s , L ( X α s , α s |F 0 s )) − G F ( X α ′ s , p s , α ′ s , L ( X α ′ s , α ′ s |F 0 s ))  ·  α s − α ′ s X α s − X α ′ s  ds      F 0 T # ≥ c F E " Z T 0    F inv ( X α s , p s , α s , L ( X α s , α s |F 0 s )) − F inv ( X α ′ s , p s , α ′ s , L ( X α ′ s , α ′ s |F 0 s ))    2      F 0 T # a.s. The r est of the pro of follo ws from argumen ts pres ent ed in the pro o f of Theorem 2.19 b y using the ab ov e monotonicit y estimate.  Remark 3.5. The adaptation of previo us ly obtained conv ergence results to the case of FBSDEs with common noise is deceptiv ely simple. Althoug h the pro of of con vergence is quite similar, numerically this addition lea ds to s igniﬁcant changes in the algorithm considered. 4. N umerical illustra tion 4.1. numerical se tup. In this section we present some n umerical results for the method introduced in this ar ticle. W e ﬁrst discretize the time interv al co nsidered [0 , T ] with a time step ∆ t and work with discretized control ( α i ) i ∈{ 1 , ··· ,N t } = ( α ( i ∆ t )) i ∈{ 1 , ··· ,N t } . The diﬀerent pro ce sses ar e then computed with an explicit Euler scheme. Let us present the scheme more precisely in the ca se study of FBSDEs (2.8)        X t = X 0 − Z t 0 F ( X s , U s , L ( X s , U s )) ds + √ 2 σ B t , U t = g ( X T , L ( X T )) + Z T t G ( X s , U s , L ( X s , U s )) ds − Z T t Z s dB s . W e supp ose that the ma p F − 1 : R 2 d × P 2 ( R 2 d ) → R d which is such that ˜ F − 1 = ˜ F − 1 is known. T he existence of such a map follo ws from Remark 2.6, howev er as w e already ment ionned in Remark 2.20, computing it ca n be a challenge whenever F dep ends on the law of ( U t ) t ∈ [0 ,T ] . (athough this is nev er a problem for mean ﬁeld ga mes, since in this case F − 1 = ∇ α L ). First w e ﬁx N p the num b er of paths sim ulated and introduce ( X i 0 ) i =1: N p ⊂ R N p , where each X i 0 is an independent r ealization of a random v a r iable with law L ( X 0 ). W e also in tro duce ( G i,j ) i :1: N p ,j =1: N t a s e q uence of indep enden t gaussia n increments sampled from a normal distribution with mean 0 and cov ariance √ ∆ t I d . These will play the ro le of the Brownian increments W ( i +1)∆ t − W i ∆ t . A discrete control ( α i,j ) i =1: N p ,j =1: N t gives the control at the timestep j along the path i . W e deﬁne X i, 0 = X i 0 and for j = 1 : N p X i,j = X i,j − 1 + ∆ t α i,j + √ 2 σ G i,j . T o approximate the backw ard process ( U α t ) t ∈ [0 ,T ] , we ﬁrst deﬁne U i,N t = g ( X i,N t , 1 N p X k δ X k,N t ) , EXTRA GRADIENT METHODS FOR MEAN FIELD GAME S OF CONTROLS AND M E AN FIELD TYPE FBSDES 23 where δ x indicates the dirac measure centered in x ∈ R d Then the sequence is co nstructed by backw ard induction for j = 0 : ( N t − 1), U i,j = E " U i,j +1 + ∆ t G ( X i,j +1 , F − 1 ( X i,j +1 , α i,j +1 , 1 N p X k δ ( X k,j + 1 ,α k,j + 1 ) ) , 1 N p X k δ ( X k,j + 1 ,α k,j + 1 ) )      F j # , where F j = σ  ( X i 0 , G i,k , α i,k ) i =1: N p ,k ≤ j  . The conditional exp ectation is approximated as follow, letting ( f ω ) ω ∈X be a parametrized family o f functions on so me space X , for ﬁxed j we approximate ( U i,j ) i =1: N p with ∀ i ∈ { 1 , · · · N p } , U i,j ≈ f ω ∗ ( X i,j ) , where ω ∗ = arginf ω ∈X J ( ω ) , for J ( ω ) = 1 N p X k      U k,j +1 + ∆ t G ( X k,j +1 , F − 1 ( X k,j +1 , α k,j +1 , 1 N p X l δ ( X l,j +1 ,α l,j +1 ) ) , 1 N p X l δ ( X l,j +1 ,α l,j +1 ) ) − f ω ( X k,j )      2 . Clearly this is based on the idea that for t wo random v a riables X , Y ∈ H , E [ Y | X ] = inf measurable f E  | Y − f ( X ) | 2  . The para metrized family ( f ω ) ω ∈X can range from neural netw orks to any suﬃcien tly rich regression method, for g eneral results on r egressio n based methods fo r BSDEs we re fer to [28]. No w that we hav e explained how the tra jectories ar e calculated we in tro duce the follo wing discr e tized op erator v N t ,N p (( X i 0 , G i,j ) i =1: N p ,j =1: N t ) : ( R N p × N t × d − → R N p × N t × d ( α i,j ) i =1: N p ,j =1: N t 7→  F − 1 ( X i,j , α i,j , 1 N p P k δ ( X k,j ,α k,j ) ) − U i,j  i =1: N p ,j =1: N t . W e can now in tro duce the asso ciated algorithm to so lve the FBSDE (2.8) Algorithm 1 Monotone FBSDE solv er ﬁx an error lev el ε 0 . ﬁx a decrea sing sequence o f step ( γ n ) n ∈ N ⊂ R + . N iter ← 0 Sim ulate N p independent re alizations ( X i 0 ) i =1: N p of X 0 . Sim ulate N p × N t realizations ( G i,j ) i =1: N p ,j =1: N t from indepe nda n t g aussian random v ariable G i,j ∼ N (0 , √ ∆ t I d ). Cho ose an initial control ( α i,j ) i =1: N p ,j =1: N t progres sively adapted. set ¯ α = α . ε err or = k v N t ,N t (( X i 0 , G i,j ) i =1: N p ,j =1: N t )( ¯ α ) k T Initialize Y = 1 γ 1 α while ε err or > ε 0 do N iter ← N iter + 1 α ← α − γ N iter v N t ,N p (( X i 0 , G i,j ) i =1: N p ,j =1: N t )( α ) ¯ α ← (1 − 1 N iter +1 ) ¯ α + 1 N iter +1 α Y ← Y − v N t ,N p (( X i 0 , G i,j ) i =1: N p ,j =1: N t )( α ) α ← γ N iter Y ε err or ← k v N t ,N t (( X i 0 , G i,j ) i =1: N p ,j =1: N t )( ¯ α ) k T end while Return ¯ α . Where we used the notation k α k T = s ∆ t N p X i,j k α i,j k 2 , which is the corres po nding L 2 − norm for discretized random pro cesses. In general the true error is diﬃcult to compute, howev er thanks to Lemma 2.22, it can b e estimated up to a multiplicativ e cons ta n t b y k v ( α ) k T . In particular this a llows us to estimate the conv ergence rate of the algorithm eﬃcien tly . 24 MEYNARD CHARLES 4.1.1. FBSDEs with c ommon noise. Let us brieﬂy explain the main diﬀerences in this setting. Since we m ust b e able to compute an approximation o f the conditional law at e a ch bac kward step, this implies that the generation of the discrete idiosyncratic noise ( G i,j ) i =1: N p ,j =1: N t and the discrete common noise ( G k,j 0 ) k =1: N 0 ,j =1: N t m ust be done independently (where we choose to sim ulate N 0 tra jectories of the common noise). A discr ete c o nt rol is now also a collection ( α i,j,k ) i =1: N p ,j =1: N t ,k =1: N 0 which depends on the trajectories of common noise. W e now prese nt a n a lgorithm in pseudo co de to compute the function v N t ,N p ,N 0 : R N t × R N p × R N 0 → R N t × R N p × R N 0 , discretization of the asso ciated functional (3.4 ) Algorithm 2 Computation of v N t ,N p ,N 0 INPUT: ( α i,j,k ) i =1: N p ,j =1: N t ,k =1: N 0 -Compute the a sso ciated forward proce s ses ( X i,j,k , p j,k ) i =1: N p ,j =1: N t ,k =1: N 0 iteratively through  X i,j +1 ,k = X i,j,k + ∆ t α i,j,k + √ 2 σ G i,j , p j +1 ,k = p j,k + ∆ t b ( p j,k ) + √ 2 σ 0 G k,j 0 . -The backw ard pro c e s s ( U i,j,k ) i =1: N p ,j =1: N t ,k =1: N 0 is computed recursively as follows: given ( U i,j +1 ,k ) i =1: N p ,j =1: N t , ( U i,j,k ) i =1: N p ,j =1: N t is co mputed b y regr ession of U i,j +1 ,k + ∆ t G F ( X i,j +1 ,k , p j +1 ,k , α i,j +1 ,k , 1 N p X l δ ( X l,j +1 ,k ,α l,j +1 ,k ) ) in to ( X i,j,k , p j,k ) i =1: N p ,k =1: N 0 -Return  F inv  X i,j,k , p j,k , α i,j,k , 1 N p P l δ ( X l,j,k ,α l,j,k )  − U i,j,k  i =1: N p ,j =1: N t ,k =1: N 0 . Clearly this algo rithm is m uch mor e costly compared to the version without common noise. First the memory complexity is now of the order of O ( N t × N p × N 0 ). Mo r eov er a t each backw ard time step, we compute the reg ression of a function in R d × R d 0 4.2. Mean ﬁeld forwar d bac kw ard sto c hastic diﬀerential equations . Compared to the deter- ministic case, the most time consuming step cons is ts in approximating the backw ard pro cess. Although the metho d w e prop ose allo ws to iterate on decoupled FBSDEs, we still need to estimate a conditional exp e ctation at each time step. In the following test, it is estimated with p olynomial regr ession o n the ﬁrst 10 Her mite po lynomials. 4.2.1. Syst em c onsider e d. W e consider the follo wing exemple (4.1)        X t = X 0 − Z t 0 aY s ds + √ 2 σ B t , Y t = bX T + Z T t ( cX s + E [ f ( X s − E [ X s ])]) ds − Z s dB s , In this case the s olution is known explicitely . Indeed Let us ﬁrs t ass ume that ( Y s ) s ∈ [0 ,T ] can be writen in feedback form as (4.2) ∀ t ∈ [0 , T ] , Y t = η ( t ) X t + θ ( t ) , for smo oth functions of time η , θ . It follows directly that ¯ X t = X t − E [ X t ] , is given b y ¯ X t = ¯ X 0 e − a R t 0 η ( s ) ds + √ 2 σ x Z t 0 e − R t s aη ( u ) d u dW s . In particular , there exists a contin uous function e : [0 , T ] → R dep ending on η , a, L ( X 0 ) only suc h that ∀ t ∈ [0 , T ] , E [ f ( X t − E [ X t ])] = E  f ( ¯ X t )  = e ( t ) . EXTRA GRADIENT METHODS FOR MEAN FIELD GAME S OF CONTROLS AND M E AN FIELD TYPE FBSDES 25 Using the r epresentation for mul a (4.2 ), we deduce that ∀ t ∈ [0 , T ] , 0 = Z T t  ( c + dη ds ( s ) − aη 2 ( s )) X s + e ( s ) − aη ( s ) θ ( s ) + dθ ds ( s )  ds + Z T t ( √ 2 σ η ( s ) − Z s ) dW s . In the end it suﬃces to solve the follo wing system  c + dη ds ( s ) − aη 2 ( s ) = 0 , η ( T ) = b, e ( s ) − aη ( s ) θ ( s ) + dθ ds ( s ) = 0 , θ ( T ) = 0 . In par ticular if a, b, c ≥ 0 a nd f is either monotone o r Lipschitz with k f k Lip ≤ c , we are in the monotone regime a nd the system admits a unique solution on a n y interv al of time. Since mo notonicity also implies uniqueness of a so lution to (4.1), w e ha ve exhibited the unique solution to this mean ﬁeld FBSDE. F o r c = 0                    η ( t ) = b 1 + ab ( T − t ) , ¯ X t = ¯ X 0  1 − abt 1 + abT  + √ 2 σ (1 + ab ( T − t )) Z t 0 1 1 + ab ( T − s ) dW s , θ ( t ) = 1 1 + ab ( T − t ) Z T T − t e ( s )(1 + ab ( T − s )) ds e ( t ) = E  f ( ¯ X t )  . Cho osing X 0 = x ∈ R , ¯ X t ∼ N  0 , 2 σ ab  1 + ab ( T − t ) − (1 + ab ( T − t )) 2 1 + abT  , and e ( t ) ca n b e computed very accurately with gaussian quadrature. 4.2.2. Nu meric al r esults. W e now present some numerical res ult for the system (4.1) for the para meters                σ = 1 , a = 1 , b = 1 , x 0 = 1 , c = 0 , T = 1 0 , , with a step size γ = 0 . 0 8 and ∀ x ∈ R , f ( x ) = atan ( x − 1 ) . F o llowing the algorithm 1 with N t = 1 00 , N p = 1000 0 and letting ε err or ( n ) = k v N t ,N p ( α n ) k T , the sequence of errors on the las t iterate, we plot its logarithm in function of the n um b er of iterations 26 MEYNARD CHARLES 0 20 40 60 80 100 120 140 160 180 200 -15 -13 -11 -9 -7 -5 -3 -1 1 3 5 + ln( ε err or ) + linea r ﬁt with slop e − 0 . 0764 Numb er of iteration s Figure 1. log-linear plot of the last iterate er r or in function of the num be r of iter a- tions After a small num ber of iterations, t h e linear ﬁt is almo st p erfect. This emphasizes that th e last iterates con verge exp onentially fast when the conditions for applying Theore m 2.11 are met. Let us emphasize that the num b er of itera tio ns may appear large only b ecause w e considered a time interv al of [0 , 10 ] and ev en t hen, computations are quite fast. Since the functions θ , η ca n b e computed v ery accurately in this setting let us a lso compare our numerical results to the solution of (4.1). F or the sa me parameters, the functions η , θ are estimated numerically b y doing a linear regress io n o f  Y i,j  j =1: N p onto  X i,j  j =1: N p for ﬁxed 1 ∈ { 1 , · · · N t } . Figure 2. η ( t ) 0 2 4 6 8 10 0 0.2 0.4 0.6 0.8 1 Time Figure 3. θ ( t ) 0 2 4 6 8 10 -2.5 -2 -1.5 -1 -0.5 0 Time Plot of th e functions t 7→ θ ( t ) , t 7→ η ( t ) in blue and of the computed v alues u sing our algorithm with the red crosses. Since ε err o r is v ery small ( ∼ 10 − 13 ) the diﬀerence b etw een th e tru e curve and the computed va lues does not come from the num ber of itera tions but is rather inherent to the discreti zation (b oth in time and on the space of random paths) of the problem. 5. Perspectives In this article we hav e presented a reinterpretation of monotone FBSDEs as the solution of a monotone v ariational inequalit y in a Hilbert space. This po int of view allo ws for a straig ht forward EXTRA GRADIENT METHODS FOR MEAN FIELD GAME S OF CONTROLS AND M E AN FIELD TYPE FBSDES 27 adaptation of extra-g radient metho ds to mean ﬁeld t yp e FBSDEs. Under suﬃciently str ong ass ump- tions we present a decoupling algorithm conv erging exp onentially fast to the solution o f the problem. How ev er in g eneral the con vergence rate is of the order of 1 √ n for n the num ber of iterations. A c hal- lenge for future r e s earch is to improv e this conv ergence ra te. T o that end an interesting lead c o nsists in studying a v ersion o f the algo rithm with an ada ptiv e step size. F or example, a lthough the context is very diﬀerent it is shown in [35] that the con vergence ra te o f ﬁctitious play [18] can b e tr emendously improv ed b y adding a linear search for the best s tep size at each step. Another p ossibility is to ﬁnd better parameter izations of the problem. In [26] the authors pro ved that better conv ergence rates can be obta ined by studying FBSDEs under diﬀerent proba bilit y meas ures us ing Girsanov Theorem. Perhaps such an idea can also b e applied in our setting. ackno wledgements I would like to thank F rançois Delarue for the discussions w e had while w orking on this pap er, Charles Meyna r d ackno wledge the ﬁnancial supp or t of the Europe an Research Co uncil (ERC) under the Euro pe a n Union’s Horizo n Europ e resear ch and innov a tion pr ogram (ELISA pro ject, Grant agreement No. 10105474 6). Views and opinions expressed are how ever those of the author only and do no t necessarily reﬂect those of the Europ ea n Union or the Europ ean Research Council Executive Agency . Neither the E urop ean Union no r the gra nting authorit y can b e held resp onsible for them Reference s [1] Y ves Ac hdou, F abio Camill i , and Italo Capuzzo-Dolcetta. Mean ﬁeld games: Conv ergence of a ﬁnite diﬀerence method. SIAM Journal on Numeric al A nalysis , 51(5):2585–2 612, 2013. [2] Y ves Ac hdou and Ziad Kobeissi. Mean ﬁeld games of con trols: Finite di ﬀerence approximations. Mathematics in Engine ering , 3(3):1–35 , 2021. [3] Y ves A c hdou and Alessio Po rretta. Conv ergence of a ﬁnite di ﬀerence sc heme to we ak solutions of the system of partial diﬀeren tial equations arising in mean ﬁeld games. SIAM Journa l on Numeric al A nalysis , 54(1):161– 186, 2016. [4] Andrea Angiuli , Christy V. Grav es, Houzhi Li, Jean-F rançois Chassagneux, F rancco is D elar ue, and René A. Car- mona. C emr acs 2017: numerical probabilistic approach to mfg. ESAIM: Pr o c e ed ings and Surveys , 2018. [5] Kimon Antonak opoulos, Elena V eronica Belmega, and Pana y otis Mertiko p oulos. A daptiv e extra-gradient metho ds for min-max optimization and games. In International Confer enc e on L e arning R epr esentations , 202 1. [6] Kimon Antonak opoulos, Thomas Pethic k, Al i Kavis, Pan ay otis Mertiko p oulos, and V olkan Cevher. Sifting through the nois e: Uni versal ﬁrst-order metho ds for stochast ic v ariational inequalities. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P . S. Liang, and J. W ortman V aughan, editors, A dvanc es in Neur al Inform ation Pr o c essing Systems , v olume 34, pages 1309 9–13111. Curran Asso ciates, Inc., 2021. [7] W aïss Azizian, Ioannis Mi tliagkas, Simon La coste-Julien, and Ga uthier Gidel. A tigh t and uniﬁed a nalysis of gradien t-based methods for a whole sp ectrum of diﬀerentiable games. In Silvia Chiappa and Roberto Calandra, editors, Pr o c e e dings of the Twent y Thir d International Confer enc e on A rtiﬁcial Intel ligenc e and Statistics , volume 108 of Pr o c e e dings of Machine Le arning R ese ar ch , pages 286 3–2873. PMLR, 2020. [8] F rancis Bac h and Kﬁr Y Levy . A unive rsal algorithm for v ariational inequalities adaptive to smo othness and noise. In Alina Beygelzimer and Daniel Hsu, edit ors, Pr o c e ed ings of the Thirty- Se c ond Confer enc e on Le arning The ory , v olume 99 of Pr o c ee dings of Machine L e arning R ese ar ch , pages 164–1 94. PMLR, 2019. [9] A. Bensoussan, S.C.P . Y am, and Z. Zhang. W ell-p osedness of mean-ﬁeld type forward–ba ckw ard stochastic di ﬀer- en tial equations. Sto chastic Pr o c esses and their A pplic ations , 125( 9):3327–3354 , 2015. [10] A lain Bensoussan, Jens F rehse, and Philip Y am. Me an Field Games and Me an Field T yp e Contr ol The ory . Springer, New Y ork, 2013. [11] C. Bertucci and C. Meynard. Noise through on additionnal v ariable for mean ﬁeld games master equation on ﬁnite state space, 2024. [12] C. Bertucc i and C. Meynard. A study of common noise in mean ﬁeld games, 2024. [13] Char l es Bertucci. Monotone solutions for mean ﬁeld games master equa tions: cont inuous state space and common noise. Communic ations in Partial Diﬀer ential Equa tions , 48(10-12):1245–128 5, 2023. [14] Char l es Bertucci, Jean-Mic hel Lasry , and Pierre-Louis Lions. On lipschitz solutions of mean ﬁeld games master equations. Journal of F unct ional A nalysis , 287(5):110486, 2024. [15] Jean-Baptiste Caillau, Rob erto F erretti, Emmanuel T r élat, and Hasnaa Zidani. Chapter 15 - an algorithmic guide for ﬁnite-dimensional optimal control problems. In Emm anuel T r élat and Enrique Zuazua, editors, Numeric al Contr ol: Part B , volume 24 of Handb o ok of Numeric al A nalysis , pag es 559–6 26. Elsevier, 2023. [16] P . Cardaliaguet, F. Delarue, J.-M . Lasry , and P .-L. Lions. The m aster equation and the con v ergence problem i n mean ﬁeld games. Princ eton University Pr ess , 201 9. [17] Pi erre Cardali aguet and P anagiotis Souganidis. Monotone solutions of the master equation for mean ﬁeld games with i diosyncratic noise. SIAM Journal on Mathematic al A nalysis , 54(4):419 8–4237, 2022. [18] Car daliaguet, Pierre and Hadikhanloo, Saeed. Learning in mean ﬁeld games: The ﬁctitious play . ESAIM: COCV , 23(2):569 –591, 2017. 28 MEYNARD CHARLES [19] R ené Car mona and F rançois Delarue. Mean ﬁeld forw ard-backw ard stochastic diﬀeren tial equations. Ele ctr onic Communic ations in Pr ob ability , 18:1 – 15, 2013. [20] R ené Carmona and F rançois D elarue. Pr ob abilistic The ory of Me an Field Games with A pplic ations II . Spri nger, 2018. [21] R ené Carmona, F rançois Delarue, and Daniel Lack er. Mean ﬁeld games with common noise. The A nnals of Pr ob a- bility , 44(6):3740 – 3803, 2016. [22] R ené Carmona and Mathieu Laurière. Conv ergence analysis of machine learning algorithms for the numerical solu- tion of m ean ﬁeld con trol and games: Ii–th e ﬁnite horizon case. The Anna ls of Ap plie d Pr ob ability , 32(6):406 5–4105, 2022. [23] R ené Carmona and Mathieu Laurière. Conv ergence analysis of machine learning algorithms for the numerical solu- tion of m ean ﬁeld con trol and games: Ii–th e ﬁnite horizon case. The Anna ls of Ap plie d Pr ob ability , 32(6):406 5–4105, 2022. [24] Jean-F r ançois Chassagneux, Dan Crisan, and F r ançois D elarue. Numeri cal method f or FBSDEs of M cKean–Vlasov t yp e. The A nnals of A pplie d Pr ob ability , 29(3 ):1640–1684, 2019 . [25] M icha el G. Crandall, Hitoshi Ishii, and Pierr e-Louis L ions. User’s guide to viscosity solutions of second order partial diﬀeren tial equations. Bul leti n of the A meric an Mathematic al So ciety , 27( 1):1–67, 1992. [26] F rançois Delarue and A thanasios V asileiadis. Ex ploration noise for learning li near-quadratic mean ﬁ eld games. Mathematics of Op er ations R ese ar ch , 50(3):1762–1 831, 2025. [27] W. Gangbo, A. Mészáros, C. Mou, and J. Zhang. Mean ﬁeld gam es master equations with non-separable hamilto- nians and displacement monotonicit y . A nnals of Pr ob ability , 50, 2022. [28] Em man uel Gob et, Jean-Phili ppe Lemor, and Xavier W ari n. A regression-based mon te carl o method to solve back- wa rd sto c hastic diﬀeren tial equations. The A nnals of Ap plie d Pr ob ability , 15( 3):2172–2202, 2005 . [29] Eduard Gorbuno v, Hugo Berard, Gauthier Gidel, and Nicolas Loizou. Stochast ic extragradien t: General analysis and improv ed r ates. In Gustau Camps-V alls, F rancisco J. R. Ruiz, and Isabel V alera, editors, Pr o ce e dings of The 25th International Confer enc e on A rtiﬁcial Intel ligenc e and Statistics , volume 151 of Pr o c e e dings of Machine L e arning R ese ar ch , pages 7865 –7901. PMLR, 2022. [30] Saeed Hadikhanloo, Rida Laraki, Pana y otis Mertikopoulos, and Sylv ain Sorin. Learning in nonatomic games, part i: Finite action space s and population games. Jour nal of Dynamics and Games , 9(4):433–460, 2022. [31] Jo e Jac kson and Alpár R. Mészáros. Quan titativ e conv ergence for displacemen t monoton e mean ﬁeld games of con trol, 2025. [32] G. M. Korpelevich. The ext ragradient metho d for ﬁnding saddle points and other problems. Ekono mika i Matem- aticheskie Meto dy , 12:747–756, 1976. [33] N . V. Krylo v. L e ctur es on El liptic and Par ab olic Equations in Hölder Sp ac es , v olume 12 of Gr aduate Studies in Mathematics . A merican Mathematical So ciet y , 1996 . [34] Jean-Mi c hel Lasry and Pi erre-Louis Lions. Mean ﬁeld games. Jap anese Journal of M athematics , 2(3):229–260, 2007. [35] Pi erre La vigne and Lauren t Pfeiﬀer. Ge neralized conditional gradient and learning in p oten tial mean ﬁeld games. A pplie d Mathematics & Optimization , 88(3), 2023. [36] P .-L. Lions. Lectures at collège de f rance, 2007-2011. [37] P .-L. Lions and P . Soug anidis. Exten ded mean-ﬁeld games. R endic onti Linc e i , 3:611–625, 2020. [38] A lpár R. M észáros M. Bansil and C. Mou. Global w ell-p osedness of displacemen t monotone degene rate mean ﬁeld games master equations, 2023. [39] C. Meynard. Monotone solutions to m ean ﬁeld games master equa tion in the l2-monoto ne setting, 2025 . [40] Chenchen Mou and Jianfeng Zhang. W ellp osedness of second order master equations for mean ﬁeld games wi th nonsmooth dat a. Memoirs of the A meric an Mathematic al So ciety , 2019. [41] Chenchen Mou and Jianfeng Zhang. Mean ﬁeld games of controls: Propagation of monotonicities, 2022. [42] Chenchen Mou and Jianfeng Zhang. M ean ﬁeld games of con trols: Pr opagation of monotonicities. Pr ob ability, Unc ertainty and Quantitative R isk , 7(3):247–274, 2022. [43] A . Mészáros and C. Mou. Mean ﬁeld games systems under displacemen t monotonicity . SIAM Journal on Mathe- matic al A nalysis , 56(1):529–553, 2024 . [44] Y urii Nestero v. Dual extrapolation and its applications to solving v ariational inequalities and related problems. Mathematic al Pr o gr amming , 109(2) :319–344, 2007. [45] R . Tyrrell Rockafellar and Peter R. W olenski. Con vexit y in hamilton–jacobi theory i: Dynamics and duality . SIAM Journal on Contr ol and Optimization , 39( 5):1323–1350 , 2000. [46] Ji anfeng Zhang. Backwar d Sto chastic Diﬀer ential Equations: F r om Line ar to F ul ly Nonline ar The ory , volume 86 of Pr ob ability The ory and Sto chastic Mo del ling . Springer, 2017. A. W ellposedness of a monotone FBSDE Lemma A.1. L et ( F , G, g ) : R 2 d × P 2 ( R 2 d ) → R d b e Lipschitz f unctions and c onsider the fol lowing FBSDE (A.1)  dX t = − ˜ F ( X t , U t ) dt + σ dB t , X 0 = X , dU t = ˜ G ( X t , U t ) dt − Z t dB t , U T = g ( X T ) . EXTRA GRADIENT METHODS FOR MEAN FIELD GAME S OF CONTROLS AND M E AN FIELD TYPE FBSDES 29 If the fol lowing monotonicity c ondition holds ∃ c F > 0 , ∀ ( X, Y , U , V ) ∈ H 4 d , h ˜ g ( X ) − ˜ g ( Y ) , X − Y i ≥ 0 , h  F G  ( X, Y ) −  ˜ F ˜ G  ( Y , V ) ,  X − Y U − V  i ≥ c F k ˜ F ( X, U ) − ˜ F ( Y , V ) k 2 , then for any T > 0 , and for any initial c ondition X ∈ H indep endent of ( B t ) t ≥ 0 ther e exists a unique solution to the FBSDE (A.1) . Pr o of. Let us presen t an a priori estimate for the forward backw ard system (A.1 ). W e ﬁx t wo ini- tal conditions X , Y ∈ H d an ass ume that there exist strong solutions to (A.1) ( X t , U t ) t ∈ [0 ,T ] (resp. ( Y t , V t ) t ∈ [0 ,T ] ) with initial condition X (resp. Y ). W e now in tro duce the follo wing pro cess ∀ t ∈ [0 , T ] , I t = ( U t − V t ) · ( X t − Y t ) . By Ito’s lemma, ∀ t ∈ [0 , T ] , E [ I T ] = E [ I t ] − Z T t h  ˜ F ˜ G  ( X s , U s ) −  ˜ F ˜ G  ( Y s , V s ) ,  X s − Y s U s − V s  i ds. Using the mono tonicit y of c o eﬃcient s, w e get that ∀ t ∈ [0 , T ] , 0 ≤ E [ I T ] ≤ E [ I t ] , and c F E " Z T 0 | ˜ F ( X s , U s ) − ˜ F ( Y s , V s ) | 2 # ds ≤ E [ I 0 ] . By deﬁnition o f ( X s , Y s ) s ∈ [0 ,T ] , it follows naturally that ∀ t ∈ [0 , T ] , k X t − Y t k ≤ k X − Y k + t c F p E [ I 0 ] . Using this estimate and b y a backw ard application of Gronw all lemma, there exists a co nstant C depending only on k ( G, g ) k Lip , c F such tha t ∀ t ∈ [0 , T ] , k U t − V t k 2 ≤ C e C T  k X − Y k 2 + E [ I 0 ]  . Since by Cauch y sc hw artz inequalit y ∀ λ > 0 , E [ I 0 ] ≤ λ 2 k X − Y k 2 + 1 2 λ k U 0 − V 0 k 2 , choosing λ = C e C T and ev aluating for t = 0 w e get k U 0 − V 0 k 2 ≤ (2 C e C T + C 2 e 2 C T ) k X − Y k 2 . This sho ws that there ca n be at most one strong solution for a given initial co ndition. Since this a priori e s timate is v alid for any initial condition and on a n y time in terv al, it follows naturally from ideas introduced in [14, 39] that the FBSDE (A.1) admits a Lipsc hitz decoupling ﬁeld U : [0 , T ] × R d × P 2 ( R d ) → R d . In particular, this allows to conclude tha t there indeed exists a unique stro ng solution to (A.1) for an y initial condition X ∈ H d . 

Extragradient methods for mean field games of controls and mean field type FBSDEs

Original Paper

Comments & Academic Discussion

Leave a Comment

Original Paper

Related Papers

Comments & Academic Discussion

Leave a Comment