Feedback Nash equilibria for scalar N-player linear quadratic dynamic games

F eedbac k Nash equ ilibria for sc alar N-pla y er linear quadratic dynamic games ⋆ Benita Nor tmann a , Mari o Sassano b and T h ulasi Mylv agana m a , a Dep artment of A er onautics, Imp erial Col le ge L ondon, L ondon SW7 2AZ, UK b Dip artimento di Inge gneria Civil e e Inge gneria Informatic a, Universit` a di R oma T or V er gata, 00133 R oma, Italy Abstract Considering infinite-horizon, discrete-time, linear qu ad ratic, N -play er dynamic games with scalar dynamics, a graphical representa tion of feedb ack Nash equilibrium solutions is p rovided. This represen tation is utilised to d erive conditions for th e num ber and prop erties of different feedback N ash equilibria a game may admit. The results are illustrated via a numeri cal example. Keywor ds: Linear quadratic discrete-time dy namic games, F eedback Nash equilibria 1 In tro duction Dynamic game theory (Ba¸ sar & O lsder (1998)) provides powerful to ols to mo del dynamic in teractions betw een strategic decision makers, with applications inc luding , for instance, econo mics a nd ecolo gy (Jørgensen et al. (2007)), rob otics (Li et al. (2 0 19)), m ulti-agent systems (Mylv aga na m et al. (2017), Capp e llo & Mylv agana m (2022)), p ower systems (Singh et al. (20 17)), and cyb er- security ( Etesami & Ba¸ sar (2019)). The Nash e quilib- rium is a commonly considered solution concept and of natural in terest in non-co op era tive settings, s inc e a unilateral deviation from the equilibrium by an y play er results in a worse outco me for the deviating play er (Ba¸ sar & Olsder (19 98)). F or infinite-horizon lin- ear quadratic dynamic g ames, (linear) feedback Nash equilibrium (FNE) solutions, that is Nash equilibria inv o lv ing linea r static state-feedback stra tegies, are characterised via the solutions of coupled algebra ic matrix equations. While re miniscent of the alg ebraic Riccati equations ar ising in linear q ua dratic optimal control, the coupled equations asso ciated with FNE are generally difficult to solve and ma y a dmit mult iple solutions with differen t outcomes (see, e.g. Sta r r & Ho ⋆ The w ork of B. Nortmann has been partially supp orted by UKR I DTP 2019 gran t no. EP/R513052/1. Email addr esses: benit a.nortmann1 5@imperial.a c.uk (Benita Nortmann), mario.sas sano@unirom a2.it (Mario Sassano), t.mylvaganam@im perial.ac.uk ( Thula si Mylv aganam). (1969), Ba¸ sar & Ols der (1 998), Engwerda (2005)). F or the contin uous- time setting, namely linear quadr a tic differ ential games, the existence, n um ber and prop erties of FNE solutio ns ha v e b een extensively studied (see, e.g. Papa v assilo p o ulos et al. (1979), Ba¸ sar & Olsder (1998), Engwerda (2005), Possieri & Sas sano (2015), Engwerda (2016)), ho wev er , the discrete- time eq uiv alent has re c e ived significantly less atten tion (Ba¸ sar & Olsder (1998), Pac h ter & Pham (201 0)). Additional pro d- uct terms of the decisio n v ariables introduce further challenges specific to the disc rete-time ca se a nd make this case an in teresting problem to study (see, e.g. Monti et al. (2024), Nortmann et al. (2024)). With the aim of developing int uition reg arding FNE solutions of dis c rete-time, linear qua dratic, dynamic games, the foc us of this note lies on a cla ss of games in which the sta te and the pla yers’ inputs a re sca lar v ari- ables. F or this cla ss o f games, a gra phical re pr esentation of the coupled equations asso ciated with FNE solutio ns is prop o sed. Utilising geo metric argument s, c onditions in terms o f the system and cost pa rameters characteris- ing the num b er and certain prop er ties of FNE solutions a given game admits are deriv ed. The a nalysis can be considered a discrete-time counterpart to the study of scalar differential games presented in Engwerda (2016) and (Eng werda 2005, Chapter 8 .4). Despite s imila rities in the underlying constr uctions, a few notew orthy dif- ferences b etw een the contin uous-time and the discrete- time settings ar e observed and discussed in this pap er . Some preliminar y res ults limited to games in volving t w o play ers ha ve appeared in Nortmann et al. (2023). In this note, the r esults ar e genera lised to the N -play er cas e and a wide r range of cost para meters are conside r ed. The remainder of the pa p er is organis ed a s follows. Pre- liminaries on linear quadratic dynamic g ames are re- called in Section 2. A graphical interpretation of FNE solutions is pro p o sed in Section 3. In Se c tion 4, this is utilised to derive conditions for the n umber a nd pro p- erties of FNE solutio ns a game may admit. The res ults are illustrated via a nu merical e x ample in Section 5 and concluding remar ks are provided in Section 6. 2 Preliminaries Consider a discrete-time linear time-in v ar iant system in- fluenced by the control actio ns u i ∈ R , i = 1 , . . . , N , of N ∈ N players, describ ed b y the sca lar dynamics x ( k + 1) = ax ( k ) + N X i =1 b i u i ( k ) , (1) with x (0) = x 0 , where x ∈ R denotes the s ystem state and a ∈ R , b i ∈ R \ { 0 } , i = 1 , . . . , N , are constant system para meter s 1 . Let the quadr atic cost functional J i ( x 0 , u 1 , . . . , u N ) = ∞ X k =0  q i x ( k ) 2 + r i u i ( k ) 2  , (2) with q i ∈ R , r i ∈ R > 0 , be asso ciated with play er i , i = 1 , . . . , N . The dynamics (1) and the cost functionals (2), i = 1 , . . . , N , constitute a linear quadr atic dy na mic game. Definition 1. An admissible 2 set of str ate gies { u ⋆ 1 , . . . , u ⋆ N } c onstitutes a N ash e quilibrium solut ion of the game (1) , (2) , i = 1 , . . . , N , if J ⋆ i = J i ( x 0 , u ⋆ i , u ⋆ − i ) ≤ J i ( x 0 , u i , u ⋆ − i ) , (3) holds for al l admi ssible { u i , u ⋆ − i } , for i = 1 , . . . , N , wher e u − i = { u 1 , . . . , u i − 1 , u i +1 , . . . , u N } . The str ate gy u ⋆ i is r eferr e d to as a N ash e quilibrium str at- e gy of player i , i = 1 , . . . , N , and the set { J ⋆ 1 , . . . , J ⋆ N } is the Nash e quilibrium outc ome. F o cusing on linear sta te- feedback strategies , Nash equi- librium s olutions can be characterised via the stabilising solutions of a se t o f coupled algebra ic equations. This characterisation of FNE solutions is r ecalled in the fol- lowing statement. 1 If b i = 0, for any i = 1 , . . . , N , then u i does not influence the dynamics (1) and play er i can b e disregarded. H ence, this case is excluded without loss of generalit y . 2 A set of feedback strategies { u 1 , . . . , u N } , with u i = k i x , i = 1 , ..., N , is admissible if it renders t h e zero equilibrium of system (1) asymptotically stable. Theorem 1. Consider the game (1) , (2) , i = 1 , . . . , N . The set of str ate gies { u ⋆ 1 , . . . , u ⋆ N } , wher e u ⋆ i ( k ) = k i x ( k ) , (4) for i = 1 , . . . , N , c onstitu tes a FNE of the game if and only if | a cl | < 1 , (5) wher e a cl := a + P N j =1 b j k j , and t her e exist p i ∈ R , for i = 1 , . . . , N , satisfying the set of e quations 0 =  a 2 cl − 1  p i + q i + k 2 i r i , (6a) 0 =  r i + b 2 i p i  k i + b i p i   a + N X j =1 ,j 6 = i b j k j   , (6b) and such t hat  r i + b 2 i p i  > 0 , (6c) for i = 1 , . . . , N . The c ost incurr e d by player i starting fr om initial c ondition x (0) = x 0 is J ⋆ i = p i x 2 0 . Pr o of. T he pr o of follows from dyna mic programming (Bellman (1957)) via analo g ous a rguments as used in the pro of o f (Mon ti et a l. 2 024, Theor em 3.2), which consid- ers t wo-pla yer games with q i ≥ 0, for i = 1 , 2. In the more general case consider ed herein the condition (6c ) ensures that the minimisation problem fo r player i is well-posed, for i = 1 , . . . , N . Even in the sca la r cas e co nsidered herein, the coupled algebraic eq uations (6), i = 1 , . . . , N , a re g e ne r ally chal- lenging to solve a nd ther e may b e multiple stabilising solutions, as is the ca se for the contin uo us-time coun- terpart, see e.g. Engwerda (20 05). In the follo wing sec- tion, w e use geometric a rguments to derive conditions for the num b er and prop er ties of FNE of the g ame (1), (2), i = 1 , . . . , N . Remark 1 . It i s inter esting to note that c ontr ary to their c ontinuou s - time c ounterp art ( t he c ouple d algebr aic Ric- c ati e quations, se e e.g . (Eng werda 201 6, Equation (3), i = 1 , . . . , N ) ), (6) , i = 1 , . . . , N , ar e no t quadr atic e qua- tions. In fact, they ar e cubic in t he de cision variables. Mor e over, while in the c ontinu ous-time c ase t he e qu ilib- rium gain of player i dep ends on the gains of the other players only implicitly via the c oupling of the e quivalent to (6a) , namely (Engwer da 2016, Equation (3)), it is ev- ident fr om (6b) that in the discr ete-time c ase k i explic- itly dep ends on the e quilibrium gains of al l other players. Final ly, while in the c ontinuous-time setting t he c ondi- tion r i > 0 is sufficient to ensur e the s olut ions of the e quivalent of (6a) , (6b) c orr esp ond to minimising (2) for player i (and henc e any stabilising solution of (Engwer da 2016, Equation (3), i = 1 , . . . , N ) c onstitutes a FNE), the additio nal c ondition (6c) is ne e de d in the discr ete- time c ase, for i = 1 , . . . , N . These differ enc es int ro duc e additiona l chal lenges in finding FNE solutions c omp ar e d to the c ontinu ous-time sett ing, and make the discr ete- time c ase inter est ing to stu dy. 2 3 Graphical i nterpretation of FNE With the aim o f deriving conditions for the existence of no, a unique or multiple FNE s olutions of the g a me (1), (2), i = 1 , . . . , N , we intro duce a reformulation o f Theo- rem 1. T o this end, consider the following as sumptions. Assumption 1. Let σ i := b 2 i q i r i , for i = 1 , . . . , N . The play ers ar e order ed such that σ 1 ≥ σ 2 ≥ . . . ≥ σ N . Assumption 2. Let σ N > − 1. Lemma 1. C onsider the game (1) , (2) , i = 1 , . . . , N . If Assu m ptions 1 and 2 hold, then (6c) holds fo r any solution of (5) , (6a) , (6b) , for i = 1 , . . . , N . Pr o of. F or (6c) to hold we r equire p i > − r i b 2 i . Consider first the case a cl 6 = 0. Combin ing (6a) and (6 b), and solving (6a) for p i gives that p i > − r i b 2 i is equiv alent to b i k i a cl < 1 , (7) for i = 1 , . . . , N . Combining again (6a) and (6b) b y solv- ing (6a) for p i , substituting this into (6b) and mult iply- ing by b i r i yields 0 = a cl ( b i k i ) 2 +(1 − a 2 cl ) b i k i + a cl σ i . Then, noting that a cl = a i + b i k i , with a i := a + P N j =1 ,j 6 = i b j k j , gives the condition 0 = ( b i k i ) 2 +  a i − ( σ i + 1) a i  ( b i k i ) − σ i , for i = 1 , . . . , N . Solving this quadratic equation g ives b i k i = − a i + γ i ± p γ 2 i − 1 and hence a cl = γ i ± p γ 2 i − 1, where γ i := 1 2  a i + σ i +1 a i  . Thus, (7) is in turn equiv a- lent to 1 − a i γ i ± p γ 2 i − 1 < 1 , which holds if a i and γ i hav e the same sign. This in turn holds tr ue if σ i > − 1. If a cl = 0, (6a) and (6b) imply p i = q i . Hence, (6c) holds if q i > − r i b 2 i , whic h is again equiv alent to σ i > − 1. By Assumption 1, Assumption 2 implies σ i > − 1, for i = 1 , . . . , N . Remark 2. While As s umption 1 c an b e intr o duc e d with- out loss of gener ality, Assum ption 2 only dep ends on sys- tem and c ost p ar ameters and c an h enc e b e verifie d prior to the c omputation of solutions. The r elevanc e of Assump- tion 2 is highl ighte d in L emma 1. Note that the r esults pr esente d in t he re mainder of t he p ap er ar e st il l r elevant if Assum ption 2 do es not hold. However, in this c ase they c onc ern any solutions of (5) , (6a) , (6b) , i = 1 , . . . , N . Henc e, t he c ondition (6c) , or alternatively (7) , ne e ds to b e che cke d to ensure su ch a solution c orr esp onds to a FNE. Lemma 2. C onsider the game (1) , (2 ) , i = 1 , . . . , N . L et Assumptions 1 and 2 hold and c onsider the function ˆ f ( ξ ) = ( − ξ − p ξ 2 + 1 if ξ < 0 , − ξ + p ξ 2 + 1 if ξ > 0 . (8) The set of str ate gies { u ⋆ 1 , . . . , u ⋆ N } , wher e u ⋆ i ( k ) is given by (4 ) with k i = − ξ − t i p ξ 2 − σ i b i , (9) for i = 1 , . . . , N , c onst itutes a FNE of the game with a cl 6 = 0 , if and only if ther e exist t i ∈ {− 1 , 1 } , for i = 1 , . . . , N , and ξ ∈ R \ { 0 } , satisfying a = ˆ f ( ξ ) + N ξ + t 1 p ξ 2 − σ 1 + . . . + t N p ξ 2 − σ N . (10) Pr o of. B y Theorem 1, FNE s olutions of the g ame are characterised b y the stabilising solutions of (6), i = 1 , . . . , N . The pro of lies in showing that s olving (10) is equiv alent to solving (5), (6), i = 1 , . . . , N . Eliminating p i in (6) (namely solving (6a) for p i and substituting this int o (6b)) gives the conditio n 0 = b i 2 k 2 i + ξ k i + σ i 2 b i , (11) with ξ := 1 2  1 a cl − a cl  . The equation (11) admits the solutions (9), t i ∈ {− 1 , 1 } , for i = 1 , . . . , N . Hence, a cl can b e wr itten as a cl = − ξ ± p ξ 2 + 1 = a + N X j =1 b j k j , = a − N ξ − t 1 p ξ 2 − σ 1 − . . . − t N p ξ 2 − σ N . (12) By Definition 1 , we are interested in solutions (9), t i ∈ {− 1 , 1 } , suc h that (5) holds. Hence, ther e is a one- to- one corr esp ondence betw een ξ and a cl given b y a cl = ˆ f ( ξ ) as defined in (8). Substituting this into (12) gives (10). Thus, an y solution to (5), (6), i = 1 , . . . , N , is such that (10) holds. C o nv e rsely , let { t 1 , . . . , t N } and ξ b e a s o lution to (10 ). Note that this solution is such that (11) holds with k i as in (9), for i = 1 , . . . , N . Via a cl = ˆ f ( ξ ) as defined in (8), (5) holds and (11 ) implies that (6b) holds with p i = q i + k 2 i r i 1 − a 2 cl , which is the unique solution of the Lyapuno v equation (6a) for fixed a cl , for i = 1 , . . . , N . By Lemma 1, Ass umption 2 ensures that (6c) holds, for i = 1 , . . . , N . Hence, (10) implies (5), (6), i = 1 , . . . , N . 3 Remark 3. The r esult of L emma 2 intr o duc es the as- sumption a cl 6 = 0 . N ote that if a cl = 0 , t hen The or em 1, in p articular (6 b) , implies k i = 0 , for i = 1 , . . . , N , and henc e a = 0 . Thus, the assumption a cl 6 = 0 is only r e- strictive in the sp e cial c ase in which a = 0 . In this c ase, a set of FNE stra te gies is given by (4) , with k i = 0 , for i = 1 , . . . , N . However, this trivial solution c annot b e found via the r esult of L emma 2. Al l FNE solutions of the game (1) , (2) , i = 1 , . . . , N , with a = 0 ar e henc e given by t he solut ions satisfying the c onditions of L emm a 2 (if ther e ar e any) and, in addition, the solution (4 ) , with k i = 0 , i = 1 , . . . , N . In Le mma 2 FNE solutions of the g ame (1), (2), i = 1 , . . . , N , are c haracter ised via the co ndition (10 ). Con- sider the a uxiliary functions f ℓ ( ξ ) = ˆ f ( ξ ) + N ξ + τ ℓ, 1 p ξ 2 − σ 1 + . . . + τ ℓ,N p ξ 2 − σ N , (13) for ℓ = 1 , . . . , L , where L = 2 N and τ ℓ = ( τ ℓ, 1 , . . . , τ ℓ,N ) is an N -tuple ov er the set {− 1 , 1 } . The functions f ℓ ( ξ ), ℓ = 1 , . . . , L , in (13) capture all possible com binations of the v a lues whic h t i , for i = 1 , . . . , N , can tak e in (10). Hence, b y Lemma 2, FNE solutions of the game (1), (2), i = 1 , . . . , N , are r epresented graphically by the intersections of the functions f ℓ ( ξ ), ℓ = 1 , . . . , L , as defined in (1 3), with the horizo nt al line at level a . Remark 4. If the horizontal line at level a interse cts multiple auxiliary fu n ctions f ℓ ( ξ ) , ℓ = 1 , . . . , L , as de- fine d in (13) , in a p oint in which they c oincid e, then this interse ction p oint gener al ly c orr esp onds t o multiple dis- tinct FNE solutions of the game (1) , (2 ) , i = 1 , . . . , N , r esult ing in the same closed-lo op dy na mics a cl . H ow- ever, c onsider t he sp e cial c ase of two funct ions f ℓ ( ξ ) and f w ( ξ ) , for ℓ = 1 , . . . , L , w = 1 , . . . , L , ℓ 6 = w , which c o- incide in the p oint ( ¯ ξ , ¯ f ) with ¯ ξ = ± √ σ j , j = 1 , . . . , N , and ¯ f = f ℓ ( ¯ ξ ) = f w ( ¯ ξ ) . Mor e over, f ℓ ( ξ ) and f w ( ξ ) ar e such that τ ℓ,i = τ w, i , for al l i = 1 , . . . , N , i 6 = j , or for any i 6 = l , i 6 = j , if the game is such that any σ l = σ j , l = 1 , . . . , N , l 6 = j . If the line at level a interse cts t hese two fun ctions in ( ¯ ξ , ¯ f ) , then ther e exists only one c orr e- sp onding set of gains { k 1 , . . . , k N } . This set of gains is given by k j = − ¯ ξ b j , k l = − ¯ ξ b l and k i as define d in (9) with t i = τ ℓ,i = τ w, i , for i 6 = j , i 6 = l . Henc e, in this sp e- cial c ase the int erse ction p oint ( ¯ ξ , ¯ f ) c orr esp onds to one FNE solution ra ther than two distinct FNE solutions. With the aim of utilising this graphical represe ntation to characterise the possible n um ber a nd prop erties of FNE, w e first take a closer lo ok a t the functions (13), ℓ = 1 , . . . , L . Let T = { τ 1 , . . . , τ L } denote the s et of all N -tuples over {− 1 , 1 } a nd consider the function T ℓ : { 1 , . . . , N } → { τ ℓ, 1 , . . . , τ ℓ,N } , defined by T ℓ ( i ) = τ ℓ,i , for i = 1 , . . . , N . Consider in particular the functions f l , l = 1 , 2 , 3 , L − 2 , L − 1 , L , a nd define the N- tuples τ l ∈ T , for l = 1 , 2 , 3 , L − 2 , L − 1 , L , such that T 1 ( i ) = − 1 , i = 1 , . . . , N , T 2 (1) = 1 , T 2 ( i ) = − 1 , i = 2 , . . . , N , T 3 (2) = 1 , T 3 ( i ) = − 1 , i = 1 , . . . , N , i 6 = 2 , T L − 2 (2) = − 1 , T L − 2 ( i ) = 1 , i = 1 , . . . , N , i 6 = 2 , T L − 1 (1) = − 1 , T L − 1 ( i ) = 1 , i = 2 , . . . , N , T L ( i ) = 1 , i = 1 , . . . , N . With this nota tion in place, let us highlight s ome prop- erties o f the auxiliary functions (13), ℓ = 1 , . . . , L , in the following statement 3 . Lemma 3. Consider the functions (13) , ℓ = 1 , . . . , L , and let As s umption 1 hold. Then, i. f 1 ≤ f 2 ≤ f 3 ≤ f ℓ ≤ f L − 2 ≤ f L − 1 ≤ f L , for any ℓ = 4 , . . . , L − 3 . ii. lim ξ →−∞       f ℓ ( ξ ) −   N − N X j =1 τ ℓ,j   ξ       = 0 , and lim ξ →∞       f ℓ ( ξ ) −   N + N X j =1 τ ℓ,j   ξ       = 0 . iii. f ℓ ( ξ ) is define d over t he r e al n umb ers for (a) ξ 6 = 0 , for ℓ = 1 , . . . , L , if σ 1 ≤ 0 . (b) ξ ≤ − √ σ 1 and ξ ≥ √ σ 1 , for ℓ = 1 , . . . , L , if σ 1 > 0 and σ 1 6 = σ 2 . (c) ξ ≤ − √ σ 1 and ξ ≥ √ σ 1 , for ℓ = 1 , L , and for  ξ 6 = 0 if σ 3 ≤ 0 , ξ ≤ − √ σ 3 and ξ ≥ √ σ 3 if σ 3 > 0 , for ℓ = 2 , 3 , L − 2 , L − 1 , if σ 1 > 0 and σ 1 = σ 2 . iv. if σ 1 > 0 and σ 1 = σ 2 , and ther e exist σ j > 0 such that σ j 6 = σ i , for i = 1 , . . . , N , j = 3 , . . . , N , i 6 = j , t hen let ¯ σ = max j ( σ j ) . No function f ℓ ( ξ ) , ℓ = 1 , . . . , L , is define d over the r e al numb ers for − √ ¯ σ < ξ < √ ¯ σ . v. if f ℓ ( ξ ) is define d for ξ 6 = 0 , then lim ξ → 0 − f ℓ ( ξ ) = − 1 + N X i =1 τ ℓ,i √ − σ i = ¯ a − ℓ , and lim ξ → 0 + f ℓ ( ξ ) = 1 + N X i =1 τ ℓ,i √ − σ i = ¯ a + ℓ . vi. if for al l ξ 6 = 0 ( N − 1) + | ξ | p ξ 2 + 1 6 = N X i =1 | ξ | p ξ 2 − σ i , then f L ( ξ ) is st rictly monotone for ξ < 0 and f 1 ( ξ ) is strictly monotone for ξ > 0 . Pr o of. Rec all tha t by Assumption 1 σ 1 ≥ σ 2 ≥ . . . ≥ σ N . Item i. follows from the definition o f T ℓ , for 3 Note that if N ≥ 3 and h ence L > 6, t h e N -tuples τ ℓ for ℓ = 4 , . . . , L − 3, and hence the functions f ℓ , ℓ = 4 , . . . , L − 3, are not defined ex plicitly , since they are not relev an t for the follo wing analysis. 4 l = 1 , 2 , 3 , L − 2 , L − 1 , L , and Assumption 1. Item ii. follows from (13) by noting that lim ξ →∞ | p ξ 2 + c − ξ | = 0 , ∀ c ∈ R , | c | < ∞ . Item iii. follows b y noting that (13 ) is defined ov er the real n um b ers if ξ 2 − σ i ≥ 0, for i = 1 , . . . , N , and utilising Ass umption 1 a nd the definition of T ℓ , l = 1 , 2 , 3 , L − 2 , L − 1 , L . Similarly , to demonstra te item iv. note that if σ 1 = σ 2 , then the cor resp onding terms in (13), ℓ = 1 , . . . , L , may cancel out, which affects the interv a l o ver whic h some functions are defined. I f a ny σ j > 0, j = 3 , . . . , N , such that σ j 6 = σ i , for i = 1 , . . . , N , i 6 = j , and ¯ σ = max j ( σ j ), then the corre s p o nding terms do not cancel in any func- tions. Utilising Assumption 1, the functions in which all terms c o rresp o nding to repe a ted σ i , i = 1 , . . . , N , ca n- cel out a r e defined over the real num b er s if ξ 2 − ¯ σ ≥ 0 . Item v. follows from (13 ) by noting that (8 ) is such that lim ξ → 0 − ˆ f ( ξ ) = − 1, and lim ξ → 0 + ˆ f ( ξ ) = 1. Finally , item vi. is shown by noting that d dξ f ℓ ( ξ ) = ( N − 1) ± ξ √ ξ 2 +1 + P N i =1 τ ℓ,i ξ √ ξ 2 − σ i , and using the definition of T ℓ , ℓ = 1 , L . 4 Conditions for exis te nce, n um b er and prop er- ties of FNE Via the result of Lemma 2, FNE solutions o f the game (1), (2), i = 1 , . . . , N , ca n b e r epresented graphically by the intersections of the functions (13 ), ℓ = 1 , . . . , L , with the horizontal line at level a . The prop er ties of the func- tions (13), ℓ = 1 , . . . , L , highlighted in L e mma 3 allow us to c haracter ise the p oss ible num b er and lo cation of such intersection p o ints, a nd hence to derive conditions for the existence of a FNE, a s well as the num b er and prop erties of differen t FNE solutions of the game, in the following statement. Theorem 2. Consider the game (1) , (2) , i = 1 , . . . , N , and let As s umptions 1 and 2 hold. Then, i. if system (1) is op en-lo op unstable with a fast r ate of diver genc e, i.e. | a | ≫ 1 , then t her e exist 2 N − 1 FNE solutions. ii. if σ 1 > 0 , then ther e always exist s at le ast one FNE solution for any value of a . iii. if σ 1 > 0 and ther e ex ist σ j > 0 su ch that σ j 6 = σ i , for i = 1 , . . . , N , j = 1 , . . . , N , j 6 = i , and ¯ σ = max j ( σ j ) , then the system (1) in close d lo op with any FNE solution is such that | a ⋆ cl | ≤   √ ¯ σ − √ ¯ σ + 1   . iv. if σ N ≥ 0 and the system (1) is op en- lo op stable, i.e. | a | < 1 , then ther e ex ists a unique FNE solution. v. if σ N < 0 , but σ 1 > 0 , σ 1 > σ 2 and the system (1) is op en-lo op stable with a fast r ate of c onver genc e, i.e. | a | ≪ 1 , then ther e exists a unique FNE solution if P N i =2 √ σ 1 − σ i < ( N − 1) √ σ 1 + √ σ 1 + 1 and ( N − 1) + | ξ | √ ξ 2 +1 < P N i =1 | ξ | √ ξ 2 − σ i . vi. if σ 1 ≤ 0 , and the system (1) is op en-lo op stable with a fast r at e of c onver genc e, i.e. | a | ≪ 1 , then ther e exists a unique FNE solution if  P N i =2 √ − σ i  − √ − σ 1 < 1 and ( N − 1) + | ξ | √ ξ 2 +1 6 = P N i =1 | ξ | √ ξ 2 − σ i . vii. if σ 1 = . . . = σ N = 0 and | a | = 1 ther e exists no FNE solution. Pr o of. B y Lemma 2 the intersection p oints of the func- tions (1 3 ), ℓ = 1 , . . . , L , with the horizontal line at level a co r resp ond to distinct FNE s olutions of the game (1), (2), i = 1 , . . . , N . The claims are hence shown by char- acterising the p o s sible intersection p oints. Item i. is a result of item ii. of Lemma 3 . F or la rge v alues of a , the line a t level a intersects once with L − 1 = 2 N − 1 of the L functions (13), ℓ = 1 , . . . , L . Item ii. follows from item ii. and item iii.(b),(c) of Lemma 3. Note that b y definition (13), ℓ = 1 , . . . , L , are contin uo us for ξ 6 = 0 and that f L − 1 ( − √ σ 1 ) = f L ( − √ σ 1 ) and f 1 ( √ σ 1 ) = f 2 ( √ σ 1 ). Hence, for any v a lue of a 6 = 0, the hor izontal line at lev el a intersects at least once with either o f the functions f ℓ ( ξ ), ℓ = 1 , 2 , L − 1 , L . T he case a = 0 is dis cussed in Remark 3. Item iii. follo ws from item iv. o f Lemma 3. If the stated conditions hold, any intersection p o ints of (13 ), ℓ = 1 , . . . , L , with the horizontal line at level a are such that ξ ⋆ ≤ − √ ¯ σ or ξ ⋆ ≥ √ ¯ σ . T he cla im is s hown b y recalling that a cl = ˆ f ( ξ ), with ˆ f ( ξ ) defined in (8). Item iv. follows from items i. , ii. , iii. and vi. of Lemma 3. If σ N ≥ 0, then by Assumption 1 σ i ≥ 0, for i = 1 , . . . , N , and the line at le vel a intersects only o nce with one of the functions (13), ℓ = 1 , L , for max ξ< 0 ( f L − 1 ( ξ )) ≤ − 1 < a < 1 ≤ min ξ> 0 ( f 2 ( ξ )). Item v. follows from items i. , ii. , iii.(b) and vi. of Lemma 3. If the stated conditions hold, then f L ( − √ σ 1 ) < f 1 ( √ σ 1 ) and f L ( ξ ) for ξ < 0 a nd f 1 ( ξ ) for ξ > 0 are strictly decreasing . Hence, the line at level a in tersects only onc e with one o f the functions (13), ℓ = 1 , L , for v ery small v alues o f a . Item vi. follo ws from items i. , ii. , iii. (a) , iv. and vi. of Lemma 3. If the stated conditions hold, then ¯ a − L − 1 < ¯ a + 2 and f L ( ξ ) for ξ < 0 and f 1 ( ξ ) for ξ > 0 ar e b oth either strictly decrea sing or str ictly incr easing. Hence, the line at level a intersects only once with one of the functions (13), ℓ = 1 , L , fo r very small v alues o f a . Finally , item vii. follows from items i. , ii. , iii.(a) and v. of Le mma 3. Note that if σ i = 0, for i = 1 , . . . , N , then f L ( ξ ) > − 1, f L − 1 ( ξ ) < − 1 for ξ < 0, f 1 ( ξ ) < 1, f 2 ( ξ ) > 1 for ξ > 0, and ¯ a − ℓ = − 1 , ¯ a + ℓ = 1 for ℓ = 1 , . . . , L . Hence, since (13), ℓ = 1 , . . . , L , are not defined at ξ = 0, the hor izontal lines at a = 1 and a t a = − 1 do not int ersect with any o f the functions. Remark 5. It is inter esting to c omp ar e the r esults of The or em 2 with the c ontinuous- t ime c ount erp art. While analo gous r esult s to items i. - iv. and vii. c an b e derive d for 5 sc alar line ar quadr atic differential games , s e e Engwer da (2016), note that for this class of c ontinuous-time dy- namic games ther e always exist s a unique fe e db ack Nash e quilibrium for games involving op en - lo op st able systems with fast r ate of c onver genc e, irr esp e ctive of the the signs of σ i , i = 1 , . . . , N , (Engwer da 2016, The or em 3.1). In the discr ete-t ime setting, however, t his only holds tru e for sp e cial c ases if σ N < 0 , such as under the c onditions of items v. and vi. of The or em 2. 5 Example T o illustr ate the presen ted r esults, cons ider the ga me defined b y the dynamics (1) with N = 3, a = ˜ a , and b 1 = b 2 = b 3 = 1, and the cost functionals (2), i = 1 , 2 , 3, with q 1 = 0 . 1, q 2 = ˜ q 2 , q 3 = ˜ q 3 and r 1 = r 2 = r 3 = 1. Note that σ i = q i , for i = 1 , 2 , 3. Firstly , let ˜ q 2 = 0 . 05 and ˜ q 3 = 0 . The co rresp o nding a ux iliary functions (1 3), ℓ = 1 , . . . , 8, ar e plotted in Fig ure 1. The horizontal y el- low lines indicate the levels ˜ a = 0 . 3 and ˜ a = − 5 and the yello w cr osses hig hlight the intersection p oints b etw ee n the horizontal lines and the functions (13), ℓ = 1 , . . . , 8 . By Lemma 2 these int ersection points corr e sp ond to FNE so lutions o f the ga me for the resp ec tive v alue of ˜ a . F or ˜ a = 0 . 3 there is only a sing le intersection po int. In line with item iv. of Theo r em 2, this single intersection po int co rresp o nds to a unique FNE solution of the game. F or ˜ a = − 5 there ar e 2 3 − 1 = 7 intersection po int s, which in line with item i. of Theor e m 2 corr esp ond to 7 FNE solutions. Se c o ndly , let ˜ q 2 = − 0 . 8 and ˜ q 3 = − 0 . 9 . The cor r esp onding auxilia ry functions (13), ℓ = 1 , . . . , 8, are plotted in Figur e 2. As ab ove, the horizontal yello w lines indicate the levels ˜ a = 0 . 3 and ˜ a = − 5 and the yel- low cro sses highlight the in tersection p o ints b etw een the lines a nd the functions (13), ℓ = 1 , . . . , 8. F o r ˜ a = − 5 there a re again 2 3 − 1 = 7 intersection p oints. How e ver, for ˜ a = 0 . 3 there are 3 intersection po int s. By ins p ec t- ing the functions f 1 ( ℓ ), f 2 ( ℓ ), f 7 ( ℓ ) and f 8 ( ℓ ) it is evi- dent that there ar e mor e than 1 in tersection points for any | a | ≪ 1 . Note that the c o nditions P 3 i =2 √ σ 1 − σ i < 2 √ σ 1 + √ σ 1 + 1 and 2 + | ξ | √ ξ 2 +1 < P 3 i =1 | ξ | √ ξ 2 − σ i are not satisfied for this example. Hence this obser v ation is in line with item v. of Theor em 2 and highlights the differ- ence b etw een discr ete-time a nd con tinu ous-time sca lar linear quadr a tic dynamic games c alled to a tten tion in Remark 5. Note that for all descr ib e d cases it holds that | a ⋆ cl | ≤ | √ σ 1 − √ σ 1 + 1 | = 0 . 7326 in line with item iii. of Theor em 2. 6 Conclusion Considering in finite-horizo n, discrete-time, line a r quadratic, N - play er dynamic g ames inv olving dynam- ics in which the state and the input of each play er ar e scalar v aria bles, a graphical representation of the co ndi- tions ch arac ter ising FNE solutions is prop o s ed. Via g e- ometric arguments, this repr esentation allows to derive -5 -4 -3 -2 -1 0 1 2 3 4 5 -10 -8 -6 -4 -2 0 2 4 6 8 10 Fig. 1. Plot of the auxiliary functions for ˜ q 2 = 0 . 05 and ˜ q 3 = 0, f 1 ( ξ ) (red), f 2 ( ξ ) (green), f 3 ( ξ ) (blue), f ℓ ( ξ ), ℓ = 4 , 5 (grey), f 6 ( ξ ) (cyan), f 7 ( ξ ) (dark green) and f 8 ( ξ ) (ma- genta ), and their intersectio n p oints (yello w crosses) with the horizontal lines at a = ˜ a (y ello w) for ˜ a = 0 . 3 and ˜ a = − 5. The black dashed lines indicate the linear asymp- totes of f ℓ ( ξ ), ℓ = 1 , . . . , 8, and the grey dotted lines indicate ξ = ± √ σ 1 = ± 0 . 3162. -5 -4 -3 -2 -1 0 1 2 3 4 5 -10 -8 -6 -4 -2 0 2 4 6 8 10 Fig. 2. Plot of th e auxiliary functions for ˜ q 2 = − 0 . 8 and ˜ q 3 = − 0 . 9, f 1 ( ξ ) (red), f 2 ( ξ ) (green), f 3 ( ξ ) (blue), f ℓ ( ξ ), ℓ = 4 , 5 (grey), f 6 ( ξ ) (cyan), f 7 ( ξ ) (dark green) and f 8 ( ξ ) (magen ta), an d their intersection p oints (yello w crosses) with the horizontal lines at a = ˜ a (y ello w) for ˜ a = 0 . 3 and ˜ a = − 5. The black dashed lines indicate the linear asymp- totes of f ℓ ( ξ ), ℓ = 1 , . . . , 8, and the grey dotted lines indicate ξ = ± √ σ 1 = ± 0 . 3162. conditions in terms of the system and cost par ameters characterising the nu mber and prop erties of solutions. The results are illustrated via a numerical example. References Ba¸ sar, T. & Ols der , G. J. (1998), Dynamic N onc o op- er ative Game The ory , V ol. 23 o f Classi cs in Applie d Mathematics , 2 edn, So ciety for Industrial and Ap- plied Mathematics. Bellman, R. (195 7), Dynamic Pr o gr amming , Princeton Univ ersity Pres s. 6 Capp ello, D. & Mylv aganam, T. (2022 ), ‘Distributed differential games for control of mu lti-agent systems’, IEEE T r ansactions on Contr ol of Net work Systems 9 (2), 63 5–64 6. Engwerda, J. (2005), LQ dynamic optimization and dif- fer ential games , John Wiley & Sons. Engwerda, J . (2016 ), ‘Prop erties of feedback Nash equi- libria in scalar LQ differential games’, A utomatic a 69 , 364– 374. Etesami, S. R. & Ba ¸ sar, T. (2019), ‘Dynamic ga mes in cybe r-physical secur ity: An overview’, Dynamic Games and Applic ations 9 , 884–91 3. Jørgens en, S., Q uincamp oix, M. & Vincent, T. (2 007), A dvanc es in Dynamic Game The ory: Numeric al Meth- o ds, Al gorithms, and Applic ations to Ec olo gy and Ec o- nomics , Annals of the In ternational So cie t y of Dy- namic Games, Birkh¨ a user Boston. Li, Y., Carb oni, G., Gonzalez, F., Camp olo, D. & Bur - det, E. (2019), ‘Differential game theory for versatile ph ysical human–rob ot int eraction’, Natu re Machine Intel ligenc e 1 (1), 36–43 . Monti, A., Nortmann, B., Mylv aganam, T. & Sas- sano, M. (20 24), ‘F eedback and op en-lo op Nash equi- libria for LQ infinite-horizon discrete- time dynamic games’, SIA M Journal on Contr ol and Optimization 62 (3), 141 7 –143 6. Mylv aga nam, T., Sass ano, M. & Astolfi, A. (20 17), ‘A differential game approach to multi-agent c ollision av oidance’, IEEE T r ansactions on Automatic Contr ol 62 , 4229 –423 5. Nortmann, B., Monti, A., Sassano, M. & Mylv aganam, T. (2023), ‘F eedback Nash eq uilibria for scalar t wo- play er linear- q uadratic discr ete-time dynamic games’, IF AC -Pap ersOnLine 56 (2), 1772 –177 7 . Nortmann, B., Monti, A., Sassano, M. & Mylv a- ganam, T. (2 024), ‘Nash equilibr ia for linear qua dratic discrete-time dynamic games via iterative and data- driven algo rithms’, IEEE T r ansactions on Automatic Contr ol 6 9 (10), 656 1–65 75. Pac h ter, M. & P ham, K. D. (2010), ‘Discrete-time line a r- quadratic dynamic games’, J ournal of O ptimization The ory and Applic ations 146 (1), 1 51–1 7 9. Papa v assilo p o ulos, G. P ., Medanic, J . V. & Cruz, J. B . (1979), ‘O n the existence of Nash strategies and solu- tions to coupled Riccati equa tio ns in linear- q uadratic games’, Jour n al of Optimization The ory and Applic a- tions 28 (1), 4 9–76 . Possieri, C. & Sas sano, M. (2015), An algebraic g e ome- try approa ch for the computation o f all linea r feedback Nash equilibria in LQ differential games, in ‘IEEE Conference on Decisio n and Control’, IEEE . Singh, V. P ., Kishor , N. & Samuel, P . (2017 ), ‘Dis- tributed multi-agent system-based lo ad frequency control for multi-area p ow er system in smart grid’, IEEE T r ansactions on Industrial Ele ctr onics 64 (6), 515 1 –516 0. Starr, A. W. & Ho, Y. C. (19 69), ‘Nonz e r o-sum differen- tial games’, Journal of Optimization The ory and Ap- plic ations 3 , 18 4–206 . 7