Stochastic Games for Security in Networks with Interdependent Nodes

Stochastic Games f or Security in Networks with Interdependent Nodes Kien C. Nguye n, T ansu Alpcan, and T a mer Bas ¸ar Abstract — This paper studies a stochastic game theoretic approach to security and in trusion detection in communication and computer netw orks. Specifically , a n Attack er and a Defe nder take part in a two -player game over a network of nodes whose securit y assets and vulnerabili ties are correlated. Such a network can be modeled using weighted d irected gra phs wi th the edges repr esenting the influence among the nodes. The game can be form ulated as a non-cooperative zero-sum or nonzero- sum stochastic game. Howev er , due t o correlation among the nodes, if some nodes ar e compro mised, the effective security assets and vulnerabilities of the r emaining ones will not stay the same in general, wh ich leads t o complex system dynamics. W e examine existence, uniq ueness, and structure of th e solu tion and also provide nu merical examples to illustrate our model. I . I N T RO D U C T I O N T oda y , as computer networks beco me u biquitou s, network security and intrusio n detection (ID) play a mo re and more importan t role. The main task o f an intrusion detection sys- tem (IDS) is to detect intru sions and repo rt them to a system administrator . Among various app roaches, non- coopera ti ve game theory has recen tly been employed extensi vely to stu dy ID problem s [1] –[6]. In a general setting, a security gam e is defined between two players: an Attac ker and a Def ender (the ID S). A formu lation of security games as static games can be foun d in [1]. I n [3], the authors co nsider security gam es with imperfect observations an d use the finite-state Markov chain framework to analyze such games. The work in [4] employs the framework of Bayesian games to address the intrusion detection pro blem in wire less ad ho c networks, where a mobile node viewed as a playe r confronts an o pponen t wh ose type is unknown. In [5], the autho r e xamines the intrusion detection problem in hetero genou s n etworks as a nonzer o-sum static game. In a co mplex network, n odes are of dif ferent levels of impor- tance to the Defend er , and a lso ap pear variably attractive to the Attacker . Heterogeneity also stems from hierarchy and cor relation amon g nodes. It is thus essential to consid er scenarios where nodes have dif ferent secur ity assets. Also, apart from a node ’ s secu rity ass et, if we take into acc ount the play ers’ motiv ations, the cost of attacking, the cost of monitorin g, and o ther factors, th e gam e is no longer a zero- sum one. Using th e Nash E quilibrium (NE) so lution con cept, This w ork w as supported by Deutsche T elek om Laboratori es and in part by the Boeing Company and the V ietna m Educatio n Found ation. T ansu Alpcan is with Deut sche T elek om Laborator ies, Ernst-Reute r- Platz 7, D-1058 7 Be rlin, German y tansu.alpcan@teleko m.de T amer Bas ¸ ar and Kien C. Nguyen ar e with the De partment of Elec trical and Computer Engineerin g and the Coordina ted Scienc e Laborato ry , Uni ver - sity of Illinois at Urbana-Champai gn, 1308 W Main St., Urbana, IL 61801, USA basar1 @illinois.edu , knguyen4@illinois.edu the analy sis allows one to compu te the Attacker’ s optimal strategy as a prob ability mass d istribution o n the nodes to attack. Similar ly , the Defender’ s optimal strategy is a probab ility mass distrib ution on the nodes to mon itor (to collect and p rocess data and detect attack s). H owe ver, in this work [5 ], the security assets a re still assume d to be indepen dent. Also, the d ynamics of the ID prob lem when nodes are com promised along the play have not b een taken into account. The work in [6] add resses this pro blem using the fra me- work of zero- sum stochastic games [8] . The network is now modeled as a discrete-time o r con tinuous-tim e Markov chain where the network states are defined by the states (com- promised or not) o f the constituent node s. This formulation thus takes into acco unt the d ynamics of th e p roblem and allows on e to incorporate correlation among nodes in terms of vulnerability . The analysis is nonetheless limited to zero- sum games and again, the security assets ar e co nsidered to be independen t. This paper attempts to e xtend these earlier works to co n- struct a mo re c ompreh ensiv e network security and intrusion detection model. W e develop a network mod el based on linear influence networks p roposed in [ 7]. This m odel, when used under the framework of stochastic g ames, p ermits us to take into co nsideration the correlation among the no des in terms of both security assets a nd vulnerabilities. The rest of this pa per is organized as follows. I n the remaining part of th is section, we sum marize the notations and variables used thro ughou t this paper . Next, in Sectio n II, we introdu ce two linear influence network mo dels f or security assets and vuln erabilities. In Section III, we formu- late the security game based on th ese models as a zero- sum stochastic g ame and pr esent results o n existence, un iqueness, and stru cture of the so lution. W e the n p rovide a numerical example in Section IV. Finally , some concludin g re marks of Section V end the paper . Summary of notations and variables u sed in this paper • N : Set of nodes in th e network. • n : Number of nodes in the n etwork. • E s : Set of edges representing the influen ce among node security assets. • E v : Set of edges repr esenting the influence among node vulnerab ilities. • e i j : A directed edge fro m node i to nod e j , e i j ∈ E s or e i j ∈ E v . • G s : W eighted dir ected gr aph fo r n ode security assets, G s = { N , E s } • G v : W eighted directed grap h for node vulnerab ilities, G v = { N , E v } • I , I i j : Influ ence matrix for security assets an d its en tries. • w i j : Influence of node i on node j in terms of security assets, where i , j ∈ N • s = { s 1 , s 2 , . . . , s n } : V ector of in depend ent secu rity as- sets. • x = { x 1 , x 2 , . . . , x n } : V ector of effective security assets. • H , h i j : Suppo rt matrix and its en tries, h i j signifies th e support tha t node i g iv es nod e j (against attacks), 0 ≤ h i j ≤ 1 ∀ i , j ∈ N . • h j : Support to node j , j ∈ N , h j = ∑ n i = 1 h i j . • p j n 1 : Probability that node j is comp romised whe n player 1 ( the Attacker) attacks, player 2 (th e Defender) d oes not defend the node, and the supp ort to node j is equa l to 1 (full support). • p j n 0 : Probab ility th at node j is comp romised when the Attacker attacks, th e Defender does not defend th e node, and the support to node j is equal to 0 (no support). • p j d 1 : Probab ility that n ode j is co mpromised wh en the Attacker attack s, th e Defender defends the node , and the support to node j is equa l to 1 (full support). • p j d 0 : Probab ility that n ode j is co mpromised wh en the Attacker attack s, th e Defender defends the node , and the support to node j is equa l to 0 (no support). •  S 1 , S 2 , . . . S p  : States in th e state spac e of the system. •  Γ 1 , Γ 2 , . . . Γ p  : Ga me eleme nts o f the stochastic game, each of which corresponds to a state of the system. • p k r : Pro bability th at the n etwork goes back to state S 1 , giv en that it is currently in state S k , the Attacker attacks one node and the attack fails. • p k e : Probability that the game ends gi ven that it is currently in state S k , the Attacker attacks one nod e and the attack f ails. • p k / 0 r : Probability that th e network goes back to state S 1 , giv en that it is curr ently in state S k and th e Attacker does not attack a ny node. • p k / 0 e : Probability th at the game ends given that it is currently in state S k and the Attacker does not attack any no de. • a k i j : Instant amount that player 2 pays play er 1 at game element Γ k , if player 1 plays pu re strategy i and player 2 plays pure strate gy j . • q kl i j : Proba bility that both p layers have to play ga me element Γ l next, g iv en that they are currently at g ame element Γ k , if player 1 plays pu re strategy i and player 2 plays pure strate gy j . • q k 0 i j : Pro bability that the game ends given that they are currently at game elemen t Γ k , if play er 1 plays pure strategy i and player 2 plays pure strategy j . • m k : Numb er of p ure strategies for player 1 at game element Γ k . • n k : Nu mber of pure strategies for play er 2 at gam e element Γ k . • p ( p = 2 n ) : Number of game elements of the stochastic game, or the number of states of the state space. • α k i j : A collecti ve en try that includes the instant payoff and th e tr ansition p robabilities to all game elements, α k i j = a k i j + ∑ p l = 1 q kl i j Γ l , given that the playe rs are cur- rently at g ame elemen t Γ k , pla yer 1 p lays p ure strategy i , and player 2 play s pure strate gy j . • b k i j : V alue of α k i j when we replace game elements Γ l ’ s with their v alues. b k i j = a k i j + ∑ p l = 1 q kl i j v l . • y kt i : Probability that player 1 plays pure strategy i wh en playing game element Γ k at the t -th stage of the gam e. For stationar y strategies [8], the superscript t will be omitted. • z kt j : Probability that p layer 2 play s pure strategy j when playing game element Γ k at the t -th stage of the gam e. • y kt , ( k = 1 , . . . , p , t = 1 , 2 , . . . ) : Strategy for p layer 1, a set of m k -vectors each of which is a m ixed strategy of player 1 at ga me elem ent Γ k and t -th stage of the g ame. • z kt , ( k = 1 , . . . , p , t = 1 , 2 , . . . ) : Strategy for player 2, a set of n k -vectors each of which is a mixed strategy of player 2 at ga me elem ent Γ k and t -th stage of the g ame. • c k i : Pure strategy i for the Attacker at game element Γ k . • d k j : Pure strategy j for the Defender at game elemen t Γ k . • p k s ( c k i , d k j ) : Pr obability that the a ttack is successful given that the Attacker play s pure strategy c k i and the Defender plays pure strate gy d k j at game element Γ k . • v = ( v 1 , v 2 , . . . , v p ) : V alue vector o f the stocha stic ga me. • val ( B ) : V alu e of the ze ro-sum matrix game gi ven by the matrix B . I I . L I N E A R I N F L U E N C E N E T W O R K M O D E L S F O R S E C U R I T Y A S S E T S A N D F O R V U L N E R A B I L I T I E S W e present in this section a network model based o n the concept of linear influe nce networks [7]. The network will be represented by two weighted directed graphs, one signifying the relatio nship of secu rity assets an d the o ther denoting vulnerab ility corre lation a mong the nodes. A. Linear influence network model for security assets For a particular node, the gen eral term security asset is used to signify how impo rtant the node is to th e network. All the secu rity assets of a n etwork can be modeled as a weighted dire cted gra ph G s = { N , E s } wher e N is the set of nodes, and the elements of set E s represent the influence among the nod es. Let n be the cardin ality of N . For each edge e i j ∈ E s , we denote an associated s calar w i j that signifies the influence of node i on node j , where i , j ∈ N . The entries of the influence matrix I are then given as follows: I i j =  w i j if e i j ∈ E s 0 otherwise, (1) where 0 < w i j ≤ 1 ∀ i , j ∈ N and ∑ n i = 1 w i j = 1 , ∀ j ∈ N . Note that here we allo w for the edges of the form w j j = 1 − ∑ n i = 1 , i 6 = j w i j , which signifies the portion of influence of a node on the independen t security asset of itself. Let s = { s 1 , s 2 , . . . , s n } be the vector of ind ependen t secu- rity assets . The vector of effective security assets , denoted by x = { x 1 , x 2 , . . . , x n } can then be co mputed by th e infl uence equation : x = I s . (2) W ith th e condition ∑ n i = 1 w i j = 1 , ∀ j = ∈ N , we have that n ∑ i = 1 x i = n ∑ i = 1 n ∑ j = 1 w i j s j = n ∑ j = 1 n ∑ i = 1 w i j s j = n ∑ j = 1 s j n ∑ i = 1 w i j = n ∑ j = 1 s j . (3) Therefo re, the sum of a ll the ef fective security assets is equal to the su m of all the indepen dent security assets. The influ ence matr ix thus sig nifies the redistribution of security assets. The independent security asset of a node i is redistributed to all the nodes in th e network tha t have influence on i (inclu ding itself). When a no de is down, the node itself and all th e e dges connected to it will be removed from the graph. T hus th e security loss of the ne twork will be the node’ s e ffecti ve secur ity asset (instead of its indepen dent security asset). Con versely , if a node is brought back to the network, it re gains its o riginal in fluence o n oth er n odes. I n either case, th e en tries of the influence matrix h av e to be normalized to satisfy ∑ n i = 1 w i j = 1 , ∀ j ∈ N . For a quick justification of this line ar influence model, consider a GSM network, where a b ase station controller (BSC) i controls se veral base transceiver stations (BTS), in cluding BTS j . If a BSC fails, all th e BTSs con nected to it will b e out of service. On th e contrar y , if o nly one BTS is compr omised, the commun ication amo ng th e subscribers u nder other BTSs should not be affected (provided that the r est of the network is up and ru nning) . In such a situation , we can have for example, w j j = 0 . 7 and w i j = 0 . 3. If the BSC is down, there is still an amount of security asset 0 . 7 s j left, ev en though the BTS is n ot in service anymore. T he reason is that, if this BTS gets c onnected to anothe r BSC (or if the origin al BSC is u p ag ain), they will together create an added security asset for the n etwork. W e present in what fo llows an example to illustrate the linear influence network mo del. P S f r a g r e p l a c e m e n t s 1 2 3 1 2 3 0 . 9 0 . 2 0 . 1 0 . 7 1 / 8 7 / 8 w 32 w 12 w 31 w 33 w 22 w 11 Fig. 1. A linear influence network for security a ssets of a three-node netw ork. Example 1: Suppose that we h av e a network of thr ee nodes with correlation s as sh own in Fig. 1. As shown in Fig. 2, th e states of th e system are given as  S 1 , S 2 , . . . S p  ( p = 2 n ) where S k ∈ { 0 , 1 } n , k = 1 , . . . , p . Here a nod e is said to be in state 1 if it is compro mised and 0 oth erwise. Note that we con sider a discrete-time M arkov chain whe re the system can tran sit from one state to any state of the state space (includin g the orig inal state). The influen ce equation (1,1,1) (0,0,0) (1,0,0) (0,0,1) (1,1,0) (0,1,1) (0,1,0) (1,0,1) P S f r a g r e p l a c e m e n t s 1 2 3 0 . 9 0 . 2 0 . 1 0 . 7 1 / 8 7 / 8 S 1 S 2 S 3 S 4 S 5 S 6 S 7 S 8 Fig. 2. An example s tate diagram for the netwo rk in Fig. 1. P S f r a g r e p l a c e m e n t s 1 1 1 1 2 2 2 3 3 0 . 9 0 . 2 0 . 1 0 . 1 0 . 7 1 / 8 7 / 8 Fig. 3. Changes in a lin ear influence network for securi ty assets when nodes are compromised (Example 1). (2) can be written as:    x ( 1 ) 1 x ( 1 ) 2 x ( 1 ) 3    =   0 . 9 0 . 2 0 0 0 . 7 0 0 . 1 0 . 1 1      s ( 1 ) 1 s ( 1 ) 2 s ( 1 ) 3    (4) Now suppose that node 1 is compr omised; then the in- depend ent security a sset of nod e 3 will remain the same, s ( 2 ) 3 = s ( 1 ) 3 . The independ ent secur ity asset of node 2 will be decreased by a n amount corresponding to the influe nce of nod e 1 o n node 2: s ( 2 ) 2 = s ( 1 ) 2 − 0 . 2 s ( 1 ) 2 = 0 . 8 s ( 1 ) 2 . Also, the influences o n each nod e have to be norm alized to have ∑ i w i j = 1. Thu s we no w have w 32 = 1 / 8 a nd w 22 = 7 / 8, and the influence equation becomes x ( 2 ) 2 x ( 2 ) 3 ! =  7 / 8 0 1 / 8 1  s ( 2 ) 2 s ( 2 ) 3 ! (5) Thus we can see x ( 2 ) 2 = ( 7 / 8 ) s ( 2 ) 2 = 0 . 7 s ( 1 ) 2 , x ( 2 ) 3 = ( 1 / 8 ) s ( 2 ) 2 + s ( 2 ) 3 = 0 . 1 s ( 1 ) 2 + s ( 1 ) 3 . After node 1 goes down, the effecti ve security asset of node 2 remains th e same, while th at of node 3 is dec reased by an amount representing its influence o n node 1. Now if node 3 is in turn com promised, we ha ve a network with one node as in Fig. 3. W e h av e s ( 3 ) 2 = s ( 2 ) 2 − s ( 2 ) 2 / 8 = ( 7 / 8 ) s ( 2 ) 2 = 0 . 7 s ( 1 ) 2 , x ( 3 ) 2 = s ( 3 ) 2 . B. Linear influence network model for vulnerabilities In this sub section, we use the linear influe nce network model to represent the correlation of n ode vulnerabilities in a network. Beside the correlation of security assets, nodes also have influenc e on othe rs’ vuln erabilities. For example, within a corporate network, if a workstation is comp romised, the data store d in this compu ter ca n be exploited in attacks against oth er workstations; th ese latter computer s thus will become more vu lnerable to intrusion . Under the framework of stochastic games, this k ind of in fluence is readily incor- porated. F or instance, in the n etwork o f Example 1, if the Attacker attacks no de 1 , and the Defender decides not to defend th is node, the prob ability that the system g oes from ( 0 , 1 , 0 ) to ( 1 , 1 , 0 ) will be greater that the probability th at the system goes from ( 0 , 0 , 0 ) to ( 1 , 0 , 0 ) , if nod e 2 has som e influence on node 1 in terms of vulnerability . For e i j ∈ E v , we define the support matrix as fo llows H =  h i j if e i j ∈ E v 0 otherwise, (6) where h i j signifies the suppor t that no de i g i ves n ode j (against attacks), 0 ≤ h i j ≤ 1 ∀ i , j ∈ N . The sup port to node j , j ∈ N is defin ed as h j = n ∑ i = 1 h i j , (7) where 0 ≤ h j ≤ 1 , ∀ j ∈ N . Unlike the model for security assets, here we do not no rmalize h j . When a no de that supports node j is down, h j will d ecrease, an d thu s th e probab ility that node j is compr omised und er attack w ill increase. Let u s denote b y p j s the pro bability that no de j is compro mised at each state. W e assume an affine relatio nship between p j s and h j as follows: • If node j is not attacked the n p j s = 0. • If node j is attacked, and the Defen der is not defending this node, p j s = p j n 0 − ( p j n 0 − p j n 1 ) h j , where p j n 1 and p j n 0 are th e pr obabilities that th e nod e is com promised given that th e sup port is equal to 1 (full support) and 0 (no support) , respectively ( p j n 1 < p j n 0 ). • If n ode j is attacked, and the Defender is defending this node, p j s = p j d 0 − ( p j d 0 − p j d 1 ) h j , where p j d 1 and p j d 0 are the probabilities t hat the node is co mpromised gi ven that the suppo rt is eq ual to 1 an d 0, resp ectiv ely ( p j d 1 < p j d 0 ). • Also, it is assumed that p j d 1 < p j n 1 and p j d 0 < p j n 0 . A weighted d irected gr aph for network vulner abilities is shown in Fig. 4. P S f r a g r e p l a c e m e n t s 1 1 2 3 3 0 . 9 0 . 9 0 . 2 0 . 1 0 . 1 0 . 7 0 . 7 0 . 5 0 . 3 1 / 8 7 / 8 Fig. 4. A line ar influence netwo rk for vulnerabi litie s and the cha nges of supports when one node is compromised. I I I . T H E N E T WO R K S E C U R I T Y P RO B L E M A S A Z E RO - S U M S T O C H A S T I C G A M E A. A brief overview of zer o -sum stochastic games In this su bsection, we p rovide a brief ov erview of zero- sum stochastic games based on [8]. A stochastic game consists of p game elements Γ k , k = 1 , . . . , p . Each game element is associated with an m k × n k matrix, whose entries are gi ven by α k i j = a k i j + p ∑ l = 1 q kl i j Γ l , (8) where q kl i j ≥ 0 , l = 1 , . . . , p , i = 1 , . . . , m k , j = 1 , . . . , n k , p ∑ l = 1 q kl i j < 1 , ∀ k , i , j . (9) Expression (8) can be inter preted as follows. At game element Γ k , if player 1 chooses pure strategy i and player 2 c hooses pure strate gy j , player 2 has to p ay play er 1 a n amount a k i j . Furthermore, ther e is a probability q kl i j that b oth players ha ve to play g ame elem ent Γ l next, an d a probab ility q k 0 i j = 1 − p ∑ l = 1 q kl i j (10) that th e game will end. With conditio n (9), the p robab ility of infinite play is guaranteed to b e zero, and the e xpected payoff of player 1 (or th e expected loss of player 2), which is accum ulated thro ugh all the stages of th e g ame, is finite [8]. A strate gy for p layer 1 is a set of m k -vectors, den oted by y kt , k = 1 , . . . , p , t = 1 , 2 , . . . , each of which satisfies m k ∑ i = 1 y kt i = 1 , (11) y kt i ≥ 0 (12) Here y kt i is the pr obability that player 1 plays p ure strategy i if he is playin g game element Γ k at the t -th stage of the game. A strategy is said to be stationary if the vectors y kt are indepen dent of t for all k . In this case, the super script t can be omitted. Similarly , a strategy fo r player 2 is a set of n k -vectors, z kt , where ∑ n k j = 1 z kt j = 1 and z kt j ≥ 0. Given a pair of strategies, we can compute the vector of expected payoffs v = ( v 1 , v 2 , . . . , v p ) , where v k , k = 1 , . . . , p is th e expected payoff (to playe r 1) if the first stage of the game is Γ k . W ith the above setting s, it is kn own [8], that we can replace the game element Γ k by the value com ponen t v k = val ( B k ) , (13) where val ( B k ) is the value (in mixed strategies) of th e matr ix game B k , and B k is the m k × n k matrix whose entries are gi ven by b k i j = a k i j + p ∑ l = 1 q kl i j v l . (14) B. A zer o- sum stochastic game model for network security In this subsection we fo rmulate th e security problem as a zero-sum stochastic ga me. This is a m odified version of the game presented in [6], applied to the linear influence netw ork model propo sed in Section II. At each state k , k = 1 , . . . , p , the Attacker’ s pure strategies co nsist of m k = n + 1 actions, where n is the n umber of nodes in the n etwork: • Attack one of n nodes, c k i , where i = 1 , . . . , n . • Do nothing , c k m k = / 0. Note that this strategy space is fo r use with m ore genera l payoff form ulations. However , with the pay off fo rmulation in this paper, the Attacker will no t hav e motiv ation to attack a n ode that is alr eady comp romised, unless all the nodes have b een comp romised. For each k , the Defend er’ s pur e strategies ar e  d k i  , where • Defend node i , d k i , i = 1 , . . . , n k − 1, • Do nothing , d k n k = / 0, where n k = m k = n + 1. F or e ach p ossible com bination o f the Attacker’ s and th e Defender’ s pure strategies, the entries of the payoff matrix are: α k i j = a k i j + p ∑ l = 1 q kl i j Γ l , (15) where a k i j = p k s ( c k i , d k j ) x k ( i ) , p k s ( c k i , d k j ) is the pr obability that the attack is successful, an d x k ( i ) is the effective security asset of the n ode being attacked, i . Note that o nce a n ode is comprom ised, the effecti ve security assets and the sup- ports of the remaining nodes h av e to be recalculated as in Example 1 and Fig . 4. As mentioned in Subsection II- B, the prob abilities p k s , and thu s q kl i j , are depen dent on the supports to the n odes, and are therefor e af fected by the correlation in vu lnerabilities of the nodes. It can b e said that once we h av e in corpor ated node vulner abilities into our model, we have already implicitly taken care of the cost of attacking/de fending . For example, if a node is of high secur ity asset b ut difficult to compromise (the tr ansition probab ility to the compromise state is small), the Attacker may turn to ano ther node with a smaller security asset, wh ich is easier to attack. At a state S k , if the Attacker cho oses to attac k one no de and the attack fails, there is a p robab ility p k r ∈ ( 0 , 1 ) that the network will go bac k to state S 1 (which me ans the Defender h as detected the Attacker and m anaged to restore all th e comp romised nod es and the game restarts at S 1 ), and a pr obability p k e ∈ ( 0 , 1 ) that th e ga me will end (which means the Defender has detected the Attack er and stopped him fro m furth er intruding ). Note that p k r + p k e ≤ 1 with equality only when S k = S 1 ( 0 , 0 , . . . , 0 ) . Similarly , at one point, if the Attacker choo ses not to attack at all, th ere is a prob ability p k / 0 r ∈ ( 0 , 1 ) that the network will go back to state S 1 , and a probab ility p k / 0 e ∈ ( 0 , 1 ) that the game will end. Given 0 < p j d 1 , p j n 1 , p j d 0 , p j n 0 < 1 , j ∈ N , p k r , p k e , p k / 0 r , and p k / 0 e , k = 1 , . . . , p , and the supp ort matrix H , p k s and q kl i j can be calculated using the equations in Su bsection II-B. A numerical example is sh own in Section IV. C. Existence, uniquene ss, and structur e of the solution W e p resent in this subsection some analytical results for the game given in III-B, based o n zero-sum stocha stic game theory [8], [9]. Pr op osition 1: In the ze ro-sum stochastic game given in III-B, the probability of infinite p lay is zero and the exp ected payoff o f the Attacker ( which is also the e xpected cost of the Defender ) is finite. W ith the setup in III-B, we can show that q k 0 i j = 1 − ∑ p l = 1 q kl i j > 0 , ∀ k and ∀ i , j of each game elemen t Γ k . Thu s the pr opo- sition is proved using the theory of stochastic games. Pr op osition 2: (Theorem V . 3 . 3 [8]) In the zero-sum stochastic game gi ven in III-B, there exists exactly one vector v = ( v 1 , v 2 , . . . , v p ) that satis fies (13) and (14). Using the results fro m III- A, we can th en com pute th e NE of the game, which is a pair of station ary mixed strategies for the Attacker an d for the Defender at each state. Pr op osition 3: (Theorem V . 3 . 3 [8] ) The vector v = ( v 1 , v 2 , . . . , v p ) that satisfies (13) and (14) can be d erived throug h the following recursi ve equ ations: v 0 = ( 0 , 0 , . . . , 0 ) , (16) b kr i j = a k i j + p ∑ l = 1 q kl i j v r l , (17) v r + 1 k = val ( B r k ) = va l ( b kr i j ) . (18) W e ca n stop the recu rsion at a desire d le vel of accuracy and then use the curren t value of vector v = ( v 1 , v 2 , . . . , v p ) to compute B k using (14). The mixed strategies of the players at eac h game element Γ k are the NE in mixed strategies of the ma trix g ame B k . The strate gies so obtained will con verge to optimal stationary strategies of the stochastic game. I V . A N U M E R I C A L E X A M P L E In this section, we imp lement num erical simulatio n for a specific network with th ree n odes. The setup in III-B is carried over with some fu rther assumptions as follows. First, we adopt a simplified state diagram as given in Fig. 1. Basically , after each time step, we o nly allow for transitions where o ne more n ode is compr omised, th e transition th at returns to th e sam e state, and th e transition b ack to S 1 ( 0 , 0 , 0 ) . Second, sup pose th at th e influe nce eq uation is given as follows (Ex ample 1)    x ( 1 ) 1 x ( 1 ) 2 x ( 1 ) 3    =   0 . 9 0 . 2 0 0 0 . 7 0 0 . 1 0 . 1 1     10 10 20   =   11 7 22   , (19) and the support matrix is given by (Fig. 4) H =   0 . 7 0 0 0 . 2 0 . 5 0 0 . 1 0 . 3 0 . 9   . (20) Finally , p j d 1 = 0 . 2 , p j n 1 = 0 . 4 , p j d 0 = 0 . 5 , p j n 0 = 0 . 7 , ∀ j ∈ N , p k r = 0 . 2 , ∀ k 6 = 1, p 1 r = 0 . 7 , p k e = 0 . 3 , ∀ k = 1 , . . . , p , p k / 0 r = 0 . 2 , ∀ k 6 = 1, p 1 / 0 r = 0 . 7, and p k / 0 e = 0 . 3 , ∀ k = 1 , . . . , p . For example, suppo se the system is at S 1 ( 0 , 0 , 0 ) . The next state could be one in { S 1 ( 0 , 0 , 0 ) , S 2 ( 0 , 0 , 1 ) , S 3 ( 0 , 1 , 0 ) , S 5 ( 1 , 0 , 0 ) } . The Attacker’ s pure stra tegies includ e 1 , 2 , 3 , an d / 0 , wh ich mean to attack n ode 1 , node 2, node 3 , and do n othing, respectively . Similarly , th e Defende r’ s pur e strategies include 1 , 2 , 3, and / 0. Using the above results, we hav e that a 1 11 = p 1 s ( 1 , 1 ) x ( 1 ) 1 , q 11 11 = ( 1 − p 1 s ( 1 , 1 ))( 1 − p 1 e ) , q 15 11 = p 1 s ( 1 , 1 ) , q 1 j 11 = 0 ∀ j 6 = 1 , 5 , where p 1 s ( 1 , 1 ) = p d 0 − ( p d 0 − p d 1 ) 1 = p d 1 , as at th is state, node 1 still has full support. Also, there is a probability p 1 e g = ( 1 − p 1 s ( 1 , 1 )) p 1 e > 0 that the game will end . If the Attacker attacks node 1 and the Defender d efends node 2, we ha ve that a 1 12 = p 1 s ( 1 , 2 ) x ( 1 ) 1 , q 11 12 = ( 1 − p 1 s ( 1 , 2 ))( 1 − p 1 e ) , q 15 12 = p 1 s ( 1 , 2 ) , q 1 j 12 = 0 ∀ j 6 = 1 , 5 , where p 1 s ( 1 , 1 ) = p n 0 − ( p n 0 − p n 1 ) 1 = p n 1 , ag ain as at this state, no de 1 still h as fu ll support. Also, th ere is a p robability p 1 e g = ( 1 − p 1 s ( 1 , 2 )) p 1 e > 0 that the game will end. Now , suppose th at the sy stem is at S 5 ( 1 , 0 , 0 ) . Th e next state co uld be one in { S 1 ( 0 , 0 , 0 ) , S 5 ( 1 , 0 , 0 ) , S 6 ( 1 , 0 , 1 ) , S 7 ( 1 , 1 , 0 ) } . The Attacker’ s p ure strategies in clude 2 , 3 , and / 0, whic h mean to attack node 2, no de 3, and do nothing, respectively . Similarly , the Defender’ s pu re strategies include 2 , 3, an d / 0. Now we have that a 5 22 = p 2 s ( 2 , 2 ) x ( 5 ) 2 , q 57 22 = p 2 s ( 2 , 2 ) , q 51 22 = ( 1 − p 2 s ( 2 , 2 )) p 5 r , q 55 22 = ( 1 − p 2 s ( 2 , 2 ))( 1 − p 5 r − p 5 e ) , q 5 j 22 = 0 ∀ j 6 = 1 , 5 , 7 , where p 2 s ( 2 , 2 ) = p 2 d 0 − ( p 2 d 0 − p 2 d 1 ) 0 . 8, as at this state, node 2 has a su pport of 0 . 8. Also, th ere is a pr obability p 5 e g = GE Node 1 Node 2 Node 3 Do noth ing 1 ( 0 , 0 , 0 ) 0 . 61 26 0 0 . 3874 0 2 ( 0 , 0 , 1 ) 0 . 38 17 0 . 6183 0 0 3 ( 0 , 1 , 0 ) 0 . 64 15 0 0 . 3585 0 4 ( 0 , 1 , 1 ) 1 0 0 0 5 ( 1 , 0 , 0 ) 0 0 . 6568 0 . 3432 0 6 ( 1 , 0 , 1 ) 0 1 0 0 7 ( 1 , 1 , 0 ) 0 0 1 0 8 ( 1 , 1 , 1 ) 0 . 25 0 . 25 0 . 25 0 . 25 T ABLE I O P T I M A L S T R ATE G I E S F O R T H E A T TAC K E R AT E A C H G A M E E L E M E N T ( G E ) . GE Node 1 Node 2 Node 3 Do noth ing 1 ( 0 , 0 , 0 ) 0 . 07 02 0 0 . 9298 0 2 ( 0 , 0 , 1 ) 0 . 66 14 0 . 3386 0 0 3 ( 0 , 1 , 0 ) 0 . 08 69 0 0 . 9131 0 4 ( 0 , 1 , 1 ) 1 0 0 0 5 ( 1 , 0 , 0 ) 0 0 . 034 0 . 966 0 6 ( 1 , 0 , 1 ) 0 1 0 0 7 ( 1 , 1 , 0 ) 0 0 1 0 8 ( 1 , 1 , 1 ) 0 . 25 0 . 25 0 . 25 0 . 2 5 T ABLE II O P T I M A L S T R ATE G I E S F O R T H E D E F E N D E R A T E A C H G A M E E L E M E N T . ( 1 − p 2 s ( 2 , 2 )) p 5 e > 0 that the ga me will end. The oth er entries of oth er gam e elements can be calculated in a similar way . Using the recur si ve procedu re given in Pro position 3, we can then compu te the optimal strategy of each play er and the value of the game. The value vector conv erges to an accuracy of 10 − 4 after 56 iter ations. Th e o ptimal strategies of the Attacker and th e Defend er , and the value vector are giv en in T ables I, III, and II I. As can be seen from T ab le I, for example, when all the n odes are up and ru nning, th e Attacker wants to attack node 1 with prob ability 0 . 6126 and node 3 with pr obability 0 . 3874, while the Defend er w ants to d efend no de 1 with pro bability 0 . 07 02 and node 3 with probab ility 0 . 9298. Recall tha t the effecti ve security assets of nodes 1 , 2, and 3 at th is state are 1 1 , 7 , an d 22 , re spectiv ely . It is worth notin g th at the m ixed strategies for the play ers can also be interpreted as the way to allocate their resources in the security game. GE 1 2 3 4 Payoffs 19 . 607 8 15 . 8301 17 . 9557 12 . 339 2 GE 5 6 7 8 Payoffs 17 . 965 9 13 . 0283 15 . 3228 7 . 8431 T ABLE III T H E V A L U E V E C T O R ( T H E E X P E C T E D PA YO FFS O F T H E A T TAC K E R , A L S O T H E E X P E C T E D L O S S E S O F T H E D E F E N D E R A T E A C H G A M E E L E M E N T ) . V . C O N C L U S I O N In this p aper we have pro posed a new n etwork model based o n linear influen ce network s to rep resent the inter- depend ence of nodes in terms o f secur ity assets a nd vul- nerabilities. W e to ok the first step to form ulate the secur ity game between an Attacker and a Def ender over this network using the framework o f zero-sum stochastic game theor y . The o ptimal so lution obtained allows one to c ompreh end the behavior o f a r ational attacker , as well as to provide IDSs with g uidelines on how to alloca te their resou rces. Moreover, modeling ne tworks with linear influence n etwork models helps facilitate solving the security game s using software progr ams. As mentioned earlier, apart from a node’ s secur ity asset, if we take into accou nt the players’ m otiv ations, the cost o f attackin g, the cost of monito ring, and o ther factors, the game is no lon ger a zero-sum one . This work thus can be extended to non zero-sum stoch astic g ames, wh ere we can address more flexible an d p ractical pay off formulations. Furthermo re, in many real-world scenarios, neith er th e At- tacker nor the Defender has full knowledge of the network’ s nodes and th eir corre lation. Thus studying stochastic secur ity games with incomplete inform ation is an intriguing research direction. V I . A C K N OW L E D G M E N T S W e would like to thank Deutsch e T elekom Lab oratories, the Boeing Company , and the V ietnam Education Foundation for th eir suppo rt. W e are also g rateful to four an onymous revie wers for their valuable comments. R E F E R E N C E S [1] T . Alpcan and T . Bas ¸ ar , “ A Game Theoreti c Approach to D ecisio n and Analysis in Network Intrusion Detection”, Pr oceedin gs of the 42nd IEEE Confe renc e on Deci sion and Contr ol , Hawa ii, USA, 2003, pp. 2595–2600. [2] T . Alpcan and T . Bas ¸ ar , “ A game theoreti c analysi s of intrusion detec tion in access control s ystems, ” Pr oceed ings of the 43r d IEEE Confer ence on Decisio n and Contr ol , Paradi se Island, Bahamas, 2004, pp. 1568–1573. [3] T . Alpcan and T . Bas ¸ ar , “ An intrusion detection game with limited observ ation s, ” Pro ceedings of the 12th Int. Symp. on Dynamic Games and Applications , Sophia Antipoli s, France, 2006. [4] Y . Liu, C. Comani ciu, and H. Man, “ A Bayesian game approach for intrusion detection in wirele ss ad hoc networks, ” Pro ceedings of the W orkshop on Game The ory for Net works (GameNets) , Pisa , Italy , 2006. [5] L. Chen, “ On Selfish and Malicio us Behav iors in W ir eless Networks - A Non-cooper ative Game Theor etic A ppr oach , ” Ph.D. thesis, T elecom ParisT ech, 2008. [6] K. Sallha mmar , “ Stocha stic Models for Combined Security and De- pendabil ity Evaluation , ” Ph.D. thesis, Norwegia n Unive rsity of Sci- ence and T echnology , 2007. [7] R. A. Miura-K o, B. Y olken, N. Bambos, and J. Mit chell, “ Security In vestme nt Games of Int erd ependent Organi zations , ” Proceedi ngs of the 46th Alle rton Confere nce, Illi nois, USA, Sep., 200 8. [8] G. Owen, Game Theory , 3nd Ed., Ca lifornia : Academic Press, 2001. [9] L. Shaple y , “ Stoc hastic games , ” Proc. Natl. Acad. Sci. USA 39 (1953) 1095-1100.