On Finite Exchangeability and Conditional Independence

On Finite Exc hangeability and Condit ional Indep e ndence Ka yv an Sadeghi Dep artment of Statistic al Scienc e, University Col le ge L ondon, Gower Str e et, L ondon, WC1E 6BT, Unite d Kingdom Abstract: W e study the independence structure of finitely exc hangeable distributions o ver random vec tors and random net works. In particular, w e prov i de nece ssar y and sufficient conditions for an exchange able v ector so that its elemen ts are completely independent or completely dependen t. W e also provide a sufficien t condition for an exc hangeable ve ctor so that its elemen ts are marginall y indep enden t. W e then generalize these results and conditions for exc hangeable random net wo r ks. In this case, it i s demon- strated t hat the situation is more complex. W e sho w that the independence structure of exc hangeable random netw orks l ies in one of six regimes that are t wo-fold dual to one another, represent ed by undirected and bidirected independence graphs in graphical model sense with graphs that are com- plemen t of eac h other. In addition, under certain additional assumptions, we prov i de necessary and sufficient conditions f or the exc hangeable net work distributions to b e faithful to eac h of these graphs. Keywords and phrases: conditional independence, exc hangeability , faith- fulness, random netw orks. 1. In tro duction The conce pt of exchangeability has b ee n a na tural and con venien t a ssumption to imp os e in probability theory and for simplifying s tatistical models. As de- fined or iginally for random s e q uences, it states tha t any order of a finite num ber of samples is equally likely . The concept was later generalized for binary ra n- dom ar rays [ 1 ], a nd consequently for r a ndom netw orks in statistical netw ork analysis. In this c ontext, exchangeability is tra nslated into inv aria nc e under re- lab eling o f the no des of the netw or k, whereby is omorphic graphs have the same probabilities; see, e.g., [ 14 ]. Exchangeability is closely related to the concept of independent and iden- tically distributed r andom v ariables. It is an immediate consequence o f the definition that indep endent and identically distr ibuted r andom v ariable s a re exchangeable, but the c o nv erse is not true. F or infinite s equences, the c o nv erse is established by the w ell- known de Finetti’s Theor em [ 5 ], which implies that in any infinite sequence of exchangeable random v a riables, the r andom v ariables are conditionally independent a nd identically-distributed g iven the underlying distributional form. O ther versions of deFinetti’s Theore m exist for the gener - alized definitions of e xchangeabilit y [ 27 , 7 ]. How ever, for finitely exchangeable r andom s equences (vectors) and arrays (matrices), the conv erse do es not hold, and the av ailable re s ults bas ic ally pro- 1 imsart-g eneric ver. 2014/10/16 file: exch-ind-arxiv- new.tex date: June 15, 2020 K. Sade ghi/On Finite Exchange ability and Conditional Indep endenc e 2 vide approximations of the infinite case; see, e.g ., [ 6 , 21 , 15 ]. In this pap er , we utilize a co mpletely different a pproach to study the r elationship betw een finite exchangeability and (conditiona l) independence . W e employ the theo r y of graphical mo dels (s e e e.g. [ 13 ]) in order to pro vide the independence structure of exchangeable distributions. In particular, we exploit the neces sary and sufficien t conditions, provided in [ 26 ], for faithfulness of probability distributions and g raphs, whic h determine when the conditional independence structure of the distribution is exactly the same a s tha t of a g raph in gra phical mo del sens e. Thus, we, in practice, w or k on the induced independence mo del o f an exchangeable probability dis tr ibution rather tha n the distribution itself. The sp e cialization to finitely exchangeable random vectors leads to necessar y and s ufficient conditions for a n exchangeable vector to b e co mpletely indep endent or co mpletely dep endent, meaning that tw o elements o f the vector are conditionally indep endent or dep endent, resp ectively , given any subset of the rema ining elements o f the vector. These conditions are namely intersection and comp os itio n prop erties [ 22 , 28 ]. W e also use the res ults to provide a s ufficient co nditio n for an exchangeable vector to be marginally independent. F or random net works, we follow a similar pro cedure. It was shown in [ 14 ] that if a dis tr ibution over an exchangeable ra ndom netw ork could b e faithful to a g raph then the s keleton of the gr aph is only one of the four p ossible types: the empt y graph, the complete gra ph, and the, so-called incidence gra ph, and its complement (s e e Prop ositio n 8 ). A question is then which gra phs that emerge from these s keleta a r e faithful to the exchangeable distribution. W e show that, other than complete indep endence or dep endence, there a re four other indep en- dence structur e s that can a rise for ex changeable netw orks. Thes e ar e faithful to a pair o f dual g raphs with incidence graph skeleton, and a pair of dual g raphs with the co mplemen t of the incidence graph skeleton. W e then pr ovide ass ump- tions under which int er section and comp osition prop erties a re necess ary and sufficient for the ex changeable random netw orks to b e faithful to eac h of these six cas es. The structure of the pap er is as fo llows: In the next section, we provide some definitions and known results needed for this pa p e r from g raph theory , ran- dom netw orks , and graphica l mo dels . In Sectio n 3 , w e pr ovide the results for exchangeable random vectors. In Section 4 , we provide the results for exchange- able random net works b y star ting with providing the afo r ementioned p ossible cases in Theorem 2 , a nd then in tro ducing them in se parate subs ections. W e end with a shor t discuss ion on these results in Section 5 . W e will provide techni- cal definitions and results needed for the pro of of Theo rem 2 , and its pr o of, in Appendix A . imsart-g eneric ver. 2014/10/16 file: exch-ind-arxiv- new.tex date: June 15, 2020 K. Sade ghi/On Finite Exchange ability and Conditional Indep endenc e 3 2. Definitions and prel iminary results 2.1. Gr aph-the or etic c onc epts A (lab ele d) gr aph is an order ed pair G = ( V , E ) consisting of a vertex set V , which is non-empty and finite, an e dge set E , and a re la tion that with each edge asso ciates tw o vertices, called its endp oints . W e omit the term lab eled in this pap er since the context do es not give r ise to ambiguit y . When vertices u and v are the endp oints o f a n edge, these are adjac ent and we wr ite u ∼ v ; we denote the co rresp onding edge as uv . In this pap er, we will restrict our attention to simple g raphs, i.e. graphs with- out lo ops (this a ssumption means that the endp oints of each edge are distinct) or multiple edges (each pa ir of vertices are the endp oints of at mos t o ne edge). F urther, three typ es of edg es, denoted b y arr ows , ar cs (solid lines with tw o- headed arrows) and lines (solid lines without arrowheads) hav e been used in the literature of gr a phical mo dels . Arrows can b e r epresented by order ed pairs of vertices, while arcs and lines by 2-s ubs ets of the vertex set. How ever, for our purp ose, except for Section 4.2 , we will distinguis h only lines and arcs, which resp ectively for m u ndir e cte d and bidir e cte d graphs. The graphs F = ( V F , E F ) and G = ( V G , E G ) ar e consider ed equal if a nd only if ( V F , E F ) = ( V G , E G ). A sub gr aph of a graph G = ( V G , E G ) is graph F = ( V F , E F ) suc h that V F ⊆ V G and E F ⊆ E G and the assig nment o f endpoints to edges in F is the same as in G . The line gr aph L ( G ) o f a graph G = ( V , E ) is the intersection gr aph of the edge s et E , i.e. its vertex set is E and e 1 ∼ e 2 if and only if e 1 and e 2 hav e a common endp oint [ 32 , p. 16 8]. W e will in particula r b e in tere sted in the line graph of a co mplete graph, whic h we will refer to as the incidenc e gr aph . Figure 1 displays the incidenc e graph for V = { 1 , 2 , 3 , 4 } . W e deno te by L − ( n ) 1 , 2 1 , 3 2 , 3 2 , 4 1 , 4 3 , 4 Fig 1 . The i ncidenc e gr aph for V = { 1 , 2 , 3 , 4 } . and L ↔ ( n ), the undirec ted incidence gr aph and the bidirec ted incidence gr aph for n no des, resp ectively; and by L c − ( n ) a nd L c ↔ ( n ), the undir ected co mplemen t of the incidence g r aph and the bidirected complement of the incidence graph for n no des, resp ectively , where the c omplement of gra ph G refers to a graph with the same node set as G , but with the edge set that is the complement set of the edge set of G . imsart-g eneric ver. 2014/10/16 file: exch-ind-arxiv- new.tex date: June 15, 2020 K. Sade ghi/On Finite Exchange ability and Conditional Indep endenc e 4 The skeleton o f a g raph is the undirected gr aph where all arrowheads ar e remov ed fro m the gra phs, i.e., all edges ar e repla ced by lines. W e deno te the skeleton of a g r aph G by sk( G ). A walk ω is a list ω = h i 0 , e 1 , i 1 , . . . , e n , i n i of vertices and edges such that fo r 1 ≤ m ≤ n , the edge e m has endp oints i m − 1 and i m . When indicating a sp ecific walk, we may skip the edges, and only write the nodes , when the walk we ar e considering is clear from the co nt ex t. A p ath is a w alk w ith no rep eated no des. 2.2. R andom networks Given a finite no de set N — representing individuals or actors in a given p op- ulation of interest — w e define a r andom network over N to be a collection X = ( X d , d ∈ D ( N )) o f bina r y random v a r iables taking v a lues 0 and 1 indexed by a set D ( N ), w hich is a collection of unorder ed pair s ij of no des in N . The binary random v aria bles X d are called dyads , a nd no des i and j are said to hav e a tie if the random v aria ble X ij takes the v a lue 1 , and no tie otherwise . Thus, a random netw o rk is a random v aria ble taking v alue in { 0 , 1 } ( N 2 ) and can, ther e- fore, b e see n as a random simple, undir e cte d g raph with no de set N , whereby the ties form the ra ndom edges of the gra phs . W e use the terms netw ork, no de, and tie ra ther than gra ph, vertex, and edg e to differentiate from the terminology used in the gra phical mo de l sense. Indeed, as we shall discuss gra phical mo dels for netw orks , we w ill also cons ider each dyad d a s a vertex in a graph G = ( D , E ) representing the dependence structure of the random v ariables asso cia ted with the dyads, with the edge set of such graph representing Ma rko v prop erties of the distribution of X . 2.3. Pr ob abilistic indep endenc e mo dels and thei r pr op erties An indep endenc e mo del J ov er a finite set V is a set of triples h X , Y | Z i (calle d indep endenc e statements ), wher e X , Y , and Z ar e disjoin t subsets o f V ; Z may b e empt y , but h ∅ , Y | Z i a nd h X , ∅ | Z i are always included in J . The independenc e statement h X , Y | Z i is rea d a s “ X is indep endent of Y given Z ”. Independenc e mo dels may in genera l hav e a proba bilistic interpretation, but not necessarily . Similar ly , no t all indepe ndence mo dels can b e ea sily r e presented b y graphs. F or further discuss ion on gener a l indep e ndence models , s ee [ 28 ]. In order to define probabilistic indep endence mo dels, consider a set V and a collection of random v a riables { X α } α ∈ V with state spaces X α , α ∈ V and join t distribution P . W e le t X A = { X v } v ∈ A etc. for each subset A of V . F or dis joint subsets A , B , and C of V we use a short nota tion A ⊥ ⊥ B | C to denote that X A is c onditional ly indep endent of X B given X C [ 4 , 13 ], i.e. that for any measur able Ω ⊆ X A and P - almost all x B and x C , P ( X A ∈ Ω | X B = x B , X C = x C ) = P ( X A ∈ Ω | X C = x C ) . W e can now induce an indep endence mo del J ( P ) by letting h A, B | C i ∈ J ( P ) if and only if A ⊥ ⊥ B | C w.r.t. P . imsart-g eneric ver. 2014/10/16 file: exch-ind-arxiv- new.tex date: June 15, 2020 K. Sade ghi/On Finite Exchange ability and Conditional Indep endenc e 5 Similarly we use the notation A 6 ⊥ ⊥ B | C for h A, B | C i / ∈ J ( P ). W e say that no n-empty A and B are c ompletely indep endent if, for every C ⊆ V \ ( A ∪ B ), A ⊥ ⊥ B | C . Similarly , we say that A and B ar e c ompletely dep endent if, for any C ⊆ V \ ( A ∪ B ), A 6 ⊥ ⊥ B | C . If A , B , or C has only one member { u } , { v } , or { w } , for b etter reada bility , we write u ⊥ ⊥ v | w . W e also wr ite A ⊥ ⊥ B when C = ∅ , whic h denotes the mar ginal indep endenc e of A and B . A probabilistic indep endence mo del J ( P ) ov er a set V is alwa ys a semi- gr aphoi d [ 22 ], i.e., it sa tisfies the four following prop erties for disjoint subsets A , B , C , and D of V : 1. A ⊥ ⊥ B | C if and only if B ⊥ ⊥ A | C ( symmet ry ); 2. if A ⊥ ⊥ B ∪ D | C then A ⊥ ⊥ B | C and A ⊥ ⊥ D | C ( de c omp osition ); 3. if A ⊥ ⊥ B ∪ D | C then A ⊥ ⊥ B | C ∪ D and A ⊥ ⊥ D | C ∪ B ( we ak union ); 4. if A ⊥ ⊥ B | C ∪ D and A ⊥ ⊥ D | C then A ⊥ ⊥ B ∪ D | C ( c ontr action ). Notice that the reverse implication o f contraction clea rly holds by decomp osition and weak union. A s e mi- graphoid for which the reverse implication of the weak union prop er ty holds is said to be a gr apho id ; that is, it also satisfies 5. if A ⊥ ⊥ B | C ∪ D and A ⊥ ⊥ D | C ∪ B then A ⊥ ⊥ B ∪ D | C ( interse ction ). F urthermore, a graphoid o r semi-g raphoid for which the r everse implication of the decomp osition prop er ty holds is said to be c omp ositional , that is, it als o satisfies 6. if A ⊥ ⊥ B | C and A ⊥ ⊥ D | C then A ⊥ ⊥ B ∪ D | C ( c omp osition ). If, for example, P has strictly p ositive density , the induced probabilistic in- depe ndence mo del is alwa ys a gr aphoid; see e.g . Pro p o sition 3.1 in [ 13 ]. See also [ 24 ] for a ne c e ssary a nd sufficient co ndition for P in order for the intersec- tion proper ty to hold. If the dis tribution P is a regular multiv aria te Gaussian distribution, J ( P ) is a comp ositio nal grapho id; e.g. se e [ 28 ]. Pro babilistic inde- pendenc e mo dels with positive densities are not in general compos itional; this only holds fo r specia l t yp es of m ultiv a riate distributions suc h as, for exa mple, Gaussian distributions and the symmetr ic binar y distributions used in [ 31 ]. Another impor tant pro p erty that is no t necessa rily satisfied by pr obabilistic independenc e mo de ls is singleton- t r ansitivity (als o ca lled we ak tr ansitivity in [ 22 ], where it is shown that for Ga us sian and binar y distributions P , J ( P ) alwa ys satisfies it). F or u , v , a nd w , s ingle elements in V , 7. if u ⊥ ⊥ v | C and u ⊥ ⊥ v | C ∪{ w } then u ⊥ ⊥ w | C or v ⊥ ⊥ w | C ( singleton-tr ansitivity ). In a ddition, we hav e the tw o following prop erties: 8. if u ⊥ ⊥ v | C then u ⊥ ⊥ v | C ∪ { w } for every w ∈ V \ { u , v } ( upwar d-stability ); 9. if u ⊥ ⊥ v | C then u ⊥ ⊥ v | C \ { w } for every w ∈ V \ { u, v } ( downwar d- stability ). Henceforth, instead of s aying that “ J ( P ) satisfies thes e prop erties” , we simply say that “ P s atisfies these pr op erties” . Fir st we provide the following well-kno wn imsart-g eneric ver. 2014/10/16 file: exch-ind-arxiv- new.tex date: June 15, 2020 K. Sade ghi/On Finite Exchange ability and Conditional Indep endenc e 6 result [ 26 ]: Lemma 1. F or a pr ob ability distribution P the fol lowing holds: 1. If P satisfies upwar d-stability t hen P satisfies c omp osition. 2. If P satisfies do wnwar d-stability then P satisfies interse ction. 2.4. Exchange abil ity for r andom ve ctors and networks A probability distribution P ov er a finite vector ( X 1 , X 2 , X 3 , . . . , X n ) of random v a riables with the s ame shared s ample space is (finitely) exchange able if for a ny per mutation π ∈ S ( n ) o f the indices 1 , 2 , 3 , . . . , n , the probability distribution o f the p ermuted vector ( X π (1) , X π (2) , X π (3) , . . . , X π ( n ) ) is the same as P ; s e e [ 1 ]. W e shall for brevit y say that the s e q uence X is exchangeable in the meaning that its distribution is. W e are also co ncerned with probability distributions on net works that are finitely exchangeable. A distribution P o f a random matrix X = ( X ij ) i,j ∈N ov er a finite no de set N with the sa me shared s ample space is said to be (finitely) we akly exchange able [ 27 , 1 0 ] if for a ll p ermutations π ∈ S ( N ) w e hav e that P { ( X ij = x ij ) i,j ∈N } = P { ( X ij = x π ( i ) π ( j ) ) i,j ∈N } . (2.1) If the matrix X is symmetric — i.e. X ij = X j i , we say it is symmetric we akly exchange able . Ag ain, we sha ll for brevity say that X is weakly or symmetric weakly exchangeable in the mea ning that its distribution is. A symmetric binary array with zero dia gonal can b e int er preted as a matrix of ties (the adjac en cy matrix ) of a random net work and, th us, the ab ov e con- cepts c an b e translated int o netw or k s. A r andom netw or k is exchange able if its adjacency matrix is sy mmetric weakly exchangeable. Then it is easy to obse rve that a rando m net work is ex changeable if and o nly if its distr ibution is inv ariant under rela b eling of the no des of the netw or k. 2.5. Undir e cte d and bidir e cte d gr aphic al mo dels Graphical mo dels [see, e.g. 13 ] are sta tistical mo dels expressing conditional inde- pendenc e statements a mong a collec tion of ra ndom v aria bles X V = ( X v , v ∈ V ) indexed by a finite set V . A gr aphical mo del is determined by a gra ph G = ( V , E ) ov er the indexing set V , and the edge set E (which may include edges of undi- rected, directed or bidirec ted type) enco des co nditional indep endence r elations among the v a riables, or Markov pr op erties . W e say that C sep ar ates A and B in a n undirected gr aph G , denoted by A ⊥ u B | C , if ev ery pa th b etw een A a nd B has a v ertex in C , that is there is no path betw een A and B outside C . F or a bidirected gra ph G , we s ay that C sep ar ates A and B , denoted b y A ⊥ b B | C , if every path b etw een A and B has a v ertex outside C ∪ A ∪ B , that is there is no path b etw een A and B within A ∪ B ∪ C . Note the obvious duality b etw een this and separa tio n for undirected imsart-g eneric ver. 2014/10/16 file: exch-ind-arxiv- new.tex date: June 15, 2020 K. Sade ghi/On Finite Exchange ability and Conditional Indep endenc e 7 graphs. W e mig ht skip the subscripts u and b in ⊥ u and ⊥ b when it is a pparent from the context with which separ a tion we ar e dealing . A joint probability distribution P for X V is Markovian with resp ect to an undirected graph [ 3 ] with the vertex se t V if A ⊥ u B | C implies A ⊥ ⊥ B | C . P is Markovian with resp ect to a bidirected graph [ 2 , 12 ] if A ⊥ b B | C implies A ⊥ ⊥ B | C . F or exa mple, in the undirected graph o f Figure 2 (a), the globa l Mar ko v prop- erty implies that { u , x } ⊥ ⊥ w | v , whereas in the bidirected g raph of Figure 2 (b), the g lo bal Markov prop erty implies that { u, x } ⊥ ⊥ w . u v x w u v x w (a) (b) Fig 2 . (a) An undir e cted dep endenc e gr aph. (b ) a bidir e cted dep endence gr aph. If, for P and undirected G , A ⊥ u B | C ⇐ ⇒ A ⊥ ⊥ B | C then we say that P a nd G are fai t hful ; Similarly , if, for P and bidir ected G , A ⊥ b B | C ⇐ ⇒ A ⊥ ⊥ B | C then P and G a re faithful . Hence, faithfuln es s implies b eing Marko- vian, but not the other wa y aro und. F or a given probability distr ibution P , w e define the skeleton o f P , denoted b y sk( P ), to b e the undirected g raph with the vertex set V such tha t vertices u a nd v a re no t adjacent if and only if there is some subset C of V so that u ⊥ ⊥ v | C . Thu s, if P is Marko vian with resp ect to an undirected gra ph G then sk( P ) would be a subg r aph of G (since for every mis sing edge i j in G , i ⊥ ⊥ j | V \ { i , j } ); and if P is Markovian with resp ect to a bidirected gra ph G then sk( P ) is a subgr aph of sk( G ) (since for every missing edge i j in G , i ⊥ ⊥ j ). In gener al, a graph G ( P ) is induced by P with s keleton sk( P ). F or undirec ted graphs, let G u ( P ) = s k( P ), whereas for bidirected gra phs, let G b ( P ) be sk ( P ) with all e dges being bidir ected. W e sha ll need the following res ults fro m [ 26 ] (where the first pa rt w as first shown in [ 2 3 ]): Prop ositi on 1. L et P b e a pr ob ability distribution define d over { X α } α ∈ V . It then holds that 1. P and G u ( P ) ar e faithful if and only if P satisfies interse ction, s ingleton- tr ansitivity, and upwar d-stability. 2. P and G b ( P ) ar e faithful if and only if P satisfi es c omp osition, singleton- tr ansitivity, and do wnwar d-stability. 2.6. Duality i n i ndep endenc e mo dels and gr aphs The dual o f an indep endence mo del J (defined in [ 20 ] under the name o f dual r elation ) is the indep endence mo del defined by J d = { h A, B | V \ ( A ∪ B ∪ C ) i : imsart-g eneric ver. 2014/10/16 file: exch-ind-arxiv- new.tex date: June 15, 2020 K. Sade ghi/On Finite Exchange ability and Conditional Indep endenc e 8 h A, B | C i ∈ J } ; see als o [ 9 ]. W e will need the following lemma r egarding the duality of indep endence mo dels: Lemma 2. F or an indep endenc e mo del J and its dual J d , 1. J is semi-gr aphoid if and only if J d is semi-gr aphoid; 2. J satisfies interse ction if and only if J d satisfies c omp osition; and vic e versa; 3. J satisfies singleton-tra nsitivity if and only if J d satisfies singleton- tr ansitivity; 4. J satisfies upwar d-stability if and only if J d satisfies downwar d-st ability; and vic e versa. Pr o of. 1 ., 2., and 3 . are proven in [ 18 ], whic h showed that if J is singleton- transitive c o mp ositional graphoid, so is the dual co uple o f J (although this is prov en base d on the so-ca lled Gaussoid form ulation). An alter na tive pro of for these, w ith the s ame formulation a s in this pap er, is found in [ 19 ]. (In fact the statement in the mentioned article was prov en for a ge ne r alization of singleton- transitivity , ca lle d dual de c omp osable tra nsitivity ). In order to prov e 4., suppose that J satisfies up ward-stability , and assume that h i, j | C i ∈ J d and k ∈ C . Therefor e, h i, j | V \ ( { i, j } ∪ C ) i ∈ J . Hence, h i, j | V \ ( { i, j } ∪ C ) ∪ { k }i ∈ J becaus e of upw ard-stability . Therefore h i, j | C \ { k }i ∈ J d , whic h implies down ward-stabilit y of J d . The other direction is similar. As an additional statement to the lemma, it can also be shown that J is closed under mar ginalization if and only if J d is closed under c onditioning ; and vice versa; but this re s ult is not needed in this pap er . In addition, for separation in gra phs, we will use the following lemma related to duality: Lemma 3. L et G u and G b b e an undir e cte d and a bidi r e cte d gr aph su ch that sk( G u ) = sk( G b ) . Then, the indep en denc e mo del J ( G b ) induc e d by G b is J d ( G u ) and vic e versa. Pr o of. Since the separation s atisfies the comp osition prop erty , it is sufficient to prov e the statement for singletons. Suppose that there is a connecting path ω betw een i and j g iven C in G u . This means that no inner vertex of ω is in C ; th us they ar e all in V \ ( { i, j } ∪ C ). Ther efore, in G b , i and j a re connecting given V \ ( { i, j } ∪ C ). The o ther directio n (where i 6⊥ j | C in G b ⇒ i 6 ⊥ j | V \ ( A ∪ B ∪ C ) in G u ) is proven in a similar wa y . In fact, the second part of Pr op osition 1 , co uld b e implied b y the first part, and vice versa, using Lemmas 2 and 3 ; and the second par t of Lemma 1 , could be implied by the first part, a nd vice versa, using Lemma 2 . 3. Re sults for vector exc hangeabi lity W e shall study the rela tionship b etw een vector exchangeability and conditiona l independenc e b y us ing the definitions and results in the previous section. In the imsart-g eneric ver. 2014/10/16 file: exch-ind-arxiv- new.tex date: June 15, 2020 K. Sade ghi/On Finite Exchange ability and Conditional Indep endenc e 9 ent ir e sectio n, w e a ssume that P is a proba bility distribution defined over the vector ( X v ) v ∈ V . First, notice that exchangeabilit y is c lo sed under mar g inaliza- tion and conditioning: Prop ositi on 2. If X V is an ex change able r andom ve ctor t hen so ar e the mar ginal ve ct ors X A , for A ⊆ V , and the c onditional ve ctors X A | X C = x ∗ C , for disjoi nt A, C wher e V = A ∪ C , if they exist. Pr o of. Fir st we prov e clo sedness under mar ginalization: F or a p ermutation ma- trix π of A , let the p ermutation matrix over V b e π ∗ ( a ) = π ( a ), fo r a ∈ A , and π ∗ ( b ) = b fo r b ∈ V \ A . The pro o f then follows from ex changeabilit y of X V . Now we prove closedness under co nditioning: By the definition of co nditio n- ing, we need to pr ov e that P ( x A , x ∗ C ) = P ( x π ( A ) , x ∗ C ) for e very π . This is again true by consider ing π ∗ ( a ) = π ( a ), for a ∈ A , a nd π ∗ ( c ) = c for c ∈ C . Notice tha t the ab ov e r esult implies that the mar g inal/conditio na l X A | X C , (i.e., when A ∪ C ⊂ V ) is also exchangeable. W e now hav e the following results: Prop ositi on 3. If P satisfies ve ctor exchange ability then the fol lowing holds: 1. P satisfies upwar d-stability if and only if it satisfies c omp osition. 2. P satisfies downwar d-stability if and only if it s atisfies interse ction. Pr o of. 1 . ( ⇒ ) follows fr o m L e mma 1 . T o prov e ( ⇐ ), let i ⊥ ⊥ j | C . By exchange- ability , for an arbitrary k / ∈ C ∪ { i , j } , w e hav e i ⊥ ⊥ k | C . Comp o s ition implies i ⊥ ⊥ j ∪ k | C . W eak union implies i ⊥ ⊥ j | C ∪ { k } . 2. follows from 1 . and the consequence of duality provided in Lemma 2 . Prop ositi on 4. If P is exchange able then it satisfies singleton-tr ansitivity. Pr o of. If i ⊥ ⊥ j | C and k / ∈ C ∪ { i , j } then clearly i ⊥ ⊥ k | C a nd k ⊥ ⊥ j | C . W e see that the skeleton of an exchangeable dis tr ibution can only take a very sp ecific form: Prop ositi on 5. If P is exchange able then sk( P ) is either an empty or a c om- plete gr aph. Pr o of. If ther e is any indep endence statemen t of form i ⊥ ⊥ j | C then by p ermu- tation for a ll v aria ble s , we o btain k ⊥ ⊥ l | C ′ for all k and l . Therefore, sk( P ) is empt y . If ther e is no indep endence statemen t of this fo r m then sk( P ) is c o m- plete. Notice that for faithfulness to empty or co mplete gr aphs, it is immaterial whether one considers undirected or bidirected interpretation o f gr aphs. Corollary 1. Le t P satisfy ve ctor exchange ability. If P is faithful to a gr aph (b oth under un dir e cte d interpr etation and under bidir e cte d int erpr etation) then the gr aph is empty or c omplete. Hence, there ar e tw o reg imes av ailable: if there is no indep endence statement implied by P then we a re in the complete g r aph regime; and if there is at least one conditional indep endence statement implied by P then we are in the empty imsart-g eneric ver. 2014/10/16 file: exch-ind-arxiv- new.tex date: June 15, 2020 K. Sade ghi/On Finite Exchange ability and Conditional Indep endenc e 10 graph regime. The following a lso pr ovides conditions for the o ppo site direction of the ab ov e r esult: Theorem 1. If P is ex change able then P is faithful to a gr aph (b oth under undir e cte d interpr etation and under bidir e cte d interpr etation) if and only if P satisfies the int erse ction and c omp osition pr op erties. The gr aph must then b e either empty or c omplete. Pr o of. The fir st result follows from Pro po sitions 1 , 3 , and 4 . The s e cond follows from Pr op osition 5 . Indeed other exchangeable distr ibutio ns may exist but they are not faithful to a graph. W e then have the following c orollar ies: Corollary 2. L et P b e exchange able and t her e exists an indep endenc e st atement induc e d by P . It then ho lds that al l varia bles X v ar e c ompletely inde p endent of e ach other if and only if P satisfies interse ction and c omp osition. Corollary 3. L et P b e a re gular ex change able Gaussian distribution. If t her e is a zer o element in its c ovarianc e matrix then al l X v ar e c ompletely indep endent; and otherwise t hey ar e c ompletely dep endent. Pr o of. The pro of follows from the fact that a regular Gauss ian distribution satisfies the intersection and compo sition pr o p erties. Notice that the ab ov e sta tement could b e shown otherwis e since a zero off- diagonal entry of the cov ar iance matrix can b e p ermuted by exchangeability to every other off-diago nal entry , making the cov a riance matrix diagonal. Prop ositi on 6. If P is exchange able, satisfies interse ction (this ho lds when P has a p ositive density), and if ther e exists one indep endenc e s t atement induc e d by P then al l variables X v ar e mar ginal ly indep endent of e ach other. Pr o of. B y P rop osition 3 , P satisfies downw ard-sta bilit y . An indep endence sta te- men t A ⊥ ⊥ B | C , b y the use of decomp osition implies i ⊥ ⊥ j | C for an arbitrary i ∈ A and j ∈ B . Down ward-stability implies i ⊥ ⊥ j . Exchangeability implies all pairwise mar g inal indep endences. Example 1. Consider an exchange able distribution P over four variables ( i, j, k , l ) with i ⊥ ⊥ j | k . By ex change ability, al l indep endenc es of form π ( i ) ⊥ ⊥ π ( j ) | π ( k ) hold for any p ermutation π on ( i, j, k , l ) . It is e asy to se e that none of the semi-gr aphoi d axioms c an gener ate new indep endenc e statements fr om these. If interse ction holds then, for example, fr om i ⊥ ⊥ j | k and i ⊥ ⊥ k | j , we obtain i ⊥ ⊥ { j, k } , which by de c omp osition implies i ⊥ ⊥ j , and henc e al l mar ginal indep en- denc es b etwe en singletons. If c omp osition hol ds then, for ex ample, fr om i ⊥ ⊥ j | k and i ⊥ ⊥ l | k , we obtain i ⊥ ⊥ { j, l } | k , which by we ak union implies i ⊥ ⊥ j | { k , l } , and henc e al l c onditional indep endenc es b etwe en singletons given the r emaining variables. imsart-g eneric ver. 2014/10/16 file: exch-ind-arxiv- new.tex date: June 15, 2020 K. Sade ghi/On Finite Exchange ability and Conditional Indep endenc e 11 4. Re sults for e xc hangeability for random netw orks Henceforth, in the context o f random netw orks, we consider vectors whos e com- po nents a r e indexed by dyads (i.e. tw o - element subsets o f the no de set N ). Thus, for a conditional indep endence sta tement of the for m A ⊥ ⊥ B | C , A, B , C ⊂ D ( N ) are pa irwise disjoint s ubsets of dyads. (Later o n, we par ticularly write the condi- tioning sets as simply C , which should b e considered a s ubset of dyads.) Notice that we simply use the no tation i j for a dyad whose endpo ints ar e no des i and j . This is different from the notatio n { i, j } , which indicates the set of tw o no des i and j as use d in the previo us section. 4.1. Mar ginal ization and c onditi oning for exchange abl e r andom networks Here we focus o n marginaliza tion ov er a nd conditioning on arbitrary sets of dyads. Notice that by mar ginalizing over a set M , we mean we mar ginalize the set M out, which results in a dis tr ibution ov er D ( N ) \ M . How ever, exchangeable net works are not alwa ys closed under marg ina lization ov er or conditioning on an ar bitrary set of dyads: F or a marginal net work X A , where A is a subset of dyads, exc hangea bility and summing up all proba bilities over v a lues of the dyads that are ma rginalized ov er imply that P ( X A = x A ) = P (( X π ( i ) π ( j ) ) ij ∈ A = x A ), fo r a ny p ermutation π . How ever, this is not neces sarily equal to P ( X A = ( x π ( i ) π ( j ) ) ij ∈ A ), which is what we need for e xchangeabilit y of the marginal to hold. F or conditioning, in fact, X A | X C is no t exchangeable if a no de app ears in a dyad in A and a dyad in C (e.g. i app ear ing in ij ∈ A and ik ∈ C ). This is bec a use, for a per mutation π that maps i to a node o ther than i , exchangeability of X A | X C is equiv alent to P ( x A | x C ) = P (( x π ( i ) π ( j ) ) ij ∈ A | x C ), which itself is equiv alent to P ( x A , x C ) = P (( x π ( i ) π ( j ) ) ij ∈ A , x C ). B ut, this do es not necess arily hold as i is mapp ed to another no de in A but not in C . How ever, we hav e the following: Prop ositi on 7. L et A and C b e disjoint subsets of dyads of an exchange able r andom net work X such that A and C do not s har e any no des, i.e. if i j ∈ A then ther e is no dyad ik or j k in C for any no de k ∈ N . It then ho lds that t he c onditional/ mar ginal r andom network X A | X C is exchange able. Pr o of. Le t N ( A ∪ C ) b e the set o f all endp o int s of dyads in A and C , and define similarly N ( A ) and N ( C ). Define the p ermutation π ∗ ∈ S ( N ( A ∪ C )) such that π ∗ ( i ) = π ( i ) for i ∈ N ( A ) and π ∗ ( k ) = k for k ∈ N ( C ). No tice that this is well- defined since A and C do not share any no des. Using π ∗ and by exchangeability of X , we co nclude that P ( x A , x C ) = P (( x π ( i ) π ( j ) ) ij ∈ A , x C ), which, as ment ione d befo re, is equiv alent to the e x changeabilit y of X A | X C . imsart-g eneric ver. 2014/10/16 file: exch-ind-arxiv- new.tex date: June 15, 2020 K. Sade ghi/On Finite Exchange ability and Conditional Indep endenc e 12 4.2. T yp es of gr aphs faithfu l to exchange able distribut ions An analogous r esult to that of vector exchangeability , concerning the sk eleton of an exchangeable pro bability distr ibution, was prov en in [ 14 ]: Prop ositi on 8. If a distribution P over a r andom network X is exchange able then sk( P ) is one of t he following: 1. the empty gr aph; 2. the incidenc e gr aph; 3. the c omplement of the incidenc e gr aph; 4. the c omplete gr aph. Notice that a pairwise independence statemen t for an exchangeable P ov er a random net work X is of form ij ⊥ ⊥ k l | C , where i, j, k , l are nodes of X . De- pending on the type of sk( P ), these statements take different fo rms. Lemma 4. Supp ose that ther e exists an indep endenc e st atemen t of form ij ⊥ ⊥ k l | C , i 6 = j, k 6 = l , for an exchange able P over a r andom network X . It is then in one of the fol lowing forms dep ending on the typ e of sk( P ) : 1. empty gr aph ⇒ no c onstr aints on i, j, k , l ; 2. incidenc e gr aph ⇒ i 6 = l , k and j 6 = l , k ; 3. c omplement of the incidenc e gr aph ⇒ i = k or i = l or j = k or j = l ; if sk( P ) is t he c omplete gr aph then it is not p ossible to have such a c onditional indep endenc e statement. Pr o of. The proof follows fro m the fa ct if there is an edge b etw een ij a nd k l in G ( P ) then there is no s tatement o f fo r m ij ⊥ ⊥ k l | C . It is clear that if sk( P ) is the co mplete g raph then every dyad is completely depe ndent on every other dy a d. Thus we consider the cases of the incidence graph skeleton and the complement of the incidence gr aph skeleton separ ately . How ever, before this, w e show that among the k nown graphical mo dels , o nly undirected and bidir ected graphs can be faithful to an exchangeable distribution. In o rder to do so, we can start off by consider ing any class o f mixe d gr aphs , i.e. graphs with sim ultaneo us undirected, directed, or bidirected edges that use the (unifying) separ a tion cr iterion intro duce d in [ 16 ]. T o the b est o f o ur knowledge, the larges t class of such graphs is the class of chain mixe d gr aphs [ 16 ], which includes the classes of anc estr al gr aphs [ 25 ], L WF [ 1 7 ] and r e gr ession chain gr aphs [ 30 ], a nd several others; see [ 1 6 ]. W e require some definitions and results, including the formulation of the separation criterion, whic h we only need for the results in this subsection. W e provide these together with the proof o f the main result (Theorem 2 ) in Appendix A . If the reader is only interested in the statement of the theorem and not the pro of, we sugg e st tha t they skip the material in the a pp e ndix . Two g raphs are called Markov e quivalent if they induce the same indep en- dence mo del. imsart-g eneric ver. 2014/10/16 file: exch-ind-arxiv- new.tex date: June 15, 2020 K. Sade ghi/On Finite Exchange ability and Conditional Indep endenc e 13 Theorem 2. If a distribution P over an exchange able r andom network with n no des is faithful to a chain mixe d gr aph G then G is Markov e quivalent to one of the fol lowing gr aphs: 1. the empty gr aph; 2. the undir e cte d incidenc e gr aph, L − ( n ) ; 3. the bidir e cte d incidenc e gr aph, L ↔ ( n ) ; 4. the undir e cte d c omplement of the incidenc e gr aph, L c − ( n ) ; 5. the bidir e cte d c omplement of the incidenc e gr aph, L c ↔ ( n ) ; 6. the c omplete gr aph. Hence, for exchangeable r andom netw orks, there are six regimes av ailable, where these are thr ee pair s that a re complement of ea ch other . Within the tw o non-trivial pairs, the t wo undirec ted and bidirected cases act as dual of each other as descr ib ed in Section 2.6 . W e will ma ke use of this duality to simplify the r esults and pro ofs. Although the following metho d is not unique, here we provide a simple test to decide in whic h regime a given exchangeable dis tribution lies: Algorithm 1. F or arbitr ary fi xe d n o des i, j, k , l , m of a given exchange able net- work, test the fol lowing: • ij ⊥ ⊥ k l | C , fo r some C , and ij ⊥ ⊥ ik | C ′ , for some C ′ ⇒ Empty gra ph; • ij ⊥ ⊥ k l | C , fo r some C , and ij 6 ⊥ ⊥ ik | C ′ , for any C ′ ⇒ L ( n ) : – ik ∈ C ⇒ L − ( n ) ; – ik / ∈ C ⇒ L ↔ ( n ) ; • ij 6 ⊥ ⊥ k l | C , fo r any C , and ij ⊥ ⊥ ik | C ′ , for some C ′ ⇒ L c ( n ) : – lm ∈ C ′ ⇒ L c − ( n ) ; – lm / ∈ C ′ ⇒ L c ↔ ( n ) ; • ij 6 ⊥ ⊥ k l | C , fo r any C , and ij 6 ⊥ ⊥ ik | C ′ , for any C ′ ⇒ Complete gr aph. Prop ositi on 9. If a distribution P over an exchange able r andom network is faithful to a gr aph G then Algo rithm 1 determines the Markov e quivalenc e class of G . Pr o of. Fir st, we show that this test covers all the p ossible ca ses: The first level tests ( ij ⊥ ⊥ k l | C and ij ⊥ ⊥ ik | C ′ ) clearly cover a ll the ca ses concerning the skele- ton of the g raph. But, the seco nd level test ( ik ∈ C or lm ∈ C ′ ), which only concerns the non- trivial skeletons, migh t not b e consisten t: F or e x ample, first consider the L ( n ) case, and assume that ij ⊥ ⊥ k l | C 1 and ij ⊥ ⊥ k l | C 2 , for some C 1 , C 2 , but ik ∈ C 1 and ik / ∈ C 2 . How ever, in such ca ses, it is ea sy to show that P cannot b e faithful to e ither of the t wo undire c ted or bidirected graphs . The case of L c ( n ) is similar . The algorithm also outputs the corre c t reg imes: The tests of ij 6 ⊥ ⊥ k l | C a nd ij ⊥ ⊥ ik | C ′ clearly determine the skeleton sk( P ). The test ik ∈ C then deter- mines the type of edges since, in h i j, ik , k l i , if ik / ∈ C then ij and k l cannot imsart-g eneric ver. 2014/10/16 file: exch-ind-arxiv- new.tex date: June 15, 2020 K. Sade ghi/On Finite Exchange ability and Conditional Indep endenc e 14 be separa ted in the undir ected graph; and if ik ∈ C then ij and k l cannot be separated in the bidirected g raph. The tes t l m ∈ C ′ can b e prov en similarly . Unlik e the vector exchangeability ca se, the intersection and comp o sition pr op- erties are not in gener al sufficient for faithfulness of exchangeable net work dis- tributions to the gr aphs provided in Theorem 2 . How ever, in s p ec ia l cases this holds. W e detail this b elow for each reg ime ment io ned ab ove. 4.3. The i ncidenc e gr aph c ase In this section, w e assume that sk( P ) = L − ( n ). The following exa mple sho ws that, in principle, intersection and co mp osition are not sufficient for faithfulness of P and L − ( n ) (and P and the bidirec ted incidence gra ph L ↔ ( n )). Example 2. Supp ose that ther e is an ex change able P that induc es ij ⊥ ⊥ k l | C ij,kl , wher e C ij,kl = { i k , il , j k , j l } . Exchange ability implie s that ij ⊥ ⊥ k l | C ij,kl for al l i, j, k , l . Supp ose, in addition, t hat P satisfies upwar d-st ability. Notic e that her e we do not show that such a pr ob ability distribution ne c essarily exists – one c an tr e at this define d indep endenc e m o del as an ex ample of “a network-exchange able semi-gr aphoi d”. It is e asy to se e that sk( P ) = L − ( n ) . Mor e over, by Le mma 1 , P satisfies c omp osition. In addition, P satisfies interse ction: Notic e t hat none of the semi- gr aphoi d axioms plus upwar d-stability and c omp osition imply an indep en denc e statement of form A ⊥ ⊥ B | C wher e A, B b oth c ontain a no de i . In add ition, it c an b e se en that these axioms imply t hat if t her e is A ⊥ ⊥ B | C then for every ij ∈ A and k l ∈ B , it holds that C ij,kl ⊆ C . L et C A,B = S ij ∈ A,kl ∈ B C ij,kl . By these two observations, we c onclude that if A ⊥ ⊥ B | C ∪ D and A ⊥ ⊥ D | C ∪ B then C A,D ⊆ C . By upwar d-stability al l statement s of form i j ⊥ ⊥ k l | C wher e C ij,kl ⊆ C hold. Henc e, by c omp osition, we have that A ⊥ ⊥ B | C . Henc e, by c ontr action, A ⊥ ⊥ B ∪ D | C . However, P do es not satisfy singleton-tra nsitivity: Consider ij ⊥ ⊥ k l | C ij,kl and ij ⊥ ⊥ k l | C ij,kl ∪ { k m } . If, for c ontr adiction, singleton-tr ansitivity holds t hen, b e c ause sk( P ) = L − ( n ) , we have that ij ⊥ ⊥ k m | C ij,kl . But, it c an b e se en that this st atement is not in the indep endenc e mo del sinc e no c omp ositional gr aphoid axioms c an gener ate this statement fr om { i j ⊥ ⊥ k l | C ij,kl } . By Pr op osition 1 (1.), it is implie d that, although interse ction and c omp osi- tion ar e satisfie d, P is not faithful to L − ( n ) . If we now supp ose that P induc es ij ⊥ ⊥ k l | C d ij,kl , wher e C d ij,kl = V \ ( C ij,kl ∪ { ij, k l } ) , P is exchange able, and it satisfies downwar d-stability then, by u sing the duality (L emmas 2 and 3 ), we c onclude that although interse ction and c om- p osition ar e satisfie d, P is not faithful to L ↔ ( n ) . How ever, under cer tain assumptions, intersection and comp os ition are suffi- cient for faithfulness. W e will c onsider the tw o dual r egimes within the incidence graph case re lated to the case s of Theorem 2 : the undirected incidence graph imsart-g eneric ver. 2014/10/16 file: exch-ind-arxiv- new.tex date: June 15, 2020 K. Sade ghi/On Finite Exchange ability and Conditional Indep endenc e 15 case and the bidirected incidenc e graph case. W e make use of the duality be- t ween these to immediately extend the results in the undirec ted case to the bidirected cas e. As in E xample 2 , le t C ij,kl = { ik , il , j k , j l } . F or the faithfulness r esults to the undirected case, one assumption that is us ed is tha t for some (and beca use of ex changeability for all) i, j, k , l , ij ⊥ ⊥ k l | C implies that C ij,kl ⊆ C . F or the faithfulness results to the bidirected c ase, o ne as sumption is that for so me (and bec ause of exchangeabilit y for all) i , j, k , l , i j ⊥ ⊥ k l | C implies that C ij,kl ∩ C = ∅ . F or a separ a tion s tatement A ⊥ B | C , we define C to b e a minimal s e pa rator in the case that if we r emov e any vertex from C , the separatio n do e s not hold; we define a m ax imal separator similarly . Let also C ij = { i r , j r : ∀ r 6 = i , j } and C d ij = V \ ( C ij ∪ { ij } ) = { l m : ∀ l, m / ∈ { i, j }} . It holds tha t C ij is a minimal separato r of ij, k l in L − ( n ) and C d ij \ { k l } is a maximal s e pa rator in L ↔ ( n ): Prop ositi on 10 . 1. In L − ( n ) , it holds that ij ⊥ k l | C ij , and if ij ⊥ k l | C then | C ij | ≤ | C | . In fact, if C 6 = C ij and C 6 = C kl then | C ij | < | C | . 2. In L ↔ ( n ) , it holds t hat ij ⊥ k l | C d ij \ { k l } , and if ij ⊥ k l | C t hen | C d ij | ≥ | C | + 1 . In fact, if C 6 = C d ij \ { k l } and C 6 = C d kl \ { ij } t hen | C d ij | > | C | + 1 . Pr o of. 1 . The first cla im is straightforward to prove since every path from k l to ij must pass throug h an adjacent vertex of ij , which contains i or j . T o pr ove the seco nd statemen t, no tice that if i m / ∈ C then at least k m a nd l m must b e in C . The same vertices k m and lm appear if j m / ∈ C to o, but for no other vertices of C ij missing in C . Thus, for every miss ing mem b er o f C ij in C , there is at least a member of C kl that sho uld b e in C . Hence, | C ij | ≤ | C | . T o prov e the third statement, s upp o se, for contradiction, that C 6 = C ij and C 6 = C kl and | C | = | C ij | . Using the fact that, for every missing member of C ij in C , there is a t least a member of C kl that should b e in C , there can- not be any vertex outside C ij ∪ C kl in a C . Hence, C ⊂ C ij ∪ C kl . If n > 5 and, sa y , im, j m / ∈ C but k m, l m ∈ C then consider the vertices i h, j h, k h, l h . Without loss of generality , assume that ih , j h ∈ C but kh, l h / ∈ C . Then the path h k l , k h , mh, j m, ij i connects k l and ij , a c ontradiction. The cases where n = 3 , 4 , 5 ar e easy to chec k. 2. The pr o of follows from 1. by using the dualit y (Lemma 3 ), a nd o bserving that | C d ij \ { k l }| = | C d ij | − 1. How ever, not all minimal separa tors of ij, k l in L − ( n ) a re o f the form a b ov e. F or example, in L − (6), consider the set C = { i k , il , im , j k , j l , j m, hk , hl , mh } . It ho lds that ij ⊥ k l | C . Ho wev er , C is not of the for m C pq for a ny pair p, q ∈ { i, j, k , l , m, h } . The same can b e said ab out L − ( n ): Conside r C d = V \ ( C ∪ { ij, k l } ). By Lemma 3 , we have that ij ⊥ k l | C d in L ↔ (6), a nd, in addition, C d is maximal. Prop ositi on 11. L et a distribution P b e define d over an exchange able r andom network and s k( P ) = L − ( n ) , and c onsider some n o des i, j, k , l . 1. Supp ose t hat for every minimal C su ch that i j ⊥ ⊥ k l | C , it holds that C ij,kl ⊆ C and C is invariant under swapping k and m , and l and h , for every imsart-g eneric ver. 2014/10/16 file: exch-ind-arxiv- new.tex date: June 15, 2020 K. Sade ghi/On Finite Exchange ability and Conditional Indep endenc e 16 m, h , m 6 = h , mh / ∈ C ∪ { ij, k l } . It then holds t hat if P satisfies c omp osition then it satisfies upwar d-stability and singleton-t r ansitivity. 2. Supp ose that for every maximal C such that ij ⊥ ⊥ k l | C , it holds t hat C ij,kl ∩ C = ∅ and C is invariant under swapping k and m , and l and h , for every m, h , m 6 = h , mh ∈ C . It then holds that if P satisfies int erse ction then it satisfies downwar d-stability and singleton-tr ansitivity. Pr o of. B y Lemma 4 , we know tha t i, j, k , l ar e all different. 1. First, we pr ov e up ward-stability . Suppose that i j ⊥ ⊥ k l | C . Notice that, bec ause of exc hang eability , the a s sumptions of the statement hold for every i, j, k , l , m, h . F o r a v aria ble mh / ∈ C ∪ { ij, k l } , we prove that ij ⊥ ⊥ k l | C ∪ { mh } by induction on | C | . Notice that, by ass umption, mh / ∈ C ij,kl . In addition, without los s o f g e ne r ality , we can a ssume that m, h 6 = i, j since otherwise we can swap i and k a nd j and l and pro c eed as follows, and finally sw ap them back. The base case is when C is minimal. Because of exchangeability , by the inv a riance of C under the aforementioned swaps, and since mh / ∈ C ij,kl , w e hav e that ij ⊥ ⊥ mh | C . By comp ositio n ij ⊥ ⊥ { k l , mh } | C , which, by weak union, implies i j ⊥ ⊥ k l | C ∪ { mh } . Now s uppo se that ij ⊥ ⊥ k l | C , C is not minimal, and, for every C ′ such tha t | C ′ | < | C | , [ i j ⊥ ⊥ k l | C ′ ⇒ ij ⊥ ⊥ k l | C ′ ∪ { op } ], for every op . Conside r the inde- pendenc e statemen t ij ⊥ ⊥ k l | C 0 such that C 0 ⊂ C and C 0 is minimal. W e hav e that ij ⊥ ⊥ mh | C 0 . By induction hypo thesis, w e can add v ertices to the condi- tioning set in or der to obtain ij ⊥ ⊥ mh | C . Now, again comp o sition and weak union imply the r esult. Singleton-transitiv ity a lso follows from the ab ov e argument. W e need to s how that ij ⊥ ⊥ k l | C and i j ⊥ ⊥ k l | C ∪ { mh } imply ij ⊥ ⊥ mh | C o r mh ⊥ ⊥ k l | C . (Notice that b ecause of upw ard-s tability ij ⊥ ⊥ k l | C ∪ { mh } is immaterial.) Aga in without loss of ge nerality , we can a ssume that m, h 6 = i, j , and the ab ov e argument show ed that ij ⊥ ⊥ mh | C . 2. The pro of follows fr o m part 1. and the duality (Lemma 2 ). Theorem 3. L et a distribution P b e define d over an exchange able r andom net- work and sk( P ) = L − ( n ) , and c onsider some no des i, j, k , l . 1. Supp ose t hat for every minimal C su ch that i j ⊥ ⊥ k l | C , it holds that C ij,kl ⊆ C and C is invariant under swapping k and m , and l and h , for every m, h , m 6 = h , mh / ∈ C ∪ { ij, k l } . Then P is faithful to L − ( n ) if and only if P satisfies t he interse ction and c omp osition pr op erties. 2. Supp ose that for every maximal C such that ij ⊥ ⊥ k l | C , it holds t hat C ij,kl ∩ C = ∅ and C is invariant u nder swapping k and m , and l and h , for every m , h , m 6 = h , mh ∈ C . Then P is faithful to L ↔ ( n ) if and only if P satisfies the interse ction and c omp osition pr op erties. Pr o of. The pro of follows from Prop os itions 11 and 1 . imsart-g eneric ver. 2014/10/16 file: exch-ind-arxiv- new.tex date: June 15, 2020 K. Sade ghi/On Finite Exchange ability and Conditional Indep endenc e 17 4.4. The c omplement of the incidenc e gr aph c ase In this section, we assume that sk( P ) = L c − ( n ). W e show again that, under certain assumptions, intersection a nd comp osition ar e sufficient for faithfulness. W e again utilize the duality of the tw o additiona l reg imes in the complement of the incidence graph case related to the ca ses of Theorem 2 : the undirected co m- plement of the incidence graph c a se and the bidirected complement of incidence graph case. Let C d ij k = { l m : ∀ l , m / ∈ { i, j, k }} . F or the faithfulness results to the, r esp ectively , undirected a nd bidirected case, a n assumption that is used is that for some (and b eca use o f exchangeability for all) i, j, k , ij ⊥ ⊥ ik | C implies that C d ij k ⊆ C for undirected case a nd C d ij k ∩ C = ∅ for the bidirected c a se. Let C j = { j r : ∀ r 6 = j } . Recall also that C ij = { ir , j r : ∀ r 6 = i, j } and C d ij = { l m : ∀ l , m / ∈ { i, j }} . C d ij is a minimal separato r of ij, ik in L c − ( n ), a nd C ij \ { ik } is a maximal sepa rator in L c ↔ ( n ): Prop ositi on 12 . L et n > 4 . 1. In L c − ( n ) , it holds that i j ⊥ ik | C d ij , and if ij ⊥ i k | C then | C d ij | ≤ | C | . In fact, if C 6 = C d ij and C 6 = C d ik then | C d ij | < | C | . 2. In L c ↔ ( n ) , n > 4 , it holds that ij ⊥ i k | C ij \ { ik } , and if ij ⊥ ik | C then | C ij | ≥ | C | + 1 . In fact, if C 6 = C ij \ { i k } and C 6 = C d ik \ { ij } then | C d ij | > | C | + 1 . Pr o of. 1 . The firs t claim is straightforw ard to prove since every path from ik to ij m ust pass through an adjacen t v er tex of ij , which do e s not contain i or j . This set is C d ij . T o prov e the second sta tement , consider the sets C d ij \ C d ik = C k \ { ik , j k } and C d ik \ C d ij = C j \ { ij, j k } , i.e., the neighbour s of ij and ik with the joint neighbours r emov ed. F or every subse t S of C k \ { ik , j k } , clea rly there are at leas t the same num b er of vertices in C j \ { ij, j k } adjacent to mem b ers of S . Hence, the Ha ll’s marr iage theor em [ 11 ] implies the result. T o prov e the third statement, s upp o se, for contradiction, that C 6 = C d ij and C 6 = C d ik and | C | = | C d ij | . Using the fact that, for every missing mem b er of C d ij in C , there is at least a mem b er o f C d ik that should b e in C , there cannot b e any vertex outside C d ij ∪ C d ik in a C . Hence, C ⊂ C d ij ∪ C d ik . If n > 4 and, k m, j o / ∈ C , where o 6 = m , the pa th h ik , j o, k m, ij i connects ik a nd ij , a contradiction. Thus, say , km, j m / ∈ C (which implies j l, k l ∈ C ). Then the path h ik, j m, i l , k m, ij i connects ik and ij , a co ntradiction. 2. The pro of follows from the previous par t and Lemma 3 , and observing that | C ij \ { ik }| = | C ij | − 1. How ever, not all minimal s e parator s o f i j, ik in L c − ( n ) a re of the for m ab ove. F or example, in L c − (5), cons ider the set C = { k l , j l , il , l m } . It holds that ij ⊥ ik | C . In a ddition, C is minimal. How ever, C is not of the fo rm in the ab ov e pro po si- tion. In L c ↔ (5), for the set C d = V \ ( C ∪ { ij, ik } ), by using Lemma 3 , we see that i j ⊥ ik | C d , and C d is ma ximal. Prop ositi on 13. L et a distribution P b e define d over an exchange able r andom network and s k( P ) = L c − ( n ) , and c onsider some n o des i, j, k . imsart-g eneric ver. 2014/10/16 file: exch-ind-arxiv- new.tex date: June 15, 2020 K. Sade ghi/On Finite Exchange ability and Conditional Indep endenc e 18 1. Supp ose t hat for every minimal C such that i j ⊥ ⊥ ik | C , C d ij k ⊆ C and C is invariant u n der swapping k and m for every m with an l such that l m / ∈ C . It then holds t hat if P satisfies c omp osition then it satisfies upwar d-stability and singleton-t r ansitivity. 2. Supp ose that for every maximal C such that ij ⊥ ⊥ ik | C , C d ij k ∩ C = ∅ and C is invaria n t under swapping k and m for every m with an l such that lm ∈ C . It t hen ho lds that if P satisfies interse ction then it satisfies downwar d-stability and singleton-t r ansitivity. Pr o of. B y L e mma 4 , we know that the form of independencies for sk( P ) = L c − ( n ) is ij ⊥ ⊥ i k | C , as pr ovided in the s ta tement o f the prop osition. 1. First, we prove up ward-stability . Supp ose that ij ⊥ ⊥ i k | C . Notice that, bec ause of exc hang eability , the a s sumptions of the statement hold for every i, j, k , l , m . F or a v ariable l m / ∈ C ∪ { i j, ik } , we prove that ij ⊥ ⊥ ik | C ∪ { l m } by induction on | C | . Notice that, by assumption, l m ∈ C ij k . Without los s of generality , we can assume that l ∈ { i, j, k } , and further, l = i or l = k since otherwise w e can swap j and k and proce e d as follows, and finally swap them back. The base case is when C is minimal. W e hav e tw o c a ses: If l = i then b ecause of exchangeability , by the in v ariance o f C under swapping k and m , and since im / ∈ C d ij k , we hav e that ij ⊥ ⊥ i m | C . By comp osition i j ⊥ ⊥ { ik , im } | C , which, b y weak union, implies ij ⊥ ⊥ ik | C ∪ { im } . If l = k then by swapping k and i , we hav e that j k ⊥ ⊥ i k | C . Now, notice that the statement ij ⊥ ⊥ ik | C do es not change under the k j - swap, which implies that C is a lso inv aria nt under swapping j and m . B y this swap, we have k m ⊥ ⊥ ik | C . By this, ij ⊥ ⊥ ik | C , a nd the use o f comp osition, we obtain { ij, k m } ⊥ ⊥ ik | C , which, by weak union, implies ij ⊥ ⊥ ik | C ∪ { k m } . Now supp ose that ij ⊥ ⊥ ik | C , C is not minimal, and, for every C ′ such that | C ′ | < | C | , [ ij ⊥ ⊥ i k | C ′ ⇒ ij ⊥ ⊥ ik | C ′ ∪ { op } ], for every op . Consider the inde- pendenc e statemen t ij ⊥ ⊥ i k | C 0 such that C 0 ⊂ C and C 0 is minimal. W e hav e that ij ⊥ ⊥ i m | C 0 or k m ⊥ ⊥ ik | C 0 . By induction hypothesis, we can a dd vertices to the conditioning set in o r der to obtain ij ⊥ ⊥ im | C or k m ⊥ ⊥ ik | C . Now, a gain comp osition a nd weak union imply the result. Singleton-transitiv ity a lso follows from the ab ov e argument. W e need to s how that ij ⊥ ⊥ ik | C and i j ⊥ ⊥ i k | C ∪ { l m } imply ij ⊥ ⊥ l m | C or l m ⊥ ⊥ ik | C . (Notice that b ecause of up ward-stability ij ⊥ ⊥ ik | C ∪ { l m } is immateria l.) Ag ain without loss of generality , we c a n as sume that l = i or l = k , and the ab ov e ar gument show ed that ij ⊥ ⊥ im | C or k m ⊥ ⊥ i k | C . 2. The pro of follows fr o m part 1. and the duality (Lemma 2 ). Theorem 4. L et a distribution P b e define d over an exchange able r andom net- work and sk( P ) = L c − ( n ) , and c onsider some no des i, j, k . 1. Supp ose t hat for every minimal C such that i j ⊥ ⊥ ik | C , C d ij k ⊆ C and C is invariant u n der swapping k and m for every m with an l such that l m / ∈ C . Then P is faithful to L c − ( n ) if and only if P satisfies the interse ction and c omp osition pr op erties. imsart-g eneric ver. 2014/10/16 file: exch-ind-arxiv- new.tex date: June 15, 2020 K. Sade ghi/On Finite Exchange ability and Conditional Indep endenc e 19 2. Supp ose that for every maximal C such that ij ⊥ ⊥ ik | C , C d ij k ∩ C = ∅ and C is invaria n t under swapping k and m for every m with an l such that lm ∈ C . Then P is fai thful to L c ↔ ( n ) if and only if P satisfies the interse ction and c omp osition pr op erties. Pr o of. The pro of follows from Prop os itions 13 a nd 1 . 4.5. The empty gr aph c ase Clearly every minimal sepa rator in the empt y graph is the empt y set, and the maximal separato r is all the remaining vertices. In the case where sk( P ) is the empt y graph, w e hav e the following. Prop ositi on 14. L et a distribution P b e define d over an exchange able r andom network, and sk( P ) b e the empty gra ph, and c onsider some no des i , j, k , l . 1. Supp ose t hat ij ⊥ ⊥ k l and i j ⊥ ⊥ ik hold. It t hen holds that if P satisfies c om- p osition then it satisfies u pwar d-st ability and singleton-tr ansitivity. 2. Supp ose that ij ⊥ ⊥ k l | V \ { i j, k l } and ij ⊥ ⊥ ik | V \ { ij, ik } hold. It then holds that if P satisfies interse ction then it s atisfies downwar d-stability and singleton-t r ansitivity. Pr o of. B y Lemma 4 , we know tha t i, j, k , l ar e disjoint. 1. First, we prove upw ard- stability . Suppos e tha t ij ⊥ ⊥ k l | C or ij ⊥ ⊥ ik | C . Notice tha t, b eca use of exchangeability , the a ssumptions o f the statement hold for i , j, k , l that ar e all differ ent. W e prove that ij ⊥ ⊥ k l | C ∪ { m h } or ij ⊥ ⊥ ik | C ∪ { mh } , for every mh / ∈ C ∪ { ij, k l } or mh / ∈ C ∪ { ij, ik } , resp ectively , by induction on | C | . The base case is when C = ∅ . First, co nsider the case where ij ⊥ ⊥ k l . If mh / ∈ C ij,kl then by swapping k and m and l and h , we obtain i j ⊥ ⊥ mh . If mh ∈ C ij,kl then say mh = j l . W e use ij ⊥ ⊥ ik and firs t sw ap i a nd j to obtain ij ⊥ ⊥ j k . Now w e swap k a nd l to obtain ij ⊥ ⊥ j l . (The other three cases of mh are similar .) Now comp osition and weak-union imply the result. Now, consider the cas e w her e ij ⊥ ⊥ ik . First suppo se that mh ∈ C ij,kl . If mh = il then by swapping j and l , we obtain il ⊥ ⊥ ik . If mh = j k then by swapping i a nd k , we obta in j k ⊥ ⊥ ik . If mh = j l then by swapping i and j and then k a nd l , we obtain ij ⊥ ⊥ j l . If mh / ∈ C ij,kl then use ij ⊥ ⊥ k l and swap k and m and l and h to o bta in i j ⊥ ⊥ mh . No w comp osition and weak-union imply the result. The inductive step is similar to the inductive step o f P rop ositio n 11 o r Prop o- sition 13 dep ending on the for m ij ⊥ ⊥ k l | C or ij ⊥ ⊥ ik | C . Singleton-transitiv ity also follows fro m the ab ov e arg ument . 2. The pro of follows fr o m the first par t a nd the duality (Lemma 2 ). Theorem 5. L et a distribution P b e define d over an exchange able r andom net- work, and sk( P ) b e empty. Supp ose also that one of the fol lowing c ases holds for i, j, k , l that ar e al l differ ent: imsart-g eneric ver. 2014/10/16 file: exch-ind-arxiv- new.tex date: June 15, 2020 K. Sade ghi/On Finite Exchange ability and Conditional Indep endenc e 20 a) ij ⊥ ⊥ k l and ij ⊥ ⊥ ik ; b) ij ⊥ ⊥ k l | V \ { ij, k l } and ij ⊥ ⊥ ik | V \ { ij, ik } . Then P is faithful to the empty gr aph if and only if P satisfies the interse ction and c omp osition pr op ert ies. Pr o of. The pro of follows from Prop os itions 14 a nd 1 . 5. Summ ary and di scussion Our results concern the co nditional indep endence mo dels induced by exchange- able distributions for ra ndom vectors and r andom netw or k s. W e hav e shown that exchangeable random vectors are c o mpletely indep endent of ea ch other o r completely dep endent of each other if they satisfy intersection and comp ositio n prop erties. In a dditio n, they are mar ginally indep endent if there ex ists at least one indep endence statement and the intersection pro p er ty is satisfied. The in- tersection pr op erty is w ell- understo o d, and we know that a p ositive joint densit y is a sufficient conditio n for it to hold; thus it is particula r ly imp or tant to study the co mp o sition prop er ty fo r exchangeable random vectors, which, in this case, is simplified to A ⊥ ⊥ B | C ⇒ A ⊥ ⊥ ( B ∪ D ) | C , for every disjoint D . F or e x changeable ra ndo m netw or k s, as it turned out, the situation is m uch more co mplicated. As an impor tant extensio n o f o ur r esults in [ 14 ], we show ed that the independence str uctures of exchangeable rando m net works that can be represented by a g raph in graphical mo del sense are one of the s ix p ossible cases: co mpletely dyadic-indep endent, faithful to the undirected or bidirected incidence graphs, faithful to the undirected or bidirected complement of the incidence gr a ph, or completely dyadic-dep endent. The undir ected and bidirected versions of the incidence gr aph a nd its comple- men t are in fact dual to each other, so in a sense there are fo ur r egimes av ailable. In other words, with ex changeability a nd dualit y factor ed in, one passes from the skeleton to the graphical Markov equiv alence class . Exploiting this dualit y , all the res ults for the undirected ca se can b e extended to the bidirected case. In addition, the rema ining four cases are just tw o ca ses mo dulo gr aph complement as well. Although w e failed to do so, it would b e es p e cially nice if this duality could be understo o d, so that one can simply pr e s ent the res ults with tw o-fold duality ar guments. W e hav e pr ovided a simple test to decide in whic h of the s ix regimes an exchangeable random netw or k lies in cases when it has a “structured” inde- pendenc e structure. The main t wo elements of the four “non-trivial” cases is whether an indep endence is of fo rm ij ⊥ ⊥ k l | C or ij ⊥ ⊥ ik | C ; a nd whether C ij,kl = { ik , il , j k , j l } is in C o r is disjo int fro m C . W e, in fac t, do not hav e “necessar y” and sufficient co nditions for whether an exchangeable random netw ork is structured, but rather sufficien t conditions that, in addition to the expected intersection and comp osition pro p e rties, are mainly bas ed on whether a minimal (max ima l) s eparato r is inv ar iant under the men tioned node sw aps of the net work. F or testing purpo s es, it is important to imsart-g eneric ver. 2014/10/16 file: exch-ind-arxiv- new.tex date: June 15, 2020 K. Sade ghi/On Finite Exchange ability and Conditional Indep endenc e 21 stress that the only conditioning set that needs to b e tested are the minimal ones, whic h w ould significantly improv e the computational complexity of a ny relev ant algo rithms. Indeed, in practice, it is mor e imp or tant to understand the indep endence structures o f exchangeable statistica l netw ork mo dels fo r ra ndom netw orks with (in mo st situations) binary dyads. One p o int is that binar y distr ibutions alwa ys satisfy singleton- tr ansitivity [ 8 ], a necessa ry condition for faithfulness, although under our sufficient a ssumptions this condition is automatically satisfie d. In g en- eral, how ever, it would be useful to s tudy which actua l exchangeable mo dels for net works (such as exchangeable exp onential ra ndom gr aph mo dels [ 29 ]) sa tisfy the provided conditions, b oth when we deal with binary random net works or weigh ted ones. App endix A: Pro of of Theorem 2 As mentioned in Sectio n 4.2 , we fo cus on the clas s of chain mixed gra phs, which contains simultaneous undire c ted, directed, o r bidirected edg es, with the sepa- ration criterion intro duce d in [ 16 ]. W e r efrain from defining this cla ss explicitly as it is not needed fo r our purpo s e. How ever, we note that w e c a n fo c us o nly on simple gr aphs since it was shown in [ 16 ] that for any (non-simple) chain mixed graph there is a Markov equiv alent simple gr a ph (the collection of which constitutes the clas s of anterial gr aphs ). W e also only fo cus on maximal gr aphs , which ar e graphs where a missing edge be tw een v ertices u and v implies that there exists a separa tion sta tement of form u ⊥ v | C , for some C – again it w as shown in [ 16 ] that for a ny (non-maximal) chain mixe d gra ph ther e is a Mar ko v equiv a lent maxima l graph. First, we need the following additio na l definitions: A se ction ρ of a walk is a maximal subw alk cons is ting only of lines, meaning that ther e is no o ther subw alk that only consists of lines and includes ρ . Thus, any walk decomp os e s uniquely int o sections; these a re not necessa rily edge-disjo int and sections may also b e single vertices. A section ρ on a walk ω is called a c ol lider se ction if one of the following walks is a subw alk of ω : i ≻ ρ ≺ j , i ≺ ≻ ρ ≺ j , i ≺ ≻ ρ ≺ ≻ j . All other sections o n ω are called non-c ol lider sections. A trise ct ion is a walk h i, ρ, j i , where ρ is a section. If in the tris ections, i and j are distinct and not adjacent then the trisection is called unshielde d . W e say that a trisection is collider o r non-collide r if its section ρ is collider o r no n- collider res pe c tively . W e say tha t a walk ω in a gr aph is c onne cting given C if all collider sections of ω intersect C and a ll non- c ollider sections a re disjoint from C . F or pairwise disjoint s ubsets A, B , C , we say tha t A and B are sepa rated by C if there are no connecting walks b etw een A and B given C , and we use the no tation A ⊥ B | C . Lemma 5. I f two maximal gr aphs G and H ar e Markov e quivalent then G and H have the same unshielde d c ol lider trise ctions. Pr o of. B e cause of maximality , G a nd H hav e the same skeleton. An uns hielded trisection in these graphs cannot b e a collider in one a nd a non-collider in the imsart-g eneric ver. 2014/10/16 file: exch-ind-arxiv- new.tex date: June 15, 2020 K. Sade ghi/On Finite Exchange ability and Conditional Indep endenc e 22 other. This is bec ause if that is the c ase (say an unshielded trisection b etw een i and j and separa tion i ⊥ j | C ), by Markov equiv alence, it implies that an inner vertex of the cor resp onding section is not in C in o ne, but in C in the other, which is a co ntradiction. W e will not need the conv er se of the ab ove lemma, but only a weak er r esult: Lemma 6. If t her e ar e no unshielde d c ol lider trise ctions in G then G is Markov e quivalent to s k( G ) . Pr o of. Fir st, we show that if A 6 ⊥ B | C in G then A 6⊥ B | C in sk( G ): It holds that there is a connecting walk ω in G be tw een A and B given C . If there are no collider sections on ω then no vertex on ω is in C . Therefore, ω is a co nnec ting walk in sk( G ). If there is a collider section with endp oints i and j on ω then it has to be shielded. W e can no w replace this collider section with the ij edge. Applying this metho d r ep eatedly , we obtain a walk that has no vertex in C , a nd is, therefor e, connecting in s k( G ). Now, w e show that if A ⊥ B | C in G then A ⊥ B | C in sk( G ): C o nsider an arbitrar y pa th b etw een A and B in s k( G ). W e need to show that there is a vertex on this path that is in C . Consider this path in G a nd call it ω . If all sections on ω are non-collider then there must b e a vertex o n ω that is in C , and we are done. Hence, consider a collider sectio n with endp oints i and j on ω . This section is shielded; thus, r eplace the section with the ij edge. By rep eating this pro cedure, we either o btain a pa th, with a subset o f vertices of ω , whose sections are all non- c ollider; or we even tually obtain an edge betw een the endpoints of ω , w hich is imp o ssible. The following lemma extends the concept o f exchangeabilit y for r a ndom net- works to graphs in gra phical mo dels: Lemma 7. Supp ose that a distribution P over an exchange able r andom network with no de set N is faithful to a gr aph G , and let π b e a p ermu tation function on N . L et also H b e the gr aph obtaine d by p ermuting the vertic es of G by vertex i j b eing mapp e d to π ( i ) π ( j ) . Then P is faithful to H ; henc e, G and H ar e Markov e quivalent. Pr o of. Le t J π ( P ) b e the indep endence mo del obtained from J ( P ) by ma p- ping independence statemen ts A ⊥ ⊥ B | C to π ( A ) ⊥ ⊥ π ( B ) | π ( C ), where π ( A ) = { π ( i ) π ( j ) : ij ∈ A } , etc. It is obvious that J π ( P ) is faithful to H . Because of exchangeabilit y , w e als o have that J ( P ) = J π ( P ). Therefore , P is faithful to H . Hence, G and H are Markov equiv alent. W e are now re ady to provide the pro of of Theo rem 2 : Pr o of of The or em 2 . C a ses 1 and 6 a re trivial. Thus, we nee d to conside r cases 2 a nd 3 of Pro po sition 8 . By Lemma 6 , if there are no unshielded co llider trisec- tions in G then G is Markov equiv a lent to L − ( n ) or L c − ( n ), resp ectively . Thus, suppo se that there is a n unshielded collider trisection in G . F or case 2 of Pro p o sition 8 , we can assume that there is a n e dg e 12 , 13 in an unshielded collider trisectio n, such tha t there is an arrowhead at 13 on 12 , 13. imsart-g eneric ver. 2014/10/16 file: exch-ind-arxiv- new.tex date: June 15, 2020 K. Sade ghi/On Finite Exchange ability and Conditional Indep endenc e 23 Now, c onsider a n arbitrary edge ij, ik in G . Let π b e a p er mut a tio n that only swaps i and 1 , j and 2, and k and 3, and call the r esulted graph H . (Notice tha t it is p ossible that j = 3 a nd k = 2 .) B y Lemma 7 , H and G are Mar ko v equiv alent. In addition, ik is in an unshielded co llider trisection in H with a n ar rowhead at vertex ik on ij, ik . Hence, by Lemma 5 , ik is in an unshielded co llide r trisection in G with an a rrowhead at vertex ik on ij, ik . Since i, j, k are arbitra ry , and in particular, every edge co uld b e mapp ed to ij, ik by a per mut atio n, we conclude that ther e is an arrowhead at every vertex on every edge in G . Therefore, G is a bidirected gra ph (and Ma rko v equiv alent to L ↔ ( n )). F or case 3 of Prop os itio n 8 , we assume that there is an edge 1 2 , 34 in an unshielded collider trisectio n, suc h tha t there is an arrowhead at vertex 34 on 12 , 34. In this case, w e apply a similar metho d to the previous case, but by a per mutation that only swaps i and 1 , j and 2, k and 3, and l and 4 . Ac kno wledg ements The author is g rateful to Steffen Lauritzen and Alessandro Rinaldo for rais- ing this problem in o ne of our numerous conv ers ations. The author is also truly thankful to the tw o anonymous refer e es, whose comments substantially improv ed the pap e r. References [1] Aldous, D. (1 985). Exchangeability and Related T opics. In ´ Ec ole d’ ´ Et´ e de Pr ob abilit ´ es de Saint–Flour XIII — 1983 (P . Hennequin, ed.) 1–198 . Springer-V erla g, Heidelb erg. L ecture Notes in Ma thematics 1117. [2] Cox, D. R. and W ermuth, N. (1993). Linea r dep endencies represented by chain gr aphs (with discussion). Stat. Sci. 8 204 –218 ; 247– 277. [3] Darr och, J. N. , L auritzen, S. L. and Speed, T. P . (1 980). Markov fields and log-linea r interaction mo dels for co ntingency tables. Ann. Statist. 8 5 22–5 39. [4] Da wid, A. P. (1979). Co nditio nal indep endence in statistical theory (with discussion). J. R. S tat. So c. S er. B. S tat. Metho dol. 41 1– 31. [5] deFinetti, B. (1931). F unzione Ca ratteristica Di un F enomeno Aleatorio. 6. Memorie 251– 299. Academia Na z ionale del Linceo. [6] Diaconis, P. and Freedman, D. (19 80). Finite exchangeable sequences. Ann. Pr ob ab. 8 74 5–76 4. [7] Diaconis, P . and Janson, S. (20 08). Graph limits and exchangeable ran- dom g raphs. R endic onti 28 33 – 61. [8] Dr ton, M. , Sturmfels, B. a nd Sulliv ant, S. (200 9). L e ctur es on Al- gebr aic Statistics . O b erwolfach Seminars 39 . Spr inger. [9] Dr ton, M. and X iao, H. (2 010). Smo o thness o f Gaussia n Conditional Independenc e Models. Contemp. Math. 516 1 55–17 7. [10] Eag l eso n , G. K. a nd W eber, N. C. (197 8). Limit theorems for weakly exchangeable a rrays. Mathematic al Pr o c e e dings of the Cambridge Philo- sophic al So ciety 84 123– 130. imsart-g eneric ver. 2014/10/16 file: exch-ind-arxiv- new.tex date: June 15, 2020 K. Sade ghi/On Finite Exchange ability and Conditional Indep endenc e 24 [11] Hall, P. (19 35). On Repre s entativ es of Subse ts. J. L ondon Math. S o c. 10 26–30 . [12] Ka uermann, G. (1996). On a dualiza tion of graphical Gaus sian mo dels. Sc and. J. St at. 23 105– 116. [13] Lauritzen, S . L. (199 6 ). Gr aphi c al Mo dels . Clar endon Pr ess, Oxford, United Kingdom. [14] Lauritzen, S . L. , Rinaldo, A. and Sadeghi, K. (2018). Ra ndom Net- works, Gr aphical Mo dels, and Exch a ngeability . J. R. Stat. So c. S er. B. Stat. Metho dol. 80 48 1–50 8. [15] Lauritzen, S. L. , Rinaldo, A . and Sadeghi, K. (2 019). On Exchange- ability in Netw or k Mo dels. J . Algebr. Stat. 10 85–1 13. [16] Lauritzen, S. L. and Sadeghi, K. (2018 ). Unifying Marko v pr op erties for gr aphical mo dels. Ann. Statist. 46 . [17] Lauritzen, S . L. and Wermuth, N. (1989). Graphical mode ls for as so- ciation b etw een v ariables, so me of which are qualitative and so me quanti- tative. Ann. St atist. 17 31–57. [18] Ln ˇ eni ˇ cka, R. and Ma t ´ u ˇ s, F. (2007). On Gaussian c o nditional indep en- dence s tructures. Kyb ernet ika 43 327 – 342. [19] Malouche, D. , Rajara tnam, B. and Rolfs, B. T. (2 013). Duality in Graphical Mo dels . [20] Ma t ´ u ˇ s, F. (19 92). Ascending And Des c e nding Conditional Indep endence Relations. In in T ra ns. 11th Pr ague Conf. Information The ory, St atist ic al De cision F un ctions and R andom Pr o c esses , vol. B. 1 81–2 00. [21] Ma t ´ u ˇ s, F. (1995). Finite partially exchangeable arrays T ech- nical Repo rt No. 1 856, Ins t. of Inf. Theory and Autom., ht tp:// staff.utia.cas.cz/ matus/CsiMaGSI13 .p df. [22] Pearl, J. (198 8 ). Pr ob abilistic R e asoning in Intel ligent Systems : net- works of plausible infer enc e . Morgan Ka ufmann P ublishers, Sa n Ma teo, CA, USA. [23] Pearl, J. and P az, A . (1985). Gr aphoids: A Gr aph-b ase d L o gic for R e a- soning ab out R elevanc e R elations . University o f California (Los Angele s). Computer Science Department. [24] Peters, J. (20 1 5). On the Intersection P rop erty of Conditional Indepen- dence and its Application to Causal Discovery . J. Causal Infer enc e 3 9 7– 108. [25] Richardson, T. S. and Spir tes, P. (200 2 ). Ances tr al graph Marko v mo dels. Ann. Statist . 30 962–1 0 30. [26] Sadeghi, K. (201 7 ). F aithfulness of Probability Distributions and Gr aphs. J. Mach. L e arn. Res. 18 . [27] Sil verman, B. W. (197 6). Limit Theor ems for Disso ciated Random V ari- ables. Ad v. in Appl. Pr ob ab. 8 806 – 819. [28] Studen ´ y, M. (2005). Pr ob abilistic Conditional Indep endenc e St ru ctur es . Springer-V erla g, London, United Kingdom. [29] W asserman, S. and P a ttison, P. (1996 ). Log it models and logistic re- gressio ns for s o cial net works: I. An intro ductio n to Markov gr aphs and p ∗ . Psychometrika 61 40 1-42 5. imsart-g eneric ver. 2014/10/16 file: exch-ind-arxiv- new.tex date: June 15, 2020 K. Sade ghi/On Finite Exchange ability and Conditional Indep endenc e 25 [30] Wermuth, N. , Cox, D. R. and Pearl, J. (1 9 94). E xplanation for mul- tiv ar iate structures der ived from univ ariate recur sive regress ions T echnical Repo rt No. 94(1 ), Univ. Mainz, Ge r many . [31] Wermuth, N. , Mar chetti, G . M. a nd Cox, D. R. (2009). T ria ngular systems for sy mmetr ic binary v ariables. Ele ctr on. J. Stat. 3 9 32–9 55. [32] West, D. B. (2001). Intr o duction to Gr aph The ory . Pr entice Hall, Upp er Saddle Riv er , NJ, USA. imsart-g eneric ver. 2014/10/16 file: exch-ind-arxiv- new.tex date: June 15, 2020