Bilinear Mixed-Effects Models for Affiliation Networks

Bilinear Mixed-Effects Mo dels for Affiliation Net w orks Y anan Jia ∗ Catherine A. Calder ∗ ‡ Christopher R. Bro wning † June 7, 2021 Abstract An affiliation net work is a particular type of tw o-mode so cial netw ork that consists of a set of ‘actors’ and a set of ‘ev ents’ where ties indicate an actor’s participation in an ev ent. Although net w orks describe a v ariet y of consequential so cial structures, statistical metho ds for studying affiliation netw orks are less w ell developed than meth- o ds for studying one-mo de, or actor-actor, netw orks. One wa y to analyze affiliation net works is to consider one-mo de netw ork matrices that are derived from an affilia- tion net work, but this approach may lead to the loss of imp ortan t structural features of the data. The most comprehensive approach is to study both actors and even ts sim ultaneously . In this paper, we extend the bilinear mixed-effects mo del, a t yp e of laten t space mo del developed for one-mo de net works, to the affiliation netw ork setting b y considering the dep endence patterns in the interactions betw een actors and even ts and describe a Mark o v c hain Mon te Carlo algorithm for Ba yesian inference. W e use our mo del to explore patterns in extracurricular activity mem b ership of students in a racially-div erse high sc ho ol in a Midwestern metrop olitan area. Using techniques from spatial point pattern analysis, we show ho w our mo del can pro vide insigh t in to patterns of racial segregation in the v oluntary extracurricular activity participation profiles of adolescen ts. Keyw ords: Ba yesian mo deling, generalized linear mo del, so cial net works, Marko v c hain Mon te Carlo (MCMC), latent space, p oin t pattern, racial segregation, visualiza- tion ∗ Departmen t of Statistics, The Ohio State Univ ersity , Colum bus, OH, USA † Departmen t of So ciology , The Ohio State Universit y , Colum bus, OH, USA ‡ Email: calder@stat.osu.edu 1 1 In tro duction In typical statistical analyses, the primary goal is learning about prop erties of individual units. When the property of in terest in v olves in teractions betw een m ultiple units rather than prop erties of the individual units themselv es, the units can b e considered a netw ork. Net work data are widely used to represent relational information among interacting units. Units are referred to as no des in a netw ork, and relationships b et w een the nodes are represen ted by ties/e dges . Pairs of no des, which may be either link ed or not, are called dyads in a netw ork. W e use the term mo de to differen tiate sets of distinct nodes in a net work. The most common t yp e of netw ork is a one-mo de net w ork in whic h all units are of the same type. A t ypical example is a friendship net work where all no des are individuals, or actors , and ties b et w een all actors are well defined. Tw o-mo de netw orks contain relational information ab out t wo distinct sets of entities, sp ecifically ab out ties b et w een no des of different mo des. Tw o-mo de net works can capture more relational structure than the standard one-mo de representation of suc h data and are a natural representation of relational data in volving affiliations betw een sets of en tities. The term “affiliation” usually refers to mem b ership or participation data. Arguably , the most well known affiliation dataset is the “Southern W omen” net work collected b y Da vis and Gardner (1941), which consists of attendance records at v arious so cial even ts in a small southern town. This dataset is an affiliation net work since the ties represen t affiliations b etw een a set of actors (w omen), denoted by A , with a set of events (so cial ev ents), denoted by E . Affiliation net w orks, suc h as the Southern W omen net work, allo w the study of the dual persp ectiv es of actors and ev ents where connections among mem bers of one of the modes are based on link ages established through the second mode (i.e., women are connected b ecause they attend the same so cial ev ents and so cial even ts are connected through the w omen that participate in them). In this pap er, w e study patterns of participation in extracurricular activities within a racially diverse high sc ho ol in a Midw estern metrop olitan area of the United States. In particular, we aim to iden tify patterns of racial segregation in the extracurricular activity profiles of students. Details on the data and aims of our 2 segregation analysis are provided in Section 4.1. In this pap er, w e build on ideas from Hoff (2005) and extend the bilinear mixed-effects mo dels dev elop ed for one-mo de net w orks to the t w o-mo de settings. The bilinear effect for an actor/ev ent pair in our mo del is the inner pro duct of unobserv ed characteristic v ectors sp ecific to actors and ev ents. Our mo del can capture fourth order (ev en n umber order) dep endence, whic h we argue b elo w is necessary to describ e the t yp es of structures seen in real affiliation net works. Inferences from our mo del provide a visual and interpretable mo del-based spatial represen tation of affiliation relationships. If we presume the existence of a latent so cial sp ac e in whic h the positioning of actors captures similar profiles of ev ent participation, these laten t p ositions allow us to explore, as well to test hypotheses ab out, social structure within an affiliation netw ork. Here, w e describe ho w metho ds from spatial p oin t pattern analysis can b e used to in vestigate the presence of racial segregation in extracurricular activit y mem b erships of high sc ho ol students. This pap er is organized as follo ws. In the next section, w e in tro duce types of dep endence often seen in t w o-mo de net w ork datasets, discuss basic models for affiliation net work data, and argue that these basic mo dels are not sufficiently able to capture the dep endencies in tw o- mo de net w ork data. In Section 3, we state our bilinear mixed effects mo del and demonstrate ho w it is able to capture higher-order dep endence than standard mixed-effects mo dels. W e also pro vide a description of a Marko v c hain Mon te Carlo (MCMC) algorithm providing full Bay esian inference. Our analysis of student extracurricular activit y participation is presen ted in Section 4. W e conclude in Section 5 with a discussion of some directions for future researc h. 3 2 Affiliation Net w orks 2.1 Bac kground Generally , an affiliation netw ork can b e denoted b y an n a × n e affiliation matrix Y = { y ik } , whic h records the affiliation of eac h actor with eac h even t, where rows index actors and columns index ev ents, and n a and n e are the total n umber of actors and even ts, resp ec- tiv ely . The entries of this matrix, y ik , can b e binary v ariables or non-negativ e integer-v alued v ariables. If actor i is affiliated with even t k , then y ik ≥ 1 and y ik = 0 otherwise, where i = 1 , 2 , . . . , n a and k = 1 , 2 , . . . , n e . Each row of Y describ es an actor’s affiliation with the ev ents. Similarly , eac h column of Y describes the mem b ership of an ev ent. An affiliation net work can also b e represented b y a bipartite graph, or a graph in which the no des can b e partitioned into tw o subsets corresp onding to the distinct mo des, and all lines are b et w een pairs of no des b elonging to the differen t mo des. F or affiliation net works, since actors are affiliated with ev ents, and ev ents ha ve actors as mem b ers, all lines in the bipartite graph are b et w een no des represen ting actors and no des representing even ts. Statistical metho ds for one-mo de net works are fairly w ell developed. The exponential random graph mo del (ERGM) is one of the most p opular metho ds for analyzing netw orks (F rank and Strauss, 1986; W asserman and P attison, 1996; Pattison and W asserman, 1999; Robins et al., 1999). Although ER GMs are useful for mo deling global netw ork characteristics, they are known to p ossess some undesirable prop erties. Robins et al. (1999) and Handco c k et al. (2003) discussed these c hallenges asso ciated with ER GMs, including the in tractability of the normalizing constan t in the lik eliho o d function of ERGMs and mo del degeneracy . Snijders et al. (2006) prop oses an alternativ e sp ecification of ERGMs that partially addresses these issues, but requires sp ecifying v alues of tuning parameters. As an alternativ e, models built on laten t v ariables hav e attracted considerable atten tion recently . These mo dels include mixed- effects mo dels (v an Duijn et al., 2009; Zijlstra et al., 2009; Hoff, 2003, 2005, 2009), the sto c hastic blo c kmo del (W ang and W ong, 1987; Snijders and Nowic ki, 1997; Snijders, 2001), 4 and laten t space mo dels (Hoff et al., 2002). All of these latent v ariable mo dels assume conditional indep endence of the probabilit y of ties betw een dyads. That is, the elemen ts of Y are indep enden t conditional on latent v ariables. Conditional independence do es not imply that latent v ariable mo dels cannot capture net work dep endencies of interest. Indeed, some of the more sophisticated latent v ariable mo dels make clev er use of latent structures to capture t yp es of dep endence. The conditional indep endence of edges implies that mo del degeneracy is not an issue. In addition, the conditional indep endence of tie probabilities leads to computational adv an tages in mo del fitting (Hunter et al., 2012). A latent v ariable mo del that we build on in this pap er is the bilinear mixed-effects mo del prop osed b y Hoff (2005), whic h is an extension of laten t space models for one-mo de netw orks. This mo del uses in teracting latent v ariables to capture certain t yp es of higher-order dep endence patterns often presen t in so cial netw orks. While there is a rich literature on statistical metho ds for one-mo de netw orks, metho ds for t wo-mode net w orks are limited. One approach, known as the “con v ersion,” or pro jection metho d (Newman, 2001), relies on the tw o one-mo de net works that can b e derived from an affiliation net w ork: YY 0 is the one-mo de netw ork for actors and Y 0 Y is the one-mo de net work for ev ents. Information is lost, ho wev er, b y con v erting an affiliation net work in to t wo one-mo de netw orks. F or instance, if we use binary matrices to represent the one-mo de net works, then we lose information ab out b oth the num b er and the prop erties of the shared partners of the other set. Alternativ ely , w e can build mo dels for tw o-mo de net works to analyze b oth actors and ev en ts simultaneously . W ang et al. (2009) extended ER GMs to the tw o-mode situation. Ho wev er, these mo dels suffer from the limitation of the one-mo de ER GMs describ ed ab ov e. In addition, they do not readily p ermit the mo deler to inv estigate patterns in activit y participation across m ultiple ev en ts (e.g., whether certain individuals share activity profiles). As we will illustrate, the latent v ariable approach we take is muc h more amenable to this sort of study . 5 2.2 Dep endence P atterns in Affiliation Net w orks Net work data differ from other types of dep endent data in that ties often tend to b e tr an- sitive, b alanc e d , and cluster able (W asserman and F aust, 1994). In one-mo de friendship net- w orks, we often see patterns that indicate “a friend of a friend is a friend,” a statement that translates to prop erties of sets of three dy ads (triangles). In particular, this pattern is called transitivit y . Balance is a generalized version of transitivit y defined for signed relationship of the type A ij is p ositiv e if there is a tie b et w een no des i and j and is negative otherwise. F ormally , a signed relationship b etw een no des is denoted as follo ws: A ij = ( 1 if i and j are tied, − 1 if i and j are not tied. In one-mo de netw orks, a triangle formed by a triad of units i, j, k is said to b e balanced if A ij × A j k × A ki > 0. Clusterabilit y is a generalization of the concept of balance. A triangle is clusterable if it is either balanced or the pairwise relationships within the triad are all negativ e. Here w e extend these definitions to the t wo-mode setting, which to the best of our kno wledge has not been done previously . W e say a set of four p ossible ties among a tetr ad of units consisting of one pair of actors i, j and one pair of even ts k , l , { A ik , A il , A j k , A j l } , forms a cycle , and offer the following definitions. Definition 2.1 F or signe d affiliation r elations, a cycle { A ik , A il , A j k , A j l } is tr ansitive if whenever A ik = A il = A j k = 1 , we have A j l = 1 . T ransitivit y implies that if actors i and j b oth hav e a tie with ev ent k and actor i is tied with another ev ent l , then we exp ect actor j also has a tie with even t l . Definition 2.2 F or signe d affiliation r elations, a cycle { A ik , A il , A j k , A j l } is said to b e b al- anc e d if A ik × A il × A j k × A j l = 1 . Since the n umber of elements in a cycle is an even num ber (4), we note that balance and clusterabilit y are identical conceptually in the t wo-mode setting. 6 F or general signed relations among units, many theories of so cial systems suggest that the relationships within a cycle tend to b e balanced. F or example, if A ik = 1 and A j k = 1, which means the relationships betw een actor i and ev ent k and b etw een actor j and ev ent k are p ositiv e, then it is more lik ely that either b oth A il = 1 and A j l = 1 or b oth A il = − 1 and A j l = − 1. In other w ords, if actors i and j b oth participate in even t k , then they are likely to either b oth participate in ev ent l or b oth not participate in even t l . In real affiliation net works, we exp ect to see more evidence of balance than we would exp ect if the presence of ties is completely random. This translates in to the presence of particular balanced patterns among cycles, whic h are illustrated in Figure 1. Figure 1: All p ossible balanced cycles among a tetrad. Solid lines connecting actors and ev ents denote ties (p ositiv e relationship) and dashed lines denote the absence of ties (negativ e relationship). These configurations, 0-tw o-path (0- L 2 ), 1-tw o-path (1- L 2 ), actor 1-tw o-path (1- L A 2 ), even t 1-t wo-path (1- L E 2 ), and four-cycles ( C 4 ), sho wn in Figure 1 are the balanced cycles often seen in affiliation netw orks. Generalizations of these structures, the actor k -t w o-path ( k - L A 2 ) and the even t k -t wo-path ( k - L E 2 ), are used by W ang et al. (2009) to define ERGMs for t wo-mode netw orks. Consider the case where actor-even t ties within an affiliation netw ork are assumed to b e indep enden t and identically distributed with tie probabilit y π 0 . In this case, it can b e sho wn that the exp ected prop ortion of balanced cycles is π = π 4 0 + (1 − π 0 ) 4 + 6 π 2 0 (1 − π 0 ) 2 . In many real net w orks, the observ ed prop ortion of balanced cycles is greater than this theoretical v alue from this mo del (i.e., indep enden t and identically distributed ties). That is, p > 7 p 4 0 + (1 − p 0 ) 4 + 6 p 2 0 (1 − p 0 ) 2 , where p is the observed prop ortion of balanced cycles and π 0 is the observed prop ortion of actor-even t pairs that are tied. F rom this, we can see the imp ortance of capturing fourth-order dep endence (dep endence b et w een tetrads) in mo dels for affiliation net work data. 2.3 Basic Mo dels Our data consist of an n a × n e so ciomatrix Y , with en tries y ik denoting the v alue of the relation b etw een actor i and ev ent k and additional co v ariate information asso ciated with actors, ev ents, and dy ads. 2.3.1 Fixed-Effects Mo del Since most affiliation net w ork data, y ik , are binary or (non-negativ e) in teger v alued, w e sp ecify mo dels using the standard generalized linear mo del framework. W e let P r ( Y = y | β β β ) = n a Y i =1 n e Y k =1 P r ( Y ik = y ik | β β β ) , where eac h component of Y follo ws an exp onential family distribution. W e relate µ ik ≡ E ( Y ik | β β β ) to a set of cov ariate v ariables x ik via a link function denoted by g ( · ): θ ik = g ( µ ik ) = β β β 0 x ik , where β β β is a r -dimensional vector of unknown regression co efficien ts. W e decomp ose x ik in to x ik = ( x d ik , x a i , x e k ), where x d ik is an r d -dimensional cov ariate v ector asso ciated with (actor i , even t k ) dy ad, x a i is an r a -dimensional cov ariate vector asso ciated with actor i , x e k is an r e -dimensional co v ariate v ector associated with ev ent k , implying r d + r a + r e = r . The mo del can then b e rewritten as θ ik = g ( µ ik ) = β β β 0 d x d ik + β β β 0 a x a i + β β β 0 e x e k . (1) where β β β d , β β β a , and β β β e are v ectors of unknown regression coefficients with dimension r d , r a and r e , resp ectiv ely . The affiliation netw ork data are measured on a set of actors and a set of 8 ev ents. Since actors and even ts comprise multiple dyads, the observ ations y ik s are lik ely not (conditionally) indep enden t giv en the regression co efficien ts, and we need a mo del whic h can capture dep endence induced by the shared actors and ev ents making up the dy ads. 2.3.2 Mixed-Effects Mo dels F or affiliation netw ork data, an actor can attend m ultiple even ts and an even t can hav e m ultiple actors. T o mo del the within-no de dep endence, w e consider mixed mo dels with actor and ev ent random effects of the form θ ik = g ( µ ik ) = β β β 0 d x d ik + β β β 0 a x a i + β β β 0 e x e k + a i + e k , (2) where µ ik ≡ E ( Y ik | θ ik ), and a i and e k represen t the actor and even t random effects, resp ec- tiv ely . F or discrete data sub ject to ov erdisp ersion (Poisson, binomial), an observ ation level residual is also present, so that, θ ik = g ( µ ik ) = β β β 0 d x d ik + β β β 0 a x a i + β β β 0 e x e k + a i + e k + γ ik , (3) with γ ik usually tak en as indep endent and identically distributed errors (Congdon, 2010). W e can in terpret the γ ik s as dy ad random effects. The observ ations { Y ik : i = 1 , . . . , n a , k = 1 , . . . , n e } are mo deled as conditionally inde- p enden t given the random effects, denoted b y a = ( a 1 , . . . , a n a ) 0 , e = ( e 1 , . . . , e n e ) 0 , and γ γ γ = vec( Γ ) for the n a × n e matrix Γ with elemen ts γ ik for i = 1 , . . . , n a and k = 1 , . . . , n e . That is, P r ( Y = y | β β β , a , e , γ γ γ ) = n a Y i =1 n e Y k =1 P r ( Y ik = y ik | β β β , a i , e k , γ ik ) . W e tak e the different t yp es of random effects to b e mutually independent and Gaussian with mean zero and v ariances σ 2 a , σ 2 e and σ 2 γ , resp ectiv ely: a | σ 2 a ∼ MVN(0 , σ 2 a I n a × n a ) , e | σ 2 e ∼ MVN(0 , σ 2 e I n e × n e ) , 9 and γ γ γ | σ 2 γ ∼ MVN(0 , σ 2 γ I n γ × n γ ) , where I n × n denotes the n -dimensional identit y matrix and n γ = n a × n e . The mo del given b y (2) is a sp ecial case of (3), where σ 2 γ is equal to zero. Therefore, we refer to the mo del giv en by (3) as the generalized linear mixed effects mo del for affiliation netw orks. Letting  ik denote the ( i, k ) random effect (i.e.,  ik = a i + e k + γ ik ) and marginalizing ov er the a i s, e k s, and γ ik s, it follo ws that Co v( y ik , y il ) = E(  ik  il ) = σ 2 a Co v( y ik , y j k ) = E(  ik  j k ) = σ 2 e Co v( y ik , y ik ) = E(  2 ik ) = σ 2 a + σ 2 e + σ 2 γ where σ 2 a and σ 2 e capture the comp onen ts of the total v ariation in the  ik s explained b y dyads con taining the same actor or even t, resp ectiv ely . This mo del is able to capture dep endence b et w een elements of Y due to shared no des using a standard random-effects sp ecification. Ho wev er, as we will discuss in the next section, this mixed mo del is unable to capture the fourth order (or higher even order) dep endence frequen tly encoun tered in real affiliation net works. 3 A Bilinear Mixed-Effects Mo del 3.1 Mo del Sp ecification In order to capture more transitivit y and balance than the generalized linear mixed effects mo del allo ws, w e add a bilinear random effect to the mo del giv en by (3). As with Hoff (2005)’s bilinear mixed effects mo del for one-mo de netw ork data, this addition enables us to capture the exp ected balanced tendencies in tw o-mo de net work relations. W e presume the existence of a latent so cial space of dimension t . Both actors and ev ents ha ve p ositions in this laten t space, denoted b y the v ectors u i and v k , resp ectiv ely . If w e 10 consider the pair of actors with p osition vectors ( u i , u j ) (or the pair of even ts with p osition v ectors ( v k , v l )), and they ha ve similar direction and magnitude, then the inner pro ducts u 0 i v k and u 0 j v k (or u 0 i v k and u 0 i v l ) will not b e to o differen t. A probability measure o ver these unobserved c haracteristics induces a mo del in whic h the presence of a tie b et w een an actor and an even t is dep endent on the presence of other ties. Relations mo deled as suc h are probabilistically balanced. W e add this inner pro duct of latent v ectors u i and v k to (3) so that  ik = a i + e k + γ ik + ε ik , where the random effects a i , e k and γ ik are still taken to b e m ultiv ariate normal with means and cov ariances are as giv en in Section 2.3.2. This set of bilinear terms { ε ik = u 0 i v k , i = 1 , . . . , n a , k = 1 , . . . , n e } allows us to capture balance. T o see this further, consider the case in which t = 1, where t is the dimension of the u i and v k v ectors. In this case, the ε ik s corresp ond to the residuals from the v ersion of the mo del without the bilinear term. Since ε ik × ε il × ε j k × ε j l = ( u i u j v k v l ) 2 ≥ 0, the bilinear term can b e seen to capture p ositive residual cycles. Of course in a real dataset, w e do not exp ect net works to b e completely balanced. By taking t > 1, the bilinear term captures the balanced tendencies of real netw orks without forcing ev ery residual cycle to b e p ositive. W e assume the u i s and v k s are mutually indep enden t and follow t -dimensional m ultiv ariate normal distributions so that u i | Σ Σ Σ u ∼ MVN(0 , Σ Σ Σ u ) v k | Σ Σ Σ v ∼ MVN(0 , Σ Σ Σ v ) . In addition, w e assume u i ⊥ u j for { i, j = 1 , . . . , n a : i 6 = j } and v k ⊥ v l for { k , l = 1 , . . . , n e : k 6 = l } . It follo ws that ε ik s ha ve momen ts E( ε ik ) = 0 E( ε 2 ik ) = trace( Σ Σ Σ u Σ Σ Σ v ) E( ε ik ε j k ε j l ε il ) = trace( Σ Σ Σ u Σ Σ Σ v Σ Σ Σ u Σ Σ Σ v ) . 11 The other second, third, and fourth order moments are all equal to zero. F or simplicit y , we assume Σ Σ Σ u = σ 2 u I t × t , Σ Σ Σ v = σ 2 v I t × t . In this case, the moments of bilinear term b ecome E( ε 2 ik ) = tσ 2 u σ 2 v , E( ε ik ε j k ε j l ε il ) = tσ 4 u σ 4 v . This gives the following nonzero second and forth order moments for the bilinear random- effects comp onen ts,  ik = a i + e k + γ ik + u 0 i v k : E(  ik  il ) = σ 2 a , E(  ik  j k ) = σ 2 e , E(  2 ik ) = σ 2 a + σ 2 e + σ 2 γ + tσ 2 u σ 2 v , E(  ik  j k  j l  il ) = σ 4 a + σ 4 e + tσ 4 u σ 4 v . The bilinear effect ε ik = u 0 i v k can b e interpreted as a mean-zero random effect that is able to capture particular fourth order dep endence in affiliation net work data. 3.2 P arameter Estimation The parameters w e w ant to estimate are { β β β d , β β β a , β β β e , σ 2 a , σ 2 e , σ 2 γ , σ 2 u , σ 2 v } . F ollowing Hoff (2005), w e work with the follo wing representation of our mo del: θ ik = β β β 0 d x d ik + ( β β β 0 a x a i + a i ) + ( β β β 0 e x e k + e k ) + γ ik + u 0 i v k = β β β 0 d x d ik + µ a i + µ e k + γ ik + u 0 i v k , (4) where µ a i = β β β 0 a x a i + a i and µ e k = β β β 0 e x e k + e k can b e view ed as actor and even t sp ecific effects, resp ectiv ely . W e then define z ik = θ ik − u 0 i v k = β β β 0 d x d ik + µ a i + µ e k + γ ik , and let z = v ec( Z ), where Z is the n a × n e matrix with elements z ik for i = 1 , . . . , n a and k = 1 , . . . , n e . W e take θ θ θ to b e the n a × n e matrix with elemen ts θ ik , and let u b e a n a × t matrix with rows u i for i = 1 , . . . , n a and v b e a n e × t matrix with ro ws v i for i = 1 , . . . , n e . Then w e can write z = vec( θ θ θ − uv 0 ) = X D    β β β d µ µ µ a µ µ µ e    + γ γ γ (5) where X D is the appropriate design matrix constructed using (4) and γ γ γ is a v ector with dimension n γ as describ ed in Section 2.3.2. F rom (5), it is clear that conditional on the θ θ θ s, 12 u s and v s, the other parameters can b e sampled using a standard Bay esian normal-theory regression approac h. Our general Gibbs sampler giv en b elo w is similar to Hoff (2005)’s algorithm, with the ex- ception of the first part of step 1 and all of steps 2 and 3 in the outline b elo w: 1. Sample linear effects: Sample β β β d , µ µ µ a , µ µ µ e | β β β a , β β β e , σ 2 a , σ 2 e , σ 2 γ , θ θ θ , u , v (linear regression) Sample β β β a , β β β e | µ µ µ a , µ µ µ e , σ 2 a , σ 2 e , σ 2 γ (linear regression) Sample σ 2 a , σ 2 e , and σ 2 γ from their full conditionals 2. Sample bilinear effects: F or i = 1 , . . . , n a sample u i | u − i , v , θ θ θ , β β β d , µ µ µ a , µ µ µ e , σ 2 u (linear regression ) F or k = 1 , . . . , n e sample v k | v − k , u , θ θ θ , β β β d , µ µ µ a , µ µ µ e , σ 2 v (linear regression ) Sample σ 2 u and σ 2 v from their full conditionals 3. Update θ θ θ : F or actor i and even t k Prop ose θ ∗ ik ∼ N( β 0 X ik + µ a i + µ e k + u 0 i v k , σ 2 γ ) Accept θ ∗ ik with probabilit y [ p ( y ik | θ ∗ ik ) /p ( y ij | θ ik )] V 1 The full conditional distributions of β β β d , µ µ µ a , µ µ µ e , β β β a , β β β e , σ 2 a , σ 2 e , σ 2 γ , σ 2 u , σ 2 v , u i m and v k are given in App endix A. F or binary affiliation netw ork cases, we can use the same algorithm as ab o ve with σ 2 γ set to a fixed v alue since o ver-dispersion is not appropriate for generalized linear mo dels for binary resp onses. W e use this algorithm to fit the mo del to a binary affiliation netw ork dataset in Section 4 with σ 2 γ = 1. In addition to the parameters, the dimension of the laten t bilinear effects, t , is unknown. Choice of t will generally dep end on the goal of the analysis. If we w an t to visualize the bilinear terms in order to understand laten t structure in an affiliation netw ork, w e can simply c ho ose t = 1 , 2, or 3. If the goal is prediction, we can examine higher dimensions 13 and compare models using the Deviance Information Criterion (DIC; Spiegelhalter et al., 2002a), or p erhaps formally include t in the model space and employ a rev ersible jump MCMC algorithm for mo del fitting (Green, 1995). In Section 4, w e try different v alue for t and rep ort the corresp onding DIC. Lastly , if we are in terested in whether the model captures particular features of the observ ed netw ork (W asserman and F aust, 1994) or in examining particular asp ects of lac k-of-fit, we can ev aluate the mo del with p osterior predictive c hecks (Besag, 2001). 4 Application: Racial Segregation in Extracurricular Activities 4.1 Motiv ation and Data Description The presence of members of different racial and ethnic groups within a so cial unit is referred as interr acial c ontact . In terracial contact in the educational setting is an imp ortant so cial issue, but extan t researc h primarily focuses on the racial comp osition of the sc ho ol as a whole or within a classro om. In high school, how ever, extracurricular activities pla y a significant role in students’ school exp eriences, but contact patterns b et ween races within extracurric- ular activities ha v e b een largely unexplored (Granov etter, 1986; Clotfelter, 2002). Sc ho ols that are integrated comp ositionally ma y not necessarily result in integrated so cial interac- tions if students’ so cial netw orks are segregated by race/ethnicit y . Mo o dy (2001) finds that sc ho ols in whic h extracurricular activities are in tegrated by race/ethnicity exhibit low er lev els of race/ethnic segregation in friendship netw orks, suggesting that extracurricular activities pla y an imp ortan t role in diversifying the so cial exp eriences of youth. Consequen tly , accu- rate characterization of segregation patterns in extracurricular activities by race/ethnicit y is necessary in order to understand the features of sc ho ol so cial structure that shap e actual so cial interactions and friendship formation. In this section, we examine in terracial con tact in high school extracurricular organizations b y applying our prop osed bilinear mixed effects 14 mo del to student extracurricular activit y netw ork data. W e consider a binary affiliation net w ork of student participation in extracurricular activ- ities collected b y Daniel McF arland as part of his do ctoral dissertation at the Univ ersit y of Chicago (McF arland, 1999). The data is av ailable in NetData R pac k age (Now ak et al., 2012). The extracurricular activit y data were collected as part of a larger observ ational study of tw o high schools that included classro om observ ations, surveys, school records, and in terviews. W e use the data from “Magnet” High, an elite magnet school located in an inner- cit y neigh b orho od of a large Midwestern metrop olitan area, whose actual name is redacted to protect the confiden tiality of the study participants. It is an in tegrated high school with high-abilit y students from predominantly low er-income households. While heterogeneous in racial bac kground, Magnet High is rather homogeneous in terms of student abilit y . The extracurricular activit y data w as collected from information on volun tary participation in clubs and sp orts in yearbo oks. Gender and racial background on students was ascertained based on y earb o ok photos, coupled with observ ation and school records. The full affiliation netw ork data for Magnet High consists of 1295 studen ts and 91 student organizations, in whic h participation is recorded o ver three y ears (1996-1998) along with individual-lev el attributes of grade, gender, and race. W e com bine similar clubs together (see App endix B) and the newly constructed netw ork has n e = 37 activities (even ts). W e fo cus on the n a = 905 studen ts (actors) in grade 8-12 with non-missing race information listed as Hispanic, Asian, blac k, and white. W e only consider netw ork as it exists in 1996 for our analysis. The data used in our analysis is sho wn in Figure 2. While this data is nearly 20 years old, w e are not aw are of more recent data of a similar nature nor has this data b een studied from the p ersp ectiv e of interracial con tact. Magnet High is comp osed of 6 p ercen t Hispanic, 2 p ercen t Asian, 35 p ercent black, and 57 p ercen t white students. Approximately 72 p ercen t of the studen ts participated in at least one activity . A descriptive plot of activity by race is given in Figure App endix C.1, and a summary of the data by race is provided in T able 1. 15 Figure 2: Illustration of the tw o-mo de netw ork of extracurricular activities, with isolated studen ts omitted. The large blue circles represent the n e = 37 activities (even ts), and the small circles represent the studen ts (actors), where the colors of the plotting sym b ol indicate the students’ races. If a student participates in an activity , a line is dra wn connecting the studen t and activity . This figure w as constructed using functions in the iGraph R pac k age (Csardi and Nepusz, 2006). 16 Hispanic Asian blac k white All Num b er of studen ts 54 19 314 518 905 Num b er of extracurricular activities 28 17 30 35 36 P ercent Male 53.7 47.4 35.0 44.8 42.0 P ercent F emale 46.3 52.6 65.0 55.2 58.0 P ercent of participating students 84.2 70.4 70.7 73.0 72.3 T able 1: V arious summary statistics b y race. The second row is the num b er of extracurricular with members of each race. The last row is the p ercen t of students who participate in at least one extracurricular activity by race. W e fit our prop osed bilinear mixed-effects mo dels to the extracurricular affiliation netw ork dataset collected from Magnet High. There are no dyad-specific cov ariates in our analysis, so β d corresp onds to the intercept. W e use gender and race as the actor-sp ecific cov ariates. T aking “b o y” and “white” as the resp ective base categories for these t wo categorical co v ari- ates, w e hav e actor effect parameters β G a (“girl” effect), β B a (“blac k” effect), β A a (“Asian” effect), and β H a (“Hispanic” effect). Note that these effects are in terpreted relativ e to the baseline log o dds of an activity tie for white b o ys. The size of the clubs is the ev ent-specific co v ariate, with corresp onding effect parameter β e . F rom our inferences about the positioning of studen ts in the latent so cial space, w e examine the degree of interracial contact in high sc ho ol extracurricular clubs using techniques from p oin t pattern analysis. 4.2 Evidence of Higher-Order Dep endence Before w e fit the bilinear mixed-effect mo del, w e chec k the balance in the data and examine the dep endence patterns as describ ed in Section 2.2. F or this affiliation dataset, the fraction of ties b et w een actors and even ts is p 0 = 0 . 0705 (distribution is highly righ t skew ed) and the prop ortion of balanced cycles, p , is 0 . 8842. Under an assumption of indep endence of the actor/ev ent ties, the exp ected proportion of p ositiv e residual cycles is p 4 0 + (1 − p 0 ) 4 + 6 p 2 0 (1 − p 0 ) 2 = 0 . 7724. If w e randomly generate an n a × n e matrix with 7.05 p ercen t p ositiv e v alues 17 t 0 1 2 3 4 5 6 7 Lik eliho o d -5625 -5103 -4893 -4668 -4364 -4258 -4051 -3950 DIC 9985 9533 9485 9516 9437 9370 9287 9255 DIC alt 24841 25858 31395 30809 59157 76318 147347 208044 T able 2: DIC v alues for mo dels with v arying dimension of the comp onents of the bilinear term. (+1) and 92.95 p ercen t negative v alues (-1) 100 times, the greatest observed prop ortion of balanced cycles in the 100 matrices is 0 . 8697. Therefore, we conclude that the observed prop ortion of balanced cycles is significantly greater than exp ected under indep endence (p- v alue ≈ 0). 4.3 Priors Prior distributions for the random effect v ariances ( σ 2 a , σ 2 e , σ 2 u , and σ 2 v ) are taken to b e indep enden t and distributed as IG(1 , 1), where IG( a, b ) denotes the in verse gamma dis- tribution with shap e a > 0 and scale b > 0. The priors for β β β are normally distributed β β β ∼ MVN( 0 , I r × r ) . The v ariance of the prior distribution of β β β is small since w e are in the logistic regression setting. 4.4 Results The MCMC algorithm described in Section 3.2 was run for 150,000 iterations for v alues of t = 0 (no bilinear term) , 1 , 2 , 3 , 4 , 5 , 6 , and 7. T race plots suggest that the Marko v chain reac hes its stationary distribution w ell b efore 100,000 iterations, so we conserv ativ ely base our inferences on the last 50,000 iterations. DIC (Spiegelhalter et al., 2002b) and alternativ e DIC (Gelman et al., 2004) are used to assess our mo dels. The results corresp onding to differen t v alues of t are listed in T able 2. In terms of the DIC criterion, mo del fit generally impro ves as the dimension of the bilinear terms’ comp onen ts increases, and the largest decrease in 18 P arameters β d β G a β b a β A a β H a β e σ a σ e σ u σ v Mean -4.33 0.11 -0.13 -0.09 0.07 0.01 0.02 0.06 0.62 0.90 SD 0.29 0.08 0.08 0.16 0.24 0.003 0.001 0.01 0.34 0.41 lo wer 95% CI -4.91 -0.05 -0.29 -0.40 -0.39 0.01 0.01 0.03 -0.04 0.09 upp er 95% CI -3.75 0.28 0.02 0.23 0.53 0.02 0.02 0.09 1.29 1.72 T able 3: P osterior means and standard deviations of mo del parameters when t = 2. DIC o ccurs when t changes from 0 to 1. Ho w ever, the DIC alt with half the v ariance of the deviance as an estimate of the num b er of free parameters in the mo del keeps increasing and jumps considerably when t increases from 3 to 4. Based on the DIC and DIC alt and considering our ability to plot in t w o dimensions, w e c ho ose to rep ort inferences on the t = 2 mo del. T able 3 provides the p osterior mean and standard deviations for all scalar mo del parameters when t = 2. The 95 p ercen t credible in terv als of all actor-sp ecific co v ariate co efficien ts co ver 0, which implies that student extracurricular participation generally do es not app ear to dep end on gender and race. As exp ected, the relationship b et ween club size and the exp ected log o dds of participation is p ositiv e with E [ β β β e | Y ] = 0 . 01, implying that for ev ery additional mem b er, the o dds of a particular individual b eing in the club increases by one p ercen t (since e 0 . 01 = 1 . 01). After eliminating the effects of gender, race, and club size on log o dds of a student partici- pating in an activity , we explore the structure of the laten t so cial space through the bilinear term, uv 0 , whic h captures dep endence b et ween the students through common extracurricu- lar activity profiles. First, note that the dimension of the bilinear term uv 0 is n a × n e and that the bilinear mo del dep ends on u and v only through the inner pro ducts uv 0 , whic h is in v ariant under rotations and reflections of u and v . T o appropriately compare p osterior samples of u and v , we first rotate them to a common orientation using a “Pro crustean” transformation (Sibson, 1978) to a Mon te Carlo estimate of p osterior mean (sample av erage of the p osterior samples), then summarize our inferences b y the plot of the p osterior mean of 19 u and v after rotation whic h represents the p ositions of students and activities in the latent so cial space shown as sho wn in Figure 3. The structure of the latent so cial space can b e inv estigated b y examining the p osition of the activit y latent v ectors, v k , within this space. T o facilitate in terpretation in Figure 3, the extracurricular activities are colored based on the assigned categories of activities listed in App endix B. (Note that these categories w ere not used in the mo del fitting.) The p ositions of students are shown as p oin ts in the laten t so cial space. T riangles represen t male studen ts, and dots represent female students. The plotting sym b ol color for the students corresp onds race, where white, blac k, Asian and Hispanic studen ts are colored red, blac k, yello w, and blue, resp ectiv ely . F rom Figure 3, we can see that generally the activities corresp onding in the same category tend to b e lo cated nearby each other. F or example, w e see that Drill, Cheer, and Pep, the three activities in the Cheer category , are lo cated in the upp er left capturing the apparen t tendency of students to either participate in all or none of these activities. On the other hand, there are examples where the clubs do not cluster based on category . F or example, in Music category , Orchestra and Choir lo cate in the opp osite direction from Band indicating a lack of o v erlap in participants in these groups. F rom the distribution of the triangles and dots, w e see that males dominate the b ottom righ t quadran t of the plot, which makes in tuitive sense since the b o ys-only sp orts (F o otball, Baseball, and W resting) are oriented in this direction. W e can also see that girls tend to b e more active in the Service, News, and Cheer categories. 4.5 Racial Segregation T o detect patterns of racial segregation within the latent so cial space, w e use tec hniques from spatial p oin t pattern analysis. Here we consider the students as p oin ts in a compact subspace, D , defined to b e the 6 × 6 square centered at (0 , 0) in R 2 . Each p oin t in D has “mark” defined b y the corresp onding student’s race. In order to examine racial segregation 20 Figure 3: Plot of p osterior mean of the bilinear terms. The v s are shown as the lo cations of the extracurricular activity names with colors corresp onding to the categorization in Ap- p endix B, and the u s corresp onding to male and female students are represen ted as triangles and dots, resp ectiv ely . 21 of studen ts in terms of the extracurricular activity profiles, after controlling for the relative prop ensit y to participate in activities by race and gender, w e lo ok for evidence of clustering b y race in D . Our inv estigation of clustering by race is based on Ripley’s multi-t yp e K -function (Diggle, 2003, pages 123-124), where K r 1 r 2 ( h ) is defined as the exp ected num b er of students of race r 2 within a distance h of a t ypical studen t of race r 1 , divided by λ r 1 r 2 , where λ r 1 r 2 = λ r 1 + λ r 2 is sum of the in tegrated in tensity functions of p oin ts with marks r 1 and r 2 o ver D . F or a particular posterior realization, m , let u [ m ] i denote the p osition of student i in the latent activit y space after rotation. W e can estimate K r 1 r 2 ( h ) for this realization as ˆ K [ m ] r 1 r 2 ( h ) = 1 ˆ λ [ m ] r 1 r 2 × P n a i =1 P n a j =1 I ( r ( i ) = r 1 & r ( j ) = r 2 ) I ( | u [ m ] i − u [ m ] j | < h ) n r 1 + n r 2 = area( A ) ( n r 1 + n r 2 ) 2 × n a X i =1 n a X j =1 I ( r ( i ) = r 1 & r ( j ) = r 2 ) I ( | u [ m ] i − u [ m ] j | < h ) , where I ( · ) is the indicator function, r ( · ) returns the race of the student indexed b y the func- tion’s argumen t, n r 1 and n r 2 are the num b er of students of race r 1 and r 2 , resp ectiv ely . W e can estimate E [ K r 1 r 2 ( h ) | Y ] b y av eraging ov er the ˆ K [ m ] r 1 r 2 ( h ) p oin t-wise ov er h . Alternativ ely , to get a sense of p osterior v ariabilit y , Figure 4 sho ws ten p osterior multi-t yp e K -functions (in red) for eac h pair of races. (Note that the differences b et ween these red curves is difficult to discern due to the scale of the plots.) F or reference, w e also recomputed the estimated ˆ K [ m ] r 1 r 2 ( h ) under randomization (sampling without replacement) of the race of students with race either r 1 and r 2 , ten times for eac h m . These estimated multi-t yp e K -functions are sho wn in grey in Figure 4. F or all pairs of races, each of the p osterior estimated K -functions fall within the p oin t-wise as a function of h low er and upp er b ounds determined by the estimated K functions simulated under randomization, although the Hispanic curv es are near the upp er b ounds. Therefore, we do not find there to b e strong evidence of clustering b y race within the latent so cial space determined by extracurricular activity participation. Returning to Figure 2, we can see that in fact the Hispanic students do not app ear to b e distributed uniformly in this bipartite graphical summary of the data. Our w ork pro vides a 22 mo del-based metho dology for formally exploring whether or not studen ts are segregated by race in their extracurricular activit y profiles. Lastly , we note that these conclusions are not affected b y changing t . 5 Discussion Extracurricular activities pla y an imp ortan t role in shaping the so cial exp eriences of high sc ho ol students. Our con tribution is a statistical mo del that can allow a deep er exploration in to the patterns of racial and ethnic segregation in extracurricular activity participation within a school. Using a regression framework, we are able to con trol for differences in participation generally by b oth activity characteristics and attributes of the students. Our approac h allows us to mo ve b ey ond the examination of sp ecific activities separately . Through inferences on the bilinear comp onent of our mo del, w e are able to uncov er patterns in shared participation across multiple activities. Thus, unlike previous w ork which fo cuses on differ- ences in the amount of participation b y race (Clotfelter, 2002), our metho dology can unco ver differences in the ov erall patterning of participation b y race. Our prop osed mo del is based on a generalized linear mixed-effects mo del with the addition of an inner pro duct of tw o latent v ectors. As w e ha ve sho wn, this latent structure allows us to capture the types of dep endence patterns (balance) usually seen in affiliation netw orks. Our mo del impro v es on existing ad ho c metho ds for analyzing affiliation netw orks in that w e coheren tly capture the uncertain ty in our inferences on the latent structure of a net work. Visualizing this uncertaint y is a c hallenging task, and we plan to w ork on this important issue in future work. As is often the case for fitting complex Ba yesian mo dels using MCMC, computation can b e c hallenging when the sample size (num b er of actors and ev en ts) is large. Accordingly , future w ork will seek to exploit computational tricks to take adv an tage of sparsit y in affiliation net works, as w ell as explore approximate computation tec hniques. 23 Hispanic Asian black white white black Asian Hispanic Figure 4: Posterior samples of the m ulti-t yp e K -functions for pairs of races (red). References curv es (grey) are computed under randomization of race (see Section 4.4). 24 Lastly , the laten t structure of our mo del pro vides a natural mec hanism for incorp orating grouping structure in either the actors or ev ents. F or example, in the extracurricular activ- it y example, we could ha ve built in the exp ected similarity b etw een activities in the same category by placing a hierarc hical prior on the activity random effects and the comp onen ts of the v s. In this wa y , w e would b e able to explain the dep endences in affiliation net works b oth within and across different groups of ev ents. Ac kno wledgemen ts Supp ort for this w ork w as pro vided b y gran ts from the National Science F oundation (NSF DMS-1209161), the National Institutes of Health (NIH R01DA032371), the William T. Gran t F oundation, and The Ohio State Universit y Institute for Population Research (NIH P2CHD058484). References Besag, J. (2001). Markov chain Monte Carlo for statistic al infer enc e . Univ ersity of W ash- ington, Cen ter for Statistics and the So cial Sciences. Clotfelter, C. T. (2002). “Interracial contact in high sc ho ol extracurricular activities.” The Urb an R eview , 34. Congdon, P . D. (2010). Applie d b ayesian hier ar chic al metho ds . Chapman & Hall/CR C, Bo ca Raton, FL. Csardi, G. and Nepusz, T. (2006). “The igraph softw are pack age for complex netw ork researc h.” InterJournal , Complex Systems, 1695. Da vis, A. and Gardner, B. B. (1941). De ep south . Universit y of Chicago Press, Chicago, IL. 25 Diggle, P . J. (2003). Statistic al analysis of sp atial p oint p atterns . 198 Madison Aven ue, New Y ork, NY 10016: Oxford Universit y Press Inc. F rank, O. and Strauss, D. (1986). “Marko v graphs.” Journal of the A meric an Statistic al Asso ciation , 81, 832–842. Gelman, A., Carlin, J. B., Stern, H. S., and Rubin, D. B. (2004). Bayesian data analysis . New Y ork: Chapman and Hall. Grano vetter, M. (1986). “The micro-structure of sc ho ol desegregation.” In Scho ol Dese g- r e gation R ese ar ch: New Dir e ctions in Situational Analysis , eds. J. Prager, D. Longshore, and M. Seeman. New Y ork: Plenum Press. Green, P . J. (1995). “Rev ersible jump Marko v chain Mon te Carlo computation and Ba y esian mo del determination.” Biometrika , 82, 4, 711–732. Handco c k, M. S., Robins, G., Snijders, T., Mo o dy , J., and Besag, J. (2003). “Assessing degeneracy in statistical mo dels of so cial net works.” Journal of the A meric an Statistic al Asso ciation , 76, 33–50. Hoff, P . D. (2003). “Random effects mo dels for netw ork data.” In Dynamic So cial Network Mo deling and Analysis: Workshop Summary and Pap ers , 303–312. — (2005). “Bilinear mixed-effects mo dels for dyadic data.” Journal of the A meric an Statis- tic al Asso ciation , 100, 469, 286–295. — (2009). “A hierarchical eigenmo del for p o oled co v ariance estimation.” Journal of the R oyal Statistic al So ciety, Series B , 71, 971–992. Hoff, P . D., Raftery , A. E., and Handco c k, M. S. (2002). “Latent space approac hes to so cial net work analysis.” Journal of the Americ an Statistic al Asso ciation , 97, 1090–1098. Hun ter, D. R., Krivitsky , P . N., and Sc hw einberger, M. (2012). “Computational statistical metho ds for so cial netw ork mo dels.” Journal of Computational and Gr aphic al Statistics , 21, 856–882. 26 McF arland, D. A. (1999). “Organized behavior in so cial systems: a study of student en- gagemen t and resistance in high schools.” Ph.D. thesis, Universit y of Chicago, Chicago, IL. Mo ody , J. (2001). “Race, sc ho ol integration, and friendship segregation in America.” Journal of the R oyal Statistic al So ciety , 107, 3, 679–719. Newman, M. E. J. (2001). “Scientific collab oration netw orks. I. Netw ork construction and fundamen tal results.” Physic al R eview , 64, 016131, 1–8. No wak, M., W est woo d, S. J., Messing, S., and McF arland, D. (2012). “Netw ork data for McF arland’s SNA R labs.” P attison, P . and W asserman, S. (1999). “Logit mo dels and logistic regressions for so cial net works: I I. Multiv ariate relations.” British Journal of Mathematic al and Statistic al Psycholo gy , 52, 169–193. Robins, G., Pattison, P ., and W asserman, S. (1999). “Logit mo dels and logistic regressions for so cial netw orks: I I I. V alued Relations.” Psychometrika , 64, 371–394. Sibson, R. (1978). “Studies in the robustness of multidimensional scaling.” Journal of the R oyal Statistic al So ciety , 40, 234238. Snijders, T. A. B. (2001). “The statistical ev aluation of so cial netw ork dynamics.” So ciolo g- ic al Metho dolo gy , 31, 361–395. Snijders, T. A. B. and No wicki, K. (1997). “Estimation and prediction for sto chastic blo c k- mo dels for graphs with latent blo c k structure.” Journal of Classific ation , 14, 75–100. Snijders, T. A. B., Pattison, P . E., Robins, G. L., and Handco c k, M. S. (2006). “New sp ecifications for for exp onen tial random graph mo dels.” So ciolo gic al Metho dolo gy , 36, 1, 99–153. 27 Spiegelhalter, D. J., Best, N. G., Carlin, B. P ., and V an Der Linde, A. (2002a). “Bay esian measures of mo del complexit y and fit.” Journal of the R oyal Statistic al So ciety, Series B , 64, 4, 583–639. Spiegelhalter, D. J., Best, N. G., Carlin, B. P ., and v an der Linde, A. (2002b). “Bay esian measures of mo del complexity and fit.” Journal of the R oyal Statistic al So ciety , 64. v an Duijn, M. A., Gile, K. J., and Handco c k, M. S. (2009). “A framew ork for the comparison of maxim um pseudo likelihoo d and maximum likelihoo d estimation of exp onential family random graph mo dels.” So cial Networks , 31, 52–62. W ang, P ., Sharp e, K., Robins, G. L., and P attison, P . E. (2009). “Exp onen tial random graph mo dels for affiliation netw orks.” So cial Networks , 31, 12–25. W ang, Y. J. and W ong, G. Y. (1987). “Sto c hastic blo c kmo dels for directed graphs.” Journal of the A meric an Statistic al Asso ciation , 82, 397, 8–19. W asserman, S. and F aust, K. (1994). So cial network analysis: metho ds and applic ations . Cam brige Universit y Press, Cambrige. W asserman, S. and P attison, P . (1996). “Logit mo dels and logistic regression for so cial net works: I. An introduction to Marko v graphs and p*.” Psychometrika , 61, 401–425. Zijlstra, B. J. H., v an Duijn, M. A. J., and Snijders, T. A. B. (2009). “MCMC estimation for the p2 net w ork regression mo del with crossed random effects.” British Journal of Mathematic al and Statistic al Psycholo gy , 62, 143–166. 28 APPENDICES A F ull Conditional Distributions F ull conditional distribution of ( β β β d , µ µ µ a , µ µ µ e ) The full conditional distribution of ( β β β d , µ µ µ a , µ µ µ e ) is prop ortional to p ( z | β β β d , µ µ µ a , µ µ µ e , σ 2 γ ) × p ( µ µ µ a , µ µ µ e | β β β a , β β β e , σ 2 a , σ 2 e ) × p ( β β β d ) . Assume the prior distribution of β β β d follo ws a multiv ariate normal distribution β β β d ∼ MVN( µ µ µ β β β d , Σ Σ Σ β β β d ) . W e already kno w that µ a i = β β β 0 a x a i + a i and µ e k = β β β 0 e x e k + e k . So, w e hav e µ µ µ a , µ µ µ e | β β β a , β β β e , σ 2 a , σ 2 e ∼ MVN( X ae β β β ae , Σ Σ Σ ae ) , where X ae is a ( n a + n e ) × 2 matrix, β β β ae = ( β β β a , β β β e ) 0 , and Σ Σ Σ ae = σ 2 a I n a 0 0 σ 2 e I n e ! . Since β β β d , µ µ µ a , µ µ µ e are indep enden t and Gaussian, we can rewrite their joint distribution as β β β d , µ µ µ a , µ µ µ e | β β β a , β β β e , σ 2 a , σ 2 e ∼ MVN " µ µ µ β β β d X ae β β β ae ! , Σ Σ Σ β β β d 0 0 Σ Σ Σ ae !# . Let z ik = θ ik − u 0 i v k = β β β 0 d x d ik + µ a i + µ e k + γ ik so z | β β β d , µ µ µ a , µ µ µ e , σ 2 γ ∼ MVN    X D    β β β d µ µ µ a µ µ µ e    , σ 2 γ I n γ    It follows that the full conditional distribution ( β β β d , µ µ µ a , µ µ µ e ) is multiv ariate normal with the follo wing mean and cov ariance: Σ Σ Σ = " Σ Σ Σ − 1 β β β d 0 0 Σ Σ Σ − 1 ae ! + X 0 D X D /σ 2 γ # − 1 29 µ µ µ = Σ Σ Σ " Σ Σ Σ − 1 β β β d β β β d Σ Σ Σ − 1 ae X ae β β β ae ! + X 0 D z /σ 2 γ # F ull conditional distribution of ( β β β a , β β β e ) The full conditional distribution of ( β β β a , β β β e ) is prop ortional to p ( µ µ µ a , µ µ µ e | β β β a , β β β e , σ 2 a , σ 2 e ) × p ( β β β a , β β β e ) . Assume the combined regression parameter has a multiv ariate normal prior: ( β β β a , β β β e ) ∼ MVN( µ µ µ β β β ae , Σ Σ Σ β β β ae ). Therefore, the full conditional is a multiv ariate normal distribution with the following mean and v ariance: Σ Σ Σ =  X 0 ae Σ Σ Σ − 1 ae X ae + Σ Σ Σ − 1 β β β ae  − 1 µ µ µ = Σ Σ Σ " X 0 ae Σ Σ Σ − 1 ae µ µ µ a µ µ µ e ! + Σ Σ Σ − 1 β β β ae µ µ µ β β β ae # . F ull conditional distribution of σ 2 a , σ 2 e W e restrict Σ Σ Σ a = σ 2 a I n a × n a and Σ Σ Σ e = σ 2 e I n e × n e . F or σ 2 a ∼ IG( α a 1 , α a 2 ), and σ 2 e ∼ IG( α e 1 , α e 2 ), the full conditionals are indep endent and σ 2 a | µ µ µ a ∼ IG( n a / 2 + α a 1 , α a 2 + ( µ µ µ a − X a β β β a ) 0 ( µ µ µ a − X a β β β a ) / 2) and σ 2 e | µ µ µ e ∼ IG( n e / 2 + α e 1 , α e 2 + ( µ µ µ e − X e β β β e ) 0 ( µ µ µ e − X e β β β e ) / 2) . 30 F ull conditional distribution of σ 2 γ W e restrict Σ Σ Σ γ = σ 2 γ I n γ × n γ . Using prior distribution of σ 2 γ ∼ IG( α γ 1 , α γ 2 ), the full conditional distribution of σ 2 γ is σ 2 γ | β β β d , µ µ µ a , µ µ µ a , z ∼ IG    α γ 1 + n γ / 2 , α γ 2 +    z − X D    β β β d µ µ µ a µ µ µ e       0    z − X D    β β β d µ µ µ a µ µ µ e       / 2    F ull conditional distribution of u i Let θ ik = β β β 0 d x d ik + µ a i + µ e k + γ ik + u 0 i v k , as b efore, and ˆ θ ik = E ( θ ik | β β β d , µ a i , µ e k , x d ik ) = β β β 0 d x d ik + µ a i + µ e k . Then e ik = θ i,k − ˆ θ ik = v k 0 u i + γ γ γ i , Considering the full conditional of u i , w e hav e      e u i, 1 e u i, 2 . . . e u i,n e      | {z } e u i =      v 1 v 2 . . . v n e      | {z } v u i +      γ i, 1 γ i, 2 . . . γ i,n e      | {z } γ γ γ u i , so e e e u i | v , u i , σ 2 γ ∼ MVN( vu i , σ 2 γ I n e ). Therefore, sampling u i from its full conditional is equiv- alen t to a Bay esian linear regression problem. Assuming the u i s are a priori indep enden t and u i ∼ MVN(0 , Σ Σ Σ u ) , the full conditional of u i is m ultiv ariate normal with Σ Σ Σ = ( Σ Σ Σ − 1 u + v 0 v /σ 2 γ ) − 1 and µ µ µ = Σ Σ Σ v 0 e u i /σ 2 γ . F ull conditional distribution of v k Similar to the deriv ation of the full conditional distribution of v i , we hav e e ik = θ ik − ˆ θ ik = u 0 i v k + γ γ γ i . 31 Considering the full conditional of v k , w e hav e      e 1 ,k e 2 ,k . . . e n a ,k      | {z } e v k =      u 1 u 2 . . . u n a      | {z } u v k +      γ 1 ,k γ 2 ,k . . . γ n a ,k      | {z } γ γ γ v k e e e v k | u , v i , σ 2 γ ∼ MVN( uv k , σ 2 γ I n a ). Therefore, sample v k from its full conditional is also equiv- alen t to a Ba y esian linear regression problem. Assuming the v k s are a priori indep enden t and eac h v k ∼ MVN(0 , Σ Σ Σ v ) , . the full conditional of v k is m ultiv ariate normal with Σ Σ Σ = ( Σ Σ Σ − 1 v + u 0 u /σ 2 γ ) − 1 and µ µ µ = Σ Σ Σ u 0 e v k /σ 2 γ . F ull conditional distribution of σ 2 u , σ 2 v W e restrict Σ Σ Σ u = σ 2 u I t × t and Σ Σ Σ v = σ 2 v I t × t and let σ 2 u ∼ IG( α u 1 , α u 2 ) and σ 2 v ∼ IG( α v 1 , α v 2 ). Then the full conditionals are σ 2 u | u ∼ IG( n a t/ 2 + α u 1 , α u 2 + trace( u 0 u ) / 2) and σ 2 v | v ∼ IG( n e t/ 2 + α v 1 , α v 2 + trace( v 0 v ) / 2) . 32 B Data Pro cessing In our analysis of McF arland (1999)’s data for y ear 1996, w e collapsed sev eral extracurricular activit y categories as describ ed b elo w. F or example, our club “Pep” includes b oth mem b ers of “P ep.Club” and “P ep.Club.Officers.” In addition, we group ed the activities in to one of eigh t t yp es lab eled as 1-8 b elo w. These groups were not used in fitting our mo del, but w ere helpful in in terpreting our fitted mo del. 1. Language (a) Asian (b) Sp anish includes Hispanic.Club, Spanish.Club, Spanish.Club..high.,Spanish.NHS (c) L atin (d) F r ench includes F rench.Club..lo w., F rench.Club..high., F rench.NHS (e) German includes German.Club, German.NHS 2. Academic Comp etition (a) Deb ate (b) F or ensics includes F orensics, F orensics..National.F orensics.League. (c) Chess (d) Scienc e.Olympiad (e) Quiz.Bow l (f ) A c ademic.De c athalon 3. News (a) Newsp ap er (b) Y e arb o ok includes Y earb o ok.Con tributors, Y earb o ok.Staff 33 4. Cheer (a) Pep includes Pep.Club, P ep.Club.Officers (b) Dril l (c) Che er includes Cheerleaders..8th, Cheerleaders..9th, Cheerleaders..Spirit.Squad, Cheerleaders..JV, Cheerleaders..V 5. Service (a) National Honor So ciety (b) Drunk.Driving includes Drunk.Driving, Drunk.Driving.Officers (c) Key 6. Art/Theater (a) A rt (b) The atr e (c) Thespian 7. Music (a) Band includes Band..8th, Band..Marching..Symphonic., Band..Jazz (b) Or chestr a includes Orc hestra..8th, Orc hestra..F ull.Concert, Orc hestra..Symphonic (c) Choir includes Choir..treble, Choir..concert, Choir..women.s.ensem ble, Choir..a.capella, Choir..c hamber.singers, Choir..v o cal.ensem ble..4.women., Choir..barb ershop.quartet..4.men. 8. Sports (a) F o otb al l includes F o otball..8th, F o otball..9th, F o otball..V (b) So c c er (c) V ol leyb al l includes V olleyball..8th, V olleyball..9th, V olleyball..JV, V olleyball..V 34 (d) Basketb al l includes Basketball..boys.8th, Basketball..boys.9th, Basketball..boys.JV, Bask etball..b o ys.V, Basketball..girls.8th, Basketball..girls.9th, Basketball..girls.JV, Bask etball..girls.V (e) Baseb al l includes Baseball..JV..10th., Baseball..V (f ) Softb al l includes Softball..JV..10th., Softball..V (g) Cr oss.Country includes Cross.Country ..b o ys.8th, Cross.Coun try ..girls.8th , Cross.Coun try ..b oys. V, Cross.Coun try ..girls.V (h) Golf (i) Swim includes Swim...Dive.T eam..boys, Swim...Dive.T eam..girls (j) T ennis includes T ennis..b o ys.V, T ennis.girls.V (k) T r ack includes T rack..boys.8th, T rac k..girls.8th, T rack..boys.V, T rack..girls.V (l) Wr estling includes W restling..8th, W restling..V 35 C Club affiliations b y race Figure C.1: Bar plot of the student club affiliations by race. 36