Array Variate Elliptical Random Variables with Multiway Kronecker Delta Covariance Matrix Structure

Arra y V ariate Elliptical Random V ariables wi th Multiw a y Kronec k er Delta Co v ariance Matrix Structure Deniz Akdemir Department of Statistics Universit y of Central Florida Orlando, FL 32816 August 13, 2018 Abstract Standard statistical metho ds applied to matrix random v a riables often fail to describ e the underlying structure in m ultiwa y data sets. In this pap er we will d iscuss the concept of an arra y vari ate random v ariable and introduce a class of elliptical arra y den sities which hav e elliptical contours. 1 In tro du ction The array v ariate ra ndom v ariable up to 2 dimensions has b een studied in ten- sively in [3] and by many others. F or a rrays observ ations of 3 , 4 or in g eneral i dimensions a suitable nor mal probablity mo de l has b een recent ly pr o p osed in [1]. In this pap er w e will gener alize the notion o f elliptical random v aria bles to the a r ray case. In Se c tio n 2, we first study the algebra of ar r ays. In Sectio n 3, we in tro duce the concept o f an array v ariable ra ndom v ar iable. In s e ction 4, the densit y of the elliptical array r andom v ariable is provided. Then, in Section 5, w e define the a r ray rando m v ariable with elliptica l density . 2 Arra y A lgebra In this pap er we will only study arrays with real elements. W e will write e X to say tha t e X is an ar ray . When it is necessary w e ca n write the dimensions of the a rray as subindices, e.g., if e X is a m 1 × m 2 × m 3 × m 4 dimensional a rray in R m 1 × m 2 × ... × m i , then w e can write e X m 1 × m 2 × m 3 × m 4 . Arrays w ith the us ual element wise summation and scalar m ultiplication oper a tions can b e sho wn to be a v ector space. 1 T o r efer to an element o f an array e X m 1 × m 2 × m 3 × m 4 , we write the p os itio n of the element as a subindex to the array na me in pa r enthesis, ( e X ) r 1 r 2 r 3 r 4 . If we wan t to refer to a sp ecific column vector obtained by keeping all but an indica ted dimension constant, we indicate the consta nt dimensions as b efor e but we will put ’:’ for the non constant dimension, e.g., for e X m 1 × m 2 × m 3 × m 4 , ( e X ) r 1 r 2 : r 4 refers to the the column vector (( X ) r 1 r 2 1 r 4 , ( X ) r 1 r 2 2 r 4 , . . . , ( X ) r 1 r 2 m 3 r 4 ) ′ . W e will now revie w s ome basic principles and tec hniques of array alg ebra. These res ults and their pro ofs ca n b e found in Rauhala [4], [5] a nd Bla ha [2]. Definition 2.1 . Inv erse K roneck er pr o duct of two matric es A and B of dimen- sions p × q and r × s c orr esp ondingly is written as A ⊗ i B and is define d as A ⊗ i B = [ A ( B ) j k ] pr × qs = B ⊗ A, wher e ′ ⊗ ′ r epr esents the or dinary Kr one cker pr o du ct . The following prop er ties of the inv er se Kronecker pro duct ar e useful: • 0 ⊗ i A = A ⊗ i 0 = 0 . • ( A 1 + A 2 ) ⊗ i B = A 1 ⊗ i B + A 2 ⊗ i B . • A ⊗ i ( B 1 + B 2 ) = A ⊗ i B 1 + A ⊗ i B 2 . • αA ⊗ i β B = αβ A ⊗ i B . • ( A 1 ⊗ i B 1 )( A 2 ⊗ i B 2 ) = A 1 A 2 ⊗ i B 1 B 2 . • ( A ⊗ i B ) − 1 = ( A − 1 ⊗ i B − 1 ) . • ( A ⊗ i B ) + = ( A + ⊗ i B + ) , wher e A + is the Mo or e -Penrose inv erse of A. • ( A ⊗ i B ) − = ( A − ⊗ i B − ) , where A − is the l -inv erse of A defined as A − = ( A ′ A ) − 1 A ′ . • If { λ i } and { µ j } are the eigenv alues with the co rresp onding eigenv ectors { x i } and { y j } for ma tr ices A a nd B resp ectively , then A ⊗ i B ha s eigen- v alues { λ i µ j } with corresp o nding eigenv ectors { x i ⊗ i y j } . • Given t wo matrices A n × n and B m × m | A ⊗ i B | = | A | m | B | n , tr ( A ⊗ i B ) = tr ( A ) tr ( B ) . • A ⊗ i B = B ⊗ A = U 1 A ⊗ B U 2 , for some p ermutation matrices U 1 and U 2 . It is well k nown that a ma tr ix equatio n AX B ′ = C can b e rewritten in its monolinear form as A ⊗ i B v ec ( X ) = v ec ( C ) . (1) F urthermore, the matrix e quality A ⊗ i B X C ′ = E 2 obtained by stacking equa tions o f the form (1) can b e wr itten in its monolinear form as ( A ⊗ i B ⊗ i C ) ve c ( X ) = v ec ( E ) . This pro cess of s tacking equations could be contin ued and R-matrix multiplica- tion oper a tion introduced by Rauhala [4] provides a co mpact wa y of r epresenting these equatio ns in a rray form: Definition 2.2. R-Matrix Multiplica tion is define d elementwise: (( A 1 ) 1 ( A 2 ) 2 . . . ( A i ) i e X m 1 × m 2 × ... × m i ) q 1 q 2 ...q i = m 1 X r 1 =1 ( A 1 ) q 1 r 1 m 2 X r 2 =1 ( A 2 ) q 2 r 2 m 3 X r 3 =1 ( A 3 ) q 3 r 3 . . . m i X r i =1 ( A i ) q i r i ( e X ) r 1 r 2 ...r i . R-Matrix multiplication generalizes the matrix m ultiplication (array mult i- plication in tw o dimensions)to the case of k -dimensio nal arrays. The following useful pr op erties of the R-Ma trix multiplication are reviewed by Blaha [2]: 1. ( A ) 1 B = AB . 2. ( A 1 ) 1 ( A 2 ) 2 C = A 1 C A ′ 2 . 3. e Y = ( I ) 1 ( I ) 2 . . . ( I ) i e Y . 4. (( A 1 ) 1 ( A 2 ) 2 . . . ( A i ) i )(( B 1 ) 1 ( B 2 ) 2 . . . ( B i ) i ) e Y = ( A 1 B 1 ) 1 ( A 2 B 2 ) 2 . . . ( A i B i ) i e Y . The o p e rator r v ec describ es the relatio nship betw een e X m 1 × m 2 × ...m i and its monolinear for m x m 1 m 2 ...m i × 1 . Definition 2 .3. r v ec ( e X m 1 × m 2 × ...m i ) = x m 1 m 2 ...m i × 1 wher e x i s the c olumn ve ctor obtaine d by st acking t he element s of the arr ay e X in the or der of its dimensions; i. e., ( e X ) j 1 j 2 ...j i = ( x ) j wher e j = ( j i − 1) n i − 1 n i − 2 . . . n 1 + ( j i − 2) n i − 2 n i − 3 . . . n 1 + . . . + ( j 2 − 1) n 1 + j 1 . Theorem 2.1. L et e L m 1 × m 2 × ...m i = ( A 1 ) 1 ( A 2 ) 2 . . . ( A i ) i e X wher e ( A j ) j is an m j × n j matrix for j = 1 , 2 , . . . , i and e X is an n 1 × n 2 × . . . × n i arr ay. Write l = r v ec ( e L ) and x = rv e c ( e X ) . Then, l = A 1 ⊗ i A 2 ⊗ i . . . ⊗ i A i x . Therefore, there is an eq uiv alent e xpression of the arr ay equa tion in mono- linear for m. Definition 2.4. The squar e norm of e X m 1 × m 2 × ...m i is define d as k e X k 2 = m 1 X j 1 =1 m 2 X j 2 =1 . . . m i X j i =1 (( e X ) j 1 j 2 ...j i ) 2 . Definition 2.5 . The distanc e of f X 1 m 1 × m 2 × ...m i fr om f X 2 m 1 × m 2 × ...m i is define d as q k f X 1 − f X 2 k 2 . Example 2.1. L et e Y = ( A 1 ) 1 ( A 2 ) 2 . . . ( A i ) i e X + e E . Then k e E k 2 is minimize d for b e X = ( A − 1 ) 1 ( A − 2 ) 2 . . . ( A − i ) i e Y . 3 3 Arra y V ari ate Random V ariables Arrays can be consta nt arr ays, i.e. if ( e X ) r 1 r 2 ...r i ∈ R are constant s for a ll r j , j = 1 , 2 , . . . , m j and j = 1 , 2 , . . . , i then the array e X is a c onstant a rray . Array v ariate random v ariables are arrays with all elements ( e X ) r 1 r 2 ...r i ∈ R random v a riables. If the sample space for the r a ndom outcome s is S , ( e X ) r 1 r 2 ...r i = ( e X ( s )) r 1 r 2 ...r i where each of ( e X ( s )) r 1 r 2 ...r i is a r e al v a lued function from S to R . If e X is an array v ariate ra ndom v ariable, its densit y (if it exists ) is a scalar function f e X ( e X ) such that: • f e X ( e X ) ≥ 0; • R e X f e X ( e X ) d e X = 1; • P ( e X ∈ A ) = R A f e X ( e X ) d e X , where A is a subset of the space of realizatio ns for e X . A scala r function f e X , e Y ( e X , e Y ) defines a joint (bi-array v aria te) probability density function if • f e X , e Y ( e X , e Y ) ≥ 0; • R e Y R e X f e X , e Y ( e X , e Y ) d e X d e Y = 1; • P (( e X , e Y ) ∈ A ) = R R A f e X , e Y ( e X , e Y ) d e X d e Y , where A is a subset of the spa ce of r ealizations for ( e X , e Y ) . The marg inal pro bability density function of e X is defined b y f e X ( e X ) = Z e Y f e X , e Y ( e X , e Y ) d e Y , and the conditional pr obability de ns it y function of e X g iven e Y is defined b y f e X | e Y ( e X | e Y ) = f e X , e Y ( e X , e Y ) f e Y ( e Y ) , where f e Y ( e Y ) > 0 . Two r andom ar rays e X a nd e Y are indep endent if and only if f e X , e Y ( e X , e Y ) = f e X ( e X ) f e Y ( e Y ) . Theorem 3.1. L et ( A 1 ) 1 , ( A 2 ) 2 , . . . , ( A i ) i b e m 1 , m 2 , . . . , m i dimensional p ositive definite matr ic es. The Jac obian J ( e X → e Z ) of t he tr ansformation e X = ( A 1 ) 1 ( A 2 ) 2 . . . ( A i ) i e Z + f M is ( | A 1 | Q j 6 =1 m j | A 2 | Q j 6 =2 m j . . . | A i | Q j 6 = i m j ) − 1 . 4 Pr o of . The result is proven using the equiv alence of mono linear form obtained through the r v ec ( e X ) and ar ray e X . Let e L m 1 × m 2 × ...m i = ( A 1 ) 1 ( A 2 ) 2 . . . ( A i ) i e Z where ( A j ) j is an m j × n j matrix for j = 1 , 2 , . . . , i a nd e X is an n 1 × n 2 × . . . × n i array . W rite l = rv ec ( e L ) and z = r v e c ( e Z ) . Then, l = A 1 ⊗ i A 2 ⊗ i . . . ⊗ i A i z . The result follows from no ting that J ( l → z ) = | A 1 ⊗ i A 2 ⊗ i . . . ⊗ i A i | − 1 , and using in- duction with the rule | A ⊗ i B | = | A | m | B | n for n × n matrix A and m × m matrix B to show tha t | A 1 ⊗ i A 2 ⊗ i . . . ⊗ i A i | − 1 = ( | A 1 | Q j 6 =1 m j | A 2 | Q j 6 =2 m j . . . | A i | Q j 6 = i m j ) − 1 . Corollary 3. 1. L et e Z ∼ f e Z ( e Z ) . D efi ne e X = ( A 1 ) 1 ( A 2 ) 2 . . . ( A i ) i e Z + f M wher e ( A 1 ) 1 , ( A 2 ) 2 , . . . , ( A i ) i b e m 1 , m 2 , . . . , m i dimensional p ositive definite m atric es. The p df of e X is given by f e X ( e X ; ( A 1 ) 1 , ( A 2 ) 2 , . . . , ( A i ) i , f M ) = f ( A − 1 1 ) 1 ( A − 1 2 ) 2 . . . ( A − 1 i ) i ( e X − f M )) | A 1 | Q j 6 =1 m j | A 2 | Q j 6 =2 m j . . . | A i | Q j 6 = i m j . The main adv an tage in cho osing a K roneck er s tr ucture is the decrea se in the nu mber of pa rameters. 4 Arra y V ari ate Normal Distribution By using the results in the pr evious section on arr ay algebra, mainly the rela - tionship of the arr ays to their monolinear forms describ ed by Definition 2.3 , we can write the density of the standar d nor ma l array v ariable. Definition 4.1. If e Z ∼ N m 1 × m 2 × ... × m i ( f M = 0 , Λ = I m 1 m 2 ...m i ) , then e Z has arr ay variate standar d normal distribution. The p df of e Z is given by f e Z ( e Z ) = exp ( − 1 2 k e Z k 2 ) (2 π ) m 1 m 2 ...m i / 2 . (2) F or the scalar case, the density for the standard nor ma l v ariable z ∈ R 1 is given a s φ 1 ( z ) = 1 (2 π ) 1 2 exp ( − 1 2 z 2 ) . F or the m 1 dimensional standar d no rmal vector z ∈ R m 1 , the density is given by φ m 1 ( z ) = 1 (2 π ) m 1 2 exp ( − 1 2 z ′ z ) . Finally the m 1 × m 2 standard matrix v ariate v ariable Z ∈ R m 1 × m 2 has the density φ m 1 × m 2 ( Z ) = 1 (2 π ) m 1 m 2 2 exp ( − 1 2 trace ( Z ′ Z )) . With the ab ove definition, we hav e g e neralized the notion of nor mal random v ariable to the ar r ay v ariate cas e. 5 Definition 4.2. We write e X ∼ N m 1 × m 2 × ... × m i ( f M , Λ m 1 m 2 ...m i ) if r v ec ( e X ) ∼ N m 1 m 2 ...m i ( rv e c ( f M ) , Λ m 1 m 2 ...m i ) . Her e, f M is the exp e cte d value of e X , and Λ m 1 m 2 ...m i is the c ova rianc e matrix of the m 1 m 2 . . . m i -variate ra ndom variable r v ec ( e X ) . The fa mily of normal densities with Kr oneck er Delta Co v ariance Structure are obtained b y co nsidering the densities obtained b y the lo cation-sca le trans- formations of the standard normal v aria bles. This kind o f mo de l is defined in the next. Theorem 4 .1. L et e Z ∼ N m 1 × m 2 × ... × m i ( f M = 0 , Λ = I m 1 m 2 ...m i ) . Define e X = ( A 1 ) 1 ( A 2 ) 2 . . . ( A i ) i e Z + f M wher e A 1 , A 2 , . . . , A i ar e non singular matric es of or ders m 1 , m 2 , . . . , m i . T hen the p df of e X is given by φ ( e X ; f M , A 1 , A 2 , . . . A i ) = exp ( − 1 2 k ( A − 1 1 ) 1 ( A − 1 2 ) 2 . . . ( A − 1 i ) i ( e X − f M ) k 2 ) (2 π ) m 1 m 2 ...m i / 2 | A 1 | Q j 6 =1 m j | A 2 | Q j 6 =2 m j . . . | A i | Q j 6 = i m j . (3) Pr o of . The result is easily obtained using the densit y in D efinition 4.2 with Corollar y 3 .1. 5 Arra y V ari ate Elliptical Distribution A spher ically symmetric r a ndom v ector x has a density of the form f ( x ′ x ) for some kernel p df f ( t ) , defined for t ∈ R + . A stochastic representation for the random v ector x is giv en b y x d = r u , where r is a nonnegative random v aria ble with cdf K ( r ) that is indep endent of u which is uniformly dis tributed over the unit spher e in R k , deno ted by S k , where r d = p ( x ′ x ) , u d = x √ ( x ′ x ) . If b oth K ′ ( r ) = k ( r ) ∈ K the pdf of r , and f ( x ′ x ) the pdf of x exist, then they are r elated a s follows: k ( r ) = 2 π k 2 Γ( k 2 ) r k − 1 f ( r 2 ) . (4) Some examples follow: 1. The standard m ultiv ariate no rmal N k ( 0 k , I k ) distr ibutio n with pdf φ k ( x ) = 1 (2 π ) k/ 2 e − 1 2 x ′ x . 2. The multiv ariate spherical t dis tribution with v degr ees of freedom with density function f ( x ) = Γ( 1 2 ( v + k )) Γ( 1 2 v )( v π ) k/ 2 1 (1 + 1 v x ′ x ) ( v + k ) / 2 . 6 3. When v = 1 the m ultiv ariate t distribution is called spherical Cauch y distribution. The a rray v aria te rando m v ariable with spherical con tour s is defined using the relationship betw een the array and its monolinear for m. This amoun ts to saying that the density of the arr ay v ariable e X is spheric a l if and only if it ca n be written in the for m f ( k e X k 2 ) for some kernel p df f ( t ) , defined for t ∈ R + . The elliptically contoured random v a riables with unstructured cov a riance matrices can b e obtained by apply ing a linea r tra ns formation to the mo nolinear for m of an ar ray v ariate spher ical r andom v ariable. In general w e can define an arr ay v ar iable r andom v ariable with unstructured cov aria nce matrix, i.e. w e ca n say e X has a rray v ar iate distribution if r v e c ( e X ) has elliptical distribution. Array v ariate densities with elliptical contours tha t have Kronecker delta cov ariance structure are also ea s ily constructed using Co rrola ry 3.1. Definition 5.1. L et A 1 , A 2 , . . . , A i b e non singular matric es with or ders m 1 , m 2 , . . . , m i and f M b e a m 1 × m 2 × . . . × m i c onstant arr ay. Also , let m = m 1 m 2 . . . m i . Then the p df of an m 1 × m 2 × . . . × m i el liptic al ly c ontour e d arr ay variate r andom variable e X with kernel p df f ( t ) , t > 0 is f e X ( e X ; ( A 1 ) 1 , ( A 2 ) 2 , . . . , ( A i ) i , f M ) = f ( k ( A − 1 1 ) 1 ( A − 1 2 ) 2 . . . ( A − 1 i ) i ( e X − f M ) k 2 ) | A 1 | Q j 6 =1 m j | A 2 | Q j 6 =2 m j . . . | A i | Q j 6 = i m j . F or ex ample, the following definition provides a g eneralizatio n of the multi- v ariate t dis tribution to the a rray v ariate case. Definition 5.2. L et A 1 , A 2 , . . . , A i b e non singular matric es with or ders m 1 , m 2 , . . . , m i and f M b e a m 1 × m 2 × . . . × m i c onstant arr ay. Also , let m = m 1 m 2 . . . m i . Then the p df of an m 1 × m 2 × . . . × m i arr ay varia te t r andom variable e T with de gr e es of fr e e dom v is given by f ( e T ; f M , A 1 , A 2 , . . . A i ) = c (1 + k ( A − 1 1 ) 1 ( A − 1 2 ) 2 . . . ( A − 1 i ) i ( e T − f M ) k 2 ) − ( v + m ) / 2 | A 1 | Q j 6 =1 m j | A 2 | Q j 6 =2 m j . . . | A i | Q j 6 = i m j (5) wher e c = ( vπ ) m/ 2 Γ(( v + m ) / 2) Γ( v/ 2) . Distributional prop er ties of a arr ay normal v a riable with density in the form of given b y Theorem 4.1 or a arr ay v aria te elliptical ra ndom v ariable with densit y of the form in (5.1) ca n o btained by using the equiv alent monolinear repres en- tation o f the rando m v ariable. The momen ts, the marginal and conditiona l distributions, independence of v ar iates can b e studied cons idering the equiv- alent monolinear form o f the array v ariable and the well kno wn pr op erties o f the multiv aria te normal random v aria ble and the multiv a r iate elliptical random v ariable. 7 References [1] D. Akdemir a nd A.K . Gupta. Arra y V ariate Rando m V a riables with Multi- wa y Kroneck er Delta Cov a riance Matrix Struc tur e. T e chnic al R ep ort , 11 (2), 2011. [2] G. Blaha. A F ew Basic Principles and T echniques of Array Algebra. Journal of Ge o desy , 51 (3):177– 202, 1 977. [3] A.K. Gupta and D.K. Nagar . Matrix V ariate Distributions . Chapman & Hall CRC Monog raphs and Surveys in Pure and Applied Mathematics. Chapman & Hall, 2000 . [4] U.A. Rauhala . Arr ay A lgebr a with Applic ations in Photo gr ammetry and Ge o desy . Divisio n of P hotogra mmetry , Roy al Institute of T echnology , 1974. [5] U.A. Rauhala. Intro duction to Array Algebra. Photo gr ammetric Engine ering and R emote Sensing , 46(2):177– 192, 198 0. 8