A multivariate generalization of Costas entropy power inequality

A multi v ariate generaliza tion of Costa’ s entrop y po wer inequality Miquel Payar ´ o and Daniel P . Palomar Departmen t of E lectronic an d Com puter En gineerin g, Hong Kong University of Science an d T e chnolog y , Clear W ater Bay , K owloon, Hong Kong. { miquel.pa yaro, palomar } @ust.hk Abstract — A simple multivariate v ersion of Costa’ s entro py power inequ ality is p ro ved. In particul ar , it i s shown that if independ ent white Gaussian noise is added to an arbitrary multivar iate signal, th e entropy power of the resulting random variable is a mul tidimensional concav e function of the individual variances of the components of the signal. As a side result, we also giv e an expression for the Hessian matrix of the entropy and entropy power fu nctions with respect to the varia nces o f th e signal components, which i s an interesting result in i ts own right. I . I N T R O D U C T I O N The entropy power of t he rand om v ector Y ∈ R n was first introdu ced b y Shann on in his seminal work [1] and is, since then, defined as N ( Y ) = 1 2 π e exp  2 n h ( Y )  , (1) where h ( Y ) r epresents the differential entropy , which, for continuo us random vecto rs read s as 1 h ( Y ) = E  log 1 P Y ( Y )  . For th e case where the distribution of Y assigns positiv e mass to one or more singletons in R n , the above definition is extend ed with h ( Y ) = −∞ . The entropy power of a random vector Y represents the variance (or power) of a standard Gaussian random vector Y G ∼ N  0 , σ 2 I n  such that both Y an d Y G have identical differential entropy , h ( Y G ) = h ( Y ) . A. Shann on’s entr opy po wer in equality (E PI) For any two indep endent ar bitrary rando m vector s X ∈ R n and W ∈ R n , Shannon g ave in [1] the fo llowing in equality: N ( X + W ) ≥ N ( X ) + N ( W ) . The first rigorous proof o f Shannon ’ s EPI was gi ven in [2] by Stam, and was sim plified by Blachman in [3]. A simple and very elegant pro of by V erd ´ u and Guo based on estimation theoretic consideration s has recen tly appear ed in [4]. Among m any oth er impor tant results, Bergmans’ pr oof of the converse for the degraded Gaussian bro adcast chann el [ 5] and Oohama ’ s partial solution to the rate distortio n regio n problem for Gaussian multiterm inal source co ding systems [6 ] follow from Sh annon ’ s EPI. 1 Throughout this paper we work with natural logarithms. B. Costa’ s EPI Under the setting of Shan non’ s EPI, Costa proved in [7] that, provided th at the random vector W is white Gaussian distributed, then Shannon’ s EPI can be strengthened to N ( X + √ t W ) ≥ (1 − t ) N ( X ) + tN ( X + W ) , (2) where t ∈ [0 , 1 ] . As Costa noted , the above EPI is equiv alent to the c oncavity of the en tropy power f unction N ( X + √ t W ) with resp ect to the param eter t , or, for mally 2 d 2 d t 2 N  X + √ t W  ≤ 0 . (3) Due to its inherent interest and to the fact that the proo f b y Costa was rather inv olved, simplified pr oofs of his result h av e been subsequently given in [8 ]–[1 1]. Additionally , i n h is pap er Costa pre sented two extensions of his main r esult in (3). Precisely , he showed that the E PI is also valid when the Gaussian vector W is not white, and also fo r the case where th e t parameter is multiplying the arbitrarily distributed random vector X , d 2 d t 2 N  √ t X + W  ≤ 0 . (4) Similarly to Shan non’ s EPI, Costa’ s EPI has been used successfully to deri ve im portan t info rmation -theoretic results concern ing, e.g. , Gaussian interfer ence c hannels in [12] or multi-anten na flat fading chann els with memory in [13] . C. A im o f the pa per Our objecti ve is to extend the particular case in (4) of Costa’ s E PI to the mu lti variate case, allo wing the real par am- eter t ∈ R to becom e a matrix T ∈ R n × n , which , to the b est of th e authors’ k nowledge, h as not bee n consider ed befo re. Beyond its theoretical inter est, the motivation b ehind this study is due to the fact that the concavity of th e en tropy power with respect to T implies the con cavity of th e entr opy and mutual informa tion quan tities, which would be a very desirable property in op timization procedure s in o rder to be able to, e. g., design th e linear p recoder that max imizes th e 2 The equi v alence between equations (2) and (3) is due to the fact that the functio n N ` X + √ t W ´ is twice dif ferenti able almost e very where thanks to the smoothing properties of the added Gaussian noise. mutual in formatio n in the linear vector Gaussian cha nnel with arbitrary input d istributions. Consequently , we inv estigate the co ncavity of the f unction N  T 1 / 2 X + W  , (5) with r espect to the symmetric matrix T = T 1 / 2 T T / 2 . Un - fortun ately , th e con cavity in T of the entro py power can be easily disproved by findin g simple c ounterexamp les as in [1 4] or even throug h nume rical comp utations of the entro py power . Knowing this negati ve re sult, we th us focus our study on the next possible mu lti variate candid ate: a diagon al matrix. Our o bjective now is to study th e concavity o f N  Λ 1 / 2 X + W  , (6) w .r .t. the diagona l matrix Λ = diag( λ ) , with [ λ ] i = λ i . For the sake of no tation, thr ougho ut this work we defin e Y = Λ 1 / 2 X + W , where we recall that the r andom vector W is assumed to follow a white zero -mean Gau ssian d istribution and the distribution o f the rando m vector X is arbitra ry . In particular, the distribution of X is allowed to assign po siti ve mass to one or more sing letons in R n . Con sequently , the results presented in Theorems 1 and 2 in Section III also hold for the case where the random vector X is discrete. I I . M AT H E M A T I C A L P R E L I M I N A R I E S In this section we presen t a number of lemmas fo llowed by a proposition that will pr ove useful in the proof of ou r multidimen sional EPI. In our de riv ations, the identity matrix is denoted by I , the vector with all its entr ies eq ual to 1 is represented by 1 , and A ◦ B repr esents the Hadamar d (or Schur) elem ent-wise m atrix product. Lemma 1 (B hatia [15, p. 15]): Let A ∈ S n + be a positi ve semidefinite matrix, A ≥ 0 . Then it fo llows th at  A A A A  ≥ 0 . Pr o of: Since A ≥ 0 , consider A = CC T and write  A A A A  =  C C   C T C T  . Lemma 2 (B hatia [15, Exer cise 1 .3.10 ]): Let A ∈ S n ++ be a positi ve definite matrix, A > 0 . Then it follows th at  A I I A − 1  ≥ 0 . (7) Pr o of: Conside r ag ain A = CC T , then we have A − 1 = C − T C − 1 . Now , simply wr ite ( 7) as  A I I A − 1  =  C 0 0 C − T   I I I I   C T 0 0 C − 1  , which, from Sylvester’ s law of in ertia for congrue nt m atrices [15, p . 5] an d Lem ma 1, is positiv e semidefin ite. Lemma 3 (S chur Theor em): I f th e matrices A and B are positive semide finite, then so is the produ ct A ◦ B . If, bo th A and B are p ositiv e de finite, then so is A ◦ B . In other words, the class of p ositiv e (semi)defin ite matrices is closed und er the Hadamard product. Pr o of: See [16, Th. 7.5.3 ] or [17, Th. 5 .2.1]. Lemma 4 (S chur complement) : Let the matrices A ∈ S n ++ and B ∈ S m ++ be po siti ve definite, A > 0 and B > 0 , and not necessarily of th e same dimension . Then the following statements are eq uiv alent 1)  A D D T B  ≥ 0 , 2) B ≥ D T A − 1 D , 3) A ≥ DB − 1 D T , where D ∈ R n × m is any arbitrary matrix. Pr o of: See [16, Th. 7.7.6 ] and th e second exercise following it or [1 8, Pro p. 8.2 .3]. W ith the ab ove lemm as at h and, we ar e now read y to prove the fo llowing p ropo sition: Pr o position 5 : Consider two p ositiv e de finite matrices A ∈ S n ++ and B ∈ S n ++ of the sam e dimen sion, an d let D A be a diagona l matrix containing the d iagonal elements of A , (i.e., D A = A ◦ I ). Then it f ollows that A ◦ B − 1 ≥ D A ( A ◦ B ) − 1 D A . (8) Pr o of: From Lemmas 1, 2, and 3, it follows that  A A A A  ◦  B I I B − 1  =  A ◦ B D A D A A ◦ B − 1  ≥ 0 . Now , from Le mma 4, the result follows directly . Cor o llary 6: Let A ∈ S n ++ be a positi ve definite matrix. Then, d T A ( A ◦ A ) − 1 d A ≤ n, (9) where we have defin ed d A = D A 1 = ( A ◦ I ) 1 as a column vector with the d iagonal e lements o f matrix A . Pr o of: Particularizing the result in Propo sition 5 with B = A and pre- and p ost-multiply ing it by 1 T and 1 we obtain 1 T  A ◦ A − 1  1 ≥ 1 T D A ( A ◦ A ) − 1 D A 1 . The result in (9) now follows straigh tforwardly fro m the fact 1 T  A ◦ A − T  1 = n , [19] (see also [1 8, Fact 7.6.1 0], [17 , Lemma 5.4.2(a)]) . Note that A is symmetric and thus A T = A and A − T = A − 1 . Remark 7 : Note th at the p roof of Cor ollary 6 is based on the r esult o f Pro position 5 in (8). An alternative p roof cou ld follow similarly from a different inequ ality by Styan in [2 0] R ◦ R − 1 + I ≥ 2 ( R ◦ R ) − 1 , where R is con strained to be a correlation matrix R ◦ I = I . Pr o position 8 : Consider n ow the positive semide finite ma- trix A ∈ S n + . Then, A ◦ A ≥ d A d T A n . Pr o of: For th e case where A ∈ S n ++ is p ositiv e definite, from (9 ) in Coro llary 6 and Lemma 4 , it f ollows th at  A ◦ A d A d T A n  ≥ 0 . Applying again Lemm a 4, we get A ◦ A ≥ d A d T A n . (10) Now , assume that A ∈ S n + is po siti ve sem idefinite. W e thus define ǫ > 0 an d co nsider the po siti ve d efinite matrix A + ǫ I . From (10), we know that ( A + ǫ I ) ◦ ( A + ǫ I ) ≥ d A + ǫ I d T A + ǫ I n . T akin g the limit as ǫ tends to 0, from continuity , the validity of ( 10) for positive semidefinite matrices fo llows. Finally , to en d this section about mathematical prelim ina- ries, we g iv e a very brie f overview on some basic definitions related to minim um mea n-square er ror ( MMSE) estimation . These d efinitions are u seful in our further der iv ations due to the relation b etween the entropy and the MM SE unv eiled in [ 21]. 3 Next, we give a lemm a conce rning the positive semidefiniteness of a certain class of matrices closely related with MMSE estimation . Consider the setting describe d in the introdu ction, Y = Λ 1 / 2 X + W . For a giv en re alization of the observations vector Y = y , the MMSE estimator, c X ( y ) , is g iv en b y the condition al mean c X ( y ) = E { X | Y = y } . W e n ow define the c ondition al MMSE matrix, Φ X ( y ) , a s the mean-squ are er ror matr ix co nditioned o n the fact that the received vector is eq ual to Y = y . Formally Φ X ( y ) , E n ( X − c X ( y ))( X − c X ( y )) T   Y = y o = E  X X T   Y = y  (11) − E { X | Y = y } E  X T   Y = y  . From this definition , it is clear that Φ X ( y ) is a positiv e semi- definite m atrix. Now , th e MMSE m atrix E X can b e calculated by averaging Φ X ( y ) in (11) with r espect to th e distribution o f vector Y as E X = E { Φ X ( Y ) } . (12) See below th e last lemm a in this section. Lemma 9 : For a given ra ndom vector X ∈ R n , it f ollows that E  X X T  ≥ E { X } E  X T  . Pr o of: Simply note that E  X X T  − E { X } E  X T  = E  ( X − E { X } )( X − E { X } ) T  ≥ 0 , where last ine quality follows from th e fact that the expectation operator preserves positive semid efiniteness. 3 Strictl y speaking the rela tion found in [21] concerns the quantiti es of mutual information and MMSE, but it is still useful for our proble m because the entropy h ( Y ) and the mutual information I ( X ; Y ) hav e the same depende nce on Λ up to a constant additi v e term. I I I . M A I N R E S U LT O F T H E PA P E R Once all the mathe matical preliminaries ha ve been pre- sented, in this section we give the main re sult o f the paper, namely , the con cavity of the entr opy power functio n N ( Y ) in (6), with respect to the diag onal elements of Λ . Prior to proving this result, we present a weaker result concer ning the co ncavity of the en tropy fu nction h ( Y ) , which is ke y in proving the concavity of the en tropy power . A. W arm u p: An entr opy in equality Theor em 1: Assume Y = Λ 1 / 2 X + W , where X is arbi- trarily distributed an d W f ollows a zero-m ean white Gaussian distribution. Th en th e entropy h ( Y ) is a conc av e fun ction of the d iagonal elem ents of Λ , i.e., ∇ 2 λ h ( Y ) ≤ 0 . Furthermo re, the entr ies of the Hessian matrix of the en tropy function h ( Y ) with respect to λ are given by  ∇ 2 λ h ( Y )  ij = − 1 2 E n  E { X i X j | Y } − E { X i | Y } E { X j | Y }  2 o , (13) which can be written more co mpactly as ∇ 2 λ h ( Y ) = − 1 2 E { Φ X ( Y ) ◦ Φ X ( Y ) } . (14) Pr o of: For the co mputation s leading to (13) and (14) see Appen dix I. Once the expression in (14) is obtained, concavity (o r n egativ e semidefiniteness of th e Hessian matrix) follows straigh tforwardly ta king into ac count that the m atrix Φ X ( y ) d efined in (11) is positive sem idefinite ∀ y , Lem ma 3, and from th e fact that the expectation operator pr eserves the semidefiniteness. B. Multivariate extension of Co sta’ s EPI Theor em 2: Assume Y = Λ 1 / 2 X + W , where X is arbitrarily distributed and W follo ws a zero -mean white Gaussian distribution. Th en the en tropy power N ( Y ) is a concave fu nction of the diagonal elem ents of Λ , i.e., ∇ 2 λ N ( Y ) ≤ 0 . Moreover , the Hessian matrix o f the entro py power function N ( Y ) with respec t to λ is gi ven by ∇ 2 λ N ( Y ) = N ( Y ) n d E X d T E X n − E { Φ X ( Y ) ◦ Φ X ( Y ) } ! , (15) where we recall th at d E X is a column vector with the d iagonal entries o f the matr ix E X defined in (1 2). Pr o of: First, let u s p rove (15). From the defin ition of the entropy power in (1) and applying the chain rule we obtain ∇ 2 λ N ( Y ) = 2 N ( Y ) n  2 ∇ λ h ( Y ) ∇ T λ h ( Y ) n + ∇ 2 λ h ( Y )  . Now , rep lacing ∇ λ h ( Y ) by its expression fro m [ 21, Eq. (61)] [ ∇ λ h ( Y )] i = 1 2 [ E X ] ii = 1 2 E { [ Φ X ( Y )] ii } , and incorporatin g the expression for ∇ 2 λ h ( Y ) calculated in (14), th e result in (15) follows. Now that a explicit expression f or the Hessian matrix has been obtained, we wish to pr ove that i t is ne gative s emidefinite. Note fro m (1 5) that, except for a positive factor, the Hessian matrix ∇ 2 λ N ( Y ) is the sum of a rank one positiv e semidefinite matrix and the H essian matrix of the entropy , which is negativ e semidefin ite accordin g to Theorem 1. Con sequently , the definiteness of ∇ 2 λ N ( Y ) is unk nown a priori, and some further developments are need ed to determine it, which is what we do next. Consider a family of positi ve s emidefinite matrices A ∈ S n + , characterized by a certain vector par ameter v , A = A ( v ) . Applying Proposition 8 to each matrix in this family , we obtain A ( v ) ◦ A ( v ) ≥ d A ( v ) d T A ( v ) n . ( 16) Since (16) is true for all po ssible values o f v , we have E { A ( V ) ◦ A ( V ) } ≥ E n d A ( V ) d T A ( V ) o n , (17) where now th e parameter v h as been consider ed to b e a random v ariable, V . Note that the distribution of V is arbitrary and d oes no t affect the v alidity of (17). Fro m Lem ma 9 we know th at E n d A ( V ) d T A ( V ) o ≥ E  d A ( V )  E n d T A ( V ) o , from wh ich it f ollows th at E { A ( V ) ◦ A ( V ) } ≥ E  d A ( V )  E n d T A ( V ) o n . Since the o perators d A and expectation commute we fin ally obtain E { A ( V ) ◦ A ( V ) } ≥ d E { A ( V ) } d T E { A ( V ) } n . Identify ing A ( V ) with the ran dom covariance error matrix Φ X ( Y ) and using (12) th e result in the theorem f ollows as d E X d T E X n − E { Φ X ( Y ) ◦ Φ X ( Y ) } ≤ 0 , and N ( Y ) ≥ 0 . I V . C O N C L U S I O N In this pap er we have proved that, for Y = Λ 1 / 2 X + W the function s N ( Y ) and h ( Y ) are concave with r espect to the d iagonal entries of Λ and have also gi ven explicit expressions f or the elements of the Hessian matrice s ∇ 2 λ N ( Y ) and ∇ 2 λ h ( Y ) . Besides its theo retical interest and inherent beauty , the importan ce of the results p resented in this work lie mainly in th eir p otential app lications, such as, the c alculation o f th e optimal power allocation to maximize the mutual info rmation for a g iv en non- Gaussian constellation as d escribed in [1 4]. A P P E N D I X I C A L C U L A T I O N O F ∇ 2 λ h ( Y ) In this sectio n we ar e interested in the calcu lation of the elements of the Hessian matrix  ∇ 2 λ h ( Y )  ij , which are defined b y  ∇ 2 λ h ( Y )  ij = ∂ 2 h ( Λ 1 / 2 X + W ) ∂ λ i ∂ λ j . First o f all, using the prop erties of d ifferential entropy we write h ( Λ 1 / 2 X + W ) = h ( X + Λ − 1 / 2 W ) + 1 2 log | Λ | , and recalling that we work with natural logarithm s we have ∂ 2 h ( Λ 1 / 2 X + W ) ∂ λ i ∂ λ j = ∂ 2 h ( X + Λ − 1 / 2 W ) ∂ λ i ∂ λ j − δ ij 2 λ 2 i . (18) W e ar e now interested in expanding the first term in the r ight hand side of last eq uation, so we define th e diagon al matr ix Γ = Λ − 1 and the ran dom vecto r Z = Λ − 1 / 2 Y . T hus [ Γ ] ii = γ i = 1 /λ i and Z = X + Λ − 1 / 2 W = X + Γ 1 / 2 W . Applying the chain rule we obtain ∂ 2 h ( X + Λ − 1 / 2 W ) ∂ λ i ∂ λ j = 1 λ 2 i λ 2 j ∂ 2 h ( X + Γ 1 / 2 W ) ∂ γ i ∂ γ j    Γ = Λ − 1 + 2 δ ij λ 3 i ∂ h ( X + Γ 1 / 2 W ) ∂ γ i    Γ = Λ − 1 . (19) The exp ressions for the two terms ∂ 2 h ( X + Γ 1 / 2 W ) ∂ γ i ∂ γ j and ∂ h ( X + Γ 1 / 2 W ) ∂ γ i are g iv en in App endix II, where we also sketch how they can be comp uted, for f urther details see [1 4]. Using th ese results, the rig ht h and side o f the expression in (1 9) can be rewritten as 1 λ 2 i λ 2 j − 1 2 E ( ( E { X i X j | Z } − E { X i | Z } E { X j | Z } ) 2 γ 2 i γ 2 j ) − δ ij 2 γ 2 i + E ( E  X 2 i | Z  − ( E { X i | Z } ) 2 γ 3 i ) δ ij !      Γ = Λ − 1 + 2 δ ij λ 3 i  1 2 γ 2 i  γ i − E  ( X i − E { X i | Z } ) 2        Γ = Λ − 1 . Simplifying terms we obtain − 1 2 E n ( E { X i X j | Z } − E { X i | Z } E { X j | Z } ) 2 o − δ ij 2 λ 2 i + δ ij λ i E  E  X 2 i | Z  − ( E { X i | Z } ) 2  + δ ij λ 2 i − δ ij λ i E  ( X i − E { X i | Z } ) 2  . (20) Finally , n oting that E  ( X i − E { X i | Z } ) 2  = E  E  X 2 i | Z  − ( E { X i | Z } ) 2  E { f ( X ) | Z } = E  f ( X ) | Λ 1 / 2 Z  = E { f ( X ) | Y } , and plugg ing (2 0) in (18) we obtain the d esired result in ( 13): ∂ 2 h ( Λ 1 / 2 X + W ) ∂ λ i ∂ λ j = − 1 2 E n ( E { X i X j | Y } − E { X i | Y } E { X j | Y } ) 2 o . By simple inspection of the entries of the Hessian matrix above, the re sult in (14) can be f ound. A P P E N D I X I I G R A D I E N T A N D H E S S I A N O F h ( Z = X + Γ 1 / 2 W ) The elements of th e g radient o f h ( Z = X + Γ 1 / 2 W ) with respect to the diagonal elem ents of Γ can be found thank s to the co mplex multivariate de Bruijn’ s iden tity fo und in [ 22, Th . 4] ad apted to the real case ∂ h ( X + Γ 1 / 2 W ) ∂ γ i = 1 2 E (  ∂ log P Z ( Z ) ∂ z i  2 ) . (21) The elements of the Hessian matrix can be foun d qu ite directly f rom the expression s foun d in [7, Eq. (50)] and in V illani’ s Le mma in [9] for the single dimensional second deriv ati ve d 2 h ( X + √ t W ) / d t 2 (see [14] for fu rther d etails on th e specific gen eralization to the multidime nsional case): ∂ 2 h ( X + Γ 1 / 2 W ) ∂ γ i ∂ γ j = − 1 2 E (  ∂ 2 log P Z ( Z ) ∂ z i ∂ z j  2 ) . (22) T o furth er elaborate th e expressions in (21) and (22) we see that we need to compu te the gr adient and Hessian of the fun ction log P Z ( z ) . The expression fo r the gradien t h as already b een g iv en in [21, Eq. (56)], [22 , Eq . (1 05)] ∂ log P Z ( z ) ∂ z i = E { X i | Z = z } − z i γ i . (23) The e xpression for the Hessian of log P Z ( z ) requires slightly more elabor ation and here we only give a sketch, more details can be fo und in [14]. Differentiating ( 23) with respec t to z j we obtain ∂ 2 log P Z ( z ) ∂ z i ∂ z j = 1 γ i γ j ( E { X i X j | Z = z } − E { X i | Z = z } E { X j | Z = z } ) − δ ij γ i , (24) where we h av e used that [ 14] ∂ E { X i | Z = z } ∂ z j = 1 γ j ( E { X i X j | Z = z } − E { X i | Z = z } E { X j | Z = z } ) . Plugging (2 3) into (21) and o perating acco rding to the deriv ation in [22, E q. ( 106)] we obtain ∂ h ( X + Γ 1 / 2 W ) ∂ γ i = 1 2 γ 2 i  γ i − E  ( X i − E { X i | Z } ) 2  . Similarly , plu gging ( 24) into (22) we obtain the d esired expression for the Hessian as ∂ 2 h ( X + Γ 1 / 2 W ) ∂ γ i ∂ γ j = − 1 2 E (  E { X i X j | Z } − E { X i | Z } E { X j | Z } γ i γ j − δ ij γ i  2 ) , which can be expanded as ∂ 2 h ( X + Γ 1 / 2 W ) ∂ γ i ∂ γ j = − 1 2 E ( ( E { X i X j | Z } − E { X i | Z } E { X j | Z } ) 2 γ 2 i γ 2 j ) − δ ij 2 γ 2 i + E ( E  X 2 i | Z  − ( E { X i | Z } ) 2 γ 3 i ) δ ij . R E F E R E N C E S [1] C. E. Shannon, “A mathematic al theory of communicati on, ” Bell Syst T ech. J , vol. 27, no. 3, pp. 379–423, 1948. [2] A. J. Stam, “Some inequali ties satisfied by the quantities of information of Fisher and Shannon, ” Inform. and Contr ol , vol. 2, pp. 101–112, 1959. [3] N. Blachman, “The con volution inequa lity for entropy powers, ” IEEE T rans. on Information Theory , vol. 11, no. 2, pp. 267–271, 1965. [4] S. V erd ´ u and D. Guo, “A simple proof of the entrop y-po w er inequa lity, ” IEEE T rans. on Informat ion Theory , vol. 52, no . 5, pp. 2165– 2166, 2006. [5] P . Bergmans, “A simple conv erse for broadcast channe ls with additi ve white Gaussian noise, ” IEEE T ra ns. on Informati on Theory , vol. 20, no. 2, pp. 279–280, 1974. [6] Y . Oohama, “Rate-dist ortion theory for Gaussian multiterminal source coding systems with sev era l s ide informations at the decoder, ” IEEE T rans. on Information Theory , vol. 51, no. 7, pp. 2577–2593, 2005. [7] M. H. M. Costa, “A new entrop y po wer inequali ty, ” IEEE T r ans. on Informatio n Theory , vol. 31, no. 6, pp. 751–760, 1985. [8] A. Dembo, “Simple proof of the concavi ty of the entrop y powe r with respect to added Gaussian noise, ” IEEE Tr ans. on Information Theory , vol. 35, no. 4, pp. 887–888, 1989. [9] C. V illa ni, “A s hort proof of the “conca vit y of entropy power”, ” IEEE T rans. on Information Theory , vol. 46, no. 4, pp. 1695–1696, 2000. [10] D. Guo, S. Shamai, and S. V erd ´ u, “Proof of entrop y power inequa litie s via MMSE, ” in Proc . IEE E Internationa l Symposium on Information Theory (ISIT’06) , July 2006. [11] O. Rioul, “Information theoret ic proofs of entrop y power inequalit ies, ” arXiv:0704.1751v 1 [cs.IT] , 2007. [12] M. H. M. Costa, “On the Gaussian interfe rence channel , ” IEE E T rans. on Information Theory , vol. 31, no. 5, pp. 607–615, 1985. [13] A. Lapido th and S. M. Moser , “Capacity bounds via duality with applic ations to multipl e-ante nna systems on flat-fading channe ls, ” IE EE T rans. on Information Theory , vol. 49, no. 10, pp. 2426–2467, 2003. [14] M. Payar ´ o and D. P . Palomar , “Hessian matrix and concavity properties of mutual informatio n and entropy in linear vec tor Gaussian channels, ” Submitte d to IEE E T r ans. on Information Theory , 2008. [15] R. Bhatia , P ositive definite matrice s . Princeton Uni ve rsity Press, 2007. [16] R. A. Horn and C. R. Johnson, Matrix analysis . Cambri dge Univ ersity Press, 1985. [17] ——, T opics on matrix analysis . Cambridg e Uni ve rsity Press, 1991. [18] D. S. Bernstei n, Matrix mathemat ics . Princeton Uni ve rsity Press, 2005. [19] C. R. Johnson and H. M. Shapiro, “Mathe matica l aspects of the relat i ve gain array A ◦ A − T , ” SIAM Journal on Algebra ic and Discrete Methods , vol. 7, p. 627, 1986. [20] G. P . H. St yan, “Ha damard products and mult i v ariat e statistic al analysis, ” Linear Algebr a Appl. , vol. 6, pp. 217–240, 1973. [21] D. Guo, S. Shamai, and S. V erd ´ u, “Mutual information and m inimum mean-square error in Gaussian channel s, ” IEEE T rans. on Information Theory , vol. 51, no. 4, pp. 1261–1282, 2005. [22] D. P . Palomar and S. V erd ´ u, “Gradie nt of mutual information in linea r vec tor Gaussian channel s, ” IEEE T ra ns. on Informati on Theory , vol. 52, no. 1, pp. 141–154, 2006.