On optimal precoding in linear vector Gaussian channels with arbitrary input distribution

On optimal precoding in linear vect or Gaussian channels with arbitrary input d istrib ution Miquel Payar ´ o Departmen t of Rad iocommu nications Centre T ecnol ` ogic de T elecomunicacion s de Catalunya Castelldefels, Barcelo na, Spain Email: miquel.payaro @cttc.cat Daniel P . Palomar Departmen t of E lectronic an d Com puter En gineerin g Hong Kong University of Scien ce and T echnolog y Clear W ater Bay , Ko wloo n, Hong K on g Email: palomar@ust.h k Abstract —The design of the precoder the maximizes the mutual information in linear vector Gaussian channels w ith an arbitrary input distribution is studied. Prec isely , the precoder optimal left singular vectors and singul ar values are d eriv ed. The characterization of the right singu lar vector s is left, in general, as an open pr oblem whose computational co mplexity is then studied in three cases: Ga ussian signalin g, low SNR, and high SNR. For the Gaussian signalin g case and the low SNR regime, the dependen ce of the mutual informa tion on t he right singular vector s vanishes, making the optimal p recoder design p roblem easy to solve. In the high SNR r egime, however , the dependence on the right sin gular vector s cannot be av oided and we sh ow the difficulty of computin g th e op timal precoder through an NP - hardness analysis. I . I N T R O D U C T I O N In linear vector Gaussian chan nels with an a verage p ower constraint, c apacity is achieved by zero -mean Gaussian inputs, whose covariance is alig ned with the ch annel eigenmod es and where th e power is distributed among the covariance eigenv alues ac cording to the waterfilling po licy [1], [ 2]. Des- pite the inform ation theor etic o ptimality of G aussian inputs, they are seld om used in p ractice due to their implem entation complexity . Rather , system designers of ten resor t to simple discrete co nstellations, such as BPSK or QAM. In this context, the scalar relation ship between mutual informa tion and minimu m mean square error ( MMSE) fo r linear vecto r Gau ssian channels p ut fo rth recently in [3 ], and extended to the vector case in [ 4], has becom e a fundam ental tool in transmitter design beyond the Gaussian signaling c ase. In [5], the authors der iv ed the op timum diag onal pr ecoder, or power allocation, in q uasi-closed form, coin ing th e term mercury /waterfilling. Th eir r esults were fo und for the partic- ular ca se o f a dia gonal ch annel corrup ted with A WGN and imposing inde penden ce on the co mpone nts of the inpu t vector . The mercury/waterfilling p olicy was later extended to non- diagona l channels in [6 ] thr ough a numer ical algorith m. The linear transm itter design (or linear prec oding) problem was recently studied in [7], [8] with a wider sco pe by considerin g fu ll (non-d iagonal) pr ecoder and chan nel matrices and arbitrary inputs with possibly dep endent co mpon ents. In [7], [8 ] th e auth ors gave necessary cond itions for th e op timal This work was supported by the RGC 618008 and the TEC2008-06327- C03-03/TEC research gra nts. precod er and optim al tran smit covariance matrix an d p roposed numerical iterativ e methods to compu te a (in gen eral sub op- timal) solution. De spite all th ese r esearch efforts, a gen eral solution for this problem is stil l missing. In this work, we make a step tow ar ds th e characterizatio n o f its solution and g iv e some h ints and ide as on why this p roblem is so challengin g. The c ontributions of the pr esent p aper are: 1) The expression for the op timal left singular vector matrix of the preco der th at maximizes a wide family of objective fu nctions (includin g the mutual inform ation) is g iv en. 2) W e gi ve a necessary and su fficient con dition for the optimal singular values of the pre coder that maximizes the mutual information and p ropo se an efficient me thod to nu merically co mpute it. 3) W e sho w that th e depe ndence of th e mutu al in formatio n on the righ t singular vector matrix o f the precod er is a ke y element in the intractability of computing the precod er that maximizes the mutual informa tion. 4) W e g iv e an expression f or the Jacob ian of the mutual informa tion with respect to the transmitted signal co - variance, correctin g the expression in [ 4, E q. ( 24)]. F ormalism: In th is work we d efine a prog ram according to { f ⋆ 0 , x ⋆ 1 , . . . , x ⋆ m } = Na me ( a 1 , . . . , a p ) := max/min x 1 ,...,x m f 0 ( x 1 , . . . , x m , a 1 , . . . , a p ) (1) subject to f i ( x 1 , . . . , x m , a 1 , . . . , a p ) ≤ 0 , ∀ i, where ( a 1 , . . . , a p ) are the p arameters and ( x 1 , . . . , x m ) are the o ptimization variables. Observe that the first returne d argument, f ⋆ 0 , correspon ds to the optimal v alue of the objectiv e function . W e also make use of the Jaco bian op erator D ap plied to a m atrix valued fu nction F of a matrix argument X defined as D X F = ( ∂ vec F ) / ( ∂ vec T X ) [ 9, Sec . 9.4], where vec X is the vector obtained stacking th e co lumns of X . This notatio n requires some modificatio ns when either F or X are symmetric matrices, see [9] for details. In Section VI we u se some concepts of co mputation al c omplexity and program reductions. See [10], [1 1] for referen ce. I I . S I G N A L M O D E L W e con sider a gen eral discrete-time linear vector Gaussian channel, whose outp ut Y ∈ R n is represented by the follo win g signal m odel Y = HP S + Z , (2) where S ∈ R m is the input vector d istributed according to P S ( s ) , th e matrices H ∈ R n × p and P ∈ R p × m represent the channel and precoder linear transformation s, r espectively , and Z ∈ R n represents a ze ro-mean Gaussian no ise w ith iden tity covariance m atrix R Z = I 1 . For the sake of simplicity , we assume that E { S } = 0 and E  S S T  = I . The tr ansmitted p ower ρ is th us given by ρ = T r  PP T  . W e will also m ake u se of the notation P = √ ρ ¯ P , with T r  ¯ P ¯ P T  = 1 and also define R H = H T H . Moreover , we defin e the SVD decomp osition of the precoder as P = U P Σ P V T P , the entries of Σ P as σ i = [ Σ P ] ii , and also the eigend ecomposition of the ch annel covariance as R H = U H Λ 2 H U T H . Finally , we de fine the MMSE matrix as E S = E  ( S − E { S | Y } )( S − E { S | Y } ) T  . I I I . P R O B L E M D E FI N I T I O N A N D S T RU C T U R E O F T H E S O L U T I O N In th is p aper we are in terested in study ing the prop erties of the precoder P that maximize s the mu tual information under a n av erage transmitted power constraint. Howe ver , in this section we consider the more g eneric pr oblem setup {P ⋆ 0 , P ⋆ P 0 } = Ma xPerf ormac e  ρ, P S ( s ) , R H  := max P P 0 (3) s.t. T r  PP T  = ρ, where P 0 is a gener ic perfo rmance measure th at dep ends on the precoder P throu gh the received vector Y . In th e fo llowing lemma we char acterize the depende nce of P 0 on the p recoder matr ix P . Lemma 1 : Con sider a per forman ce measur e P 0 of the sys- tem Y = HP S + Z , su ch that P 0 depend s on the distribution of the rand om ob servation Y con ditioned o n th e input S . It then follows that the depende nce of P 0 on the p recoder P is only thr ough P T R H P and we can thus write without loss of generality P 0 = P 0  P T R H P  . Pr oof: The pro of follows quite ea sily b y no ting th at P T H T Y is a sufficient statistic of Y , [2, Section 2.1 0]. The sufficient statistic is th us P T H T HP S + P T H T Z . T he first term o bviously depends on P only thr ough P T R H P . Sinc e the second term P T H T Z is a Gaussian rand om vector, its behavior is co mpletely d etermined by its m ean ( assumed zero ) and its covariance matrix, gi ven by P T R H P . From all the p ossible ch oices f or the perform ance measure function P 0 , we ar e now goin g to focus o ur attentio n on th e specific class of reasonable p erform ance measur es, which is defined n ext. 1 The assumption R Z = I is made w .l.o.g., as, for the case R Z 6 = I , we could alw ays conside r the whitened recei ved signal R − 1 / 2 Z Y . Definition 1 : A perfor mance measure P 0  P T R H P  is said to be rea sonable if it f ulfills that P 0  α P T R H P  > P 0  P T R H P  , for any α > 1 a nd P T R H P 6 = 0 , which implies tha t P 0 is a power efficient p erform ance measure. Remark 1 : The gen eric cost fu nction 2 f 0 considered in [12] was assumed to be a f unction o f the e lements of the vector diag  ( I + P T R H P ) − 1  and increasing in each argumen t. Recalling th at, fo r any α > 1 an d P T R H P 6 = 0 , we have  diag  ( I + α P T R H P ) − 1  i <  diag  ( I + P T R H P ) − 1  i . (4) It is straightfor ward to see that the perf orman ce measure defined as P 0 , − f 0 is a re asonable pe rforman ce mea sure accordin g to Definition 1 . Based o n a result in [12] f or th e design of op timal linear precod ers, we charac terize the left singu lar vectors of an optimal precoder of (3). Pr opo sition 1: Consider the optimization p roblem in (3). It then fo llows that, f or any rea sonable perform ance measure P 0 , the left sing ular vectors of the optima l precoder P ∈ R p × m can alw ay s be cho sen to coincid e with the eigen vector s of the channel c ovariance R H associated w ith the min { p, m } largest eigenv alues. Pr oof: For simp licity we consider the case m ≥ p . Th e case m < p follows similarly . From the SVD of the pre coder P = U P Σ P V T P and the eigen -decomp osition of the matrix Σ P U T P R H U P Σ P = Q∆Q T , (5) with ∆ diag onal and Q orthon ormal, it f ollows that Q T Σ P U T P R H U P Σ P Q (6) is a diagonal ma trix. From [12, Lemma 1 2], we can state that there exists a matrix M = U H Σ M , with Σ M having non-zero elements o nly in the main d iagonal, su ch th at M T R H M = ∆ and that T r  MM T  ≤ T r  Σ P  = T r  PP T  . Now , we o nly need to check that P T R H P = V P Q∆Q T V T P = V P QM T R H MQ T V T P . Defining e P = M Q T V T P = U H Σ M e V T , with e V = V P Q , we have shown by co nstruction that fo r any given matrix P we can find an other matrix e P such that the objective function in (3) is the same, e P T R H e P = P T R H P ⇒ P 0  e P T R H e P  = P 0  P T R H P  , (7) which follows from Lem ma 1, whereas the required transmit- ted power is not larger , T r  e P e P T  = T r  MM T  ≤ T r  PP T  . Since the p erform ance measure P 0 is reasonab le, the result follows directly . From th e result in Proposition 1, it follows th at, the channel model in (2) can be simplified, witho ut loss of optimality , to Y ′ = Λ H Σ P V T P S + Z , (8) where now the on ly op timization variables are Σ P and V P . 2 Observe that, while a performance measure P 0 is to be maximize d, a cost functio n f 0 is usually to be minimized . I V . O P T I M A L S I N G U L A R V A L U E S In this section we particularize the generic performanc e measure con sidered in the previous section to th e inpu t-outpu t mutual information in (8), i.e., P 0 = I ( S ; Y ′ ) . T o compute the optimal Σ ⋆ P we define { I ⋆ , Σ 2 ⋆ P } = Op tPowe rAllo c  ρ, P S ( s ) , Λ H , V P  := max { σ 2 i } I ( S ; Y ′ ) (9) s.t. X i σ 2 i = ρ. Observe that the op timization is do ne with respect to the optimal squ ared singu lar values. The optim al singular values are then defined up to a sign, which d oes no t affect the mutual inf ormation . Consequ ently , w e defin e σ ⋆ i = + p σ 2 ⋆ i and [ Σ ⋆ P ] ii = σ ⋆ i . Let us n ow present an app ealing pro perty of I ( S ; Y ′ ) . Lemma 2 ([13 ]): Conside r the mo del in (8) an d fix V P . Then it follows that the mu tual in formatio n I ( S ; Y ′ ) is a concave function of the squared diagonal en tries of Σ P . W ith this result, we can now o btain a necessary and su fficient condition for th e squa red entries of Σ ⋆ P . Pr opo sition 2: The entries o f the squ ared singular value matrix σ 2 ⋆ i = [ Σ 2 ⋆ P ] ii of th e solutio n to (9 ) satisfy σ 2 ⋆ i = 0 ⇒ [ Λ 2 H ] ii mmse i ( Σ ⋆ P , V P ) < 2 η σ 2 ⋆ i > 0 ⇒ [ Λ 2 H ] ii mmse i ( Σ ⋆ P , V P ) = 2 η , (10) where η is such that the power constraint is satisfied and where we h av e u sed mmse i ( Σ ⋆ P , V P ) to define the i -th diagon al entry of the MMSE matrix E b S correspo nding to the model Y ′ = Λ H Σ ⋆ P b S + Z with b S = V T P S . Pr oof: The pro of is based on obtain ing the KKT condi- tions of th e op timization pr oblem in (9) together with d I ( S ; Λ H Σ P b S + Z ) d ( σ 2 i ) = [ Λ 2 H ] ii E    b S  i − E  b S  i   Λ H Σ ⋆ P b S + Z   2  , (11) which follows from [4, Cor . 2]. Remark 2 : The set of non-lin ear equ ations in (10) can be numerically solved with, e.g ., the Newton metho d becau se it has quadratic con vergence and the concavity prop erty stated in Lemma 2 guar antees th e global optimality of the obtain ed solution. The expression for the en tries o f the Jacobian vector of mmse i ( Σ ⋆ P , V P ) with respect to the squ ared entries of Σ P , which is need ed at each iteration, is g iv en by [13] d mmse i ( Σ ⋆ P , V P ) d ( σ 2 j ) = − [ Λ 2 H ] j j E  [ Φ ( Y ′ )] 2 ij  , where Φ ( y ′ ) = E n b S b S T    y ′ o − E n b S    y ′ o E n b S T    y ′ o . At this point, we have obtain ed th e optim al le ft singu lar vectors and the optimal singular values of the linear pre- coder that maximiz es the mutual inf ormation fo r a fixed V P . Unfor tunately , th e optimal solution for the rig ht singular vectors V P seems to b e an extremely difficult prob lem. A simple suboptim al solutio n consists in o ptimizing V P based on standar d numerical methods guaran teed to converge to a local optimu m. See further [14 ] f or details o n the practical algorithm to comp ute the preco der . From the re sults pr esented in this section, it is app arent that th e difficulty of the pr oblem in (3) wh en optimizin g the mutual infor mation lies in the computa tion of the optimal right singular vectors matrix, V ⋆ P . T o suppo rt th is statement, in the following sections we deal with three cases: the Gaussian signaling case, and the low a nd high SNR regimes. In the Gaussian signaling case and low SNR regime, we recover the well-known result that the mutu al info rmation de pends only on the squar ed precode r Q P = PP T and is ind ependen t of the right singular vectors m atrix V P , which f urther implies tha t, in bo th cases, the optima l precoder can be easily comp uted. In Section VI we will show that, for the high SNR regime, the precoder design problem beco mes com putationally difficult throug h a NP- hardne ss analysis. V . S I T U A T I O N S W H E R E T H E M U T UA L I N F O R M A T I O N I S I N D E P E N D E N T O F V P 1) Gau ssian sig naling case: For the Gaussian sign aling case, we recover the we ll k nown expression for the mutual informa tion [2] I ( S ; Y ) = 1 2 logdet  I + Q P R H  , (12) from which it is clear that the o nly dependence of the mutual informa tion o n the preco der is th roug h Q P = U P Σ 2 P U T P and, th us, it is independen t of V P . As we ha ve pointed out in the introductio n and generalized in Proposition 1, the optimal covariance Q P is alig ned with the chann el eigenmo des U H . Also the p ower is distributed among the covariance eigenv a- lues Σ 2 P accordin g to the waterfilling policy [1], which can be co mputed ef ficiently . 2) Low SNR re gime: For the low SNR regime, a first-order expression of the m utual in formatio n is [4] I ( S ; Y ) = 1 2 T r  Q P R H  + o  k Q P k  . (13) Just as in th e previous case, from this expression it is clear that the mutual infor mation is insensiti ve to the right singular vector matrix V P . Moreover, th e op timal matrix Q P is easy to ob tain in clo sed fo rm [15 ] 3 . Remark 3 : The expression in (13) was derived in [4 ] throug h the expression of the Jacobian o f the m utual infor- mation with respect to Q P . Alth ough the result in (13) is correct, the expression for the Jacobian D Q P I ( S ; Y ) giv en in [4, Eq. (24)] is o nly valid in the lo w SNR regime. Th e c orrect expression for D Q P I ( S ; Y ) valid for all SNRs is [16] D Q P I ( S ; Y ) = 1 2 vec T  R H PE S P − 1  D n − vec T ( E S P T R H U P Σ P ) ΩN n ( P − 1 ⊗ P T ) D n , (14) 3 The optimal signaling strategy in the low SNR reg ime was studied in full general ity in [15]. W e recall that, in this work, we are assuming that the signali ng is fixed and the only remaining degree of freedom to maximize the mutual informati on is the precoder matrix P . with Ω =      v T 1 ⊗ V P ( σ 2 1 I − Σ 2 P ) + V T P v T 2 ⊗ V P ( σ 2 2 I − Σ 2 P ) + V T P . . . v T n ⊗ V P ( σ 2 n I − Σ 2 P ) + V T P      , (15) where v i is th e i - th colum n of m atrix V P , N n and D n are th e symmetrization and du plication m atrices defined in [ 9, Secs. 3.7, 3. 8], A + denotes the Moore-Penr ose pseud o-inv erse, and where fo r the sake of clar ity , we h av e assumed th at P − 1 exists and that n = m = p . V I . H I G H S N R R E G I M E In this section we conside r that the signaling is discrete, i.e., the input can only take values fr om a finite set, S ∈ S , { s ( i ) } L i =1 . As discussed in [5] , [8], f or discrete inp uts and high SNR, th e maximization o f the p roblem in (3) with th e mutual information as perform ance measure is asymptotically equiv alent to the maxim ization of the squ ared minimu m dis- tance, d min 4 , among the receiv ed co nstellation points defined as d min = min e ∈E e T P T R H P e , where E is the set contain ing all the p ossible d ifferences between the inpu t points in S . Consequently , let u s b egin by consider ing the optimization problem of fin ding the precoder that maxim izes the min imum distance a mong th e receiv ed con stellation points { d ⋆ , P ⋆ d } = MaxMi nDist ( ρ, E , H ) := max P min e ∈E e T P T R H P e (16) s.t. T r  PP T  = ρ. In the following, we give th e pro of th at the p rogr am in (16) is NP- hard with respect to the d imension m of the signa ling vector , S ∈ R m , for the case where the set E is considered to be unstructured (i.e., not constrained to be a dif fe rence set). W e are now p reparin g the proof without th is assumption in [1 4]. The p roof is based on a series o f Coo k red uctions. W e say that progr am A can be Coo k redu ced to progra m B , A C O O K − − − → B , if progr am A can be com puted w ith a polyn omial time algo rithm that calls p rogram B as a subroutine assuming that the ca ll is perfor med in one clock cycle. W e have th at, if A C O O K − − − → B and A is NP-h ard, then B is also in NP-har d, [11 ]. Before giving the actual p roof we de scribe two more progr ams and giv e some of their pr operties. A. I ntermediate p r ograms and their pr operties W e first pr esent the MinNorm progr am, wh ich com putes the minimum nor m vector that fu lfills a set of con straints on its scalar product with a given set of vectors { w i } m i =1 { t ⋆ , z ⋆ } = Mi nNorm  { w i } m i =1  := min z ∈ R m k z k 2 (17) s.t. | w T i z | ≥ 1 , i = 1 , . . . , m. Lemma 3 ([17 ]): Min Norm is NP-har d. 4 Although we use the symbol d min , it denotes squared distance. Algorithm 1 Reduction of Mi nNorm to MinPower Input: S et of weig ht vectors { w i } m i =1 . Output: V ector z ⋆ that achieves the minimu m norm , f ulfill- ing all the con straints | w T i z ⋆ | ≥ 1 . V alue of the minimum n orm t ⋆ = k z ⋆ k 2 . 1: Assign H =  1 0 . . . 0  ∈ R 1 × p . 2: Assign E = { w 1 , . . . , w m } . 3: Call { ρ ⋆ , P ⋆ } = MinPo wer (1 , E , H ) . 4: t ⋆ = ρ ⋆ . 5: z ⋆ = ( Fir stRow ( P ⋆ )) T . The second prob lem is Mi nPower and it computes the precod er tha t m inimizes the transmitted power such tha t the minimum distance is above a certain thr eshold: { ρ ⋆ , P ⋆ } = Mi nPowe r ( d, E , H ) := min P T r  PP T  (18) s.t. min e ∈E e T P T R H P e ≥ d. Lemma 4 : Assum e that { d ⋆ 0 , P ⋆ 0 } is the ou tput to the pro - gram MaxMinD ist ( ρ 0 , E , H ) . I t th en fo llows that the output to Mi nPowe r ( d ⋆ 0 , E , H ) is giv en by { ρ 0 , P ⋆ 0 } . Similarly , assum e that { ρ ⋆ 0 , P ⋆ 0 } is the outp ut to th e pro- gram Mi nPowe r ( d 0 , E , H ) . It then follows that th e o utput to MaxMin Dist ( ρ ⋆ 0 , E , H ) is given by { d 0 , P ⋆ 0 } . Pr oof: See [18]. Lemma 5 : Assum e that { d ⋆ 0 , P ⋆ 0 } is the ou tput to the pro - gram MaxMi nDist ( ρ 0 , E , H ) . It then fo llows that th e out- put to MaxMi nDist ( αρ 0 , E , H ) with α > 0 is given by { αd ⋆ 0 , √ α P ⋆ 0 } . Pr oof: The proof follows easily , e .g., by considering the change of optimizatio n variable P = √ α e P and noting tha t the solutio n to the optimization pr oblem remains unchang ed if the o bjective function is scaled by a constant param eter . In the f ollowing we prove the following chain of r eduction s: MinNor m C O O K − − − → MinP ower C O O K − − − → MaxM inDis t . B. R eduction of Min Norm to MinP ower In Algorithm 1 we pr esent our proposed Cook re duction of MinNor m to MinPowe r . Pr opo sition 3: Algorith m 1 is a po lynom ial time Cook reduction of Mi nNorm to MinPo wer . Pr oof: Under the assumption th at Mi nPower can be solved in one clock cycle, it follows that Algorithm 1 ru ns in polyno mial time as well. It remain s to che ck th at the outpu t of th e algorithm corr esponds to the solution to Min Norm . Note th at fo r the particu lar values assigned to the chann el matrix H an d the set E in Step s 1 and 2 in Algo rithm 1, the progr am MinPow er (1 , E , H ) in (18) particular izes to min P T r  PP T  (19) s.t. min i ∈ [1 ,m ] w T i p 1 p T 1 w i ≥ 1 , (20) where p 1 is a column vector with the ele ments of the first row of th e pr ecoder matrix P . Ob serving that the co nstraint in (2 0) Algorithm 2 Reductio n of MinPower to MaxMinD ist Input: Des ired squared minimum d istance, d . Set o f vector s E . Channel matr ix, H . Output: Pre coder P ⋆ that m inimizes the transmitted p ower , fulfilling min e ∈E e T P ⋆ T R H P ⋆ e ≥ d . T r ansmitted power ρ ⋆ = T r  P ⋆ P ⋆ T  . 1: Call { d ⋆ 0 , P ⋆ 0 } = Ma xMinD ist (1 , E , H ) . 2: Assign ρ ⋆ = d d ⋆ 0 . 3: Assign P ⋆ = q d d ⋆ 0 P ⋆ 0 . only affects the elemen ts of the first row of matrix P , it is clear that the o ptimal solution to (19) fulfills [ P ⋆ ] ij = 0 , ∀ i 6 = 1 , as this assignmen t minimizes the tra nsmitted p ower . Recalling that w T i p 1 p T 1 w i = | w T i p 1 | 2 , it is now straightforward to see that the first row o f matr ix P ⋆ , which is the solutio n to the problem in (19 ), is also the so lution to M inNor m in (17 ). Cor ollary 1: For the case where the set E is un constrained , the progra m MinPowe r is NP-h ard. C. Reductio n of MinPowe r to Max MinDi st In Algorithm 2 we pr esent our proposed Cook red uction of MinPow er to M axMin Dist . Pr opo sition 4: Algorith m 2 is a po lynom ial time Cook reduction of Mi nPowe r to Max MinDi st . Pr oof: Under th e a ssumption that MaxMi nDist can b e solved in one clock cycle, it follows that Algorithm 2 run s in polyno mial time as well. It remain s to che ck th at the outpu t of th e algorithm co rrespon ds to th e solutio n to MinP ower . Assume that the output to M axMin Dist (1 , E , H ) is giv en by { d ⋆ 0 , P ⋆ 0 } as in Step 1 in Alg orithm 2. Note that, from the power constraint in (1 6), we have that T r  P ⋆ 0 P ⋆ T 0  = 1 . Fr om Lemma 5, cho osing α = d/d ⋆ 0 , it fo llows that  d, q d/d ⋆ 0 P ⋆ 0  = Ma xMinD ist ( d/d ⋆ 0 , E , H ) . (21) Now , applying Lem ma 4 , we have that  d/d ⋆ 0 , q d/d ⋆ 0 P ⋆ 0  = MinPo wer ( d , E , H ) , (22) from wh ich it imm ediately fo llows th at ρ ⋆ = d/d ⋆ 0 and P ⋆ = p d/d ⋆ 0 P ⋆ 0 , which comp letes the proo f. Cor ollary 2: For the case where the set E is un constrained , the progra m MaxMinD ist is NP-ha rd. Although the fact that the pr ogram M axMin Dist is NP-har d is not a pr oof th at the maxim ization of the mutual in formatio n is also NP-hard, it gives a powerful hint on its expected computatio nal comp lexity in the hig h SNR regime where the minimum distance is the key perfor mance parameter . From this expected comp lexity o n the precod er design at high SNR and th e fact that, in Section III, we characte rized the optimal lef t sing ular vector s an d the singular values of the precod er that maximize s the mutual information as a fun ction of the rig ht singular vector matrix V P , it seems reasonable to p lace the computatio nal co mplexity burden of the optimal precod er design in the compu tation of V ⋆ P . V I I . C O N C L U S I O N W e have studied th e pro blem of findin g the precode r tha t maximizes th e mutual inf ormation f or an arb itrary (but given) input distribution. W e have found a closed-for m expression for the left singular vectors of the optimal precoder and ha ve gi ven a sufficient a nd necessary con dition to comp ute the optimal singular values. W e have also recalled th at, in the low SNR or Gaussian sign aling scenarios, the o ptimal pre coder can be easily fo und as the mutual infor mation does not d epend o n the right singular vector s. Finally , we have argued that in the high SNR regime, the computation al co mplexity of the calculation of th e optimal righ t singu lar vectors is expec ted to be h ard. R E F E R E N C E S [1] E. T elatar , “Capa city of multi-ante nna Gaussian channels, ” Eur opean T rans. on T elecomm. , vol. 10, no. 6, pp. 585–595, Nov .-Dec. 1999. [2] T . M. Cove r and J. A. Thomas, Elements of Informatio n Theory . New Y ork: John Wi ley & Sons, 1991. [3] D. Guo, S. Shamai, and S. V erd ´ u, “Mutua l information and minimum mean-square erro r in Gaussian channels, ” IEEE T rans. on Informat ion Theory , vol. 51, no. 4, pp. 1261–1282, 2005. [4] D. P . Palomar and S . V erd ´ u, “Gradi ent of m utual information in line ar vec tor Gaussian channels, ” IEEE T rans. on Informatio n Theory , vo l. 52, no. 1, pp. 141–1 54, 2006. [5] A. Lozano, A. Tu lino, and S. V erd ´ u, “Optimum power allocation for parall el Gaussian channels with arbitrary input distrib utions, ” IEE E T rans. on Informati on Theory , v ol. 52, no. 7, pp. 3033–3051, July 2006. [6] F . P ´ erez-Cr uz, M. R. D . Rodrigues, and S. V erd ´ u, “Generali zed mer- cury/w aterfill ing for multiple -input multipl e-output channels, ” in Proc . of the 45th Annual Allerton Confere nce on Communication, Contr ol, and Computing , All erton (Illinois, USA), Sept. 2007. [7] ——, “Optimal precodi ng for digital subscriber lines, ” in Pr oc. IEEE Internati onal Conf ere nce on Communicati ons (ICC’08) , 2008. [8] M. R. D. Rodrigues, F . P ´ erez-Cruz, and S. V erd ´ u, “Multipl e- input multiple-o utput Gaussian channels: Optimal cov ariance for non- Gaussian inputs, ” in Proc. IEEE Information Theory W orkshop (ITW ’08) , 2008, pp. 445–449. [9] J. Magnus and H. Neudeck er, Matrix Dif fer ential Calculus with Appli - cations in Sta tistics an d E conometri cs , 3rd ed. New Y ork: Wi ley , 2007. [10] C. H. Papadimitri ou, Computation al Comple xity . Readin g, Mass: Addison-W esle y , 1994. [11] O. Goldreich, Computati onal Complexi ty: A Conceptu al P erspective . Cambridge : Cambri dge Univ ersity Press, 2008. [12] D. P . Palomar , J. M. Cioffi, and M. A. L agunas, “Joint T x-Rx beam- forming design for multicarrier MIMO channels: A unified frame work for con ve x optimizati on, ” IEEE T rans. on Signal P r ocessing , vol. 51, no. 9, pp. 2381– 2401, Sep. 2003. [13] M. Payar ´ o and D. P . Palomar , “He ssian and concavi ty of mutual informati on, entropy , and entropy powe r in linear vect or Gaussian channe ls, ” accepted in IEEE T rans. on Information Theory , 2009. [Online]. A v ailab le: htt p://arx i v .org/abs/09 03.1945 [14] ——, “On linear preco ding strat egie s in l inear v ector Gaussian ch annels with perfect channel sta te information, ” in pre parati on , 2009. [15] S. V erd ´ u, “Spectral effici ency in the wideband regime, ” IEEE T rans. on Informatio n Theory , vol. 48, no. 6, pp. 1319 – 1343, 2002. [16] M. Payar ´ o and D. P . Palo mar , “A note on “Gradient of mutual in- formation in linear vec tor Gaussian chan nels”, ” unpublished document (availa ble upon request ) , 2008. [17] Z .-Q. L uo, N. D. Sidiropoul os, P . T seng, and S. Zhang, “ Approximation bounds for quadrati c optimizati on with homogene ous quadrat ic con- straints, ” SIAM J. on Optimization , vol. 18, no. 1, pp. 1–28, 2007 . [18] S. P . Boyd and L. V andenbergh e, Con vex Optimization . Cambridge Uni versity Press, 2004.