Gaussian Belief Propagation Based Multiuser Detection

Gaussian Belief Propagation Based Multiuser Detection Danny Bickson 1 and Da nny Dolev School of Computer Science and E ngineer ing Hebrew Univ ersity of Jerusalem Jerusalem 91904, Israel { daniel51 ,dolev } @cs.huji.ac.il Ori Sh ental 1 , Paul H. Siegel and J ack K. W olf Center f or Magnetic Recording Research University of California - San Diego La Jolla, CA 920 93 { oshental,p siegel,jwolf } @ucsd.edu Abstract — In this work, we present a nov el constru ction fo r solving the linear multiuser detection problem using the Gaussian Beli ef Propagation algorithm. Ou r algorithm yield s an efficient, iterativ e and distributed i mplementation of the MMSE detector . Compared t o our pre vious formulation, the new algorithm offers a reduction in memory requirements, the number of computational steps, and t he number of messages passed. W e prov e that a d etection method recently proposed by Montanari et al. is an in stance of ours, and we provide new con ve rgence results app licable to both. I . I N T R O D U C T I O N Belief propaga tion (BP), also known as the sum-pro duct algorithm , is a powerful and effi cient tool in solvin g, exactly or approximately , inference problems in probabilistic grap hical models. The underlying essence of estimation theory is to detect a hidden input to a channel from its observed outp ut. The chan nel can be represented as a c ertain gr aphical mode l, while the detection of the channel input is equiv alent to perfor ming in ference in th e co rrespon ding grap h. The use of BP [ 1] for detection p urposes has been pr oven to be very bene ficial in se veral app lications in co mmun ications. For ra ndomly -spread code -division multiple-access (CDMA) in the large-system limit, Kabashima has introduc ed a tractable BP-based multiu ser d etection (M UD) scheme, whic h exh ibits near-optimal error perf ormanc e for binary- input additi ve white Gaussian noise (BI-A WGN) chann els [2]. This message- passing scheme h as recently been exten ded to the case where the ambient noise le vel is unknown [3], [4]. As for sub-optimal detection, the nonlinear soft para llel inter ference c ancellation (PIC) detector was refor mulated b y T anaka and Okad a as an approx imate BP solution [5] to the MUD pro blem. In contr ast to the dense, fully-co nnected nature of th e graphica l model o f the non-o rthogo nal C DMA channe l, a o ne- dimensiona l (1-D) intersymbol interf erence (ISI) chann el can be interpreted as a cycle-fr ee tree grap h [6] . Thus, detectio n in 1- D ISI ch annels (term ed equ alization) can be perfo rmed in an optimal maximu m a-posterior i (MAP) mann er via BP , also known in th is context as the forward/bac kward, or BCJR, algorithm [ 7]. Also, Kurkoski et al. [8], [9] h av e propo sed an 1 Contrib uted equally to this work. Supported in part by NSF Grant N o. CCR-0514 859 and EVERGR O W , IP 1935 of the EU Sixth Frame wo rk. iterativ e BP-like d etection algo rithm for 1-D ISI channels that uses a parallel message-pa ssing schedule an d ac hieves near- optimal p erforma nce. For th e intermediate regime of non-den se graph s but with many r elativ ely short loops, extensio ns o f BP to two- dimensiona l ISI channels have been co nsidered by Mar row and W olf [10], and recen tly Shental et a l. [11]– [13] ha ve demonstra ted the near-optimality of a g eneralized version of BP for such cha nnels. Recently , BP has b een proved to asymptotically achieve optimal MAP detection f or sp arse linear systems with Gaussian no ise [1 4], [ 15], for example, in CDMA with sparse spread ing codes. An important class of practical sub-optimal detectors is based on linear detection. This class includes, for instance, the con ventional single-u ser m atched filter (MF), deco rre- lator (a.k.a. zero- forcing equalizer), linear minimum mean- square error (MMSE) detecto r and many other detectors with widespread app licability [16], [1 7]. In ge neral, linear d etection can be viewed as the solution to a (deter ministic) set o f linear eq uations d escribing the orig inal (pr obabilistic) estima- tion prob lem. Note that the math ematical o peration behind linear detection extends to other tasks in com municatio n, e.g. , channel pre coding at th e tran smitter [18]. Recently , linear detection has been explicitly linked to BP [ 19], using a Gaussian belief prop agation (GaBP) algo- rithm. T his allo ws for a distributed implementation of the linear detector [2 0], circumventing the need of, potentially cumberso me, direct matrix inv ersion (via, e.g. , Gaussian elim- ination). The derived iterati ve fram ew ork was compar ed quan - titati vely with ‘cla ssical’ iterative method s for solving systems of linear equ ations, such as those in vestigated in th e co ntext of linear implementatio n of CDMA demo dulation [2 1]–[2 3]. GaBP is shown to yield faster co n vergence than these stan dard methods. Another imp ortant work is the BP-based MUD, recently derived and analy zed by Mon tanari et al. [24] for Gaussian inpu t symbols. There a re se veral drawbacks to the linear detection tech- nique of [ 19]. First, th e input matrix R n × n = S T n × k S k × n (the chip cor relation matrix) needs to be computed pr ior to runn ing the algorithm . This compu tation require s n 2 k operation s. In case where the m atrix S is spa rse [15], the m atrix R might not no long er b e sparse. Second, GaBP uses 2 n 2 memory to store the messages. For a large n this could be prohib iti ve. In this pape r , we prop ose a new con struction that addresses those two dr awbacks. I n o ur improved con struction, given a non-r ectangular CDMA matrix S n × k , we compute the MMSE detector x = ( S T S + Ψ) − 1 S T y where Ψ is the A WGN diagona l covariance ma trix. W e utilize the GaBP algor ithm which is an efficient iterative distributed algo rithm. The new construction uses only 2 nk memor y for storing the message s. When k ≪ n this represents significant saving r elativ e to the 2 n 2 in our previously proposed algorithm . Furthermore, we d o not explicitly co mpute S T S , saving an extra n 2 k overhead. W e show that Montan ari’ s algorith m [ 24] is an in stance of our me thod. By showing th is, we are ab le to prove new con- vergence results fo r Montan ari’ s algo rithm. Montan ari proves that his method con verges on norm alized ran dom-sp reading CDMA seque nces, assuming Gau ssian signaling. Using binar y signaling, he conjectures conver gence to the large system limit. Here, we extend M ontanari’ s result, to show that his algorithm conv erges also for non -rando m CDMA sequenc es when binary signaling is used, und er weaker co nditions. Anoth er advantage of our work is that we allow different noise levels p er bit transmitted. The paper is organized as follows. Sectio n II fo rmulates the problem of linear detection an d p resents the distributed GaBP- based lin ear d etection scheme. Section III describes a novel construction for efficiently computing the MMSE detector . The relation to a factor graph co nstruction is explored in Section IV. New conv ergence resu lts for Montanari’ s work are presented in Section V. W e con clude in Section VI. I n the Append ix we further explo re the relation to M ontanar i’ s work. W e shall use th e following notations. T he oper ator {·} T stands f or a vector or matr ix tr anspose, the matrix I N is a N × N identity matrix, while the symbols {·} i and {·} ij denote entries of a vecto r an d matrix, respectively . N( i ) is the set o f graph no de con nected to nod e i . I I . L I N E A R D E T E C T I O N V I A B E L I E F P RO PAG A T I O N Consider a discrete-time channel w ith a real input vec- tor x = { x 1 , . . . , x K } T governed by an arbitrary p rior distribution, P x , and a correspo nding real outpu t vector y = { y 1 , . . . , y K } T = f { x T } ∈ R K . Here, th e fu nction f {·} denotes the c hannel transfo rmation. By defin ition, linear de- tection com pels the d ecision rule to be ˆ x = ∆ { x ∗ } = ∆ { A − 1 b } , (1) where b = y is th e K × 1 observation vector and the K × K matrix A is a positiv e-definite symmetric matr ix approx imating the chann el transformatio n. T he vector x ∗ is the solution (over R ) to Ax = b . Estimation is completed by adjusting the (inverse) m atrix-vector produ ct to the input alphabet, dictated by P x , acc omplished by using a proper clipping fu nction ∆ {·} ( e.g. , for binary signaling ∆ {·} is the sign fun ction). For example, linear channels, which app ear extensively in many applications in communication and data storage systems, are chara cterized by th e linear r elation y = f { x } = Rx + n , where n is a K × 1 additi ve noise vector and R = S T S is a positive-definite symmetric matrix, often known as the correlation ma trix. The N × K matrix S d escribes th e p hysical channel medium wh ile the vector y co rrespon ds to th e outp ut of a bank of filters match ed to th e phy sical channel S . Assuming linear chan nels with A WGN with variance σ 2 as the ambient noise, the g eneral linear detection rule (1) can describe known linear detectors. For example [1 6], [17] : • The conventional matche d filter (MF) detector is o btained by taking A , I K and b = y . This detector is optimal, in the MA P-sense, for the case of zero cross-cor relations, i.e. , R = I K , as happe ns fo r orthog onal CDMA or wh en there is no ISI effect. • The decorrelato r (zero forcin g eq ualizer) is achieved by substituting A , R and b = y . It is optimal in the noiseless case. • The lin ear minim um mean-square err or (MMSE) d etector can also be de scribed by using A = R + σ 2 I K . This de- tector is k nown to be optim al when the input distribution P x is Gaussian. In gen eral, line ar detectio n is suboptimal becau se of its deterministic und erlying mechanism ( i.e. , solving a given set of linear equation s), in co ntrast to other estimatio n scheme s, such as MAP or maxim um likelihood , tha t e merge from an optimization criter ion. In [19], linear detection, in its g eneral form (1), was implemented using an efficient message-passing algorithm. The linea r detectio n problem was shifted from an algebraic to a probab ilistic d omain. Instead o f solving a de terministic vector-matrix line ar equation, an in ference problem is so lved in a graph ical mode l describing a certain Gaussian d istribution function . Gi ven the overall channel m atrix R and the o bser- vation vector y , on e knows how to write explicitly p ( x ) and the correspon ding graph G with edg e potentials (‘comp atibility function s’) ψ ij and self-potentials (‘evidence’) φ i . These grap h potentials are dete rmined accordin g to th e f ollowing pair wise factorization of th e Gaussian distribution p ( x ) p ( x ) ∝ K Y i =1 φ i ( x i ) Y { i,j } ψ ij ( x i , x j ) , resulting in ψ ij ( x i , x j ) , ex p( − x i R ij x j ) and φ i ( x i ) = ex p  b i x i − R ii x 2 i / 2  . The set of edges { i, j } correspo nds to the set of all non-zero entries of A for which i > j . Hence, we would like to calcu late the marginal densities, which must also be Gaussian, p ( x i ) ∼ N ( µ i = { R − 1 y } i , P − 1 i = { R − 1 } ii ) , where µ i and P i are the marginal mean a nd inv erse variance (a.k.a. precision), respectively . It is shown that the infer red mean µ is identical to the desired so lution x ∗ = R − 1 y . T able I lists the GaBP algorithm up date rules. T ABLE I C O M P U T I N G A − 1 b V I A G A B P. O N L I N E M ATL A B I M P L E M E N TA T I O N I S P R OV I D E D I N [ 2 5 ] . # Stage Operation 1. Initialize Compute P ii = A ii and µ ii = b i / A ii . Set P ki = 0 a nd µ ki = 0 , ∀ k 6 = i . 2. Iterate Propagate P ki and µ ki , ∀ k 6 = i such that A ki 6 = 0 . Compute P i \ j = P ii + P k ∈ N ( i ) \ j P ki and µ i \ j = P − 1 i \ j ( P ii µ ii + P k ∈ N( i ) \ j P ki µ ki ) . Compute P ij = − A ij P − 1 i \ j A j i and µ ij = − P − 1 ij A ij µ i \ j . 3. Check If P ij and µ ij did no t converge, retur n to # 2. Else, co ntinue to #4. 4. Infer P i = P ii + P k ∈ N( i ) P ki , µ i = P − 1 i ( P ii µ ii + P k ∈ N( i ) P ki µ ki ) . 5. Decide ˆ x i = ∆ { µ i } I I I . D I S T R I B U T E D I T E R AT I V E C O M P U TA T I O N O F T H E M M S E D E T E C T O R In this section, we efficiently extend the a pplicability of th e propo sed GaBP-based solver fo r sy stems with symm etric ma- trices [19] to systems with any sq uare ( i.e. , also nonsymme t- ric) or rectang ular matrix. W e first co nstruct a new symmetric data matrix ˜ R b ased on an arbitrary ( non-r ectangular ) matrix S ∈ R k × n ˜ R ,  I k S T S − Ψ  ∈ R ( k + n ) × ( k + n ) . (2) Additionally , we define a new vector of variables ˜ x , { ˆ x T , z T } T ∈ R ( k + n ) × 1 , whe re ˆ x ∈ R k × 1 is th e (to b e shown) solution vector and z ∈ R n × 1 is an aux iliary hidden vector , and a new ob servation vector ˜ y , { 0 T , y T } T ∈ R ( k + n ) × 1 . Now , we would like to show that so lving the symmetric linear system ˜ R ˜ x = ˜ y and ta king the first k entries of the correspo nding solution vector ˜ x is equivalent to solving the original (not necessarily symmetric) system Rx = y . Note that in th e new construc tion the ma trix ˜ R is sparse again, and has only 2 nk off-diagonal nonzer o elements. When run ning the Ga BP a lgorithm we have o nly 2 nk messages, instead of n 2 in th e previous c onstruction . Writing explicitly the sym metric linear system’ s equation s, we ge t ˆ x + S T z = 0 , S ˆ x − Ψ z = y . Thus, ˆ x = Ψ − 1 S T ( y − S ˆ x ) , and extracting ˆ x we hav e ˆ x = ( S T S + Ψ) − 1 S T y . Note, that wh en the n oise level is zero, Ψ = 0 m × m , we get the Mo ore-Pen rose pseudo in verse solution ˆ x = ( S T S ) − 1 S T y = S † y . I V . R E L A T I O N T O FAC T O R G R A P H In this section we g iv e a n alternate proo f of th e corr ectness of our con struction. Given the in verse covariance m atrix ˜ R defined in (2), and the shift vector ˜ x we can deriv e the matching self and edg e poten tials ψ ij ( x i , x j ) , ex p( − x i R ij x j ) φ i ( x i ) , ex p( − 1 / 2 x i R 2 ii x i − x i y i ) which is a factorization o f the Gaussian system distribution p ( x ) ∝ Y i φ i ( x i ) Y i,j ψ ij ( x i , x j ) = = Y i ≤ k φ i ( x i ) Y i>k φ i ( x i ) Y i,j ψ ij ( x i , x j ) = = Y i ≤ k prior on x z }| { exp( − 1 2 x 2 i ) Y i>k exp( − 1 2 Ψ i x 2 i − x i y i ) Y i,j exp( − x i R ij z}|{ S ij x j ) Next, we show the relation of o ur co nstruction to a factor graph. W e will use a factor gr aph with k nodes to the left ( the bits tr ansmitted) an d n n odes to the r ight ( the sign al received), shown in Fig. 1 . Usin g the defin ition ˜ x , { ˆ x T , z T } T ∈ R ( k + n ) × 1 the vector ˆ x represents the k input bits and th e vector z represents the signal received. Now we can wr ite the system prob ability as: p ( ˜ x ) ∝ Z ˆ x N ( ˆ x ; 0 , I ) N ( z ; S ˆ x , Ψ) d ˆ x It is known that the marginal distribution over z is: = N ( z ; 0 , S T S + Ψ) This distribution is G aussian, with the f ollowing parame ters: E ( z | ˆ x ) = ( S T S + Ψ) − 1 S T y C ov ( z | ˆ x ) = ( S T S + Ψ) − 1 It is interesting to note that a similar co nstruction was u sed by Frey [26] in his seminal 1999 work when discussing the factor analysis learning p roblem. While his work is beyond the scope of this paper, it can be shown that h is alg orithm can be mod elled using the GaBP alg orithm. V . N E W C O N V E R G E N C E R E S U LT S One of the benefits of using our new con struction is th at we prop ose a new mechanism to provide future con vergence results. I n the Appe ndix we prove that M ontanari’ s algorithm is an instance of ou r algorithm , th us our co n vergence r esults apply to Montanari’ s algor ithm as well. W e know that if the matrix ˜ R is strictly diag onally do mi- nant, then Ga BP co n verges and the marginal mean s conv erge K bits sent n bits received CDMA channel linear transformation K bits sent n bits received CDMA channel linear transformation Fig. 1. Fact or graph describing the linear channel to the tru e m eans [ 27, Claim 4]. No ting that the matrix ˜ R is symmetric, we can determine the applicability of this condition by examining its c olumns. Ref erring to (4) we see tha t in the first k co lumns, we h av e the k CDMA sequences. W e assume rando m-spread ing binary CDMA seq uences which are normalized to o ne. In other word s, the absolute sum of each column is √ n . In that case, the matrix ˜ R is n ot diagon ally dominan t ( DD). W e can add a regularization ter m o f √ n + ǫ to force the m atrix ˜ R to b e DD, but we pa y in ch anging the problem . In the next n column s o f th e matrix ˜ R , we h av e the diagona l covariance matrix Ψ with different no ise le vels per bit in the main diagon al, and z ero elsewhere. Th e ab solute sum of each column of S is k / √ n , thus when the noise level of each bit satisfies Ψ i > k / √ n , we have a conver gence g uarantee. Note, that the convergence con dition is a sufficient c ondition . Based on Montan ari’ s work, we also know th at in the large system limit, th e algorithm conver ges for bin ary signaling, ev en in th e ab sence of noise. An area of f uture work is to utilize this observation to identify CDMA sch emes with matr ices S that wh en fitted into the m atrix ˜ R are eith er DD, or c omply to the spectral radius conv ergence cond ition of [28]. V I . C O N C L U S I O N W e pr esented a novel distributed algo rithm for co mput- ing the MMSE d etector for the CDMA multiuser d etection problem . Our work utilizes the Gaussian Belief Propaga tion algorithm while imp roving two existing constru ctions [ 19], [24] in this field. Althou gh we describ ed our a lgorithm in the c ontext of m ultiuser de tection, it h as wider applicability . For example, it p rovides an efficient iterative method fo r computin g the M oore-Pen rose pseu doinv erse, and it can also be applied to th e factor analy sis learning problem [26]. A P P E N D I X : M O N TA NA R I ’ S A L G O R I T H M I S A N I N S TA N C E O F O U R A L G O R I T H M In this section we show that Montanari’ s alg orithm is an instance of o ur alg orithm. Our a lgorithm is more general. First, we allo w different noise le vel f or each recei ved bit, unlike his work which u ses a single fixed noise fo r the whole system. I n p ractice, the bits are tran smitted using different frequen cies, thus suffering from d ifferent noise lev els. Seco nd, the upda te ru les in his paper are fitted o nly to the rand oml- spreading CDMA cod es, wh ere the matrix A contains on ly values which ar e drawn uniform ly from { − 1 , 1 } . Assuming binary sign alling, h e co njectures convergence to the large system limit. Our new con vergence proof holds for an y CDMA matrices pr ovided th at the absolu te sum of the ch ip sequ ences is one, und er weaker c ondition s on the n oise level. Third , we pro pose in [ 19] an e fficient br oadcast version for saving messages in a bro adcast suppor ting network . The pr obability distribution of the factor graph used by Montanar i is: dµ N ,K y = 1 Z N ,K y N Y a =1 exp( − 1 2 σ 2 ω 2 a + j y a ω a ) K Y i =1 exp( − 1 2 x 2 i ) · Y i,a exp( − j √ N s ai ω a x i ) dω Extracting the self and edge potentials from the ab ove probab ility distribution: ψ ii ( x i ) , ex p( − 1 2 x 2 i ) ∝ N ( x ; 0 , 1) ψ aa ( ω a ) , ex p( − 1 2 σ 2 ω 2 a + j y a ω a ) ∝ N ( ω a ; j y a , σ 2 ) ψ ia ( x i , ω a ) , ex p( − j √ N s ai ω a x i ) ∝ N ( x ; j √ N s ai , 0) For conv enience, T able I I p rovides a translation b etween the notations used in th is paper (taken fro m [ 19]) and that u sed by Mon tanari et a l. in [24] : T ABLE II S U M M A R Y O F N O TA T I O N S This work [19] Montanari el al. [24] Descripti on P ij λ ( t +1) i → a precisio n msg from left to right ˆ λ ( t +1) a → i precisio n msg from right to left µ ij γ ( t +1) i → a mean m sg from left to right ˆ γ ( t +1) a → i mean m sg from right to left µ ii y i prior mean of left node 0 prior m ean of right node P ii 1 prior precision of left node Ψ i σ 2 prior precision of right node µ i G i L i posterior m ean of node P i L i posterior precision of node A ij − j s ia √ N cov ariance A j i − j s ai √ N cov ariance j j = √ − 1 Now we derive Mon tanari’ s up date rules. W e star t with the precision message from left to right: P ij z }| { λ ( t +1) i → a = 1 + 1 N Σ b 6 = a s 2 ib λ ( t ) b → i = = P ii z}|{ 1 +Σ b 6 = a P ki z }| { 1 N s 2 ib λ ( t ) b → i = P ii z}|{ 1 − Σ b 6 = a − A ij z }| { − j s ib √ N ( P j \ i ) − 1 z }| { 1 λ ( t ) b → i A ji z }| { − j s ib √ N . By lookin g at T able 1 , it is easy to verif y th at this precision update rule is equ iv alen t to that in #2 of T able I. Using the same logic w e get th e precision message from right to left: P ji z }| { ˆ λ ( t +1) i → a = P ii z}|{ σ 2 + − A 2 ij P − 1 j \ i z }| { 1 N Σ k 6 = i s 2 ka λ ( t ) k → a The mean message fro m left to right is given by γ ( t +1) i → a = 1 N Σ b 6 = a s ib λ ( t ) b → i ˆ γ ( t ) b → i = = µ ii z}|{ 0 − Σ b 6 = a − A ij z }| { − j s ib √ N P − 1 j \ i z }| { 1 ˆ λ ( t ) b → i µ j \ i z }| { ˆ γ ( t ) b → i . The same ca lculation is done for the mean fro m r ight to lef t: ˆ γ ( t +1) i → a = y a − 1 N Σ k 6 = i s ka λ ( t ) k → a γ ( t ) k → a Finally , the lef t no des calculated th e pr ecision and m ean by G ( t +1) i = 1 √ N Σ b s ib λ ( t ) b → i ˆ γ ( t ) b → i , J i = G − 1 i L ( t +1) i = 1 + 1 N Σ b s 2 ib λ ( t ) b → i , µ i = L i G − 1 i . The ke y dif ference be tween the two construction s is th at Montanar i u ses a directed factor gr aph while we use an und i- rected g raphical model. As a con sequence , our construction provides additional convergence results and simpler u pdate rules. R E F E R E N C E S [1] J. Pearl, P r obabili stic R easoning in Intellig ent Systems: Networks of Plausible Inferenc e . San Francisco: Morgan Kaufmann, 1988. [2] Y . Kabashima, “ A CDMA multiuser detection algorithm on the basis of belie f propagat ion, ” J. Phys. A: Math . Gen. , vol. 36, pp. 11 111–11 121, Oct. 2003. [3] J. P . Neirotti and D. Saad, “Improved message passing for inference in densely conne cted systems, ” Europhy s. Lett. , vol . 71, no. 5, pp. 866–872, Sept. 2005. [4] ——, “Efficie nt Bayesian inference for learning in the Ising linear percept ron and signal detect ion in CDMA, ” P hysica A , vol. 365, pp. 203–210, Feb. 2006. [5] T . T anaka and M. Okada, “ Approximat e bel ief propagation, density e volu tion, and statisti cal neurodynamics for CDMA multiu ser detect ion, ” IEEE Tr ans. Inform. Theory , vol. 51, no. 2, pp. 700–706, Feb . 2005. [6] F . Kschischang, B. Frey , and H. A. Loeliger , “Factor graphs and the sum-product algorit hm, ” IEEE T rans. Inform. Theory , vol. 47, pp. 498– 519, Feb . 2001. [7] L. R. Bahl, J. Cocke, F . Jelinek, and J. Ravi v , “Optimal decodi ng of linea r codes for minimizing symbol error rate, ” IE EE T rans. Inform. Theory , vol. 20, no. 3, pp. 284–287, Mar . 1974. [8] B. M. Kurk oski, P . H. Siege l, and J. K. W olf, “Joint message-passing decodin g of LDPC codes and partial -response chann els, ” IEEE T rans. Inform. Theory , vol. 48, pp. 1410–1422, June 2002. [9] ——, “Correct ion to ’Joint message-passing decoding of LD PC codes and part ial-respo nse channel s’, ” IEEE T rans. Inform. Theory , vol . 49, p. 2076, Aug. 2003. [10] M. Marrow and J. K. W olf, “Itera ti ve detect ion of 2 -dimensional ISI channe ls, ” in Pro c. IEEE Inform. T heory W orkshop (ITW) , Paris, France, Mar . 2003, pp. 131–134. [11] O. Shental, N. Shental, A. J. W eiss, and Y . W eiss, “General ized belief propagat ion rec ei ver for near -optimal detection of two-dimensiona l channe ls with m emory , ” in P r oc. IEEE Information Theory W orkshop (ITW) , San Antonio, T exas, USA, Oct. 2004, pp. 225–229. [12] O. Shental, N. Shental, and S. Shamai (Shitz), “On the achie vab le informati on rates of two-dimensiona l channel s with memory , ” in P r oc. IEEE Int. Sy mp. Inform. Theory (ISIT) , Ade laide, Australia, Sept. 2005 . [13] O. Shental, N. Shental, S. Shamai (Shitz), I. Kanter , A. J. W eiss, and Y . W eiss, “Finite -state input two-dimensi onal Gaussian channel s with memory: Estimation and information via graphical m odels and statistic al mechanic s, ” IEE E T rans. Inform. Theory , vol. 54, pp. 1500 – 1513, April 2008. [14] A. Mon tanari and D. Tse, “ Analysis of belief propagati on for non-l inear problems: The e xample of CDMA (or: Ho w to prov e T anaka’ s formula), ” in Proc . IEEE Inform. Theory W orkshop (ITW) , Punta del Este, Uruguay , Mar . 2006. [15] C. C. W ang and D. Guo, “ Belief propaga tion is asymptotic ally equiv alent to MAP dete ction for sparse linear systems, ” in Proc . 44th Allert on Conf. on Communication s, Contr ol and Computing , Mont icello , IL , USA, Sept. 2006. [16] S. V erd ´ u, Mult iuser Detection . Cambridg e, UK: Cambridge Uni versity Press, 1998. [17] J. G. Proakis, Digital Communicat ions , 4th ed. Ne w Y ork, USA: McGra w-Hill, 2000. [18] B. R. V oj ˇ ci ´ c and W . M. Jang, “Tra nsmitter precoding in synchronous multiuser communicat ions, ” IEEE T rans. Commun. , vol. 46, no. 10, pp. 1346–1355, Oct. 1998. [19] D. Bickson, O. Shental, P . H. Siegel, J. K. W olf, and D. Dole v , “Linear dete ction via belief propagati on, ” in Pr oc. 45t h Allerton Conf. on Communicat ions, Contr ol and Computing , Monticell o, IL, USA, Sept. 2007. [20] O. Shen tal, D. Bickson, P . H. Siegel , J. K. W olf, and D. Dole v , “Gaussian belie f propagati on solver for systems of linear equatio ns, ” in IEEE Int. Symp. Inform. Theory (ISIT) , T oronto, Canada, July 2008. [21] A. Grant and C. Schlege l, “It erati ve impleme ntation s for linear multiuser detec tors, ” IEEE T rans. Commun. , vol. 49, no. 10, pp. 1824–1834 , Oct. 2001. [22] P . H. T an and L. K. Rasmussen, “Linear inter ference cance llatio n in CDMA based on iterati ve techniques for linear equati on systems, ” IEEE T rans. Commun. , vol. 48, no. 12, pp. 2099–2108, Dec. 2000. [23] A. Y ener , R. D. Y ates, , and S. Ulukus, “CDMA multiuser detecti on: A nonlinea r programming approa ch, ” IE EE T rans. Commun. , vol. 50, no. 6, pp. 1016–1024, June 2002. [24] A. Montanari , B. Prabhakar , and D. T se, “Belie f propagation based multi-user detecti on, ” in Pr oc. 43th Allerton Conf. on Communications, Contr ol and Computing , Monticello, IL, USA, Sept. 2005. [25] Gaussian Beli ef Propagat ion implementatio n in m atlab [online] http://www.cs. huji.ac.il/labs /danss/p2p/gab p/ . [26] B. J. Frey , “Local probability propagati on for factor analysi s, ” in N eural Informatio n Pr ocessing Systems (NIPS) , Den ver , Colorado, Dec 1999, pp. 442–448. [27] Y . W eiss and W . T . Freeman, “Correctn ess of belief propagatio n in Gaussian graphica l models of arbitrary topology , ” Neural Computati on , vol. 13, no. 10, pp. 2173–2200, 2001. [28] J. K. J ohnson, D. M. Malioutov , and A. S. Wil lsky , “W alk-sum inter- pretat ion and analysis of Gaussian belief propagation, ” in Advances in Neural Information Pro cessing Syst ems 18 , Y . W eiss, B. Sch ¨ olkopf, and J. Platt, E ds. Cambridge, MA: MIT Press, 2006, pp. 579–586.