Analysis of Verification-based Decoding on the q-ary Symmetric Channel for Large q

1 Analysis of V eriﬁcat ion-based Decoding on the q -ary Symmetric Channel for Lar ge q Fan Zhang and He nry D. Pﬁs ter Department of Elec trical and Comp uter Engineering, T exas A&M Univ ersity { fanzhang,hpﬁs ter } @tamu.edu Abstract —A new ve riﬁcation-based message-passing decoder fo r low-density parity-check (LDPC) codes is introduced and analyzed fo r th e q -ary symmetric channel ( q -SC). Rather th an passing messages consisting of symbol probabilities, this deco der passes lists of possib le symbols and marks some lists as veriﬁed. The d ensity evo lution (DE) equ ations for this decoder are derived and used to compute decoding thresholds. If the maximum list size is unbounded, then one ﬁnds that any capacity-a chieving LDPC code for the bin ary erasure channel can be used to achieve capacity on the q -SC fo r large q . The decoding t hresholds are also computed via DE f or th e case where each list is truncated to satisfy a maximum list-size constraint. Simul ation r esults are also presented to conﬁrm th e DE results. Durin g the simulations, we observ ed d ifferences between two v eriﬁcation-based decoding algorithms, introduced by Luby and Mitzenmacher , that were implicitly assumed to be id entical. In this paper , the node-based algorithms are e valuated via analysis and simulation. The pr obability of false veriﬁcation (FV) is also considered and techniq ues ar e discu ssed to mitigate the F V . Optimization of th e degree d istribution is also used to i mpro ve the th reshold fo r a ﬁxed maximum list size. Finally , the pr oposed algorithm i s compared with a variety of other algorithms usin g both density ev olution thresholds and simulation results. Index T erms —low density parity check codes, message passing decoding, q-ary symmetric channel, veriﬁcation decoding, l ist decoding, false veriﬁcation. I . I N T RO D U C T I O N Low-density parity-c heck (LDPC) co des ar e lin ear co des that were introduced by Gallager in 1962 [1] and re-discovered by MacKay in 1995 [2]. The ensemble of LDPC codes that we consider (e.g. see [3] an d [4]) is deﬁned by the edg e degree d istribution ( d.d.) functions λ ( x ) = P k ≥ 2 λ k x k − 1 and ρ ( x ) = P k ≥ 2 ρ k x k − 1 . The stand ard en coding and d ecoding algorithm s are based o n the b it-level operation s. Howe ver, when applied to the tra nsmission of data packets, it is natural to per form the enc oding and deco ding algorith m at the p acket lev el rather than the bit level. For example, if we are goin g to transmit 32 bits as a packet, then we can use er ror-correcting codes over the, rather large, a lphabet with 2 32 elements. Let th e r .v .s X and Y be the inp ut and o utput, respe cti vely , of a q -ary symmetric ch annel ( q -SC) whose tran sition prob a- bilities are Pr( Y = y | X = x ) =  1 − p if x = y p/ ( q − 1) if x 6 = y , This material is based upon work supported by the National Science Founda tion under Grant No. 0747470. Any opi nions, ﬁndings, conclusions, or rec ommendation s expressed in this materia l are tho se of the auth ors and do not necessarily reﬂect the views of the Nation al Science Founda tion. where x, y ∈ GF ( q ) . The cap acity of the q - SC, 1 + (1 − p ) log q (1 − p ) + p log q p − p log q ( q − 1) , is appro ximately equal to 1 − p symbols per chan nel u se for large q . This implies the numb er of symbo ls which can b e reliab ly transmitted per channel use of th e q -SC with la rge q is approxim ately equal to that of the BEC with erasur e probability p . Moreover , the behavior of the q -SC with large q is similar to the BEC in the sense that: i) incorr ectly received symb ols fro m the q -SC provide almo st no inform ation abou t the transmitted symbol and ii) err or detection (e.g. , a CRC) can be added to each symbol with negligible overhead [5]. Binary LDPC codes for the q - SC with mo derate q are propo sed and op timized based on EXIT charts in [6] a nd [7]. It is kn own that the complexity of the FFT -based belief- propag ation alg orithm, for q -ary LDPC codes, scales like O ( q log q ) . Even for mod erate sizes of q , such as q = 256 , this ren ders such alg orithms imp ractical. Howe ver, wh en q is large, an interesting effect can be used to facilitate decoding : if a symbo l is recei ved in error, then it is essentially a randomly chosen elemen t of the alphabet and it is very unlikely that the parity-ch eck equations in volving this symbol are satisﬁed. Based on this idea, Luby and Mitzenm acher dev elop a n elegant algo rithm for de coding LDPC codes on the q - SC for large q [ 8]. Howev er , their paper did no t present simulation results and lef t capacity-achieving ensembles as an interesting open p roblem. Metzn er presen ted similar ideas ea rlier in [9] and [1 0], but the focus and ana lysis is qu ite dif ferent. Dav ey and M acKay a lso de velop and an alyze a symb ol-level message-passing deco der over small ﬁn ite ﬁelds in [11]. A number of approach es to th e q -SC (for large q ) based o n interleaved Reed-Solomon codes are also possible [5] [12]. In [13], Shokro llahi and W ang discuss two ways of approachin g capacity . Th e ﬁrst uses a two-stage app roach wh ere the ﬁrst stage uses a T o rnado co de and v eriﬁcation deco ding. The second is, in fact, equiv a lent to on e of the decod ers we d iscuss in this paper . 1 When we discovered this, the authors were kind enoug h to send us an extended ab stract [14] which contains more details. Still, the authors did not conside r the theoretical perfor mance with a maximum list-size constrain t, the actual perfor mance of the decoder via simulation, or false veriﬁcation (FV) due to cycles in the decodin g graph . In this paper, we describe the algorithm in d etail and con sider those d etails. Inspired by [8], we introd uce list-message-passing ( LMP) 1 The descrip tion of the second method in [13] is very brief and we belie ve its cap acity- achie ving nature deserves furth er attent ion. 2 decodin g with veriﬁcation for LDPC co des on the q -SC. Instead of p assing a single value between symbo l and chec k nodes, we p ass a list o f candidates to im prove the decoding threshold. This mo diﬁcation also increases the p robab ility of FV . So, we analyze the causes of FV and d iscuss techniques to mitigate FV . It is worth noting that the LMP decoder we consider is somewhat d ifferent than the list extension suggested in [8]. Th eir approach uses a peelin g-style decoder based on veriﬁcation rather tha n er asures. Also, the algor ithms in [8] are proposed in a node-b ased (N B) style but analyzed using message-based (MB) d ecoders. It is implicitly assumed that the two approach es are equivalent. In fact, this is not always true. In th is p aper, we con sider the d ifferences between NB and M B decoder s and derive an asymp totic ana lysis fo r NB decod ers. The paper is organized as follows. In Section II, we descr ibe the LMP algor ithm a nd use den sity evolution (DE) [15] to analyze its p erform ance. The dif ference betwee n NB and MB decoder s f or the ﬁrst (LM1) and seco nd algo rithm (LM2) in [8] is d iscussed and th e NB d ecoder a nalysis is d erived in Section III, respecti vely . The error ﬂoor of the LMP algorithms is considered in Section IV. In Section V, dif ferential e volution is used to optimize c ode en sembles and simulation results are compare d with the theo retical thresholds. The results ar e also compare d with previously pu blished results from [ 8] and [13]. In Section V I, simulation results ar e p resented. Ap plications of the LMP algorithm are discussed and conclusion s are given in Section VII. I I . D E S C R I P T I O N A N D A N A L Y S I S A. Description o f the Deco ding Algorithm The LMP d ecoder we discuss is design ed mainly for th e q -SC and is based on local deco ding o perations ap plied to lists o f messages containing p robable co dew ord symbols. The list me ssages passed in the graph have th ree types: veriﬁed (V), unveriﬁed (U) and erasur e (E). Every V -message has a symbol v alue associated with it. Every U- message h as a list of symbols associated with it. Follo wing [8], we mark messages as veriﬁed wh en they ar e very lik ely t o be correct. In particular , we will ﬁnd that the p robability of FV appr oaches zero as q goes to inﬁnity . The LMP decod er work s by p assing list-messages aro und the decod ing grap h. Instead of passing a single co de symbol (e.g., Gallager A/B algorithm [1]) or a pr obability distribution over all possible code sym bols (e.g. , [11]), we pa ss a list of values th at are mor e likely to be correct th an th e o ther messages. At a check no de, the output list contain s all symbols which cou ld satisfy the chec k constraint fo r the given in put lists. At the check no de, the output message will be veriﬁed if and only if all the incoming messages are veriﬁed. At a node of degree d , the associativity and co mmutativity o f the node- processing oper ation allo w it to be d ecomposed into ( d − 1 ) basic 2 operation s (e.g ., a + b + c + d = ( a + b ) + ( c + d ) ). In such a schem e, the com putational com plexity of eac h basic operation is prop ortional to s 2 at the check node and s ln s 2 Here we use “basic“ to emphasize that it maps two list-messages to a single list message. at the variable node 3 , where s is the list size of the input list. The list size grows r apidly as the numbe r of iterations increases. I n or der to make th e algorithm pr actical, we have to trun cate the list to keep the list size within some maximum value, denoted S max . In ou r analysis, we also ﬁnd tha t, after the number o f iterations exceeds half the g irth o f the d ecoding graph, the probability of FV increases very rapidly . W e analyze the reasons of FV and classify the FV’ s into two types. W e ﬁnd that the codes descr ibed in [8] and [13] both suffer from type-II FV . In Section IV, we an alyze th ese FV’ s and prop ose a schem e that redu ces the prob ability of FV . The message-p assing decoding algorith m using list mes- sages (or LMP) applies the following simple rules to calculate the outpu t messages for a check n ode: • If all the inpu t m essages are veriﬁed, the n the output becomes veriﬁed with the value which makes all the incoming messages su m to z ero. • If any inpu t m essage is a n e rasure, then the outp ut message beco mes an erasur e. • If th ere is no er asure on the inp ut lists, then the outp ut list co ntains a ll sy mbols which could satisfy the check constraint for th e given input lists. • If the outp ut list size is larger th an S max , then the outp ut message is an erasure. It a pplies the following rules to calculate the output message s of a v ariable node: • If all the in put messages are er asures or there are m ultiple veriﬁed me ssages wh ich disagree, then o utput message is the chann el receiv ed value. • If any of the inpu t messages a re veriﬁed (and there is no disagreeme nt) o r a symbo l appear s mo re than once, then the output message becom es veriﬁed with the same value as th e veriﬁed input message or the symbo l which appears more than once. • If there is no veriﬁed message on the inp ut lists an d no symbol ap pears mo re tha n o nce, then the ou tput list is the union o f all input lists. • If the outp ut message has list size larger than S max , then the output message is the recei ved v alue from the channel. B. DE for Unbound ed List Size Decodin g Algorithm T o apply DE to the LMP decoder with unbou nded list sizes, d enoted LMP- ∞ (i.e., S max = ∞ ), we consider th ree quantities which ev olve with the iter ation nu mber i . Let x i be th e prob ability that th e correct sy mbol is no t on the list passed from a variable n ode to a ch eck n ode. Let y i be the probab ility that the message passed fr om a variable node to a check node is not veriﬁed. Let z i be the av erage list s ize passed from a variable nod e to a chec k no de. Th e same v ariables are “marked” ( ˜ x i , ˜ y i , ˜ z i ) to repr esent the same v alu es for messages passed fro m the check nod es to the variable nodes (i.e., the half-iteration value). W e also assume all the m essages are indepen dent, that is, we assum e that the bipartite gra ph has girth gr eater than twice the numb er of deco ding iterations. 3 The basic operation at the variab le node can be done by s binary searches of length s and the complexi ty of a binary search of lengt h s is O (ln s ) 3 First, we consider the probab ility , x i , that the correct symbol is n ot o n th e list. For any degree- d check no de, the corr ect message sym bol will only be on the edge ou tput list if all o f the other d − 1 input lists co ntain their cor respondin g cor rect symbols. This implies that ˜ x i = 1 − ρ (1 − x i ). For any degree- d variable node, the correct message symbo l is not on the edge output list only if it is not on any of th e other d − 1 edg e inp ut lists. This implies that x i +1 = pλ ( ˜ x i ) . This behavior is very similar to er asure decoding of LDPC c odes on the BEC an d giv es the id entical update eq uation x i +1 = pλ (1 − ρ (1 − x i )) (1) where p is the q - SC error p robability . Note that th rough out the DE analysis, we assume that q is sufﬁciently large. Next, we consider the p robability , y i , that the m essage is not veriﬁed. For any d egree- d check nod e, an edg e ou tput message is veriﬁed o nly if all o f the other d − 1 edge inpu t messages are veriﬁed. For a ny d egree- d variable nod e, an edge outpu t message is veriﬁed if any symbol on the other d − 1 edge input lists is veriﬁed o r occurs twice which implies ˜ y i = 1 − ρ (1 − y i ) . The event that the outpu t message is not veriﬁed can be broken into the union of two disjoint events: (i) the correct symbo l is not on any of the inpu t lists, and (ii) the sy mbol fro m the chan nel is inco rrect and the correct symbol is on exactly one of th e input lists an d not veriﬁed. For a degree- d v ariable node, th is implies that Pr( not veriﬁed ) = ( ˜ x i ) d − 1 + p ( d − 1) ( ˜ y i − ˜ x i ) ( ˜ x i ) d − 2 . (2) Summing over the d. d. giv es the u pdate equation y i +1 = λ ( 1 − ρ (1 − x i )) + p ( ρ (1 − x i ) − ρ (1 − y i )) λ ′ (1 − ρ (1 − x i )) . (3) It is impo rtant to note that (1) and (3) were published ﬁrst in [13, Thm. 2] ( by mapping x i = p i and y i = p i + q i ), but were derived in depend ently by us. Finally , we co nsider the average list-size z i . For any degree- d check node, the output list size is equal 4 to the produ ct of the sizes of the other d − 1 in put lists. Since the m ean of the prod uct of i.i.d . random variables is equal to the p roduct of th e mean s, this implies that ˜ z i = ρ ( z i ) . For any d egree- d variable nod e, the ou tput list size is equal to one 5 plus th e sum of th e sizes of th e other d − 1 input lists if th e output is not veriﬁed a nd one otherwise. Again, the mean of the sum of d − 1 i.i.d. random v ariables is simp ly d − 1 times the mean of the distribution, so the average output list size is given by 1 +  ( ˜ x i ) d − 1 + p ( d − 1) ( ˜ y i − ˜ x i ) ( ˜ x i ) d − 2  ( d − 1) ˜ z i . This gives the update eq uation z i +1 =1 + [ ˜ x i λ ′ ( ˜ x i ) + p ( ˜ y i − ˜ x i ) ( λ ′ ( ˜ x i ) + ˜ x i λ ′′ ( ˜ x i ))] ρ ( z i ) . For the LMP decoding algorithm, the threshold of an ensemb le ( λ ( x ) , ρ ( x )) is deﬁned to be p ∗ , sup  p ∈ (0 , 1]     pλ (1 − ρ (1 − x )) < x ∀ x ∈ (0 , 1]  . 4 It is actually upper bounded because we ignore the possibility of collisions betwee n incorrect entries, but the probabili ty of this occurring is negligi ble as q goes to inﬁnity . 5 A single s ymbol is alw ays recei ved from the channel . Next, we show that some cod es can achie ve cha nnel capacity using this decoding algorithm . Theor em 2.1: Let p ∗ be th e thr eshold of the d.d. p air ( λ ( x ) , ρ ( x )) an d assume that the chann el e rror rate p is less than p ∗ . In this case, th e pro bability y i that a message is no t veriﬁed in th e i -th decod ing iteration satisﬁes lim i →∞ y i → 0 . Moreover , fo r any ǫ > 0 , there exists a q < ∞ such that LMP decodin g of a long random ( λ, ρ ) LDPC code, on a q -SC with error pro bability p , results in a sym bol error rate less than ǫ . Pr oof: See Appen dix A. Remark 2.2 : Note tha t the conv ergence c ondition, p ∗ λ (1 − ρ (1 − x )) < x for x ∈ (0 , 1] , is identical to the BEC ca se but that x has a different meaning . I n the DE eq uation for the q -SC, x is the probab ility that the co rrect value is not on the list. In the DE equation for the BEC, x is the probability that the message is an erasure. This tells us any cap acity- achieving ensemb le f or the BEC is cap acity-achieving for the q -SC with LMP- ∞ algorithm an d large enough q . This also giv es some intuitio n about the behavior of the q -SC fo r large q . F or examp le, when q is large, an inco rrectly received v alue behaves like an erasure [5]. Cor ollary 2.3: The code with d .d. pair λ ( x ) = x and ρ ( x ) = (1 − ǫ ) x + ǫx 2 has a threshold o f 1 − ǫ 1+ ǫ and a rate of r > ǫ 3(1+ ǫ ) . Therefore , it achieves a rate of Θ( δ ) for a channel erro r rate of p = 1 − δ . Pr oof: Follo ws f rom  1 − ǫ 1+ ǫ  λ (1 − ρ (1 − x )) < x fo r x ∈ (0 , 1] and Theorem 2.1. Remark 2.4 : W e believe that Corollary 2.3 provides the ﬁrst linear-time decodable constru ction of rate Θ( δ ) fo r a r andom- error model with erro r pr obability 1 − δ . A discussion of linear-time en codable/d ecodable codes, for both ran dom and adversarial erro rs, can be fou nd in [16]. Th e com plexity also depend s on the required list size which may be extremely large (thoug h indepe ndent of the block length ). Un fortun ately , we do n ot have explicit bound s on the alp habet size or list size required for th is constru ction. In pra ctice, one canno t im plement a list decod er with unbou nded list size. Ther efore, we also evaluate the LMP decoder un der a bo unded list-size assump tion. C. DE for the Decod ing Algorithm with Bo unded List S ize First, we intro duce som e deﬁnitions an d n otation for the DE analysis with bo unded list-size deco ding algo rithm. No te that, in the bo unded list-size LMP alg orithm, each list may contain at most S max symbols. For convenience, we c lassify the messages into four ty pes: (V) V eriﬁed : m essage is veriﬁed an d has list-size 1. (E) Erasur e : me ssage is an erasure and has list-size 0. (L) Corr ect on list : message is not veriﬁed o r erased and the co rrect symbol is on the list. (N) Corr ect not on list : message is not veriﬁed or erased, and the correct symbol is not on th e list. For th e ﬁrst two m essage typ es, we only need to track the fractions, V i and E i , of message types in th e i -th iteration. For the third an d the fourth types of messages, we also need to track the list sizes. Ther efore, we track the gen erating function of the list size for these messages, given by L i ( x ) 4 and N i ( x ) . The coefﬁcient of x j represents the prob ability that the me ssage has list-size j . Speciﬁcally , L i ( x ) is deﬁned by L i ( x ) = S max X j =1 l i,j x j , where l i,j is the pro bability th at, in the i -th d ecoding iteration, the correct symbol is on the list an d the m essage list has size j . Th e fun ction N i ( x ) is deﬁned similarly . This implies that L i (1) is the proba bility that the list contains the correct symbol and that it is not veriﬁed. For the same reason, N i (1) gives the pro bability that the list does no t contain th e correct sym bol and that it is not veriﬁed . F or th e simplicity of expr ession, we denote the overall density as P i = [ V i , E i , L i ( x ) , N i ( x )] . The same variables are “marked” ( ˜ V , ˜ E , ˜ L, ˜ N an d ˜ P ) to represent the same values for messages passed from th e ch eck n odes to the variable nodes ( i.e., the ha lf-iteration value). Using these d eﬁnitions, we ﬁnd that DE can be compu ted efﬁciently using p olynom ial arithm etic. For convenience of analysis and implementation, we u se a sequence of basic operation s p lus a sep arate trun cation operato r to repr esent a multiple-inp ut mu ltiple-outpu t oper ation. W e use ⊞ to deno te the check- node o perator and ⊗ to den ote the variable-no de operator . Using th is, the DE for the variable-node b asic operation P (3) = ˜ P (1) ⊗ ˜ P (2) is given b y V (3) = ˜ V (1) + ˜ V (2) − ˜ V (1) ˜ V (2) + ˜ L (1) (1) ˜ L (2) (1) (4) E (3) = ˜ E (1) ˜ E (2) (5) L (3) ( x ) = ˜ L (1) ( x )  ˜ E (2) + ˜ N (2) ( x )  + ˜ L (2) ( x )  ˜ E (1) + ˜ N (1) ( x )  (6) N (3) ( x ) = ˜ N (1) ( x ) ˜ E (2) + ˜ N (2) ( x ) ˜ E (1) + ˜ N (1) ( x ) ˜ N (2) ( x ) . (7) Note th at (4)-(7) do n ot yet consider the list-size tr uncation and the chan nel value. For the basic check -node operation ˜ P (3) = P (1) ⊞ P (2) , the D E is giv en by ˜ V (3) = V (1) V (2) (8) ˜ E (3) = E (1) + E (2) − E (1) E (2) (9) ˜ L (3) ( z ) = h V (1) L (2) ( z ) + V (2) L (1) ( z ) + L (1) ( x ) L (2) ( y ) i x j y k → z jk (10) ˜ N (3) ( z ) = h N (1) ( x ) N (2) ( y ) + N (1) ( x )  V (2) y + L (2) ( y )  + N (2) ( x )  V (1) y + L (1) ( y ) i x j y k → z jk (11) where th e subscript x j y k → z j k denotes the sub stitution of variables. Finally , th e truncation of lists to size S max is h andled b y tru ncation o perators whic h map den sities to densities. W e use T and T ′ to den ote the truncation opera tion at the check and variable nodes. Speciﬁcally , we truncate terms with d egree higher than S max in th e po lynomials L ( x ) an d N ( x ) . At check nodes, the truncated probability mass is moved to E . At variable no des, lists lo nger than S max entries are replaced by the chann el v alue. Let P ′ i =  ˜ P ⊗ k − 1 i  be the inter mediate density which is the result of applying th e basic oper ation k − 1 times on ˜ P i . After co nsidering the chan nel value and list truncatio n, th e symbol nod e message density is given b y T ′ ( P ′ i ) . T o analyze this, we separa te L ′ i ( x ) into two terms: A ′ i ( x ) with d egree less than S max and x S max B ′ i ( x ) with degree a t least S max . Like wise, we separate N ′ i ( x ) into C ′ i ( x ) and x S max D ′ i ( x ) . Th e inclu sion of th e cha nnel symb ol and th e truncation are combined into a single oper ation P i = T ′  V ′ i , E ′ i , A ′ i ( x ) + x S max B ′ i ( x ) , C ′ i ( x ) + x S max D ′ i ( x )  deﬁned by V i = V ′ i + (1 − p ) ( A ′ i (1) + B ′ i (1)) (12) E i = 0 (13) L i ( x ) = (1 − p ) x ( E ′ i + C ′ i ( x ) + D ′ i (1)) + pxA ′ i ( x ) (14) N i ( x ) = px ( E ′ i + B ′ i (1) + C ′ i ( x ) + D ′ i (1)) . (15) Note th at in (12), the term (1 − p ) ( A ′ i (1) + B ′ i (1)) is due to the fact that messag es are compar ed f or possible veriﬁcation before trun cation. The overall DE recursion is easily written in terms of the forward (symbol to check) density P i and the backward (check to sym bol) density ˜ P i by taking the irregularity into acc ount. The initial density is P 0 = [0 , 0 , (1 − p ) x, px ] , where p is the err or pr obability of th e q -SC ch annel, and the r ecursion is giv en by ˜ P i = d c X k =2 ρ k T  P ⊞ k − 1 i  (16) P i +1 = d v X k =2 λ k T ′  ˜ P ⊗ k − 1 i  . (17) Note that the DE r ecursion is not one-dim ensional. This makes it difﬁcult to o ptimize the ensemb le analytically . It remain s an o pen problem to ﬁnd th e closed -form expression of th e threshold in ter ms of the max imum list size, d.d. pa irs, and the alphab et size q . In section V, we will ﬁx the max imum variable an d ch eck degrees, code rate, q and ma ximum list size and optimize the threshold over the d.d. pairs by using a numerical app roach. I I I . A N A LY S I S O F N O D E - B A S E D A L G O R I T H M S A. Differ entia l Equation Analysis of LM1-NB W e ref er to the ﬁrst and second alg orithms in [8] as LM1 and LM2, r espectiv ely . Each algorithm can be v iewed eit her as message-based (MB) or node-based (NB). The ﬁrst and s econd algorithm s in [13] and [14] are referr ed to as SW1 and SW2. These alg orithms are summarized in T able I. Note that, if no veriﬁcation occurs, the v ariable node (VN) sends the (“channel value”, U) and th e check n ode (CN) sen ds the ( “expected correct value”,U) in all th ese algorithm s. The algorithms SW1, SW2 an d LMP are a ll MB algorithms, but can be m odiﬁed to be NB algorithms. 5 T ABL E I B R I E F D E S C R I P T I O N O F M E S S A G E - P A S S I N G A L G O R I T H M S F O R q - S C Alg. Descripti on LMP- S max LMP as described in Sect ion II-A with maximum list-siz e S max LM1-MB MP decoder that passes (v alue, U / V ). [8, III.B] At VN’ s, outpu t is V if any input is V or message matches cha nnel v alue, othe rwise pass chan nel val ue. At CN’ s, output is V iff all inputs are V . LM1-NB Peeling decoder with VN s tate (value, U / V ). [8, III.B] At CN’ s, if all neighbors sum to 0, all neighbors get V . At CN’ s, if all neighbors bu t one are V , then last is V . LM2-MB The same as LM1-MB with one extra rule. [8, IV .A]. At VN’ s, if two input m essages m atch, then output V . LM2-NB The same as LM1-NB with one ex tra rule. [8, IV .A]. At VN’ s, if two neighb or value s same, then VN gets V . SW1 Identic al to LM2-MB SW2 Identic al to LMP- ∞ . [13 , Thm. 2] 1) Motivation : In [8], the a lgorithms are p roposed in the node-b ased ( NB) style [8, Section III -A and I V], but analyzed in the message-based (M B) style [8 , Section III-B and IV]. It is easy to verify th at the LM1-NB and LM1-MB have identical p erform ance, but this is not tru e fo r the NB and MB LM2 alg orithms. In this section, we will sh ow the differences between the NB decoder and MB d ecoder an d derive a p recise analysis fo r LM1-NB. First, we discuss the equi valence of LM1-MB and LM1-NB. Theor em 3.1: Let A be the set of variable nodes th at are veriﬁed when LM1-N B decoding terminates. For the same received seq uence, let B be the set of variable nodes that have at least o ne veriﬁed o utput message when LM 1-MB terminates. Th en, L M1-NB an d LM1-M B ar e equ iv alen t in the sense th at A = B . Sketch of Pr oof: Thou gh the basic idea behind this result is relativ ely straigh tforward, th e details are some what len gthy and, th erefore , deferred to a mo re gen eral treatm ent [17]. The ﬁrst observation is that the LM1-NB and LM1-MB decoders both satisfy a m onoton icity p roperty that, assuming n o FV , guaran tees co n vergence to a ﬁxed point. In particular, the messages are ord ered with respect to type (e.g., veriﬁed > correct > incorre ct) and vectors of messages are endo wed with the indu ced p artial orde ring. Using this appr oach, it can be shown that the ﬁxed point messages of the LM1 -NB decoder cannot be worse th an those o f the LM1 -MB decoder . Finally , a detailed analysis of the computation tree, for an arbitrary LM1- MB ﬁxed poin t, shows that ev ery nod e veriﬁed by LM1-NB must h av e at least one veriﬁed output message. In the NB decoder, the veriﬁcation status is associated with a n ode. Once a nod e is veriﬁed, all outgoin g messages are veriﬁed. In the MB deco der, t he status is associated with an edg e an d the outgoin g m essage on each ed ge may have a different veriﬁcation status. NB algo rithms can not, in gener al, be analyzed u sing DE be cause the ind epende nce assumption b etween messages does not hold . There fore, we develop peeling-style decod ers, which are equivalent to LM1- NB and LM2-NB, and u se differential equatio ns to analyz e them. It is worth no ting th at the threshold of a node-ba sed algo- rithm is at le ast as large as its message- based cou nterpart. This is because no de-based processing can only result in additional veriﬁcations du ring e ach step. T he intuition b ehind this can be seen by looking at a variable node of degree 3. The message- based decoder may some times outp ut a veriﬁed message on only one edge. This occur s, f or example, if the channel value is inco rrect and the three inp ut messages are V , I , I . But, the nod e-based decoder veriﬁes all output edges in this case. T he drawback is th at the co rrelation caused by node- based processing complicates the an alysis because d ensity ev o lution is based on the assumptio n that all input messages are indep endent. Follo wing [3], we an alyze the peeling -style decoder u sing differential equ ations th at track the a verage n umber of edges (grou ped into types) i n the graph as decoding progresses. From the results of [3] and [18], we know tha t th e a ctual number of e dges ( of any type), in any particular deco ding re alization is tig htly co ncentrated ar ound th e average over the lifetime of the rand om process. For peeling-style decodin g, a variable node and its ed ges are removed after veriﬁcation; each check node keeps track o f its new parity co nstraint (i.e., the value to which the attached variables mu st sum) by subtracting values associated with the removed edges. 2) Analysis of P e eling-Style Decoding : First, we in troduce some no tation an d de ﬁnitions fo r the analysis. A variable node (VN) whose channel v alue is corr ectly receiv ed is called a co rrect variable n ode ( CVN), otherwise it is called an incorrect variable n ode (IVN) . A check n ode (CN) with i edges conn ected to the CVN’ s and j edg es connected to th e IVN’ s will be said to h av e C-degree i and I-degree j , or type n i,j . W e note that the analysis of node-based algorithms ho ld for irregular L DPC code ensemble of maxim um variable node degree d v and maximu m check nod e degree d c . W e also deﬁne the following quantities: • t : d ecoding time or fraction of VNs removed fro m graph • L i ( t ) : the nu mber of edges connected to CVN’ s with degree i at time t • R j ( t ) : the n umber of ed ges co nnected to IVN’ s with degree j at time t • N i,j ( t ) : the n umber of edges connected to CN’ s with C- degree i and I- degree j • E l ( t ) : the r emaining nu mber of edges conn ected to CVN’ s at time t • E r ( t ) : the remaining number of edges connected to IVN’ s at time t • a ( t ) : the av erage degree o f CVN’ s which ha ve at least 1 edge coming from CN’ s o f type n i, 1 , i ≥ 1 , a ( t ) = P d v k =1 k L k ( t ) E l ( t ) • b ( t ) : the average degree of IVN’ s which h av e at least 1 edge coming from CN’ s o f type n 0 , 1 , b ( t ) = P d v k =1 k R k ( t ) E r ( t ) • E : nu mber of ed ges in the original graph , E = E l (0) + E r (0) . 6 Fig. 1. T anner graph for the LM1 diffe rentia l equ ation analysi s. Counting edg es in thr ee ways gives the identity X i ≥ 1 L i ( t ) + X i ≥ 1 R i ( t ) = E l ( t ) + E r ( t ) = X i ≥ 0 ,j ≥ 0 ( i,j ) 6 =(0 , 0) N i,j ( t ) . These r .v . ’ s re present a particula r realization of th e decoder . The d ifferential equation s are d eﬁned f or the normalized (i.e. , divided by E ) expected values of these variables. W e use lower - case notation ( e.g., l i ( t ) , r i ( t ) , n i,j ( t ) 6 , etc.) f or th ese deterministic trajectories. T ime is scaled so that the decod er removes exactly one v ariable node during 1 / N time units, where N is the block len gth of th e code. The descrip tion of peeling-style deco der is as follows. The peeling-style decoder removes one CVN o r I VN in each time step by the following rules: CER: If a ny CN has all its ed ges con nected to CVN’ s, remove one of these CVN’ s and all of its edges. IER1: If any I VN has at least on e edg e co nnected to a CN of type (0 , 1) , then co mpute the value o f the IVN from the attache d CN and remove th e IVN along with all its edg es. If both CER an d IER1 can be applied, then on e is chosen random ly as described be low . Since both rules remove exactly o ne VN, the decoding process either ﬁnishes in exactly N step s or stops early and canno t co ntinue. The ﬁrst case occur s o nly when either the IE R1 or CER co ndition is satisﬁed in ev ery time step. When the decoder stops early , th e p attern of CVNs and IVNs remaining is called a stopping set. W e also note that the rules above, th ough described differently , are equ iv alen t to the ﬁrst node-b ased algorithm (LM1-NB) introd uced in [8]. Recall that the no de-based a lgorithm f or LM1, fro m [ 8], has two veriﬁcation rules. The ﬁrst ru le is: if th e ne ighbors (of a CN) have values that sum to zero , then all the neighbo rs are veriﬁed to their current v alues. In this analysis, howe ver , the n eighbor s are veriﬁed on e at a tim e. Since this implies that the CN is attached only to CVNs (assuming no FV), we call this co rrect-edg e-removal (CER) and notice that it is o nly allowed if n i, 0 > 0 fo r some i ≥ 1 . Th e seco nd rule is: if all neigh bors (of a CN) are veriﬁed except for one, th en the last n eighbor is veriﬁed to th e value that satisﬁes the check . If th e last n eighbo r is an IVN, we call this typ e-I inco rrect- edge-re moval (IER1) and n otice tha t it is only allowed when n 0 , 1 ( t ) > 0 . The p eeling-style deco der perf orms one operation durin g each time step. The o peration is r andom and can be e ither 6 When we use n i,j , we refer to the type of IVN’ s. When we use n i,j ( t ) , we refer to the normali zed expec ted val ues. CER or IER1. When both operation s ar e possible, we choose random ly between these two rules by p icking CER with probab ility c 1 ( t ) an d IER1 with probab ility c 2 ( t ) , where c 1 ( t ) = P i ≥ 1 n i, 0 ( t ) P i ≥ 1 n i, 0 ( t ) + n 0 , 1 ( t ) c 2 ( t ) = n 0 , 1 ( t ) P i ≥ 1 n i, 0 ( t ) + n 0 , 1 ( t ) . This weighted sum ensur es that the expected ch ange in the decoder state is Lipschitz continuo us if either c 1 ( t ) or c 2 ( t ) is strictly positive. Ther efore, the differential e quations can be written as d l i ( t ) d t = c 1 ( t ) d l (1) i ( t ) d t + c 2 ( t ) d l (2) i ( t ) d t d r i ( t ) d t = c 1 ( t ) d r (1) i ( t ) d t + c 2 ( t ) d r (2) i ( t ) d t d n i,j ( t ) d t = c 1 ( t ) d n (1) i,j ( t ) d t + c 2 ( t ) d n (2) i,j ( t ) d t , where (1) and (2) denote, re spectiv ely , the ef fects of CER and IER1. 3) CER Analysis: If the CER op eration is p icked, then we choose ran domly an ed ge attach ed to a CN of type ( i, 0) with i ≥ 1 . This VN endpoint of this edge is distributed un iformly across the CVN ed ge sockets. Th erefore , it will be attached to a CVN of d egree k with prob ability l k ( t ) /e l ( t ) . Therefore, one has th e following differential equations for l k and r k d l (1) k ( t ) d t = l k ( t ) e l ( t ) ( − k ) , for k ≥ 1 and d r (1) k ( t ) d t = 0 . For the effect on check e dges, we can th ink of removing a CVN with degree k as ﬁr st r andomly pic king an edge of type ( k , 0) connected to that CVN and then removing all the other k − 1 edges (called reﬂected edges) attached to the same CVN. The k − 1 reﬂected edges are uniformly distrib uted over the E l ( t ) co rrect sockets of the CN’ s. These k − 1 reﬂected edges hit n i,j ( t ) i ( k − 1) ( i + j ) e l ( t ) CN’ s o f type ( i, j ) o n average. Next, we average over the VN degree, k , and ﬁnd that p (1) i,j ( t ) , n i,j ( t ) i ( a ( t ) − 1) ( i + j ) e l ( t ) is the expected nu mber o f typ e- ( i, j ) CN’ s hit by reﬂected edges. If a CN of typ e ( i, j ) is hit b y a reﬂected edge, the n the decoder state loses i + j edges of typ e ( i, j ) and ga ins i − 1 + j ed ges of type ( i − 1 , j ) . Hence, one has the following differential equation, for j > 0 and i + j ≤ d c , d n (1) i,j ( t ) d t =  p (1) i +1 ,j ( t ) − p (1) i,j ( t )  ( i + j ) . One should keep in mind th at n i,j ( t ) = 0 fo r i + j > d c . 7 For n (1) i,j ( t ) with j = 0 , the effect from above mu st be combined with effect of the ty pe- ( i, 0) initial edg e that was chosen. So the differential equation becomes d n (1) i, 0 ( t ) d t =  p (1) i +1 , 0 ( t ) − p (1) i, 0 ( t )  i +  q (1) i +1 ( t ) − q (1) i ( t )  i where q (1) i ( t ) , n i, 0 ( t ) P m ≥ 1 n m, 0 ( t ) . Note that p (1) d c +1 , 0 ( t ) , 0 and q (1) d c +1 ( t ) , 0 4) IER1 Ana lysis: I f the IER1 oper ation is picked, then we choose a rando m CN of typ e (0 , 1) and follow its only edge to the set of IVNs. The edg e is attached u niform ly to this set, so the d ifferential equations for IER1 are given b y d l (2) k ( t ) d t = 0 , d r (2) k ( t ) d t = r k ( t ) e r ( t ) ( − k ) , and d n (2) i,j ( t ) d t =  p (2) i,j +1 ( t ) − p (2) i,j ( t )  ( i + j ) − δ i, 0 δ j, 1 , where p (2) i,j ( t ) , n i,j ( t ) j ( b ( t ) − 1) ( i + j ) e r ( t ) . W e note that the removal of the initial edge, w hen ( i, j ) = (0 , 1) , is taken into accoun t by the Kr onecker delta functions. Notice that even for (3,6) co des, th ere ar e 30 differential equations 7 to solve. So we so lve the differential equations numerically an d the thresho ld for ( 3,6) co de with LM1 is p ∗ = 0 . 169 . Th is c oincides with the result from density evolu- tion analysis for LM1-MB in [8] and hints at the equivalence between LM1-NB and LM1-M B. I n the proof of Theorem 3.1 we make this eq uiv alence precise by showing that the stopping sets of LM1-NB and LM 1-MB are the same. B. Differ entia l Equation Analysis of LM2-NB Similar to the analysis of LM1-NB algo rithm, w e ana- lyze LM2-NB algo rithm b y an alyzing a pee ling-style deco der which is equivalent to the LM2-NB decoding algor ithm. The peeling-style decoder rem oves one CVN or IVN du ring each time step ac cording to the following rules: CER: If any CN ha s all its e dges co nnected to CVN’ s, pick one of the CVN’ s and remove it. IER1: I f any IVN h as any messages fr om CN’ s with ty pe n 0 , 1 , th en the IVN and all its outg oing edg es can be removed and we track the correc t value by subtract- ing the value from the chec k node. IER2: I f any IVN is attach ed to more than one CN with I- degree 1, then it will be veriﬁed and all its outgo ing edges can be removed. For simplicity , we ﬁrst introdu ce some deﬁnitions and short- hand notatio n: 7 There are 28 for n i,j ( t ) ( i, j ∈ [0 , · · · , 6] such that i + j ≤ 6 ), 1 for r k ( t ) , and 1 for l k ( t ) . Fig. 2. Graph structure of an L DPC code during LM2-NB decoding. • Correct ed ges: edges which are connected to CVN’ s • Incor rect edges: edges wh ich are con nected to IVN’ s • CER edges: edges which are connected to check nodes with type n i, 0 for i ≥ 1 • IER1 edg es: edges which ar e connected to check node s with type n 0 , 1 • IER2 edges: edges which connect IVN’ s and the check nodes with ty pe n i, 1 for i ≥ 1 • NI edges: normal incorrect edges, which are incorrect edges but neither IE R1 edges no r IER2 edg es • CER nod es: CVN’ s wh ich have at least on e CER edg e • IER1 no des: IVN’ s wh ich have at least on e IER1 edg e • IER2 no des: IVN’ s wh ich have at least two I ER2 edges • NI no des: IVN’ s which contain at most 1 IE R2 edge an d no IER1 e dges. Note th at an IVN can be bo th an IER1 node an d an IE R2 node at th e same time . The analysis of LM2-NB is mu ch more complica ted than LM1-NB because the IER2 opera tion makes the distribution of IER2 ed ges dependen t on each othe r . For example, the IER2 operation removes a random ly ch osen I VN with m ore th an 2 IE R2 edge s; this reduces the num ber of IVNs which have multiple IER2 edg es and therefo re the remaining IER2 ed ges are more likely to land o n different IVN’ s. The basic idea behind our analysis of the LM2 -NB decoder is that we can separa te the incorr ect edges into typ es and assume that m apping between sockets is giv en by a u niform random permutatio n. Strictly speak ing, th is is not true and an- other ap proach, wh ich leads to the sam e d ifferential equations, is used when co nsidering a form al pro of of correctness. In detail, we mod el the stru cture o f an LDPC code dur ing LM2- NB d ecoding as shown in Fig. 2 with one typ e for co rrect edges an d three ty pes for incorr ect edges. The following calculations assume the fou r permuta tions, labeled CER, NI, IER2, and IER1, are all uniform rand om permutatio ns. The peelin g-style deco der random ly chooses one VN fro m the set of CER IER1 and IER2 nod es and removes this node and all its edges at each step. The idea of the analysis is to ﬁrst calculate the probability of choo sing a VN with a certain type, i.e., CER, IER1 or IER2, and the nod e degree. W e then analyze how removing this VN af fects the system parameters. In the analysis, we will track the evolution o f the following system parame ters: • l k ( t ) : the fraction of edges c onnected to CVN’ s w ith degree k , 0 < k ≤ d v at time t • r i,j,k ( t ) : the fractio n of edges connec ted to IVN’ s with 8 i NI edges, j I ER2 edges and k IER1 edg es a t time t , i, j, k ∈ { 0 , 1 , . . . , d v } an d 0 < i + j + k ≤ d v • n i,j ( t ) : th e fraction o f edges connected to check n odes with i correct edges and j incorrect ed ges at tim e t , i, j ∈ { 0 , 1 , . . . , d c } and 0 < i + j ≤ d c . W e note that, when we say “fraction ”, we m ean the number of a certain typ e of edges/nod es normalized by the nu mber of edges/nod es in the origin al graph . The following quantities can be calculated from l k ( t ) , r i,j,k ( t ) an d n i,j ( t ) : • e l ( t ) , P d v k =1 l k ( t ) : the fraction of c orrect edges • e r ( t ) , P d v i =0 P d v − i j =0 P d v − i − j k =0 r i,j,k ( t ) : th e fra ction of incorrect edg es • η 0 ( t ) , P d c j =2 P d c − j i =0 j n i,j ( t ) i + j = P d v i =1 P d v − i j =0 P d v − i − j k =0 ir i,j,k ( t ) i + j + k : the f raction of NI edges • η 1 ( t ) , n 0 , 1 ( t ) = P d v k =1 P d v − k j =0 P d v − i − j i =0 kr i,j,k ( t ) ( i + j + k ) : th e fraction of I ER1 ed ges • η 2 ( t ) , P d c i =1 n i, 1 ( t ) ( i +1) = P d v j =1 P d v − j i =0 P d v − i − j k =0 j r i,j,k ( t ) ( i + j + k ) : th e fr action of I ER2 edges • s 0 ( t ) , P 1 j =0 P d v − j i =1 r i,j, 0 i + j : the fr action of NI nodes • s 1 ( t ) , P d v k =1 n k, 0 ( t ) k : the fraction of CER nodes • s 2 ( t ) , P d v k =1 P d v − k i =0 P d v − i − k j =0 r i,j,k i + j + k : the fraction of IER1 nod es • s 3 ( t ) , P d v j =2 P d v − j i =0 P d v − i − j k =0 r i,j,k i + j + k : the fraction of IER2 nod es. As in the LM1-NB analy sis, we use superscript (1) to de note the co ntribution of the CER operation s. W e use (2) to de note the co ntribution of the IER1 operation s and (3) to denote the contribution of the IER2 ope rations. Since we assume that the decoder rando mly choo ses a VN fro m the set of CER, IER1 and IER2 nodes and removes all its edges durin g each time step, the differential equations of the system parameters can be written as the weig hted sum of the con tributions o f the CER, IER1 and IER2 oper ations. The weights ar e chosen to be c 1 ( t ) = s 1 ( t ) ( s 1 ( t ) + s 2 ( t ) + s 3 ( t )) c 2 ( t ) = s 2 ( t ) ( s 1 ( t ) + s 2 ( t ) + s 3 ( t )) c 3 ( t ) = s 3 ( t ) ( s 1 ( t ) + s 2 ( t ) + s 3 ( t )) . This weighted sum ensur es that the expected ch ange in th e decoder state is Lipschitz co ntinuou s if any one c 1 ( t ) , c 2 ( t ) , or c 3 ( t ) is strictly p ositiv e. Next, we will show how CER, IER1 and IER2 oper ations af fect the sy stem par ameters. Giv en the d.d. pair ( λ, ρ ) and the channel error pro bability p , we initialize the state as follows. Since a fraction (1 − p ) λ k of th e e dges are con nected to CVN’ s of degree k , we initialize l k ( t ) with l k (0) = (1 − p ) λ k , for k = 1 , 2 , . . . d v . Noticing that e ach CN socket is conn ected to a corre ct edg e with probability (1 − p ) and incor rect edge with prob ability p , we in itialize n i,j ( t ) with n i,j (0) = ρ i + j  i + j i  (1 − p ) i p j , for i + j ∈ { 1 , 2 , . . . , d c } . The probability that an IVN socket is connected to an NI, IER1 edg e, or IER2 edge is den oted respectively by g 0 , g 1 , or g 2 with g 0 = 1 p d c X j ′ =2 d c − j ′ X i ′ =0 j ′ n i ′ ,j ′ (0) i ′ + j ′ g 1 = 1 p d c X i ′ =1 n i ′ , 1 (0) ( i ′ + 1) g 2 = 1 p n 0 , 1 (0) . Therefo re, we initialize r i,j,k ( t ) with r i,j,k (0) = pλ i + j + k  i + j + k i, j, k  g i 0 g j 1 g k 2 , for i + j + k ∈ { 1 , 2 , . . . , d v } . 1) CER an alysis: The analy sis for d l (1) k ( t ) d t is the same as the LM1-NB analysis. In the CER oper ation, the decoder random ly selects a CER edge. A CVN w ith degree k is chosen with pro bability l k ( t ) /e l ( t ) . If a degree k CVN is chosen , the number of edges of typ e l k decreases by k and there fore d l (1) k ( t ) d t = − k l k ( t ) e l ( t ) . For j ≥ 1 and i + j ≤ d c d n (1) i,j ( t ) d t =  p (1) i +1 ,j ( t ) − p (1) i,j ( t )  ( i + j ) where a ( t ) = P d v k =1 kl k ( t ) e l ( t ) is the average d egree of the CVN’ s which are hit b y th e initially cho sen CER edg e and p (1) i,j = n i,j ( t ) i ( a ( t ) − 1) ( i + j ) e l ( t ) is the average number of CN’ s with typ e n i,j hit by the a ( t ) − 1 r eﬂecting edges. For j = 0 and i ≥ 1 , we also have to consider the initially chosen CER edge. This gives d n (1) i, 0 ( t ) d t =  p (1) i +1 , 0 ( t ) − p (1) i, 0 ( t )  i +  q (1) i +1 ( t ) − q (1) i ( t )  i where q (1) i ( t ) = n i, 0 ( t ) P m ≥ 1 n m, 0 ( t ) is the pr obability that the initially chosen CER edge is of type n i, 0 . When th e removed CER node has a reﬂecting edg e that hits a CN o f type n 1 , 1 , the CN’ s IE R2 edge becom es an IER1 edge. Th is is the only way the CER op eration ca n affect r i,j,k . On a verag e, each CER operation gen erates a ( t ) − 1 reﬂecting edges. F or each reﬂecting edge, the probability that it hits a CN of type n 1 , 1 is n 1 , 1 ( t ) 2 e l ( t ) . T a king this IER2 to IER1 conversion into accou nt, for k ≥ 1 and j < d v , gives d r (1) i,j,k ( t ) d t = ( a ( t ) − 1) n 1 , 1 ( t ) 2 e l ( t )  j r i,j,k ( t ) ( i + j + k ) η 2 ( t ) ( − ( i + j + k )) − ( j + 1 ) r i,j +1 ,k − 1 ( t ) ( i + j + k ) η 2 ( t ) ( − ( i + j + k ))  = ( a ( t ) − 1) n 1 , 1 ( t ) 2 e l ( t )  − j r i,j,k ( t ) η 2 ( t ) + ( j + 1) r i,j +1 ,k − 1 ( t ) η 2 ( t )  . 9 If k = 0 or j = d v , then the I VN’ s with ty pe r i,j,k can only lose edg es and d r (1) i,j,k ( t ) d t = ( a ( t ) − 1) n 1 , 1 ( t ) 2 e l ( t )  − j r i,j,k ( t ) η 2 ( t )  . 2) IER2 ana lysis: Since the IER2 op eration do es not af fect l k ( t ) , we ha ve d l (3) k ( t ) d t = 0 . T o analyze h ow the IER2 oper ation changes n i,j ( t ) an d r i,j,k ( t ) , we ﬁrst obser ve that the probability a randomly chosen IE R2 node is of type r i,j,k is given b y Pr ( type r i,j,k | IER2 nod e ) = r i,j,k ( t ) i + j + k s 3 ( t ) , for j ≥ 2 . Other wise, Pr ( type r i,j,k | IER2 nod e ) = 0 . Let us d enote the contr ibution to d n (3) i ′ ,j ′ ( t ) d t caused by re- moving: one NI edge as u i ′ ,j ′ ( t ) , o ne I ER2 ed ge as v i ′ ,j ′ ( t ) , and one IER1 edge as w i ′ ,j ′ ( t ) . T hen, we can write th e total contribution to the der iv ative as d n (3) i ′ ,j ′ ( t ) d t = d v X i =0 d v X j =0 d v X k =0 Pr ( type r i,j,k | IER2 node ) ( iu i ′ ,j ′ ( t ) + j v i ′ ,j ′ ( t ) + k w i ′ ,j ′ ( t )) . First, we consider u i ′ ,j ′ ( t ) . If an NI e dge is chosen fro m the IVN side, then it hits a C N of type n i,j with probability j n i,j ( t ) η 0 ( i + j ) if j ≥ 2 and pro bability 0 other wise. When 2 ≤ j ≤ d v − 1 , we have u i ′ ,j ′ ( t ) = j ′ n i ′ ,j ′ ( t )( − ( i ′ + j ′ )) ( i ′ + j ′ ) η 0 ( t ) + ( j ′ + 1) n i ′ ,j ′ +1 ( i ′ + j ′ ) ( i ′ + j ′ + 1) η 0 ( t ) = − j ′ n i ′ ,j ′ ( t ) η 0 ( t ) + ( j ′ + 1) n i ′ ,j ′ +1 ( i ′ + j ′ ) ( i ′ + j ′ + 1) η 0 ( t ) . When j = d v , one ﬁnd s instead th at u i ′ ,j ′ ( t ) = − j ′ n i ′ j ′ ( t ) η 0 ( t ) . Since an NI edge cann ot be co nnected to a CN of typ e n i ′ , 1 , we mu st treat j ′ = 1 separately . No tice that ty pe n i ′ , 1 may still ga in edges from type n i ′ , 2 , so we have u i ′ , 1 ( t ) = 2 n i ′ , 2 ( t )( i ′ + 1) ( i ′ + 2) η 0 ( t ) . When j ′ = 0 , CN’ s with type n i ′ , 0 do n ot have any NI ed ges, so we hav e u i ′ , 0 ( t ) = 0 . No w we consider v i ′ ,j ′ ( t ) . Sin ce edges of typ e n i ′ ,j ′ with j ≥ 2 cannot be IER2 edges, n i ′ j ′ ( t ) with j ≥ 2 is not affected by removing IER2 ed ges. T he IER2 edge removal reduces the nu mber of edges of type n i ′ , 1 , i ≥ 1 , so we have v i ′ , 1 ( t ) = − n i ′ , 1 ( t ) η 2 ( t ) . When j ′ = 0 and i ′ ≥ 1 , w e have v i ′ , 0 ( t ) = n i ′ , 1 ( t ) η 2 ( t ) . Only CN’ s with type n 0 , 1 are affected when we remove an IER1 edge on the IVN side. So we have w 0 , 1 ( t ) = 1 and w i ′ ,j ′ ( t ) = 0 when ( i ′ , j ′ )( t ) 6 = (0 , 1) . Next, we deriv e the contribution to the r i,j,k ( t ) differential equation caused by removing an IER2 node. When the decoder removes an IER2 no de of type r i ′ ,j ′ ,k ′ , ther e are dir ect and indirect effects. The direc t effect is th e loss of i ′ NI ed ges, j ′ IER2 ed ges an d k ′ IER1 ed ges. T his edg e rem ov al affects CN degrees and the type of som e CN ed ges may also ch ange and thereby in directly af fect r i,j,k ( t ) . W e call these in directly affected edges “CN reﬂected edges”. Let u ′ i,j,k ( t ) b e the contribution of CN re ﬂected edge s to d r i,j,k ( t ) d t caused by the removal of: an NI ed ge be u ′ i,j,k ( t ) , an IER2 edge be v ′ i,j,k ( t ) , a nd an IE R2 edge b e w ′ i,j,k ( t ) . T hen, we can write the total contribution to the derivati ve as d r (3) i,j,k ( t ) d t = − Pr ( typ e r i,j,k | IER2 node ) ( i + j + k ) + d v X i ′ =0 d v X j ′ =0 d v X k ′ =0 Pr ( type r i ′ ,j ′ ,k ′ | IER2 node )  i ′ u ′ i,j,k ( t ) + j ′ v ′ i,j,k ( t ) + k ′ w ′ i,j,k ( t )  . There are two ways that the CN reﬂected edges of an NI edge can a ffect r i,j,k ( t ) . The ﬁrst occurs when the CN is of type n i, 2 and 1 ≤ i ≤ d c − 2 . In this case, removing an NI edge changes the typ e of the o ther incorr ect edge fro m NI to IER2. The second occurs when the CN is of type n 0 , 2 . Removing an NI edge ch anges the type of the other incorrect ed ge fr om NI to IER1. The proba bility that an NI ed ge h its a CN of type n i, 2 , (for 1 ≤ i ≤ d c − 2 ) is P d c − 2 i =1 2 n i, 2 ( t ) i +2 η 0 ( t ) . Likewis e, th e probab ility that a n NI ed ge hits a CN o f type n 0 , 2 is n 0 , 2 ( t ) η 0 ( t ) . Since the prob ability that an NI edge is co nnected to an IVN of type r i,j,k is ir i,j,k ( t ) ( i + j + k ) η 0 ( t ) , on e ﬁnds th at u ′ i,j,k ( t ) = P d c − 2 i =1 2 n i, 2 ( t ) i +2 η 0 ( t )  − ir i,j,k ( t ) η 0 ( t ) + ( i + 1) r i +1 ,j − 1 ,k ( t ) η 0 ( t )  + n 0 , 2 ( t ) η 0 ( t )  − ir i,j,k ( t ) η 0 ( t ) + ( i + 1) r i +1 ,j,k − 1 ( t ) η 0 ( t )  , where r i,j,k ( t ) , 0 unless i , j, k ∈ { 0 , . . . , d v } an d i + j + k ≤ d v . Since there a re no CN reﬂected edges of type IER1 and IER2, on e also ﬁnd s that v ′ i,j,k ( t ) = 0 and w ′ i,j,k ( t ) = 0 . 3) IER1 Analysis: Like the IER2 operation , the IER1 operation does not affect l k ( t ) an d therefo re d l (2) k ( t ) d t = 0 . T o analyze how IER1 changes n i,j ( t ) and r i,j,k ( t ) , we observe that the probab ility a ran domly chosen IER1 node is of type r i,j,k is g i ven by Pr ( type r i,j,k | IER1 node ) = r i,j,k ( t ) i + j + k s 2 ( t ) when k ≥ 1 . Otherwise, Pr ( type r i,j,k | IER1 node ) = 0 . Applying the same arguments used for the I ER2 operation, one ﬁnds th at, d n (2) i ′ ,j ′ ( t ) d t = d v X i =0 d v X j =0 d v X k =0 Pr ( type r i,j,k | IER1 node ) ( iu i ′ ,j ′ ( t ) + j v i ′ ,j ′ ( t ) + k w i ′ ,j ′ ( t )) 10 and d r (2) i,j,k ( t ) d t = − Pr ( type r i ′ ,j ′ ,k ′ | IER1 node ) ( i ′ + j ′ + k ′ ) + d v X i ′ =0 d v X j ′ =0 d v X k ′ =0 Pr ( type r i ′ ,j ′ ,k ′ | IER1 node )  i ′ u ′ i,j,k ( t ) + j ′ v ′ i,j,k ( t ) + k ′ w ′ i,j,k ( t )  . The pro gram used to pe rform these comp utations and comp ute LM2-NB thresh olds is available online [19]. C. Discussion o f the An alysis Consider the p eeling deco der fo r the BEC introduced in [8]. Thro ughou t th e decod ing pro cess, on e reveals an d th en removes edges one at a time f rom a h idden rand om graph . The analysis o f this d ecoder is simpliﬁed by the fact that, g iv en the curren t residu al d egree d istribution, the unrevealed portion of the graph remains uniform for e very d ecoding tr ajectory . In fact, one can build a ﬁnite-length d ecoding simulatio n never constructs the actual deco ding grap h. Instead , it trac ks only the residual d egree distribution of the gra ph an d im plicitly chooses a random deco ding graph one ed ge at a time. For asy mptotically long cod es, [8] used this ap proach to derive an analysis based on d ifferential equatio ns. Th is anal- ysis is actually quite general and can also be app lied to other peeling-style decode rs in wh ich th e unrevealed graph is n ot unifor m. One may ob serve this from its proo f of correc tness, which depends only on tw o important observations. First, the distribution of all d ecoding paths is co ncentrated very tightly aroun d its a verag e as the system size incre ases. Second, the expected change in the d ecoder state can be written as a Lipsch itz fun ction of the c urrent d ecoder state. If o ne augmen ts th e decoding state to includ e e nough informa tion so that the expected change can be comp uted from the augmented state (ev en for non-unif orm residual graphs), then the theorem still ap plies. The differential equation co mputes the average evolution, over all rand om bipar tite g raphs, of the system parameters as the block leng th n goes to inﬁnity . Wh ile the n umerical simulation of long c odes gives the e volution of the system parameters of a particu lar cod e (a p articular bipartite graph) as n goes to inﬁnity . T o p rove that the differential equatio n analysis precisely p redicts the ev olution of the system p aram- eters of a particu lar co de, on e must sh ow the concen tration of the ev olution o f th e system parameter s of a par ticular code around the ensemble average as n goes to inﬁnity . In the LM2-NB algorithm, o ne node is remov ed a t a time b u t this c an also be viewed as removin g each edg e sequ entially . The main difference for LM2-NB algorith m is that on e has more edge types and one m ust track some d etails of the edge types on both th e check no des and th e variable n odes. Th is causes a sign iﬁcant problem in the analysis because upd ating the exact effect of edg e removal req uires revealing s ome edges b efore they will be rem oved. For example, the CER operation can cause an IER2 edge to become an IER1 edge, but r ev ealing the adjacent symb ol node (or type) ren ders the analysis intr actable. Unfortu nately , ou r pr oof of co rrectness still relies on two unproven assumptions wh ich we state as a conjectures. T his section lev erages the framew o rk o f [8], [18], [20] by describ- ing only the new discrete-time random process H t associated with our a nalysis. W e ﬁrst introdu ce the deﬁn itions of the random process. In th is subsection , we use t to rep resent the discrete time. W e fo llow th e same n otation used in [8]. Let the life span of the random p rocess be α 0 n . Let Ω d enote a pro bability space and S b e a measurab le space of observations. A discrete-time random process over Ω with observations S is a seq uence Q , ( Q 0 , Q 1 , . . . ) of rand om variables wh ere Q t contains the in formation r ev ealed at t -th step. W e den ote the history of the p rocess u p to time t as H t , ( Q 0 , Q 1 , . . . , Q t ) . L et S + := ∪ i ≥ 1 S i denote the set of all h istories an d Y be the set of all deco der states. One typically uses a state space that tracks the nu mber of edges o f a certain ty pe (e.g. , the degree of the attached nodes). W e d eﬁne the rand om process as follows. The total nu mber of ed ges conne cted to IVN’ s with typ e r i,j,k at time t is denoted R i,j,k ( t ) and th e total n umber o f edges co nnected to check nodes with type n i,j is N i,j ( t ) . The main difference is th at we track the average ¯ R i,j,k ( t ) , E [ R i,j,k ( t ) | H t ] of node degree distribution rather than the exact value. Let R ( t ) , ¯ R ( t ) , and N ( t ) be vectors of random variables formed by includin g all valid i, j, k tuples f or each variable. Using this, the decoder state at time t is given b y Y t , { N ( t ) , ¯ R ( t ) } . T o co nnect this with [ 20, Theorem 5.1] , we deﬁne the h istory of ou r ra ndom pro cess as f ollows. In the beginning of the d ecoding, we label the v ariable/check nodes by th eir d egrees. When th e dec oder removes an e dge, the revealed in formation Q t contains the degree of the variable node an d ty pe of the chec k n ode to which the removed edge is connected . W e note that sometimes the edge-removal operation changes th e type of the u nremoved ed ge o n that check node. In this case, Q t also contains th e inf ormation about the type of the chec k nod e to which this CN reﬂected edge connects. But Q t does not conta in any info rmation about the IVN to whic h the CN-reﬂec ting ed ge is conn ected. By d eﬁning the history in this mann er , Y t is a deterministic fu nction of H t and can be made to satisfy th e condition s of [20, Th eorem 5.1 ]. The following con jecture, which basically says that R i,j,k ( t ) concentr ates, enca psulates one o f th e un proven assumptions needed to establish the co rrectness of this analysis. Conjectur e 3.2: lim n →∞ Pr  sup 0 ≤ t ≤ α 0 n   ¯ R i,j,k ( t ) − R i,j,k ( t )   ≥ n 5 / 6  = 0 holds for all i, j, k ∈ { 0 , . . . , d v } such that i + j + k ≤ d v . The next observation is that the expected dr ift E [ Y t +1 − Y t | H t ] can be compu ted exactly in terms of R ( t ) if the four edge-typ e p ermutation s are unifo rm. But, o nly ¯ R ( t ) can be computed exactly fro m H t . Let f ( Y t ) deno te the expected drift under the unifo rm assum ption using ¯ R ( t ) instead of R ( t ) . Since R ( t ) is concentrated around ¯ R ( t ) , by assumption, and f is Lipschitz, this is not the main difﬁculty . In stead, the uniform assumption is p roblematic and the following conjecture sidesteps the problem by assum ing th at the true 11 expected drift E [ Y t +1 − Y t | H t ] is asymptotically equ al to f ( Y t ) . Conjectur e 3.3: lim n →∞ Pr  sup 0 ≤ t ≤ α 0 n k E [ Y t +1 − Y t | H t ] − f ( Y t ) k ∞ ≥ n − 1 / 7  = 0 . If these c onjectures hold true, then [20, Theo rem 5.1] can be used to show that the dif ferential equation cor rectly mo dels the expected de coding trajectory an d actual realizations con- centrate tightly ar ound this expectation . I n particular, we ﬁn d that N i,j ( nt ) conc entrates around n i,j ( t ) an d both ¯ R i,j,k ( nt ) and R i,j,k ( nt ) concen trate ar ound r i,j,k ( t ) . These conclusion s are suppo rted by ou r simulations. I V . E R RO R F L O O R A N A L Y S I S O F L M P A L G O R I T H M S During the simulation of th e optimize d en sembles fro m T able II, we observed an un expected error ﬂoo r . While o ne might expect an er ror ﬂoo r du e to ﬁnite q ef fects, it was somewhat surprising that the er ror ﬂoor persisted even whe n q was chosen to b e large enou gh so that n o FV’ s we re obser ved in the er ror ﬂoo r regime. Analyzing th e simulatio n results shows that the error ﬂoor was caused instead b y sym bols that remain unv eriﬁed at the end of decodin g. Th is motiv ated further stud y into th e erro r ﬂoor of LMP algorithms. It is also worth noting that, when q is relati vely sma ll, the error ﬂoor is caused b y several factors in cluding as type- I FV , type-II FV (which we will discuss later) a nd the ev ent that some symbols remain un veriﬁed when the deco ding ter minates. For small q , these factors ap pear to in teract in a complex manner . In this section, we focus o nly on error ﬂoor s due to unveri- ﬁed symbols. Th ere are three reasons for this. T he ﬁrst reason is that this was the d ominan t e vent we sa w in our simulations (i.e., the error ﬂoors observed in th e simulation are not c aused by FV). The second reason is that t he assumption that “veriﬁed symbols are co rrect with h igh p robability ” is th e cornerstone of veriﬁcation- based decoding and very little can be said about veriﬁcation decoding without this assumption . For example, if FV has any sig niﬁcant impact, then both the density-evolution analysis and the differential-equation analy sis br eak down and the threshold s become meaning less. The last reason is for simplicity; one can analyze the err or ﬂo ors cau sed by e ach factor separately in this case b ecause they a re no t strongly coupled . Although the dominant contr ibution to the error ﬂo or is n ot caused by FV , an analy sis of FV is provided f or sake of th e completen ess. This analysis actually helps us unde rstand wh y the d ominant erro r events caused by FV can be av oided by increasing q . A. The Union Bound for ML Decoding First, we deriv e the u nion boun d on th e prob ability of err or with ML decoding for t he q -SC. T o match our simulations with the union bound s, we also expurgate ou r ensemble to remove all c odeword weigh ts th at have an expe cted multiplicity less than 1. Next, we sum marize a few resu lts fr om [2 1, p. 4 97] tha t characterize the low-weight c odewords of LDPC codes with degree-2 variable nodes. When the block length is la rge, all o f these low-weight codewords are cau sed, with high probab ility , by short cycles of d egree-2 nodes. For b inary codes, the n umber of codew ords with weig ht k is a random variable which co n verges to a Poisson distribution w ith mean 1 2 k ( λ 2 ρ ′ (1)) k . When the channel quality is h igh (i.e., high SNR, low e rror/erasu re rate), the probability of ML decod ing error is mainly caused by lo w-weight codewords. For non- binary GF ( q ) codes, a codeword is suppor ted on a cycle of degree-2 n odes only if the pro duct of the edge weights is 1 . This o ccurs with prob ability 1 / ( q − 1) if we choo se the i.i.d. unifo rm rand om edge weights fo r the code. Hence, the nu mber of GF ( q ) cod ew o rds of weigh t k is a ran dom v ariable, denoted B k , which conver ges to a Poisson distribution with mean b k = 1 2 k ( q − 1) ( λ 2 ρ ′ (1)) k . After expurgating the ensemb le to r emove a ll we ights with expected multip licity less than 1 , k 1 = min { k ≥ 1 | b ( n ) k ≥ 1 } becomes the minimum codeword weig ht. An upper bound on the p airwise error proba bility ( PEP) of the q -SC with er ror probab ility p is given by the fo llowing lem ma. Lemma 4.1: Let y be th e received sy mbol sequence as- suming the all-zero codeword was tran smitted. Let u be any codew ord with exactly k non-zero symbols. Then, the probab ility that the ML decod er choo ses u over th e a ll-zero codeword is upper bou nded by p 2 ,k ≤ p q − 2 q − 1 + s 4 p (1 − p ) q − 1 ! k . Pr oof: See Appen dix B. Remark 4.2 : Notice th at b k is exponential in k an d th e PEP is also exponential in k . The union boun d fo r the frame error rate, due to low-weight codew ords, can be written as P B ≤ ∞ X k = k 1 b k p 2 ,k . It is e asy to see k 1 = Ω(log q ) and the sum is domina ted by the ﬁrst ter m b k 1 p 2 ,k 1 which has th e smallest expon ent. Wh en q is large, the PEP upper boun d is o n the orde r of O  p k  . Therefo re, the order of the union bou nd on fram e err or ra te with ML decoding is P B = O     λ 2 ρ ′ (1) p  log q q log q    and the expe cted number of symbols in er ror is O     λ 2 ρ ′ (1) p  log q q    , if pλ 2 ρ ′ (1) < 1 . B. Err or Analysis for LMP Algorithms The error of LMP algorithm co mes f rom tw o types o f decodin g failure. The ﬁrst ty pe of deco ding failure is du e to u n veriﬁed symb ols. The seco nd on e is caused by FV . T o 12 understan d the perfor mance of LMP a lgorithms, we an alyze these two ty pes sep arately . Wh en we analyze e ach error typ e, we neglect the interaction for simp licity . FV’ s can be classiﬁed into tw o types. The ﬁrst typ e is, as [8] mentions, when the error mag nitudes in a single check sum to zero; we call this typ e-I FV . For single-e lement lists, it occurs with p robability rou ghly 1 /q (i.e ., the chance that tw o u niform random sym bols are equal) . For multiple lists with multiple entries, we analyze th e FV p robab ility under the assump tion that no list con tains the corr ect symb ol. I n this case, each list is un iform on th e q − 1 inco rrect symbols. For m lists of size s 1 , . . . , s m , the type-I FV prob ability is g iv en by 1 −  q − 1 s 1 ,s 2 , ··· ,s m  Q m i =1  q − 1 s i  . In general, the Birthday parado x applies an d the FV prob ability is rough ly s 2  m 2  /q for large q and equal size lists. The second typ e of FV is that me ssages beco me more and more correlate d as the n umber of iteration s grows, so that an incorrect message ma y go thr ough different paths an d return to the same no de. W e d enote th is kind o f FV as a type-I I FV . These two types of FV are q uite different. One cannot av o id type-II FV by increasing q without randomizing the edge weights an d one can not av oid ty pe-I FV by con straining the number of d ecoding iterations to be within half of th e girth (or increasing the gir th). Fig. 3 shows an example of type-II FV . In Fig. 3, there is a n 8 -cycle in the g raph (inv olving 4 variable no des) and we assume the variable n ode on th e right has an inco rrect incom ing message “ a ”. Assume that the all- zero codeword is transmitted , all the inco ming messages at each variable n ode are no t veriﬁed , the list size is less than S max , and each incoming message a t each check node co ntains the corre ct sy mbol. In this case, the inc orrect symbol will trav el alon g the cycle an d cau se FV’ s at all variable nodes along th e cycle. If the c haracteristic of the ﬁeld is 2, there are a to tal of c/ 2 FV’ s occ urring alon g the cycle, where c is the len gth o f the cycle. T his type of FV can be reduced signiﬁcantly by ch oosing e ach non -zero en try in th e parity- check matrix rand omly f rom the non-z ero elem ents of Galois ﬁeld. In this case, a cycle cau ses a type-II FV only if the produ ct of the edge-weigh ts along that cycle is 1. Ther efore, we suggest choosing the non -zero entries of the p arity-chec k matrix randomly to mitigate type-II FV . Recall that the idea of using no n-binar y e lements in the parity-ch eck matrix is q uite old and appear s in early work on LDPC codes over GF ( q ) [11]. C. An Up per Boun d on T y pe-II FV Pr obab ility for Cycles In this subsection, we analyze the probab ility of error caused by ty pe-II FV . Note th at ty pe-II FV occu rs o nly when the depth- 2 k directed neigh borho od of an edg e (or node) has cycles. Bu t type- I FV occurs at every ed ge (or node). The order o f the probability that type-I FV o ccurs is approx imately O (1 / q ) [8]. The probab ility of type-II FV is hard to analyz e because it d epends on q , S max and k in a c omplicated way . But an u pper bou nd of the pro bability o f the typ e-II FV is derived in this section. Since the probab ility of type-II FV is dominated b y short cycles of degree-2 no des, we only analyze type-II FV alo ng these sho rt cycles. As we will soon see, the pr obability of type-II FV is exponential in the leng th of the cycle. So, the error caused by type-I I FV on cycles is do minated by shor t cycles. W e also assume S max to be large en ough such that an incorrectly r eceiv ed value can pass aro und a cycle without being truncated . Th is assumption m akes our analysis an upper bound . Another condition required for an incorrectly r eceiv ed value to p articipate in a type-II FV is that th e product of the edge weigh ts along the cycle is 1. If we assume that a lmost all edg es not on the cycle are veriﬁed , then once a ny edg e on the cycle is veriﬁed, all edges will be veriﬁed in the n ext k iter ations. So we also assume that n odes along a cycle are either all veriﬁed or all u n veriﬁed. W e note that there are three possible patterns of veriﬁcation on a cycle, d ependin g on the recei ved values. The ﬁrst case is that all the nodes are recei ved in correctly . As mentioned above, the inco rrect value p asses aro und the c ycle without b eing truncated, comes back to the node again and falsely veriﬁes the outgoin g messages of the node. So all messages will be falsely veriﬁed (if they are all received in correctly ) after k iterations. Note that this happens with prob ability 1 q − 1 p k . The second case is that a ll messages are veriﬁed co rrectly , say , no FV . Note that this d oes not re quire all the nodes to have correctly received v alues. For example, if any pair of adjacent nodes are receiv ed co rrectly , it is easy to see all messages will be correctly veriﬁed . The last case is that there is at least 1 incorrectly re ceiv ed nod e in any pair of adjacent no des and that there is at least 1 n ode with cor rectly received value on the cycle. In this c ase, all m essages will be veriﬁed after k itera tions, i.e. , messages from correct no des are veriﬁed correctly and th ose from in correct nod es are falsely veriﬁed. Then the veriﬁed messages will p ropagate and half of the messages will be veriﬁed correc tly and the other half will be falsely veriﬁed. Note th at this happen s with proba bility 1 q − 1 2  p k/ 2 − p k  ≈ 2 p k/ 2 q − 1 and this appro ximation g iv es an upper bou nd e ven if we combine the pr evious 1 q − 1 p k term. Recall th at the numb er of cycles with k variable nodes conv erges to a Po isson with mean 1 2 k ( λ 2 ρ ′ (1)) k . Using the union bo und, we can up per boun d on the en semble average probab ility o f any type- II FV event with Pr( any type-II FV ) ≤ ∞ X k = k 1  λ 2 ρ ′ (1)  k 2 k ( q − 1 ) 2 p k 2 = ∞ X k = k 1  λ 2 ρ ′ (1) √ p  k k ( q − 1) = O     λ 2 ρ ′ (1) √ p  log q ( q − 1) log q    . The ensemble average numb er of nod es in volved in type-I I FV events is giv en by E [ symbols in type-II FV ] ≤ ∞ X k = k 1  λ 2 ρ ′ (1)  k 2 k ( q − 1) 2 k p k 2 = ∞ X k = k 1  λ 2 ρ ′ (1) √ p  k ( q − 1) = O    λ 2 ρ ′ (1) √ p  ( q − 1)   . 13 Notice th at both the se upp er bound s are decreasing fu nctions of q . D. Upper Bo und on Unveriﬁcation Pr oba bility for Cycles During the simulation of th e optimize d en sembles fro m T able II , we o bserved signiﬁcan t error ﬂoor s a nd all of the error e vents w ere caused by unveriﬁed symb ols when the decodin g terminates. In this subsection, W e derive the union bound for th e probability o f de coder failure caused by the symbols on short cycles which n ev er become veriﬁed. W e call this event as unveriﬁcation and we denote it by UV . As described ab ove, to match the settings of th e simulation and simplify the analy sis, we assume q is large enoug h to have arbitrarily small pro bability of both ty pe-I an d typ e-II FV . In this case, the er ror is do minated by the unveriﬁed messages because the follo wing analysis shows that the union bound on the p robability of unveriﬁcation is independen t o f q . In contrast to type-II FV , the un veriﬁcatio n event does not require cycles, i. e., unveriﬁcation occurs even o n subgr aphs without cycles. But in the lo w error-rate regime, the dominant contribution to un veriﬁcation ev ents com es f rom short cycles of degre e-2 nodes. T herefor e, we only a nalyze the probability of unveriﬁcation cau sed by sho rt cycles of d egree-2 nodes. Consider a degree-2 cycle with k variable nodes and assume that no FV o ccurs in the n eighbor hood of this cycle. Assuming the maximu m list size is S max , the cond ition for UV is th at there is at most one corre ctly receiv ed value along S max + 1 adjacent variable no des. Note that we d on’t consider typ e- II FV since type-II FV occurs with prob ability 1 q − 1 and we can choose q to be arbitrar ily large. On the o ther han d, unv eriﬁcation does no t require the produc t of the edge weights on a cycle to be 1, so one canno t mitigate it by increasing q . Let the r .v . U b e the numbe r o f symbols inv o lved in unv eriﬁcation e vents on shor t cycles of degree-2 nodes. On e can u se th e un ion bound to upper b ound the prob ability of any unv eriﬁcation e vents by Pr( U ≥ 1) ≤ ∞ X k = k 2  λ 2 ρ ′ (1)  k 2 k φ ( S max , p, k ) where k 2 = min { k ≥ 1 | 1 2 k ( λ 2 ρ ′ (1)) k ≥ 1 } and φ ( S max , p, k ) is the UV prob ability for a cycle consisting of k degre e-2 nodes. On e can also uppe r bo und the the expectation of U with E [ U ] ≤ ∞ X k = k 2  λ 2 ρ ′ (1)  k 2 φ ( S max , p, k ) . (18) The u n veriﬁcation p robability for short cycles of degree-2 nodes is gi ve by the fo llowing lem ma. Lemma 4.3: Let the cycle have k degree-2 variable nod es, the maximum list size be s , and the chan nel e rror prob ability be p . Then, the probab ility o f an u n veriﬁcation event is φ ( s, p, k ) = Tr  B k ( p )  where B ( p ) is the ( s + 1) by ( s + 1) Fig. 3. An example of type-II FV’ s. matrix B ( p ) =          p 1 − p 0 0 · · · 0 0 0 p 0 · · · 0 0 0 0 p · · · 0 . . . . . . . . . . . . . . . . . . 0 0 0 0 · · · p p 0 0 0 · · · 0          . (19) Pr oof: See Appen dix C. Let us look at (18) , we can see that the a verag e numbe r of unv eriﬁed sy mbols scales exp onentially with k . T he ensemble with larger λ 2 ρ ′ (1) will ha ve more sho rt degree-2 c ycles and m ore average unv eriﬁed symbols. Th e a verag e number o f unv eriﬁed sym bols depend s on the maximu m list-size S max in a comp licated way . Intuitively , if S max is larger, then the constraint that ”there is a t most one correct symbol along S max adjacent variable node s“ b ecomes stron ger since we assume the p robability of seeing a corr ect symbol is h igher than that of seeing a incorrect symbol. Therefo re, un veriﬁcatio n is less likely to happen and the average number of unv eriﬁed s ymbols will decr ease as S max increases. Notice also that (18) does not depend on q . One might expect that the stability condition of the LMP- S max decodin g algorithms can be u sed to analy ze the error ﬂoor . Actually , one can show tha t the stability cond ition for LMP- S max decodin g of irregular LDPC codes is identical to that of the BEC, wh ich is pλ 2 ρ ′ (1) < 1 . This d oes no t help predict the error ﬂoo r though, b ecause for codes w ith degree- 2 n odes, th e error ﬂoor is determined main ly b y sho rt cycles of degree-2 n odes. Instead , the c ondition λ 2 ρ ′ (1) < 1 simply implies that the expected number of degree-2 cycles is ﬁnite. V . C O M PA R I S O N A N D O P T I M I Z AT I O N In this section , we comp are the p roposed algo rithm, with maximum list size S max (LMP- S max ), to other message- passing deco ding algor ithms for the q -SC. W e no te that the LM2-MB algorith m is identical to SW1 for any code ensemble because th e d ecoding r ules a re the same. LM2-MB, SW1 and LMP-1 are id entical f or ( 3,6) r egular LDPC codes because the list size is always 1 and er asures never occur in LMP-1 fo r (3,6) regular LDPC co des. The LMP- ∞ algorithm is identica l to SW2. There are two impor tant d ifferences between the LMP algorithm and pr evious algo rithms: (i) erasures an d (ii) FV recovery . The LMP algorith m passes erasures b ecause, with a limited list size, it is b etter to pa ss an erasure th an to keep unlikely symbols on th e list. The LMP algorithm also detects FV ev ents and passes an erasur e if they cause d isagreemen t between veriﬁed symbols l ater in decoding, and can sometimes recover from a FV event. In contrast, LM1-NB and LM2-NB 14 T ABL E II O P T I M I Z A T I O N R E S U LT S F O R L M P A L G O R I T H M S ( R AT E 1 / 2 ) Alg. λ ( x ) ρ ( x ) p ∗ LMP-1 . 1200 x + . 3500 x 2 + . 0400 x 4 + . 4900 x 14 x 8 .2591 LMP-1 . 1650 x + . 3145 x 2 + . 0085 x 4 + . 2111 x 14 + . 0265 x 24 + . 0070 x 34 + . 2674 x 49 . 0030 x 2 + . 9970 x 10 .2593 LMP-8 . 32 x + . 24 x 2 + . 26 x 8 + . 19 x 14 . 02 x 4 + . 82 x 6 + . 16 x 8 .288 LMP-32 . 40 x + . 20 x 3 + . 13 x 5 + . 04 x 8 + . 23 x 14 . 04 x 4 + . 96 x 6 .303 LMP- ∞ . 34 x + . 16 x 2 + . 21 x 4 + . 29 x 14 x 7 .480 LM2-MB . 2 x + . 3 x 3 + . 05 x 5 + . 45 x 11 x 8 .289 LM2-NB 0 . 2972 x + 0 . 1560 x 2 + 0 . 1035 x 3 + 0 . 1813 x 5 + 0 . 2590 x 8 + 0 . 0029 x 11 x 6 .303 T ABL E III T H R E S H O L D V S . A L G O R I T H M F O R T H E ( 3 , 6 ) R E G U L A R L D P C E N S E M B L E LMP-1 LMP-8 LMP-32 LMP- ∞ LM1 LM2-MB LM2-NB .210 .217 .232 .429 .169 .210 .259 ﬁx the status of a variable n ode once it is veriﬁed and pass the veriﬁed value in all following iter ations. The results in [8] and [14] also do not consider the effects of type-II FV . Th ese FV events degrade the p erform ance in practical systems with moderate block leng ths, and therefo re we use random e ntries in th e parity-check matrix to mitigate these effects. Using the DE analysis of the LMP- S max algorithm , we can improve the thre shold by op timizing the degree distribution pair ( λ, ρ ) . Since the DE recursion is not one-d imensional, we use dif ferential evolution to optimize th e code ensem bles [2 2]. In T able II, we sho w the results of optimizing rate- 1 2 ensembles for LMP with a maximum list size of 1, 8, 32, and ∞ . Threshold s f or LM2- MB and LM2-NB alg orithms with rate 1/2 are also shown. In all but one case, the maximum variable- node d egree is 15 and the maximum chec k-nod e d egree is 9. The second table entry a llowed for larger degrees (in order to improve perfo rmance) but very little gain was o bserved. W e can a lso see that th ere is a g ain of between 0.0 4 and 0.0 7 over the thre sholds of (3,6) regular ensemble with th e same decoder . V I . S I M U L A T I O N R E S U LT S In this section, simulation results are presented f or regular and op timized L DPC codes ( see T able II) using various de- coding alg orithms and m aximum list sizes. For the simulatio n of optimized ensembles, a v ariety of ﬁnite-ﬁeld sizes are also tested. W e use no tation “LMP S max , q , X ” to deno te the sim- ulation resu lts of the L MP algo rithm with maximum list-size S max , ﬁnite ﬁeld GF ( q ) , an d ensemble X . The block length is chosen to be 100 000 an d the parity-c heck matr ices are chosen random ly while av oiding doub le edges and 4-cycles. Each non-ze ro entry in the parity-check matrix is cho sen unif ormly from GF ( q ) \ 0 to minimize the FV prob ability . Th e max imum number of decod ing iterations is ﬁxed to be 20 0 and more than 1000 b locks are run for each po int. The results ar e shown in Fig. 4 and ca n b e compar ed with the theoretical threshold s. T able III shows th e theo retical thresho lds of (3 , 6) r egular codes on the q - SC for different algorithm s an d T able II shows the thresho lds for the optimized ensemb les. The numer ical results seem to match the theoretical thresho lds well. The results of the simulation for (3 ,6) regular codes shows no apparen t error ﬂoor because ther e are no degree-2 nodes and almost n o FV oc curs is ob served durin g simulation. The LM2-NB perf orms much better th an other algo rithms with list-size 1 fo r the (3,6 ) regular ensemb le. For the optimized ensembles, there are a large number of degree-2 v ar iable nodes which cause a signiﬁcant erro r ﬂoor . By ev a luating (18), the predicted error ﬂoor cau sed b y u n veriﬁcation is 1 . 6 × 10 − 5 for the o ptimized S max = 1 ensemb le, 8 . 3 × 10 − 7 for the optimized S max = 8 ensemble, and 1 . 5 × 10 − 6 for the optimized S max = 32 ensemble. From th e results, we see th e analysis of u n veriﬁcation events match es th e numerical r esults very well. V I I . C O N C L U S I O N S In th is pape r , list-m essage-passing ( LMP) decod ing algo- rithms are discussed for the q -a ry symmetric chann el ( q - SC). It is shown that capac ity-achieving e nsembles for the BEC achieve capacity on the q - SC when th e list size is unbou nded a nd q goes to inﬁnity . Decoding thresholds are also calcu lated by density ev olution (DE). W e also de riv e a new an alysis for the node-b ased alg orithms described in [8]. The causes of false veriﬁcatio n ( FV) are also analyzed an d random en tries in the parity- check matrix are used to avoid type-II FV . Degree proﬁles are optimize d for the LMP decoder and reason able gains are obtained . Finally , simulations show that, with list sizes larger than 8, the proposed LM P algorithm outperf orms p reviously proposed algorithms. In the simulation, we also observe signiﬁcant error ﬂoors for th e optimized code ensembles. These error ﬂoors a re mainly caused by th e unv eriﬁed symb ols wh en decodin g termin ates. An analysis of this erro r ﬂoor is der i ved that matches the simulation results quite well. While we focus o n the q -SC in this work , there are a number of other ap plications of LMP d ecoding that are also quite interesting. For example, the iterativ e decoding algorithm described in [2 3] fo r comp ressed sensing is actually th e natural extension of LM1 to contin uous alphabets. For this reason, the LMP d ecoder may also b e used to imp rove the threshold of compressed sensing. This is, in some sen se, more valuable because the re are a number o f go od cod ing schemes f or the q -SC, but few low-complexity n ear-optimal decoder s for comp ressed sensing. Th is extensio n is explo red more thoro ughly in [2 4]. A P P E N D I X A P R O O F O F T H E O R E M 2 . 1 Pr oof: Given pλ (1 − ρ (1 − x )) < x f or x ∈ (0 , 1] , we start by sh owing that b oth x i and y i go to zero as i goe s to 15 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 10 −6 10 −5 10 −4 10 −3 10 −2 10 −1 10 0 Channel Error Probability p Fraction of unverified symbols LMP1/SW1/LM2−MB,2 32 ,(3,6) LMP8,2 32 ,(3,6) LMP32,2 32 ,(3,6) LM1,2 32 ,(3,6) LMP1,2 32 ,OPT LMP8,2 48 ,OPT LMP32,2 32 ,OPT LMP1,2 48 ,OPT LMP8,2 32 ,OPT LMP32,2 48 ,OPT LM2−NB,2 32 ,(3,6) Fig. 4. Simulati on result s for (3,6) regul ar code s w ith block length 100000. inﬁnity . T o d o this, we let α = sup x ∈ (0 , 1) 1 x pλ (1 − ρ (1 − x )) and note th at α < 1 b ecause p < p ∗ . It is also easy to see that, starting from x 0 = 1 , we have x i ≤ α i and x i → 0 . Next, we rewrite (3) as y i +1 = 1 p x i +1 + p ( ρ (1 − x i ) − ρ (1 − y i )) λ ′ (1 − ρ (1 − x i )) ( a ) ≤ 1 p α i +1 + p  1 − ρ ′ (1) α i − ρ (1 − y i )   λ 2 + O ( α i )  ( b ) ≤ 1 p α i +1 + pλ (1 − ρ (1 − y i ))  1 + O ( α i )  ( c ) ≤ 1 p α i +1 + αy i  1 + O ( α i )  , where ( a ) follows from ρ (1 − x ) ≤ 1 − ρ ′ (1) x , ( b ) follows from λ 2 (1 − ρ (1 − y )) ≤ λ (1 − ρ (1 − y )) , and ( c ) follows from pλ (1 − ρ (1 − y )) ≤ αy . It is easy to verify that y i +1 < y i as long as y i > α i +1 p (1 − α (1+ O ( α i ))) . Theref ore, we ﬁnd that y i → 0 because the recursion does not ha ve any p ositiv e ﬁxed points as i → ∞ . Mo reover , o ne can show that y i ev entually decreases exponentially at a rate arbitrarily close to α . Now , we con sider the p erforman ce of a rand omly chosen code and with a random error patter n. T he ap proach taken is g eneral enoug h to cover all the message-b ased decod ing algorithm s discussed in this paper . A message in the d ecoding graph is called bad if it is either unv eriﬁed o r falsely veriﬁed. Based on [1 5], on e can show that, af ter ℓ decoding iterations, the fraction o f bad messages is tightly concentr ated arou nd its average. W e n ote that the concentr ation occurs regard less o f wh ether ther e are falsely veriﬁed message s o r shor t cycles. Since the proof inv olves no new tech niques b eyond [ 15] and the extension to irregula r codes in [25], we giv e only a brief outline. The main dif ference between this scenario and [ 15] is that the algor ithms discussed in this paper pass lists wh ose size may be unbo unded and may be af fected b y FV . It tur ns out th at the size of these lists is superﬂu ous, however , if we only con sider th e fraction of messages satisfying a logical test ( e.g., the fractio n of bad messages). Let the r .v . Z ( ℓ ) denote the number of bad variable-to-ch eck messages after ℓ iter ations of decoding. Suppose the fraction of unv eriﬁed messages predicted by DE, assuming n o FV , is y ℓ . W e say that concentration fails if Z ( ℓ ) /E exceeds y ℓ by m ore than ǫ , where E is the number of edges in the graph. Follo wing [15], on e can bo und the failure p robab ility by showing that the fraction of b ad messages co ncentrates arou nd its av erage value and that the average value conv erges to the value computed by DE. The main ob servation is that the veriﬁcation status of an edge, after ℓ iterations, dep ends only on the depth- 2 ℓ neighbo rhood , N (2 ℓ ) ~ e , of the directed edge ~ e . By using a Doob’ s m artingale for th e edg e-exposure process an d apply ing Azuma’ s ineq uality , one obtain s the con centration boun d Pr Z ( ℓ ) E − E  Z ( ℓ )  E > ǫ 3 ! ≤ e − β ℓ ǫ 2 n , (20) where β ℓ is a positive constant indepe ndent of n which depend s only on the d .d. and the numbe r of iterations. The next step is showing that the expected value E  Z ( ℓ )  /E is clo se to the v alue, y ℓ , given by DE. In this case, w e must consider two sources of err or: short cycles and false veriﬁcation. In [ 15] and [25], it is shown that, when a code graph is chosen unifor mly at r andom f rom all po ssible graphs with degree distrib ution pair ( λ ( x ) , ρ ( x )) , Pr  N (2 ℓ ) ~ e is n ot tree-like  ≤ γ ℓ n , where γ ℓ is a constant indepen dent of n that depend s only on the d .d. and numbe r of iteration s. So, f or any ǫ > 0 , on e can choose n large en ough so that γ ℓ /n < ǫ/ 3 . Since type-I I FV is a failur e due to shor t cycles (see Sectio n IV -B f or d etails about type-I and type-II FV), this implies that any increase in E  Z ( ℓ )  /E du e to sho rt cycles and type-I I FV is at most ǫ/ 3 . Like wise, one can upper boun d the effect of typ e-I FV with Pr  any type-I FV occurred in N (2 ℓ ) ~ e  ≤ θ ℓ q , where θ ℓ is a c onstant indepen dent of n that depend s o nly the d .d., the n umber of iter ations, and algorithm de tails (e.g., S max ). If we ch oose q large enough so that θ ℓ /q < ǫ/ 3 , then these two bou nds imply that E  Z ( ℓ )  E − y ℓ ≤ γ ℓ n + θ ℓ q ≤ 2 ǫ 3 . (21) Finally , ( 20) and ( 21) can be com bined to bou nd the proba- bility that Z ( ℓ ) /E is gr eater than y ℓ + ǫ . A P P E N D I X B P R O O F O F L E M M A 4 . 1 Let y be th e rec ei ved symbo l sequence assuming the all- zero codeword was transmitted and let u be an other codew ord with exactly k n on-zer o symbols. Of th e k positions wh ere they dif f er , assume that i are received correctly , j are ﬂipped to other code word’ s v alue, and k − i − j are ﬂipped to a third v a lue. It is ea sy to verify that the all-zero co dew ord is not the unique ML code word whenever i ≤ j . Therefore , the pro bability that ML decoder cho oses u over the all-zero codeword is gi ven b y p 2 ,k = k X j =0 j X i =0  k i,j,k − i − j  (1 − p ) i  p q − 1  j  p ( q − 2) q − 1  k − i − j . 16 1 − p p 0 p 1 p 2 ... p p s Fig. 5. Finite -state machine for L emma 4.3. The multin omial theorem also sho ws that A ( x ) =  (1 − p ) + p q − 1 x 2 + p ( q − 2) q − 1 x  k = k X j =0 k − j X i =0  k i,j,k − i − j  (1 − p ) i  p q − 1  j  p ( q − 2) q − 1  k − i − j x k − i + j , 2 k X l =0 A l x l , where A l is the coefﬁcient of x l in A ( x ) . Next, we observe that p 2 ,k = P 2 k l = k A l is simply an unweigh ted su m of a subset of terms in A ( x ) ( namely , those wh ere k − i + j ≥ k ) . This imp lies that x k p 2 ,k = 2 k X l = k A l x k ≤ A ( x ) for any x ≥ 1 . Therefo re, we can compute the Chernoff-type bound p 2 ,k ≤ inf x ≥ 1 x − k A ( x ) . By taking derivati ve o f x − k A ( x ) over x and setting it to zero, we arr i ve at the b ound p 2 ,k ≤ p q − 2 q − 1 + s 4 p (1 − p ) q − 1 ! k . A P P E N D I X C P R O O F O F L E M M A 4 . 3 Pr oof: An unveriﬁcation ev ent occur s on a degree-2 cycle of length- k when there is a t most one correct v ariable nod e in any adjac ent set of s + 1 nodes. Let th e set of all error pattern s (i.e., 0 mean s co rrect and 1 mea ns error) of length - k whic h satisfy the UV cond ition b e Φ( s, p, k ) ⊆ { 0 , 1 } k . Using the Hamming weigh t w ( z ) , of an err or pattern as z , to c ount the number of errors, we can write the probability of UV as φ ( s, p, k ) = X z ∈ Φ( s,p,k ) p w ( z ) (1 − p ) k − w ( z ) . This expression can be evaluated using the tran sfer ma trix method to enume rate all weighted walks through a particular digraph . If we walk throu gh the nodes along the cycle by pick- ing an arbitrary node as the starting node, the UV constraint can be seen as k -steps of a p articular ﬁnite-state m achine. Since we are walking o n a cycle, the initial state must equal to the ﬁnal state. The ﬁnite-state machin e, which is shown in Fig. C, ha s s + 1 states { 0 , 1 , . . . , s } . Let state 0 be the state where we are f ree to choose either a correct or incorrect sym bol (i.e., the previous s symbols ar e all incorrect). This state has a self-loop associated with the next sym bol also being incorrect. Let state i > 0 be the state wh ere the past i values consist of one correct symbol followed by i − 1 inco rrect sym bols. Notice that o nly state 0 may ge nerate c orrect symbo ls. By d eﬁning the transfer matrix with (19), the probability that the UV con dition holds is theref ore φ ( s, p, k ) = Tr  B k ( p )  . R E F E R E N C E S [1] R. G. Gallager , “Low-de nsity parity-che ck codes, ” IRE T rans. Inform. Theory , v ol. 18, pp. 21–28, Jan. 1962. [2] D. MacKay and R. Neal, “Good codes based on v ery sparse matrice s, ” Lectur e Notes in Compute r Science , v ol. 1025, pp. 100–111, 1995. [3] M. Luby , M. Mitzenmacher , M. Shokrollah i, and D. Spielman, “Efﬁcient erasure correc ting codes, ” IEEE T rans. Inf . Theory , vol. 47, pp. 569–584, Feb . 2001. [4] T . Richardson, M. Shokrolla hi, and R. Urbanke, “Design of capacit y- approac hing irregular low-de nsity parity-check codes, ” IEEE T rans. Inf. Theory , v ol. 47, pp. 619–637, F eb . 2001. [5] A. Shokrolla hi, “Capacity-a pproachi ng codes on the q -ary symmetric channe l for lar ge q , ” in Pr oc. IEEE Inform. Theory W orkshop , (San Antonio, TX), pp. 204–208, Oct. 2004. [6] C. W eidmann, “Codi ng for the q -ary symmetric channel with moderate q , ” in Proc. IEEE Int. Symp. Information Theory , July 2008. [7] G. L echner and C. W eidmann, “Optimization of binary ldpc codes for the q -ary symmetric channel with moderat e q , ” in Pr oc. 5th Inte rnational Symposium on T urbo Codes and Related T opics , 2008. [8] M. Luby an d M. Mitzenma cher , “V eriﬁcation-ba sed decoding for pack et- based low-de nsity parity -check codes, ” IE EE T rans. Inf. Theory , vol. 51, pp. 120–127, Jan. 2005. [9] J. Metzner , “Majority-lo gic-lik e decoding of vect or symbols, ” IEEE T rans. Commun. , vol. 44, pp. 1227–1230, Oct. 1996. [10] J. Metzner , “Majorit y-logic -like vector sym bol decoding with alternat i ve symbol v alue lists, ” IEEE T rans. Commun. , vol. 48, pp. 20 05–2013, Dec. 2000. [11] M. Da ve y and D. MacKa y , “Low densit y parity check c odes ov er GF( q ), ” IEEE Commun. Lett. , vol. 2, pp. 58–60, 1998. [12] D. Bleiche nbache r , A . Kiyayias, and M. Y ung, “Decoding of interlea ved Reed-Sol omon codes ov er noisy data, ” in Pr oc. of ICALP , pp. 97–108, 2003. [13] A. Shokrollahi and W . W ang, “Low-densit y parity-chec k codes with rates very close to the capacity of the q -ary symmetric channel for large q , ” in P r oc. IEEE Int. Symp. Information Theory , (Chicago, IL), p. 275, June 2004. [14] A. Shokrollahi and W . W ang, “Low-densit y parity-chec k codes with rates very close to the capacity of the q -ary symmetric channel for large q . ” Unpublishe d exte nded abstract, 2004. [15] T . Richardson and R. Urbanke , “The capacit y of low-densi ty parity- check codes under message-passing decoding, ” IEEE T rans. Inf. Theory , vol. 47, pp. 599–6 18, Feb . 2001. [16] V . Guruswami and P . Indyk, “Linea r time encodable and list decod able codes, ” in Pr oc. of th e 35th Annual AC M Symp. on Theory of Comp. , pp. 126–135, 2003. [17] F . Z hang and H. D. Pﬁster , “On the stoppin g sets of veriﬁc ation decodin g. ” in preparati on, June 2011. [18] N. W ormald, “Diffe rentia l equation s for random processes and random graphs, ” A nnals of Applied Pr obability , vol. 5, pp. 1217–1 235, 1995. [19] F . Z hang and H. D. Pﬁster , “Software to compute the LM2-NB thresh- old. ” at http://www .ece.tamu.edu/ ˜ hpﬁster/softw are/ lm2nb threshold .m. [20] N. C. W ormald, “The diffe rentia l equati on method for random graph processes and greedy algorithms, ” in Lectur es on Appr oximation and Randomized Algorithms , pp. 73–155, Polish Scientiﬁc Publishers, 1999. [21] T . J. Richardson and R. L. Urbanke, Modern Coding T heory . Cambridge Uni ver sity Press., 2008. [22] R. Storn and K. Price, “Dif ferential ev oluti on–A simple and efﬁcie nt heuristi c for global optimization over continuou s spaces, ” J. Global Optim. , vol. 11, no. 4, pp. 341–359 , 1997. [23] S. Sarvot ham, D. Baron, and R. Baraniuk, “Sudocodes–Fast measure- ment and reconstruc tion of sparse signals, ” in Proc. IEEE Int. Symp. Informatio n Theory , (Seattle , W A), pp. 2804–2808, July 2006. [24] F . Zhang and H. D. Pﬁster, “V ericati on decoding of high-rate ldpc codes with application s in compressed sensing . ” submitted to IEEE Tr ans. on Inform. Theory a lso av ailable in Arxiv prep rint cs.IT/0903.2232v3, 2009. [25] A. Kavcic , X. Ma, and M. Mitzenmache r , “Bina ry intersymbol interfer- ence channels: Gallag er codes, density e volut ion, and code performance bounds, ” IEEE T rans. Inf. Theory , vol . 49, pp. 1636–1652, July 2003. 17 Fan Zhang (S’03) recei ve d his Ph.D. in electri cal engineeri ng from T exas A&M Uni versit y in 2010 and joine d the read-channel arth itect ure group at LSI Logic. He spent 3 years at UTStarc om Inc. be fore joining T exas A&M Uni ver sity . His current research interests include information th eory , error correcting codes and signal processing for wireless and data storage systems. Henry D. Pﬁster (S’99–M’03–SM’09) recei ve d his Ph.D. in electrica l enginee ring from UCSD in 2003 and he joined the facul ty of the School of Engineering at T exas A&M Univ ersity in 2006. Prior to that he spent two years in R&D at Qualc omm, Inc. and one year as a post-doc at EPFL. He recei ved the NSF Career A ward in 2008 and wa s a coauth or of the 2007 IEEE COMSOC best paper in Signal Proce ssing and Codin g for Data Storage. His curren t research interests include information theory , cha nnel coding, and iterati ve decoding with applicati ons in wireless communications and data storage.

Analysis of Verification-based Decoding on the q-ary Symmetric Channel for Large q

Original Paper

Comments & Academic Discussion

Leave a Comment

Original Paper

Related Papers

Comments & Academic Discussion

Leave a Comment