On Trapping Sets and Guaranteed Error Correction Capability of LDPC Codes and GLDPC Codes

SUBMITTED TO IEEE TRANSA CTIONS ON INFORMA TION THEOR Y , MA Y 2008 1 On T rapping Sets and Guaranteed Error Correction Capability of LDPC Codes and GLDPC Cod es Shashi Kiran Chilappagari, Student Member , IEEE, Dung V iet Nguyen, Student Mem- ber , IEEE, Bane V a sic, Senior Me mber , IEEE, a nd Michael W . Marcellin, F ellow , IEEE Abstract The relation between th e girth and the gu aranteed error corr ection capability of γ -left regular LDPC codes when decoded using the bit flipping (serial an d parallel) algor ithms is investigated. A lower bou nd on the size o f variable n ode sets wh ich e xpand by a factor of at least 3 γ / 4 is fou nd based o n the M oore bo und. An u pper boun d on th e guaran teed er ror correction capa bility is established b y stud ying the sizes o f smallest possible trappin g sets. The results ar e extended to ge neralized LDPC cod es. It is shown that generalized L DPC cod es can corre ct a linear fraction of err ors u nder the parallel bit flipping algo rithm when the under lying T a nner graph is a go od expan der . It is also shown that the bo und cann ot be im proved when γ is even by study ing a class of trap ping sets. A lower bound o n the size of variable n ode sets which have the req uired expansion is estab lished. Index T erms Low-density par ity-check codes, b it flipp ing algorithm s, tr apping sets, error co rrection capability I . I N T RO D U C T I O N Iterativ e algorithms for decodin g low-density parity-check (LDPC) codes [1] have been t he focus of research ov er the past d ecade and most of th eir properties are well underst ood [2], [3]. These algori thms operate by passing messages along the edges of a graphical representation of the code known as th e T anner graph, and are optimal when t he underlying graph is a tree. Message passing decoders perform remarkably well which can be attributed to their ability to correct errors beyond the traditional bounded distance decoding capability . Howe ver , in contrast to bounded distance decoders (BDDs), the guaranteed error correction capabili ty of i terativ e decoders is largely unknown. Manuscript receiv ed August 16, 2021. This work is funded by NSF under Grant CCF -0634969 , ECCS- 072540 5, ITR-0325979 and by the INSIC-EHDR program. S. K. Chilappagari, D. V . Nguyen, B. V asic and M. W . Marcellin are with the Department of Electrical and Compu ter Engineering, Univ ersity of Ar izona, Tucso n, Arizona, 85721 USA. (emails: { shashic, nguyendv , vasic, marcellin } @ece.arizona.edu. Parts of this w ork hav e been accepted for presentation at the International Symposium o n Information Theory (ISIT’08) and the International T eleme tering Conference (ITC’08). SUBMITTED TO IE EE TRANSA CTIONS ON INFORMA TION THEOR Y , MA Y 2008 2 The p roblem of recov ering from a fixed number of erasures is sol ved for i terativ e decoding on the binary erasure channel (BEC). If th e size of the minimu m stopping set in the T anner graph of a code is at l east t + 1 , t hen the d ecoder is guaranteed to recov er from any t erasures. Orlitsky et al. [4] studied the relation between stoppi ng sets and girth and deriv ed bounds on the smallest s topping set in any d -left regular T anner graph with girth g . An analogou s resul t does not exist for decoding on o ther channels such as the binary sy mmetric channel (BSC) and the addi tiv e white Gaussi an nois e (A WGN) channel. In this paper , we present such a result for hard decision decoding algorithms. Gallager [1] pro posed two bi nary message passing algorith ms, namely Gallager A and Gallager B, for decoding over the BSC. He showed that for t he column-weight γ ≥ 3 and ρ > γ , there exist ( n, γ , ρ ) 1 regular LDPC codes for w hich the bit error probability asymptotically t ends to zero wh ene ver we operate below the threshold. T he minimum distance was sho wn to increase linearly with the code l ength, but correction o f a linear fraction of errors was not shown. Zyablov and Pin sker [6] analyzed LDPC codes under a simpler decoding algorithm kn own as the bit flipping algorithm, and showed th at al most all t he cod es i n the regular ensemble wi th γ ≥ 5 can correct a constant fraction of worst case errors. Sipser and Spielman [7] used expander graph ar guments t o analyze tw o bit flipping algorithms, serial and parallel. Specifically , they showed that these algorithm s can correct a fraction of errors if the underly ing T anner graph is a goo d expander . Burshtein and Miller [8] applied expander based ar guments to s how that message passi ng algorithms can also correct a fixe d fraction of worst case errors when the degree of each variable no de is mo re than fi ve. Feldman et al. [9] showed that the lin ear programming decoder [10] is also ca pable of correcting a fraction of err ors. Recently , Burshtein [11] showed that regular codes wit h variable nodes of degree fou r are capable of correcting a linear number of errors under bit flipping algorithm . He also showed tremendous improvement in the fraction of correctable errors when t he variable node degree is at least fiv e. T anner [5 ] st udied a class of codes con structed based on bipartite graphs and short error corr ecting codes. T ann er’ s work is a generalization of the LDPC codes proposed by Gallager [1] and hence t hese codes are referred to as g eneralized LDPC (GLDPC) codes. T anner propo sed code construction techniques, decoding algorithm s and complexity and performance analy sis t o analyze these codes and derive d bounds on the rate and mini mum d istance for these codes. Sipser and Spielman [7] analyzed a special case of GLDPC codes (which they termed as expander codes) us ing expansion arguments and proposed explicit constructions of asy mptotically good codes capable of correcting a fraction of errors. Zemor [12] imp roved the fraction of correctable errors under a modified decoding al gorithm. Ba rg and Zemor in [13] analy zed the error exponents of expander codes and showed that expander codes achieve capacity over the BSC. 1 Precise definitions will be given in Section II and we f ollo w standard terminology from [1] and [5] SUBMITTED TO IE EE TRANSA CTIONS ON INFORMA TION THEOR Y , MA Y 2008 3 Janwa and Lal [14] stud ied GLDPC codes in the most general setting by consi dering u nbalanced bipartite graphs. Miladi novic and Fossorier [15] deriv ed bounds on the guaranteed error correction capability of GLDPC codes for the s pecial case of failures only decoding. The focus of this p aper is to establish lower and upper bounds on the guaranteed error correction capability o f LDPC codes and GLDPC codes as a function of their column -weight and girth. For the case of GLDPC codes, we also find the e xpansion required to gu arantee correction of a fraction of errors under the parallel bi t flippin g algorithm, as a function of t he error correction capabili ty of th e sub-code. Our app roach can be summarized as follo ws: (a) to establi sh lower bo unds, we determine the size o f var iable node sets i n a left re gular T anner graph which are guaranteed to hav e the e xpansion required by bit flipping algorithms, based on the Moore bound [16 , p.18 0] and (b) to find upper boun ds, we stud y the sizes of smallest poss ible trapping sets [17] in a left regular T anner graph. It is well known that a random g raph is a good expander with high probability [7]. Howe ver , the fraction of nodes having the required expansion is very small and hence the code lengt h to guarantee correction of a fixed number of errors must be large. Moreover , determinin g the expansion of a given graph is known to b e NP hard [18], and s pectral gap met hods cannot guarantee an expansion factor of mo re t han 1 / 2 [7]. On th e other hand, code parameters such as col umn weight and girth can be easily determined or are assumed to be known for the code under consi deration. W e prove that for a g iv en colu mn-weight, the error correction capabil ity grows exponentially in girth. Howev er , we not e that sin ce the g irth grows logarithmically i n the code length, this result does not show that the bi t flipping algorith ms can correct a linear fraction of errors. T o find an upper bound on the number of correctable errors, we study the size of sets of variable nodes which lead to decoding failures. A d ecoding failure i s said to hav e occurred if th e output of the decoder is not equal t o the transmitt ed codeword [17]. The condition s that l ead to decoding failures are well underst ood for a variety o f decodi ng algorit hms such as maximum li kelihood decoding, b ounded distance decoding and iterative decoding on the BEC. Howev er , for iterativ e decoding on the BSC and A WGN channel, the underst anding is far from complete. T wo approaches h a ve been taken i n thi s direction, namely t rapping sets [17] and p seudo-codew ords [19]. W e adopt t he trapping s et approach in this paper to characterize decoding failures. Richardson [17 ] introduced th e notion o f trapping sets to estim ate t he error floor on the A WG N channel. In [20], trapping sets were used to estimate t he frame error rate of column-weigh-three LDPC codes. In this paper , we define trapping sets with the help of fixed points for the bit flipping alg orithms (both serial and parallel). W e then find bounds on the size of trapping sets based on extremal graphs kno wn as cage graphs [21], thereby finding an upper bound on the guaranteed error correction capability . By saying that a code with column w eight γ and girth 2 g ′ is not guaranteed to SUBMITTED TO IE EE TRANSA CTIONS ON INFORMA TION THEOR Y , MA Y 2008 4 correct k errors, we mean that there exists a code with column wei ght γ and girth 2 g ′ that fails to correct k errors. The rest of the paper is organized as fol lows. In Section II, we provide a brief introducti on to LDPC codes, decoding algo rithms and trapping sets [17 ]. In Se ction III, we pro ve o ur main theorem relatin g the column weight and girth to the size of variable node sets which expand by a factor of at least 3 γ / 4 . W e deriv e bounds on the size of trappi ng sets based on cage g raphs in Section IV. In Section V, we prove that the parallel b it flipping algorithm can correct a fraction of errors if the un derlying T anner graph is a good expander . W e conclud e with a few remarks in Section VI. I I . P R E L I M I N A R I E S In t his section, we first est ablish the notation and then p roceed to giv e a brief i ntroduction to LDPC codes and hard decision decodi ng algorithm s. W e th en giv e the relation b etween the error correction capability of the code and the expansion of the underlying T anner graph. W e finall y describe trapping sets for the algorithm s. A. Graph Theory Notation W e adopt the standard notation in graph theory (see [22] for example). G = ( U, E ) denotes a g raph with set of nodes U and set of edges E . When there is no ambiguity , we simply denote the graph by G . An edge e is an u nordered pair ( u 1 , u 2 ) of nodes and is sai d to be i ncident on u 1 and u 2 . T wo nodes u 1 and u 2 are said to be adjacent (neighbors) i f th ere is an edge e = ( u 1 , u 2 ) i ncident on them. The order of the graph is | U | and th e size of t he graph is | E | . The degree of u , d ( u ) , i s the num ber of its neighbors. A nod e with degree one is call ed a leaf or a pendant node. A graph is d -regular if all the nodes have degree d . The a verage degree d of a graph is defined as d = 2 | E | / | U | . The girth g ( G ) of a graph G , is the length of smallest cycle in G . H = ( V ∪ C , E ′ ) denotes a bipartite graph with two sets o f nodes; var iable (left) nodes V and check (right) nodes C and edge set E ′ . Nodes in V have n eighbors on ly in C and vice versa. A bipartit e graph i s said to be γ -left regular if all variable nod es hav e d egree γ , ρ -right regular if all check nodes have degree ρ and ( γ , ρ ) regular if all variable nod es have degree γ and all check nodes have degree ρ . The girth o f a bi partite graph is ev en. B. LDPC Codes and Decoding Al gorithms LDPC codes [1] are a class of linear block codes which can be defined by sparse bipartite graphs [23]. Let G be a bipartite graph with two sets of nodes: n variable no des and m check nodes. This graph defines a linear block code C of l ength n and dimensio n at least n − m in the following way: The n v ariable nodes SUBMITTED TO IE EE TRANSA CTIONS ON INFORMA TION THEOR Y , MA Y 2008 5 are associated to the n coordi nates of cod e words. A vector v = ( v 1 , v 2 , . . . , v n ) is a codew ord if and o nly if for each check node, th e modul o two sum o f i ts neighbors is zero. Such a graphical representatio n of an LDPC code is called the T anner graph [5] of the code. The adjacency m atrix o f G g iv es a parity check matrix of C . An ( n, γ , ρ ) regular LDPC code has a T anner graph with n variable nodes each o f de gree γ (column weight) and nγ /ρ check nodes each of degree ρ (ro w w eight). Thi s code has length n and rate r ≥ 1 − γ / ρ [23]. W e now describe a s imple hard decision decoding algorithm known as the parallel bit flipp ing algo rithm [6], [7] to decode LDPC codes. As noted earlier , each check nod e imposes a constraint on the n eighboring var iable nodes. A constraint (check nod e) is s aid to be sat isfied by a setting of variable nod es if t he sum of the variable nodes in t he constraint is ev en; otherwise t he constraint is unsatisfied. Parallel Bit Flipping Algorithm • In parallel, flip each variable that is in more unsat isfied than sati sfied constraints. • Repeat until no such variable remains. A serial v ersion of the algorithm is also defined in [7] and all the results in this paper ho ld for the serial bit flipping algorithm also. Th e bit flipping algorit hms are iterative i n nature but do not belong t o the class of mess age passing algorithms (see [8] for an explanation). C. Expansion and Error Corr ection Capability Sipser and Spielm an [7] analyzed the performance of the bit flipping algorithms using the expansion properties of t he underlyin g T anner graph of the code. W e summarize the results from [7] below for the sake of compl eteness. W e start with t he following definitions from [7]. Definition 1: Let G = ( U, E ) with | U | = n 1 . Then every set of at most m 1 nodes expands b y a factor of δ if, for all sets S ⊂ U | S | ≤ m 1 ⇒ |{ y : ∃ x ∈ S such that ( x, y ) ∈ E }| > δ | S | . W e con sider biparti te graphs and expansion of var iable nodes o nly . Definition 2: A graph is a ( γ , ρ, α, δ ) expander i f i t is a ( γ , ρ ) regular bi partite graph in wh ich ever y subset of at mos t α fraction of the variable nodes expands by a factor of at least δ . The following t heorem from [7] relates the expansion and error correction capability of an ( n, γ , ρ ) L DPC code with T anner graph G when decoded usi ng the parallel bit flipping d ecoding algorithm . Theor em 1: [7, Theorem 11] Let G be a ( γ , ρ, α, (3 / 4 + ǫ ) γ ) expander ov er n v ariable nodes, for any ǫ > 0 . Then, the si mple parallel decoding algorith m will correct any α 0 < α (1 + 4 ǫ ) / 2 fraction of errors after log 1 − 4 ǫ ( α 0 n ) decoding rou nds. Notes: SUBMITTED TO IE EE TRANSA CTIONS ON INFORMA TION THEOR Y , MA Y 2008 6 1) The serial bit flipping algorithm can also correct α 0 < α/ 2 fraction of errors if G is a ( γ , ρ, α, (3 / 4) γ ) expander . 2) The results hold for any left regular code as expansion is needed for variable nodes only . From the above discussion, it is observed that findi ng the num ber o f variable nodes which are guaranteed to expand by a factor of at least 3 γ / 4 , giv es a l ower bou nd on the g uaranteed error correction capability of LDPC codes. D. Decoding F ailure s and T rapping Sets W e now characterize failures of the iterative decoders using fixed points and trapping sets . Some of the following discussion appears in [24], [20], [25] and we inclu de it for sake of com pleteness. Consider an LD PC code of l ength n and let x = ( x 1 x 2 . . . x n ) be th e binary vector which is the input to the i terativ e decoder . Let S ( x ) be the support of x . The support of x is defined as the set of al l p ositions i where x i 6 = 0 . The set of variable nod es (bits) whi ch differ from thei r correct v alue are referred to as corrupt variables. Definition 3: [24] A decoder failure is s aid to have occurred if th e output of the decoder is no t equal to the t ransmitted codew ord. Definition 4: x is a fixed point of the bit flipping algo rithm i f the set of corrupt v ariables remains unchanged after one round of decoding. Definition 5: [20] The support o f a fixed point is known as a trapping set. A ( V , C ) trappin g set T i s a set of V va riable nodes whos e induced su bgraph has C odd degree checks. If the variable nodes correspondi ng to a trapping set are in error , th en a decoder failure occurs. Howe ver , not all var iable nodes correspon ding to a trapping set need to be in error for a decoder failure to occur . Definition 6: [20] The m inimal num ber of variable nodes that hav e to be init ially in error for t he decoder to end up i n the trappi ng set T will be referred to as critical nu mber m for t hat trapping set. Definition 7: [24] A s et o f variable nodes which if i n error lead to a decoding failure is known as a failur e set . I I I . C O L U M N W E I G H T , G I RT H A N D E X P A N S I O N In this section, we prove ou r main theorem which relates t he column weight and g irth of a code to i ts error correction capability . W e show that the size of var iable node sets which have t he required expansion is related to the well known M oore bound [16, p.180]. W e start with a few definition s required to establish the main theorem. SUBMITTED TO IE EE TRANSA CTIONS ON INFORMA TION THEOR Y , MA Y 2008 7 A. Definitions Definition 8: The r educed graph H r = ( V ∪ C r , E ′ r ) of H = ( V ∪ C , E ′ ) is a graph with vertex set V ∪ C r and edge set E ′ r giv en by C r = C \ C p , C p = { c ∈ C : c is a p endant nod e } E ′ r = E ′ \ E ′ p , E ′ p = { ( v i , c j ) ∈ E : c j ∈ C p } . Definition 9: Let H = ( V ∪ C , E ′ ) be s uch that ∀ v ∈ V , d ( v ) ≤ γ . The γ augmented graph H γ = ( V ∪ C γ , E ′ γ ) is a graph wi th vertex set V ∪ C γ and edge set E ′ γ giv en by C γ = C ∪ C a , where C a = | V | [ i =1 C i a and C i a = { c i 1 , . . . , c i γ − d ( v i ) } ; E ′ γ = E ′ ∪ E ′ a , where E ′ a = | V | [ i =1 E ′ i a and E ′ i a = { ( v i , c j ) ∈ V × C a : c j ∈ C i a } . Definition 10: [7, Definition 4 ] Th e edge-vertex incidence graph G ev = ( U ∪ E , E ev ) of G = ( U, E ) is the b ipartite graph with vertex set U ∪ E and edge set E ev = { ( e, u ) ∈ E × U : u is an endpoint of e } . Notes: 1) The edge-vertex i ncidence graph is right regular wit h degree two. 2) | E ev | = 2 | E | . 3) g ( G ev ) = 2 g ( G ) . Definition 11: An inv erse edge-verte x i ncidence graph H iev = ( V , E ′ iev ) o f H = ( V ∪ C , E ′ ) i s a graph with verte x set V and edge set E ′ iev which is obtain ed as follows. For c ∈ C r , let N ( c ) denote the set of neighbors of c . Label one node v i ∈ N ( c ) as a root nod e. Then E ′ iev = { ( v i , v j ) ∈ V × V : v i ∈ N ( c ) , v j ∈ N ( c ) , i 6 = j, v i is a root node, for some c ∈ C r } . Notes: 1) Given a graph, the inv erse edge-verte x incidence g raph is not unique. 2) g ( H iev ) ≥ g ( H ) / 2 , | E ′ iev | = | E ′ r | − | C r | and | C r | ≤ | E ′ r | / 2 . 3) | E ′ iev | ≥ | E ′ r | / 2 with equ ality only i f all checks in C r hav e degree two. 4) The term inv erse edge-vertex incidence is used for the following reason. Suppose all checks in H hav e degree two. Then th e edge-verte x incidence g raph of H iev is H . SUBMITTED TO IE EE TRANSA CTIONS ON INFORMA TION THEOR Y , MA Y 2008 8 The Moor e bound [16, p.180] denoted by n 0 ( d, g ) is a l ower bound on the least number of ve rtices in a d -regular graph with girth g . It i s give n by n 0 ( d, g ) = n 0 ( d, 2 r + 1 ) = 1 + d r − 1 X i =0 ( d − 1) i , g odd n 0 ( d, g ) = n 0 ( d, 2 r ) = 2 r − 1 X i =0 ( d − 1) i , g even . In [26], it was shown that a simil ar bound holds for irregular graphs. Theor em 2: [26] The number of nodes n ( d, g ) in a graph of girth g and av erage degree at least d ≥ 2 satisfies n ( d, g ) ≥ n 0 ( d, g ) . Note that d need not be an integer in the above theorem. B. The Main Theor em W e n ow state and prove t he main theorem. Theor em 3: Let G be a γ ≥ 4 -left r egular T anner graph G with g ( G ) = 2 g ′ . Then for all k < n 0 ( γ / 2 , g ′ ) , any set of k variable nodes in G expands by a factor of at l east 3 γ / 4 . Pr oof: Let G k = ( V k ∪ C k , E k ) denote the subgraph i nduced by a s et of k variable nodes V k . Since G is γ -left regular , | E k | = γ k . Let G k r = ( V k ∪ C k r , E k r ) be the reduced graph. W e hav e | C k | = | C k r | + | C k p | | E k | = | E k p | + | E k r | | E k p | = | C k p | | C k p | = γ k − | E k r | . W e n eed to prove that | C k | > 3 γ k / 4 . Let f ( k , g ′ ) denote the maxim um nu mber of edges i n an arbitrary graph of order k and g irth g ′ . By Theorem 2, for all k < n 0 ( γ / 2 , g ′ ) , the avera ge d egree of a graph with k nodes and gi rth g ′ is less than γ / 2 . Hence, f ( k , g ′ ) < γ k / 4 . W e now ha ve the following lemma. Lemma 1: The number of edges in G k r cannot exceed 2 f ( k , g ′ ) i.e., | E k r | ≤ 2 f ( k , g ′ ) . Pr oof: The p roof is by contradiction. Assum e th at | E k r | > 2 f ( k , g ′ ) . Cons ider G k iev = ( V k , E k iev ) , an in verse edge verte x incidence graph o f G k . W e have | E k iev | > f ( k , g ′ ) . SUBMITTED TO IE EE TRANSA CTIONS ON INFORMA TION THEOR Y , MA Y 2008 9 This is a contradiction as G k eiv is a graph of order k and girt h at least g ′ . W e n ow find a lower bound on | C k | in terms of f ( k , g ′ ) . W e have t he following lemma. Lemma 2: | C k | ≥ γ k − f ( k , g ′ ) . Pr oof: Let | E k r | = 2 f ( k , g ′ ) − j for some integer j ≥ 0 . Then | E k p | = γ k − 2 f ( k , g ′ ) + j . W e cl aim that | C k r | ≥ f ( k , g ′ ) + j . T o see this, we note that | E k iev | = | E k r | − | C k r | , or | C k r | = | E k r | − | E k iev | . But | E k iev | ≤ f ( k , g ′ ) ⇒ | C k r | ≥ 2 f ( k , g ′ ) − j − f ( k , g ′ ) ⇒ | C k r | ≥ f ( k , g ′ ) − j. Hence we have, | C k | = | C k r | + | C k p | ⇒ | C k | ≥ f ( k , g ′ ) − j + γ k − 2 f ( k , g ′ ) + j ⇒ | C k | ≥ γ k − f ( k , g ′ ) . The theorem now follows as f ( k , g ′ ) < γ k / 4 and therefore | C k | > 3 γ k / 4 . Cor ollary 1: L et C be an LDPC code with column-weight γ ≥ 4 and girt h 2 g ′ . T hen the bit flipping algorithm can correct any error patt ern of weight l ess than n 0 ( γ / 2 , g ′ ) / 2 . I V . C AG E G R A P H S A N D T R A P P I N G S E T S In this section, we first giv e necessary and suffi cient conditi ons for a given set of v ariables to be a trapping set. W e then proceed to define a class of int eresting graphs kno wn as cage graphs [21] and establish a relation between cage graphs and trappi ng sets. W e t hen giv e an u pper bound on the error correction capability based on the si zes of cage graphs. The proo fs in this section are along t he same lines as i n Section III. Hence, we only give a sketch of the proofs. SUBMITTED TO IE EE TRANSA CTIONS ON INFORMA TION THEOR Y , MA Y 2008 10 Theor em 4: Let C be an LDPC code wit h γ -left regular T anner graph G . Let T be a set consisting of V var iable nodes with i nduced subgraph I . Let th e checks in I be partitioned into two disjoi nt sub sets; O consisting of checks with odd degree and E consistin g of checks with e ven degree. Then T is a trapping set for b it flipping algori thm iff : (a) Every variable node in I has at least ⌈ γ / 2 ⌉ neighbors in E , and (b) No ⌊ γ / 2 ⌋ + 1 checks o f O share a neig hbor out side I . Pr oof: W e first s how th at t he condi tions stated are s uffi cient. Let x T be th e in put to the bit flipping algorithm, with supp ort T . Th e only uns atisfied constraint s are in O . By t he conditi ons of th e theorem, we ob serve that no v ariable node is in v olved in m ore unsatisfied constraints than sati sfied constraints . Hence, no variable node is flipped and by definiti on x T is a fixed poi nt implyi ng that T is a trappi ng set. T o see that the conditions are necessary , o bserve that for x T to be a trapping set , no variable n ode should be inv olved in more unsatisfied const raints than satis fied constraints. Remark: Theorem 4 is a consequence of Fact 3 from [17]. T o determine w hether a given set of variables is a trapping set, i t is necessary to not only know the induced subg raph but als o the n eighbors of the odd degree checks. Howe ver , in o rder to establish general bounds on the sizes of trapping sets gi ven onl y the column weight and the girth, we consider only condition (a) of Theorem 4 which is a necessary condit ion. A set of var iable nodes satisfying condition (a) is known as a potent ial trappi ng set . A trapping set is a potential trapping set that satisfies condit ion (b). Hence, a lower bound on the size of t he potent ial trapping set is a lower bound on the si ze of a trapping set. It is worth noti ng that a pot ential t rapping set can alwa ys be extended to a trapping set by successiv ely adding a variable node til l condition (b) is satisfied. Definition 12: [21] A ( d, g ) - cage g raph , G ( d, g ) , is a d -regular graph with girth g having the minimum possible number of nodes. A lower bound, n l ( d, g ) , on t he number of nodes n c ( d, g ) in a ( d, g ) -cage graph is given by the Moore bound. An upper bound n u ( d, g ) on n c ( d, g ) (see [21] and references therein) is giv en by n u (3 , g ) =    4 3 + 29 12 2 g − 2 for g odd 2 3 + 29 12 2 g − 2 for g even n u ( d, g ) =    2( d − 1) g − 2 for g odd 4( d − 1) g − 3 for g e ven . Theor em 5: Let C be an LDPC code with γ -left regular T anner graph G and girth 2 g ′ . Let T ( γ , 2 g ′ ) denote the s ize of s mallest possible po tential trapping set of C for t he bit flipping al gorithm. Then, |T ( γ , 2 g ′ ) | = n c ( ⌈ γ / 2 ⌉ , g ′ ) . Pr oof: W e first prove the follo wing l emma and then e xhibit a po tential trappi ng s et of size n c ( ⌈ γ / 2 ⌉ , g ′ ) . Lemma 3: |T ( γ , 2 g ′ ) | ≥ n c ( ⌈ γ / 2 ⌉ , g ′ ) . SUBMITTED TO IE EE TRANSA CTIONS ON INFORMA TION THEOR Y , MA Y 2008 11 Pr oof: Let T 1 be a trapping set wit h |T 1 | < n c ( ⌈ γ / 2 ⌉ , g ′ ) and let G 1 denote the in duced s ubgraph of T 1 . W e can construct a ( ⌈ γ / 2 ⌉ , g ′′ ) - cage graph ( g ′′ ≥ g ) wi th | T 1 | < n c ( ⌈ γ / 2 ⌉ , g ′ ) nodes by removing edges (if necessary) from the in verse edge-v ertex of G 1 which is a contradictio n. W e now exhibit a potential t rapping set of size n c ( ⌈ γ / 2 ⌉ , g ′ ) . Let G ev ( ⌈ γ / 2 ⌉ , g ′ ) be th e edge-vertex inci- dence graph of a G ( ⌈ γ / 2 ⌉ , g ′ ) . Note that G ev ( ⌈ γ / 2 ⌉ , g ′ ) is a left regular bipartite graph with n c ( ⌈ γ / 2 ⌉ , g ′ ) var iable nodes of degree ⌈ γ / 2 ⌉ and all checks have degree two. Now consider G ev, γ ( ⌈ γ / 2 ⌉ , g ′ ) , the γ augmented graph of G ev ( ⌈ γ / 2 ⌉ , g ′ ) . It can be seen that G ev, γ ( ⌈ γ / 2 ⌉ , g ′ ) is a potential t rapping set. Theor em 6: There exists a code C with γ -left regular T anner graph of g irth 2 g ′ which fails to correct n c ( ⌈ γ / 2 ⌉ , g ′ ) errors. Pr oof: Let G ev, γ ( ⌈ γ / 2 ⌉ , g ′ ) be as defined in T heorem 5. Now construct a code C with col umn-weight γ and g irth 2 g ′ starting from G ev, γ ( ⌈ γ / 2 ⌉ , g ′ ) such that the set of v ariable nodes in G ev, γ ( ⌈ γ / 2 ⌉ , g ′ ) als o satisfies condition (b) of Theorem 4. Then, by T heorem 4 and Theorem 5, the set of variable nodes in G ev, γ ( ⌈ γ / 2 ⌉ , g ′ ) with cardinality n c ( ⌈ γ / 2 ⌉ , g ′ ) is a trapping set and hence C fails to decode an error pattern of weight n c ( ⌈ γ / 2 ⌉ , g ′ ) . Remark: W e n ote that for γ = 3 and γ = 4 , the above bound i s tight. Observe t hat for d = 2 , the Moore bound is n 0 ( d, g ) = g and that a cycle of lengt h 2 g wit h g variable nodes is always a potential trapping set. In fact, for a code with γ = 3 or 4 , and T anner graph of gi rth greater than eight, a cycle of the sm allest length is always a trappi ng set (see [24] for the proof). V . G E N E R A L I Z E D L D P C C O D E S In this section, we first consid er two bit flippi ng decoding algorit hms for GLDPC codes. W e then establish a relation between expansion and error correction capability . W e also establish a lower bou nd on the number of variable nodes that hav e the required expansion. W e t hen exhibit a trapping set and as a consequence sh ow that t he bound on the required expansion cannot be imp roved when γ i s even. W e also establish b ounds on the size of t rapping sets. W e b egin with the definition o f GLDPC codes by ado pting the termi nology from expander codes [7]. Definition 13 (Definitio n 6, [7]): : Let G be a ( γ , ρ ) regular bipartite graph between n variable nodes ( v 1 , v 2 , . . . , v n ) and nγ /ρ check nodes ( c 1 , c 2 , . . . , c nγ /ρ ) . Let b ( i, j ) be a function desi gned so th at, for each check node c i , the variables neig hboring c i are v b ( i, 1) , v b ( i, 2) , . . . , v b ( i,ρ ) . Let S be an error correcting code of block length ρ . The GLDPC code C ( G, S ) is the code of block length n w hose codewords are the words ( x 1 , x 2 , . . . , x n ) such that, for 1 ≤ i ≤ nγ /ρ , ( x b ( i, 1) , . . . , x b ( i,ρ ) ) is a codew ord of S . The terms column-weight, row-weight, check nodes, var iable nodes and trapping sets mean the sam e as in case of LDPC codes. T he code S at each check node is som etimes referred to as the s ub-code. SUBMITTED TO IE EE TRANSA CTIONS ON INFORMA TION THEOR Y , MA Y 2008 12 A. Decoding algori thms T anner [5 ] propo sed differe nt hard decis ion decodin g algorithms t o decode G LDPC codes. W e now describe an i terativ e al gorithm kno wn as parallel bit flipping algorithm ori ginally described in [5], which is employed when the sub-code i s capable of correcting t errors. Parallel bit flipping algorithm: Each decoding round consi sts of the following steps. • A var iable node sends its current est imate to check nodes. • A check node performs decoding on incoming messages and finds t he nearest code word. For all var iable nodes which differ from the codew ord, the check node sends a flip message. If the check node does n ot find a unique codeword, it does not send any flip messages. • A var iable node flips i f it receives more than γ / 2 flip messages. The set of va riable nodes which differ from their correct value are known as corrupt variables. Th e rest of the variable nodes are referred to as correct variables. Follo wing t he algorithms, we ha ve the following definition adopted from [7]: Definition 14: A check node is said to be confused if it sends flip messages to correct v ariable nodes, or if it does not send flip message to corrupt variable nodes, o r both. Otherwise, a check node is said to be helpful . Remarks: 1) For the parallel bit flippin g decoding algorith m, a check node with sub -code of mini mum distance at least d min = 2 t + 1 can be confused only i f it is connected to m ore than t corrupt variable nodes. 2) The parallel bit flipping algorit hm is different from t he algorithm presented by Sipser and Spielman in [7] for expander codes, b ut is similar to the algorithm propos ed by Zemor in [12]. Howe ve r , we note that the codes considered in [12] are based on d -regular bipartite graphs and are a special case of doubly generalized LDPC codes, where each v ariable node is al so associated with an error correcting code. 3) Apart from helpful checks and confused checks, Sipser and Spielman defined unhelpful checks. Howe ver , our definition of confused checks in cludes unhelpful checks as well. 4) Mil adinovic and Fossorier in [15] considered a d ecoding algori thm where the decoding at e very check either results in correct decoding or a failure b ut not miscorrection. While this assumption is reasonable when the sub-code is a long code, it is n ot true in general. W e howe ver , point out that the meth odology we adopt can be appl ied to t his case as well. 5) The work by Sipser and Spielman [7], Zemor [12], Barg and Zemor [13] and Janwa and L al [14] focused on asymptotic result s and explicit construction of expander codes. The proofs and con- structions are based on spectral gap and as noted earlier , such metho ds cannot guarantee expansion SUBMITTED TO IE EE TRANSA CTIONS ON INFORMA TION THEOR Y , MA Y 2008 13 factor of more than 1/2. Our proofs require a greater expansion factor . B. Expansion and Err or Corr ection Capabili ty W e no w prove that the above described algorithm can correct a fraction of errors if the underlying T anner graph is a good expander . Theor em 7: Let C ( G, S ) be a GLDPC code wit h a γ -left re gular T anner g raph G . Assume that t he sub-code S has minimum distance at least d min = 2 t + 1 and is capable of correcting t errors. Let G be a ( γ , ρ, α, β γ ) expander where 1 > β > t + 2 2( t + 1) . Then the parallel bit flipp ing decoding algori thm will correct any α 0 ≤ α fraction of errors. Pr oof: Let n be the numb er of variable nod es in C . Let V b e the set of corrupt variables at the beginning of a decoding round. A ssume t hat | V | ≤ αn . W e will sh ow t hat after t he decoding round, th e number of corrupt variables is strictly less than | V | . Let F be t he set of corrupt variables that fail to flip in o ne decoding round, and let C be the set of var iables that were originally uncorrupt, but which become corrupt after one decodin g round. After one decoding round , t he set of corrupt v ariables is F ∪ C . In the worst case scenario, a confused check s ends t flip messages to the uncorrupt v ariables and no flip message to t he corrupt v ariables. W e now ha ve the following lemma: Lemma 4: Let C k be the set of confus ed checks, then | C k | < (1 − β ) γ | V | t . (1) Pr oof: T he total number of edges conn ected to the corrupt variables is γ | V | . Each confused check must have at least t + 1 neigh bors in V . Let S be th e set of helpful checks that hav e at least one neighbor in V . Then, γ | V | ≥ | C k | ( t + 1) + | S | . (2) By expansion, | S | + | C k | > β γ | V | . (3) By (2) and (3), we obtain | C k | < (1 − β ) γ | V | t . SUBMITTED TO IE EE TRANSA CTIONS ON INFORMA TION THEOR Y , MA Y 2008 14 W e no w pro ve t hat | F ∪ C | < | V | . The proof is by con tradiction. Assum e that | F ∪ C | ≥ | V | . Then there exists a su bset C ′ ⊂ C such that | F ∪ C ′ | = | V | . W e observe that a v ariable node i n F can hav e at most ⌊ γ / 2 ⌋ neighbors that are not in C k . Also, a variable node i n C ′ must hav e at least ⌊ γ / 2 ⌋ + 1 neighbors in C k , and hence can have at m ost ⌈ γ / 2 ⌉ − 1 neighbors that are not in C k . Let N ( F ∪ C ′ ) be the set of neighbors of F ∪ C ′ . Then, N ( F ∪ C ′ ) ≤ | C k | + ⌊ γ 2 ⌋| F | +  ⌈ γ 2 ⌉ − 1  | C ′ | < | C k | + γ 2 | F | + γ 2 | C ′ | = | C k | + γ 2 | V | . (4) Substitutin g (1) into (4), we obtain N ( F ∪ C ′ ) <  1 − β t + 1 2  γ | V | . Now β > t + 2 2( t + 1) = > 1 − β t < 2 β − 1 2 = > 1 − β t + 1 2 < β = > N ( F ∪ C ′ ) < β γ | V | which is a contradictio n. Remark: The abov e theorem proves that th e parallel bit flipping alg orithm can correct a frac tion of errors in linear num ber of roun ds (in code length). Howe ver , if we assume an expansion of ( β + ǫ ) γ , it can be shown that the number of errors decreases by a constant factor wi th e very it eration resulting in con ver gence in logarithmi c number of rounds. The following theorem establis hes a lo wer b ound on th e number of nodes in a left regular graph which expand by a factor required by the above algorithms. Theor em 8: Let G be a γ -left regular bipartit e graph with g ( G ) = 2 g ′ . Then for all k < n 0 ( γ t/ ( t +1) , g ′ ) , any set of k variable nodes in G expands by a factor of at l east β γ , where β = t + 2 2( t + 1) . Pr oof: The p roof is similar to t he p roof of Theorem 3. Following the not ation from Theorem 3, we note that for all k < n 0 ( γ t/ ( t + 1) , g ′ ) , f ( k , g ′ ) < k γ t 2( t + 1) . Since | C k | ≥ γ k − f ( k , g ′ ) , we hav e | C k | > t + 2 2( t + 1) γ k . SUBMITTED TO IE EE TRANSA CTIONS ON INFORMA TION THEOR Y , MA Y 2008 15 Note that the above theorem holds when γ t/ ( t + 1) ≥ 2 . Cor ollary 2: L et C ( G, S ) be a GLDPC code with a γ -left regular T anner graph G and g ( G ) = 2 g ′ . Assume that th e sub-code S has minim um distance at least d min = 2 t + 1 and is capable of correcting t errors. Then the parallel bit flipping algorit hm can correct any error patt ern of weight less than n 0 ( γ t/ ( t + 1) , g ′ ) . C. T rapping Sets of GLDPC Codes W e now exhibit a trapping set for th e parallel bit flippi ng algorithm. By examining the expansion of the trappin g set, we show that the bound given i n Theorem 7 cannot be improved when γ is even. Theor em 9: Let C be a GLDPC code with γ -left regular T anner graph G . Let T be a set consist ing of V variable nodes with induced s ubgraph I with the following prop erties: (a) The degree of each check in I is either 1 or t + 1 ; (b) Each variable node in V is connected to ⌈ γ / 2 ⌉ checks of degree t + 1 and ⌊ γ / 2 ⌋ checks of degree 1 ; and (c) No ⌊ γ / 2 ⌋ + 1 checks of degree t + 1 share a v ariable node ou tside I . Then, T is a trapping s et. Pr oof: Observe t hat all the checks of degree t + 1 in I are confused. Further , each confused check does not send flip m essages to var iable nodes in V . Since any var iable node in V is connected to ⌈ γ / 2 ⌉ confused checks, it remains corrupt. Al so, no variable node outside I can recei ve more than ⌊ γ / 2 ⌋ flip messages. Hence, no variable node which is orig inally correct can get corrupted. By definition, T is a trapping set. It can be seen that the total numb er of checks in I is equal to | V | ( ⌊ γ / 2 ⌋ + ⌈ γ / 2 ⌉ / ( t + 1)) . Hence, the set of va riable nodes V expands by a factor of γ ( t + 2) / (2( t + 1)) when γ is e ven. Hence, the bound giv en in Theorem 7 cannot be improved in this case. For a set of variable nodes to be a t rapping set, it is necessary th at every variable node in th e set is connected to at least ⌈ γ / 2 ⌉ confused checks. This observation l eads to the following bo und on t he size of trapping sets. Theor em 10: Let C b e a GLDPC code with γ -left regular T anner graph G and g ( G ) = 2 g ′ . Let n c ( d l , d r , 2 g ′ ) denote the n umber of left vertices i n a ( d l , d r ) regular bipartite graph of girth 2 g ′ . Then t he size of the smallest p ossible trapping set of C is n c ( ⌈ γ / 2 ⌉ , t + 1 , 2 g ′ ) . Pr oof: Follows from Theorem 5 and Theorem 9 Cor ollary 3: L et C ( G, S ) be a GLDPC code with a γ -left regular T anner graph G and g ( G ) = 2 g ′ . Assume that th e sub-code S has minim um distance at least d min = 2 t + 1 and is capable of correcting t errors. Then the parallel bit flipping algorit hm cannot be g uaranteed t o correct all error patterns of weight greater than o r equal to n c ( ⌈ γ / 2 ⌉ , t + 1 , 2 g ′ ) . SUBMITTED TO IE EE TRANSA CTIONS ON INFORMA TION THEOR Y , MA Y 2008 16 V I . C O N C L U D I N G R E M A R K S W e deriv ed lower bounds on the guaranteed error correction capability o f LDPC and GLDPC codes by finding bounds on the num ber of nodes that ha ve the required expansion. The b ounds depend on two important code parameters namely: colum n-weight and girth. Since the relations between rate, colum n- weight, girth and code leng th are well explored in the li terature (see [1], [5] for example), bounds on the code lengt h needed to achieve certain error correction capabilit y can be derived for dif ferent column weights and sub-codes (for GLDPC codes). The bounds p resented i n the paper serv e as guidelines i n choosing code parameters in practical scenarios. The lower bound s derived in this paper are weak. Howe ver , extremal graphs av oiding th ree, four and fi ve cycles hav e b een studied in great detail (see [27], [28]) and these results can be used to derive tighter bounds when the g irth is eight, ten or twelve. Als o, since an expansion factor of 3 γ / 4 is not necessary (see [7, Th eorem 24]) for LDPC codes, it is pos sible t hat tig hter lower bounds can be derived for som e cases. The resul ts can be extended to message passing algorithm s as well. There is a considerable g ap between the lower bounds and upper bounds on the error correction capability . Deriving lower bounds based on th e sizes of trapping sets rather th an expansion may possibl y lead to bridgin g this gap. Our approach can be u sed to deriv e bounds on the guaranteed erasure recover y capability for iterative decoding on the BEC by finding t he number of variable nodes which expand by a factor of γ / 2 . In [4], the boun ds on the guaranteed erasure recove ry capabil ity were derived based on the size of t he smallest stopping set. Both approaches give the same b ounds, w hich also coi ncide wit h t he bo unds g iv en by T anner [5] for t he mini mum distance. Results si milar to the ones reported by Miladinovic and Fossorier [15] b ased on the si ze of g eneralized stoppin g sets can also be derived. R E F E R E N C E S [1] R. G. Gallager , Low Density P arity Check Codes . Cambridge, MA: M.I. T . Press, 1963. [2] T . J. Richardson and R. Urbank e, “The cap acity of low-density parity-check codes under message-pas sing decoding, ” IEEE T r ans. Inform. T heory , vol. 47, no. 2, pp. 599–618, F eb . 2001. [3] T . J. Richardson, M. Shokrollahi, and R . Urbanke, “Design of capacity-approaching irregular lo w-density parity-check codes, ” IEEE T ran s. Inform. Theory , vol. 47, no. 2, pp. 638–656 , Feb. 2001. [4] A. Orlitsky , R. Ur bank e, K. V iswan athan, and J. Zhang, “Stopping sets and the girt h of Tanner graphs, ” in Pr oc. of IEEE International Symposium on Information Theory , 2002, p. 2. [5] R. M. T anner, “ A recursiv e approach to low complexity codes, ” IE EE T rans. Inform. Theory , vol. 27, no. 5, pp. 533–547 , Sept. 1981. [6] V . V . Zyablo v and M . S. Pinsker , “Estimation of the error -correction comple xity for Gallager lo w-density code s, ” Pr oblems of Informa tion T ran smission , vol. 11, no. 1, pp. 18–28, 1976. [7] M. Si pser and D. Spielman, “E xpander codes, ” IEEE T rans. I nform. Theory , vo l. 42, no. 6, pp. 1710–17 22, Nov . 1996. [8] D. B urshtein and G. Miller, “Expander graph argumen ts for message-passing algorithms, ” IEE E T rans. Inform. Theory , vol. 47, no. 2, pp. 782–790, Feb . 2001. SUBMITTED TO IE EE TRANSA CTIONS ON INFORMA TION THEOR Y , MA Y 2008 17 [9] J. Feldman, T . Malkin, R. A. Serv edio, C. S tein, and M. J. W ainwright, “LP decoding corrects a constant fraction of errors, ” I EEE T ran s. Inform. Theory , vol. 53, no. 1, pp. 82–89, Jan. 2007. [10] J. Feldman, M. J. W ainwright, and D. R. Karger , “Using li near programming to decode binary linear codes, ” IEEE T rans. Inform. Theory , vol. 51, no. 3, pp. 954–972, March 2005. [11] D. Burshtein, “On the error correction of regular LDPC codes using the flipping algorithm, ” i n Pr oc. of IEEE International Symposium on Information T heory , June 2007, pp. 226–230. [12] G. Zemor , “On expan der codes, ” IEEE T ra ns. Inform. Theory , vol. 47, no. 2, pp. 835–837, F eb . 2001. [13] A. Barg and G. Zemor , “Error exponents of exp ander codes, ” IEEE Tr ans. Inform. Theory , vol. 48, no. 6, pp. 1725–1729 , Jun. 2002. [14] H. Janwa and A. K . Lal, “On Tanner codes: minimum distance and decoding, ” Appl. Algebr a Eng. Commun. Comput. , vol. 13, no. 5, pp. 335–347, 2003. [15] N. Miladino vic and M. Fossorier , “Generalized LDPC codes with Reed-Solomon and BCH codes as component codes for binary channels, ” vol. 3, 28 Nov . -2 Dec. 2005, pp. 6–10. [16] N. Biggs, Algebraic graph theory . Cambridge : Cambridge Univ ersity Press, 1993. [17] T . J. Richardson, “Error floors of L DPC codes, ” in Proc. of 41st Annual Allerton Conf. on Communications, Contr ol and Computing , 2003, pp. 1426– 1435. [18] N. Alon, “Spectral techniques in graph algorithms, ” in LATIN ’98: Pro ceedings of the Third Latin American Symposium on Theor etical Informatics . Lond on, UK: Springer-V erlag, 1998, pp. 206–215. [19] P . O. V ontobel and R. K oetter , “Graph-cov er decoding and finite-length analysis of message-passing iterativ e decoding of LDPC codes, ” May 2007, accepted for IEE E Transaction s on Information Theory . [ Online]. A vailable: http://www .citebase.or g/abstract?id=oai:arXi v . org:cs/051 2078 [20] S. K. Chilappagari, S. Sankaranarayanan, and B. V asic, “Error floors of LDPC codes on the binary symmetric channel, ” i n Proc . of IEEE International Confer ence on Communications , vol. 3, June 11-15 2006, pp. 1089–1 094. [21] E. W . W eisstein, “Cage graph. ” [Online]. A v ailable: http://mathworld.wolfram.com/CageGraph.h tml [22] B. Bollobas, Extremal grap h theory . Londo n: Academic Press I nc., 1978. [23] A. Shokrollahi, “ An introduction to lo w-density parity-check codes, ” in Theor etical aspects of computer science: advanced lectures . Ne w Y ork, NY , USA: Springer-V erlag New Y ork, Inc., 2002, pp. 175–19 7. [24] S. K. Chilappagari and B. V asic, “Error correction capability of column-weight-three LDPC codes, ” submitted to IEEE Tran s. Inform. Theory . [Online]. A v ailable: http://arxiv .org/abs/0710.34 27 [25] S. Sankaranarayanan, S . K. Chilappagari, R. Radhakrishn an, and B. V asic, “Failures of the Gallager B decoder: analysis and applications, ” in Proc. of UCSD Center for Information Theory and its Applications Inaugu ral W orkshop , Feb 6-9 2006. [Online]. A vailable: htpp//ita.5i.net/papers/160.pdf [26] N. Al on, S. Hoory , and M. Li nial, “The Mo ore bound f or i rregular graphs, ” Graph s and C ombinatorics , vol. 18, no. 1, pp. 53–57, 2002. [27] D. K. Garnick, Y . H. H. Kwong, and F . Lazebnik, “Extremal graphs without three-cycles or four-cy cles, ” J . Graph Theory , vo l. 17, no. 5, pp. 633–645, 1993. [28] Y . Y uansheng , L. Xiaohui, D. Guocheng , and Z. Y ongxiang, “Extremal graphs without three-cy cles, f our-c ycles or fiv e-cyc les, ” Util itas Mathematica , vol. 66, pp. 249–26 6, 2004.