A Separation Algorithm for Improved LP-Decoding of Linear Block Codes

SUBMITTED TO IEEE TRANSA CTIONS ON INF ORMA TION THEOR Y 1 A Separation Algorithm for Improv ed LP-Decoding of Linear Block Co des Akin T anatmis, Stefan Ruz ika, Horst W . Ha macher , Mayur Punekar , Frank Kienle and Norbert W ehn Abstract —Maximum Likelihood (ML) decoding is the optimal decoding algorithm f or arbitrary linear block co des and can be written as an Integer Pr ogramming ( IP) p roblem. Feldman et al. relaxed this IP problem and p resented Linear Program ming (LP) based decoding algorithm for linear block codes. In this paper , we propose a n ew IP formulation of the ML decoding problem and solve the IP with generic methods. The formulation uses indicator variables to detect violated parity ch ecks. W e derive Gomory cuts from our formulation and use them in a separation algorithm to find ML codewords. W e furth er p ropose an efficient method of finding cuts in duced by redundant parity checks ( RPC). Under certain circumstances we can g uarantee that these RPC cuts are valid and cut o ff the fractional optimal solutions of LP decoding. W e demonstrate on two L DPC codes and one B CH code that our separation algorithm perf orms significantly better than L P decoding. Index T erms —ML decodin g, LP d ecoding, Integer progra m- ming, Sep aration algorithm. I . I N T RO D U C T I O N L O W -DE NSITY P ARITY -CHECK ( LDPC) codes have attracted significan t interest in the research co mmunity in the last dec ade. LDPC code s ar e generally dec oded b y Belief Propagatio n (BP) (or Sum -Produ ct) algorithm. BP exploits the sparse structur e of the parity check matrix of LDPC codes very well and achieves g ood per forman ce. Howe ver , due to the heuristic nature of BP alg orithm, it is not possible to guaran tee the perf ormance o f BP decode rs at very low er ror rates. Moreover , the perfo rmance o f BP is very poor for arbitrary linear b lock c odes with dense pa rity ch eck matrices (which mea ns that the correspo nding T anner gr aph contains short cycles). ML deco ding of linear blo ck codes can b e modele d as an IP problem. Ho wever , since the ML decodin g is NP-hard [1], solving this IP problem is computatio nally feasible on ly for small instan ces. Nevertheless co nsidering ML deco ding as an IP p roblem yields a new approa ch to derive sub-optimal algorithm s. These algor ithms offer some ad vantages compared to BP decodin g. First, these appr oaches rely on a well-stud ied A. T anatmis, S. Ruz ika and H. W . Hamacher are with Department of Mathemat ics, Uni versit y of Kaisersla utern, Erwin-Schroe dinger - Strasse, 67663 Kaiserslautern, Germany . Email: { tanatmis, ruzika, hamacher } @mathemati k.uni-kl.de M. Punekar , F . Kien le a nd N. W ehn are with Microelectron ic Sys- tems Design Research Group, Uni versity of Kaiserslaut ern, Erwin- Schroedi nger-Stra sse, 67663 Kaiserslaut ern, Ger many . Email: { punekar , kienle , wehn } @eit.uni -kl.de This paper has been presented in part at the 5th Internat ional Symposium on Tu rbo Codes and Relate d T opics, September 1st - 5th, 2008, Lausanne, Switzerl and. Manuscript recei ved December 02, 2008; revised ; mathematical theory wh ich enables qu antitativ e statements (e.g. co n vergence, complexity , correctness, etc.) with r egard to the decodin g pr ocess and its result [8], [10], [1 3]. Secondly , they are not limited to sparse matrices. In [10] Feldman et al. propo sed a new algo rithm based on LP to decode binary linear codes. This LP deco ding alg orithm utilizes a set o f constra ints which co ntains all valid cod ew ords of a given code and a lin ear objective f unction. Min imizing this objective function over the resulting poly tope yields the ML codeword if the optimal solution is integral (known as ML certificate pr operty [10]). If th e op timal solution is not integral then LP decode r outpu ts an error . Recently , L P deco ding has b een improved towards lower complexity ([ 2], [5], [ 13], [14], [18], [ 19] ) and better perf o- mance ([3], [ 4], [8], [9]). Analysis of error correctio n perfo r- mance of LP decoding ([7], [11], [16]) an d the relation ship to iterative message p assing algorith ms ([ 10], [15], [ 17]) have also been studied in the literatur e. In th is pape r , we co ncentrate on impr oving linear pr ogram- ming de coding u sing a separa tion algor ithm. W e intro duce an alternative IP formu lation for the decoding prob lem. Instead of solving the o ptimization p roblem, we attempt to find the ML solution by an iterative separation app roach: First, we relax the IP fo rmulation and solve the resulting line ar prog ram. In case of a non-integral optimal solution, we deri ve inequalities which cut of f this non-integral solution , add these inequ alities to the LP f ormulation and r esolve the LP prob lem. This process continues until an op timal integer solution is fo und or further cuts cannot b e generated . It sho uld b e no ted tha t this g eneral integer programm ing ap proach known as separ ation problem has first been applied to LP decoding by T aghavi an d Siegel [13]. Ou r appr oach offers howe ver th e following advantages which remarkab ly facilitate LP based decod ing. 1) The nu mber of constra ints in the n ew IP formu lation is the same as the numbe r of rows in th e parity check matrix. Each p arity check equatio n which is orig inally in GF (2) is co n verted into a linear constraint in R n by means o f an a uxiliary variable. 2) The auxiliary variables serve as indica tors which can b e used for identifyin g violated par ity ch eck constra ints. W e can prove that we d etect violated inequalities faster than the adap ti ve algorith m o f T aghavi and Siegel und er some m ild assump tions. 3) W e forma lly show that the Forbidden Set Inequa lities [8] are a subset o f the set o f Gom ory cuts (see [12]) which can b e ded uced fr om o ur for mulation. 4) W e provid e emp irical evidence that ou r new separatio n SUBMITTED TO IE EE TRANSA CTIONS ON INFORMA TION THEOR Y 2 algorithm perfo rms better than LP de coding. This is mainly due to g enerating strong cuts efficiently using alternative repr esentations of the code s at hand . T o provid e emp irical evidence we ap plied the N E W S E PA R A - T I O N A L G O R I T H M to decode two LDPC c odes alo ng with on e BCH code. The rest of this p aper is organ ized as follows. W e intro duce notation in Section I I and briefly revie w relev ant litera ture in Section III. In Section IV , we introdu ce th e new I P formu lation, its LP relaxatio n, and the N E W S E PA R AT I O N A L G O R I T H M . In Section V we pre sent our numer ical results and com pare them with BP , LP d ecoding , and the lower b ound resulting fr om ML decoding. The paper is concluded with some r emarks and further r esearch ideas in Sectio n VI. I I . N OTA T I O N A N D B AC K G RO U N D A binary linear block cod e with cardinality 2 k and block length n is a k dimensional subspace of the vector space { 0 , 1 } n defined over the field GF (2) . The linear c ode C is giv en by k basis vectors o f length n which are re presented by a k × n matrix G (generator matrix). Eq uiv a lently C can be described by a pa rity check matrix H ∈ { 0 , 1 } m × n where m = n − k .W e thus have x ∈ C , i.e. x is a codew ord, if an d only if H x = 0 in GF (2) . W e de note the i th row and j th column of H by H i,. , H .,j respectively . H i,. x = 0 in GF (2) is defined as the i th parity check co nstraint. The index set I = { 1 , . . . , m } r efer to the rows and the in dex set J = { 1 , . . . , n } refer to the column s of H . The m atrix H is often re presented by a T ann er grap h G = ( V , E ) . Th e no de set V of G consists of the two d isjoint node sets indexed by I and J c alled the check nodes an d variable nod es respectively . An edge [ i, j ] ∈ E conne cts n ode i an d j if an d only if H ij = 1 . The ML d ecoding pr oblem for any bin ary code C ∈ { 0 , 1 } n can b e written in terms of the mathem atical progr am min { c T x : x ∈ C } = min { c T x : x ∈ conv ( C ) } . (1) Here, c ∈ R n is the cost vecto r obtained by th e log- likelihood r atios c i = log  P ( ˆ x i | x i =0) P ( ˆ x i | x i =1)  for a g iv en rec eiv ed bit ˆ x i and con v ( C ) den otes the conv ex hu ll of C i.e . the codeword polytop e. Th e left hand side of th e equ ation (1 ) is an integer progr amming problem which is kn own to be NP-hard [1]. Replacing C with co n v ( C ) leads to a linear progr amming pr oblem which is stated on the right hand side o f (1). Although line ar p rogram ming is poly nomially solvable in gen eral, computin g conv ( C ) is intractab le. In o ther words a con cise description of conv ( C ) by mean s of linear inequalities increases exp onentially in the block length n . Thus ML deco ding remains a challen ging task. Ne vertheless, linear programmin g decoding ca n b e applied efficiently if good approx imations of the c odeword p olytope can be fo und. Recently attempts in this direction have been made , ( e.g.[5], [10], [ 13], [ 14], [1 9]). Feldman et al. [1 0] introduced the LP decoder which minimizes c T x over a relaxation of the codeword po lytope. The relaxation is achieved by using the parity check m atrix H . Each row (check node) i ∈ I defin es a lo cal cod e C i , i.e. local cod ew ords x ∈ C i are the bit sequences which satisfy the i th parity check constra int. Note that C = C 1 ∩ . . . ∩ C m . Lemma 2 .1 ([1 4]): Let P = co n v ( C 1 ) ∩ . . . ∩ conv ( C m ) . If C = C 1 ∩ . . . ∩ C m then conv ( C ) ⊆ P . P is generally referred to as the f undamen tal polyto pe ([8], [13], [15]). This relax ation has th e advantage th at the complexity of describin g the co n vex hu ll o f any local cod e conv ( C i ) an d thu s of P is much less than th e co mplexity o f describing th e cod ew o rd poly tope C . Th e LP decoder solves the p roblem min { c T x : x ∈ P } . Sev eral app roaches are used in [5], [10], [13], [14] [ 19] to write con straints comp letely d escribing P . W e a re going to use the set of constraints already intro duced in [1 0] and referred to as Forbid den Set In equalities in [8]. The in dex set of variable nodes which are adjacen t to check node i is defined as N i := { j ∈ J : H ij = 1 } . U sing S ⊆ N i we assign values to code b its x j as follows. Set x j = 1 for all j ∈ S , an d x j = 0 f or all j ∈ N i \ S . For j / ∈ N i , x j can be cho sen arbitrarily . T hese value assignments to variables are feasible, i.e. satisfy th e parity ch eck constrain t, f or the lo cal code C i if | S | is even. If | S | is odd, they are, h owe ver, in feasible or forbid den. From this observation the so called Forbidden Set Inequa lities are der i ved. Let Σ i = { S ⊆ N i : | S | odd } . It is shown in [ 10] that co n v ( C i ) can be described by X j ∈ N i \ S x j + X j ∈ S (1 − x j ) ≥ 1 ∀ S ∈ Σ i (2) which can eq uiv ale ntly b e written as X j ∈ S x j − X j ∈ N i \ S x j ≤ | S | − 1 ∀ S ∈ Σ i . (3) Consequently the L P d ecoder solves min c T x (LPD) s.t. X j ∈ S x j − X j ∈ N i \ S x j ≤ | S | − 1 ∀ S ∈ Σ i , i = 1 , . . . , m 0 ≤ x ≤ 1 . If LPD has an in tegral optimal solution th en the LP decoder outputs the ML codeword. If LPD has a non-in tegral optimal solution the n the LP dec oder ou tputs an error . The numb er of Forbidden Set Ineq ualities induce d b y check node i is 2 δ ( i ) − 1 where δ ( i ) = P n j =1 H ij is the ch eck node degree, i.e. the number of ed ges incid ent to no de i . The LP decod er c an thus be applied successfully to low density codes. As the check node degrees increase the c omputatio nal load o f building an d solving the LP mod el is howev er in genera l prohib iti vely large. This makes th e explicit description of the fundamen - tal polytop e via Forbidden Set I nequalities ina pplicable fo r high density codes. T o overcom e this dif ficulty an alternative formu lation which req uires O ( n 3 ) constraints is prop osed in [10]. More recent for mulations o f [5] an d [19] have size linear in th e leng th an d check n ode degrees. Another appro ach applicable to hig h density codes is to solve the c orrespon ding SUBMITTED TO IE EE TRANSA CTIONS ON INFORMA TION THEOR Y 3 separation problem of LPD [13]. The separa tion pro blem over an imp licitly given po lyhedro n is defined as follows: Definition 2 .2: Given a bou nded rational polyhedr on P ⊂ R n and a rationa l vector x ∗ ∈ R n , either conclu de that x ∗ ∈ P or , if not, find a rationa l vector (Π , Π 0 ) ∈ R n × R suc h that Π T x ≤ Π 0 and Π T x < Π T x ∗ for all x ∈ P . In the latter c ase (Π , Π 0 ) is called a valid cut. In sep aration algorithm s (see [12]) one iteratively computes families Λ of valid cuts un til n o f urther cuts can be fo und. In the separation algor ithm o f [1 3], which is called ad aptive LP decodin g b y the au thors, Forb idden Set Inequalities are not added all at o nce in the beginn ing as in [ 10] but iteratively . In o ther word s, the separation problem f or th e fun damental polytop e is solved by searching violated Forbidden Set In- equalities. In the initialization step o f the LP min { c T x : 0 ≤ x ≤ 1 } is computed . An optimal solution x ∗ is ch ecked in O ( mδ max + n log n ) time, if x ∗ violates any f orbidd en set inequality where δ max is the maximum ch eck nod e d egree. If some of the Forbidden Set Ineq ualities are v iolated then these ineq ualities are added to th e f ormulatio n and the LP is resolved includ ing the new ineq ualities. Adaptive LP deco ding stops when the cur rent op timal solution x ∗ satisfies all Forbidden Set Ineq ualities. If x ∗ is integral then it is the ML codeword otherwise an error is output. Note that putting the L P d ecoder in an ad aptive setting does not yield an improvement in term s of fram e er ror rate since the same solutions are found . On the othe r ha nd the adaptive LP decoder co n verges with less c onstraints th an the LP decoder which has a po siti ve effect on comp utation time. The com munication perfo rmance of LP d ecoding motiv ated researchers to find better appro ximations of the codew o rd polytop e as p art o f M L decoding. One way is to tig hten the f undamen tal polyto pe with new valid inequalities. A mong some other gener ic tec hniques of cut gene ration, add ing so called RPC cuts is pro posed in [10]. Redundan t parity ch ecks are obtained by addin g a subset of ro ws of H matrix in GF (2) . These check s a re redun dant in the sen se that they do n ot alter the co de ( they may even degrade the perfo rmance of BP [ 10]). Howe ver they induce new con straints in the LP form ulation which m ay cut off a p articular no n-integral o ptimal solu tion thus tig htening the fundam ental p olytope. An open problem is to find metho ds to gen erate r edund ant parity checks efficiently such that the in duced constraints are guara nteed to cu t off a non-in tegral LP solution . T o th e be st of our knowledge two app roaches for generatin g potential cuts exist so far . First, ad ding redu ndant parity check cuts which result f rom adding any two ro ws of H [ 10]. Secondly , the app roach in [1 3] which makes use o f the cycles in the T an ner graph: 1 ) given a n on-integral o ptimal solution x ∗ remove a ll variable node s j form the T ann er g raph f or which x ∗ j is in tegral; 2) find a cycle by ran domly walking throug h the pruned T ann er graph ; 3) a dd the rows of the H matrix in GF (2) which corr espond to the check nodes in the cycle; 4) check if the found RPC introdu ces a cut. I I I . A N E W S E PA R AT I O N A L G O R I T H M B A S E D O N A N A LT E R N AT I V E I P F O R M U L A T I O N Our sep aration algorithm is based on the fo llowing fo rmu- lation wh ich we r efer to as I nteger Progra mming Decodin g ( I P D ) . min c T x (IPD) s.t. H x − 2 z = 0 x ∈ { 0 , 1 } n z ≥ 0 , integer IPD is an in teger pro grammin g problem wh ich works as an ML decod er . The aux iliary variable z ∈ Z m ensures the b inary constraint H x = 0 over GF (2) turns into a c onstraint over the re al num ber field R wh ich is much easier to hand le. Th is formu lation has the additional advantage that the n umber o f constraints is the same as the num ber o f rows of the parity check matrix. Note that LPD can also be used as an ML decoder by restricting x to be in { 0 , 1 } n . Y et in th is case the number o f con straints is expo nential in the check no de degree. Although our formulation IPD has less constraints, th is does not change the fact that ML decoding is NP-hard. Therefore our appro ach is to solve the separ ation problem by itera ti vely adding new cuts Π T x ≤ Π 0 accordin g to Definition 2.2 and solving the L P r elaxation o f IPD given by min c T x (RIPD) s.t. H x − 2 z = 0 Π T x ≤ Π 0 (Π , Π 0 ) ∈ Λ 0 ≤ x ≤ 1 z ≥ 0 . Note that in the initialization step ther e are no cuts of type Π T x ≤ Π 0 i.e. Λ = ∅ . If RIPD has an integral solution ( x ∗ , z ∗ ) ∈ Z n + m then x ∗ is the ML co deword. Otherwise we generate cuts of the typ e Π T x ≤ Π 0 in order to exclud e the non-in tegral solu tion fo und in the cur rent iteration. W e add these inequalities to the formu lation and so lve RIPD again. In a no n-integral solution of RIPD x or z (or both) is non- integral. If x ∈ Z n and z ∈ R m \ Z m then we add Gomo ry cuts (see [1 2]) which is a g eneric cut g eneration tech nique used in integer pr ogramm ing. Su rprisingly , in this case Gomor y cuts can b e shown to co rrespon d to Forbidden Set Inequa lities. Theor em 3 .1: Let ( x ∗ , z ∗ ) ∈ Z n × R m be the optimal solution of RIPD such that z ∗ i ∈ R \ Z for i ∈ I . Then the Gomory cut whic h is violated by ( x ∗ , z ∗ ) is the Forb idden Set Inequa lity X j ∈ S x j − X j ∈ N i \ S x j ≤ | S | − 1 (4) where S :=  j ∈ N i | x ∗ j = 1  . Proof: W e apply the general method k nown as Go mory’ s cutting plane algo rithm (see e.g. [1 2]) to our spec ial case. Gomo ry cuts ar e derived from the rows o f th e simplex tableau in order to cut off non -integral L P solutions and fin d the optimal SUBMITTED TO IE EE TRANSA CTIONS ON INFORMA TION THEOR Y 4 solution to the integer linear p rogram ming problems. Consider RIPD a t any step: min c T x (RIPD) s.t. H x − 2 z = 0 0 ≤ x ≤ 1 Ax ≤ b z ≥ 0 where c, x ∈ R n , H ∈ { 0 , 1 } m × n , z ∈ R m , A ∈ {− 1 , 0 , 1 } λ × n for some λ ∈ N 0 and b ∈ N λ 0 . Note that λ is th e number of constraints adde d iteratively u ntil the cu rrent step, i.e. λ = | Λ | . The λ × n m atrix A is the coefficient matrix of the iterativ ely ad ded co nstraints, i.e. Π T x ≤ Π 0 (Π , Π 0 ) ∈ Λ . W e denote the rig ht hand sides of these con straints with the vector b . RIPD in standar d form can be written as follows: min c T x (RIPD) (5) s.t. z − ¯ H x = 0 (6) x + s 1 = 1 (7) Ax + s 2 = b (8) z ≥ 0 , x ≥ 0 , s ≥ 0 . (9) where ¯ H := 1 2 H , s = ( s 1 , s 2 ) ∈ R n + λ . For ease of notatio n we rewrite ( 5)-(9) as min ¯ c T y (10) s.t. P y = q (11) y ≥ 0 . (12) Note th at ¯ c T = ( ¯ c 1 , . . . , ¯ c m , ¯ c m +1 , . . . , ¯ c m + n , ¯ c m + n +1 , . . . , ¯ c m +2 n + λ ) = (0 , . . . , 0 , c 1 , . . . , c n , 0 , . . . , 0) , y T = ( y 1 , . . . , y m , y m +1 , . . . , y m + n , y m + n +1 , . . . , y m +2 n + λ ) = ( z 1 , . . . , z m , x 1 , . . . , x n , s 1 , . . . , s n + λ ) and q T = ( q 1 , . . . , q m , q m +1 , . . . , q m + n , q m + n +1 , . . . , q m +2 n + λ ) = (0 , . . . , 0 , 1 , . . . , 1 , b 1 , . . . , b λ ) . The constraint m atrix P has m + n + λ rows and m + 2 n + λ columns. W e d enote th e α th row of P with P α where α ∈ { 1 , . . . , m + n + λ } and β th column of P with P β where β ∈ { 1 , . . . , m + 2 n + λ } . The comp onent in row α and column β is deno ted with P αβ . Add itionally , we define the α th unit vecto r as e α ∈ R m + n + λ . Thus, we rewrite P as P =  e 1 . . . e m P m +1 . . . P m + n e m + n +1 . . . e m +2 n + λ  . The first m colu mns of the co nstraint matrix P are the un it vectors corr espondin g to the variables { z 1 . . . z m } . Likewise, the last n + λ co lumns a re the unit vectors correspon ding to the slack variables { s 1 . . . s n + λ } . The fir st m linea r equ ations o f P y = q are of the f orm: z i − 1 2 · X j ∈ N i x j = 0 for all i ∈ { 1 , . . . , m } . Let y ∗ = ( z ∗ , x ∗ , s ∗ ) ∈ R m +2 n + λ be the optimal solution to (5)-(9). By assumption it is x ∗ ∈ { 0 , 1 } n . For i ∈ { 1 , . . . , m } , z i is given by z ∗ i = 1 2 k i , where k i =   { j ∈ N i | x ∗ j = 1 }   . It is obvio us that k i ∈ N 0 . If k i is even i.e. an ev en nu mber of variable n odes are set to 1 in th e neigh borho od of the check node i , then z ∗ i ∈ N 0 holds. Otherwise, z i is an odd mu ltiple of 1 2 . W e th en co nsider the Go mory cu t f or this r ow i . For the op timal solu tion y ∗ we can partition P into a basis sub matrix P B and a non-b asis submatrix P N , i.e. P = [ P B P N ] . Let B and N deno te the ind ex sets of the columns of P belo nging to P B and P N , respectively . An ( m + n + λ ) × ( m + n + λ ) basis m atrix, P B , cor respond ing to the op timal solution y ∗ can be co nstructed a s follows. First we take the column s e 1 , . . . , e m which are the identity vectors correspo nding to the variables { z 1 . . . z m } into P B . Secon dly for j = 1 , . . . , n , we include the column P m + j if x ∗ j = 1 or P m + n + j if s ∗ j = 1 in P B . Th ere exists n such co lumns sin ce n X j =1 ( x ∗ j + s ∗ j ) = n must hold d ue to (7). Finally we take the colu mns e m +2 n +1 , . . . , e m +2 n + λ correspo nding to the slack variables which are written for the iterativ ely add ed constraints. T he variables co rrespond ing to the column s in th e ba sis ma trix ar e called b asic variables. Th e rem aining co lumns of P fo rm the non-b asis sub matrix P N . The column s of P N are th e column s P m + j , j = 1 , . . . , n , for which x ∗ j = 0 and the co lumns e m + n + j , j = 1 , . . . , n , for which s ∗ j = 0 . The variables correspo nding to th e co lumns in P N are c alled n on-ba sic variables. The Go mory cut fo r row i o f P is given b y th e ineq uality X h ∈ N ( ¯ p ih − ⌊ ¯ p ih ⌋ ) y h ≥ ( ¯ q i − ⌊ ¯ q i ⌋ ) (13) where ¯ p ih = ( P − 1 B ) i · ( P N ) h , an d ¯ q i = ( P − 1 B ) i · q . Note that in our case i ≤ m sinc e only z ∗ has n on-integral co mpone nts. In the fo llowing we in vestigate the structure of ( P − 1 B ) i , ( P N ) h , ¯ p ih and ¯ q i . For a fixed i , it can easily be verified that the e ntries ( P − 1 B ) il , l = 1 , . . . , m + n + λ of ( P − 1 B ) i are g i ven as ( P − 1 B ) il =        1 , if l = i 1 2 , if P il = 1 , x ∗ j = 1 , l = m + j, j = 1 , . . . , n 0 otherwise (This c an b e verified by observin g th e change s o n r ow i when we app end an ( m + n + λ ) × ( m + n + λ ) iden tity matrix to P B and perform the Gauss-Jordan elimination on the appended matrix in or der to get P − 1 B .) SUBMITTED TO IE EE TRANSA CTIONS ON INFORMA TION THEOR Y 5 Having foun d ( P − 1 B ) i , ¯ q i is then compu ted b y ¯ q i = ( P − 1 B ) i · q (14) = q i + 1 2 X j : x ∗ j =1 q m + j (15) = 0 + 1 2 X j : x ∗ j =1 1 . (16) Thus, we showed tha t ¯ q i is 1 2 times the number of basic x variables in row i . Since z i is not integer, the numb er o f basic x variables in row i is o dd. It fo llows that in our case the right hand side o f the Go mory cu t, ¯ q i − ⌊ ¯ q i ⌋ , is always 1 2 . Next, we compu te ¯ p ih = ( P − 1 B ) i · ( P N ) h . The columns of P N are the colum ns of P cor respond ing to non-basic x compon ents (i.e. x ∗ j = 0 ) an d non- basic s com ponen ts (i.e. s ∗ j = 0 ) j = 1 , . . . , n . If ( P N ) h = P m + j such that x ∗ j = 0 , then for a fixed value of h , the e ntries of ( P N ) h , ( P N ) oh , o = 1 , . . . , m + n + λ ar e giv en as ( P N ) oh =    − 1 2 , if P o ( m + j ) = 1 and o ≤ m 1 , if o = m + j 0 otherwise. If ( P N ) h = P m + j such that s ∗ j = 0 , th en ( P N ) h is the unit vector e m + j . For the case that ( P N ) h = P m + j where x ∗ j = 0 , the only position where b oth ( P − 1 B ) i and ( P N ) h may have nonzero entries is position i . For all other positions l = 1 , . . . , m + n + λ and l 6 = j eith er ( P − 1 B ) il = 0 or ( P N ) lh = 0 . This implies ¯ p ih = ( P − 1 B ) i ( P N ) h =  − 1 2 , if P ih = 1 0 , if P ih = 0 . For the case that ( P N ) h = P m + j where s ∗ j = 0 , position m + j is the o nly po sition where both ( P − 1 B ) i and ( P N ) h may have a nonze ro entry . This mea ns, ¯ p ih = ( P − 1 B ) i ( P N ) h = 1 2 for all non -basic s variables correspond ing to the basic x variables in row i . If we deno te th e n on-b asic x variables in row i with th e index set N i \ S := { j : x ∗ j = 0 } and the non-b asic s variables correspon ding to the basic x variables in ro w i with th e in dex set S := { j : s ∗ j = 0 } , we can write the Gomor y cu t as X h ∈ N ( ¯ p ih − ⌊ ¯ p ih ⌋ ) y h ≥ 1 2 ⇔ X j ∈ N i \ S  − 1 2 −  − 1 2  x j + X j ∈ S  1 2 −  1 2  s j ≥ 1 2 ⇔ X j ∈ N i \ S 1 2 x j + X j ∈ S 1 2 s j ≥ 1 2 ⇔ X j ∈ N i \ S x j + X j ∈ S (1 − x j ) ≥ 1 . (17 ) Since ineq uality (17) is the forbidd en set inequality o btained from the config uration S :=  j ∈ N i | x ∗ j = 1  this conclud es the proo f.  Giv en an op timal solution o f RIPD, ( x ∗ , z ∗ ) with x ∗ j ∈ { 0 , 1 } fo r all j ∈ J and z ∗ i ∈ R \ Z for at least one i ∈ I we can efficiently der i ve Go mory cuts with the fo llowing algorithm . C U T G E N E R A T I O N A L G O R I T H M 1 Input : ( x ∗ , z ∗ ) suc h that x ∗ integral, z ∗ non-in tegral. Output : Gom ory cu t(s). 1 : Set i = 1 . 2 : If k i = 2 z ∗ i is o dd g o to 3. Othe rwise go to 5. 3 : Set con figuration S :=  j ∈ N i | x ∗ j = 1  . 4 : Construct con straint (4). 5 : If i ≤ m , set i = i + 1 go to 2 . Otherwise term inate. This algorith m has a co mputation al complexity o f O ( mδ max ) because at most m values have to be ch ecked u ntil a v iolated parity check constraint is id entified and O ( δ max ) is the complexity of constructing (4). An alg orithm to ch eck if any forbid den set inequality is violated is also given in [13]. In order to find a v iolated forbid den set inequality , the algo rithm of T aghavi a nd Siegel first sor ts x . Next, at most δ max Forbidden Set I nequalities h av e to be generated an d validated. Repeating this p rocedur e fo r m check no des leads to an algorithm of time complexity O ( mδ max + n log n ) . In contra st, we can ef ficiently deter mine the violated parity che cks using the indicator variables z . Having id entified a violated p arity check co nstraint i (if ther e exists any) we construct ( 4 ) easily by setting the coefficient of x j for { j ∈ N i : x ∗ j = 1 } to +1 , the c oefficient of x j for { j ∈ N i : x ∗ j = 0 } to − 1 and | S | = k i . Next we consider the situatio n that 0 < x ∗ j < 1 for some j ∈ J . Althoug h it is still possible to d eriv e a Go mory cu t, C U T G E N E R A T I O N A L G O R I T H M 1 is not applicable since Theorem 3.1 h olds on ly for integral x ∗ . For no n-integral x ∗ we p ropose the f ollowing separation method in o rder to find valid cutting inequ alities, the C U T G E N E R AT I O N A L G O R I T H M 2 . The idea behind C U T G E N E R A T I O N A L G O R I T H M 2 is b ased on Pr oposition 3 .2 a nd Pr oposition 3 .3. Pr opo sition 3.2: The Forb idden Set Inequ alities der i ved from row i , i ∈ { 1 , . . . , m } , of a par ity check matrix H and the inequalities 0 ≤ x ≤ 1 , c ompletely describe the con vex hull co n v ( C i ) of the lo cal codeword poly tope C i . Proof: This is shown in Theor em 4 in [10].  Pr opo sition 3.3: Let x ∗ be a non-integral optimal solu tion of R IPD and x ∗ ∈ con v ( C i ) . Then there are a t least two indices j, k ∈ J su ch that 0 < x j < 1 an d 0 < x k < 1 . In other words check node i cannot be adjacent to only one non- integral valued variable no de. Proof: If x ∗ ∈ conv ( C i ) then it ca n b e written as a conve x combinatio n of two or more e xtreme points of con v ( C i ) . Next we make use of an ob servation gi ven in the proof of Proposition 1 in [ 8]. Assume that check no de i is ad jacent to only on e non-in tegral variable n ode. Th is implies that there are two or mor e extrem e points of conv ( C i ) which differ in only o ne bit. Extreme points of conv ( C i ) differ howe ver, in at lea st two b its since they all satisfy parity che ck i which contradicts th e assumption.  A given binary linear co de C can be rep resented with some alternative, equivalent par ity check matrix which we denote SUBMITTED TO IE EE TRANSA CTIONS ON INFORMA TION THEOR Y 6 with ˆ H . Any such alter native parity check matr ix for C is obtained by perfo rming elementary ro w operations on H . Note that Proposition 3.2 is valid fo r any ˆ H . Likewise Propo sition 3.3 holds a s well for the pa rity ch eck nodes i ∈ { 1 , . . . , m } of the T ann er grap h re presenting ˆ H . Th e rows o f ˆ H may a lso be interpreted as red undant parity checks. Given a non-integral optimum x ∗ of RIPD, in C U T G E N E R AT I O N A L G O R I T H M 2 we search for a parity check which is adjacent to only one non- integral valued variable n ode. If we find such a parity check we kn ow due to Propo sition 3.3 that x ∗ can not be in the conv ex hull of th is particular parity che ck. Furth ermore d ue to Pro position 3.2 there exists a for bidden set inequality wh ich cuts off x ∗ . Note that in an exhausti ve search algorith m on e would check 2 m redund ant pa rity checks if the parity c heck is ad jacent to only one non- integral valued variable n ode. Instead of a co mputation ally expensive exhaustive sear ch we propose the C O N S T RU C T ˆ H A L G O R I T H M w hich r esem- bles Ga ussian eliminatio n. W e transfer matrix H in to a n equiv alent matrix ˆ H by e lementary r ow operation s (adding two r ows is in GF (2) ). Our aim is to represen t code C with an alternati ve par ity che ck matrix ˆ H , so that in row ˆ H i,. there exists exactly one j ∈ J where ˆ H i,j = 1 and x ∗ j is n on-integral. For all other indices h ∈ J \ { j } with ˆ H i,h = 1 , x ∗ h is integral. The C O N S T R U C T ˆ H A L G O - R I T H M tries to convert co lumns j of H with x ∗ j / ∈ Z in to un it vectors. Note that at most m colu mns of H are conv erted. C O N S T R U C T ˆ H A L G O R I T H M Input : ( x ∗ , z ∗ ) suc h that x ∗ non-in tegral Output : ˆ H . 1 : Set l = 1 , j = 1 . 2 : If x ∗ j ∈ (0 , 1) th en go to 3. Else go to 4. 3 : I f l ≤ m then do elementary row operations until H l,j = 1 and H i,j = 0 for all i ∈ I \ { l } . Set l = l + 1 . 4 : Set j = j + 1 . If j ≤ n then g o to 2. Otherwise terminate. ˆ H can be obtained in O ( m 2 n ) . The C O N S T RU C T ˆ H A L G O - R I T H M is u seful in the following sense. Suppo se i ∈ I is a check n ode adjacent to se veral variable no des j ∈ J such that x ∗ j is non- integral. If ˆ H h as such a r ow i the n we use Proposition 3.2 and Prop osition 3 .3 to construct Forbidden Set Ineq ualities which c ut off the fraction al op timal solution. Specifically we construct the inequalities (19) or (20). W e ref er to these ineq ualities a s ne w Forbidden Set In equalities. No te that N i in the original H matrix and ˆ N i in ˆ H are different index sets. First we calculate k i =    { h ∈ ˆ N i | x ∗ h = 1 }    . (18) If k i is odd we u se the ineq uality X h ∈ ˆ N i : x ∗ h =1 x h − x j − X h ∈ ˆ N i : x ∗ h =0 x h ≤ k i − 1 , (19) otherwise, k i is even, i.e. X h ∈ ˆ N i : x ∗ h =1 x h + x j − X h ∈ ˆ N i : x ∗ h =0 x h ≤ k i . (20) Theor em 3 .4: Let ( x ∗ , z ∗ ) ∈ R n × R m be the op timal solution o f the curr ent RIPD form ulation such that x ∗ is n on- integral. If there exists a ˆ H i,. such that ˆ H i,j = 1 and x ∗ j is non-in tegral for exactly one j ∈ J then th e new forbid den set inequality is a valid ineq uality which is vio lated by x ∗ . Proof: W e hav e to sh ow tha t: 1) For k i odd [ ev en ] the inequality (19) [( 20 )] is violated by x ∗ . 2) For k i odd [ e ven ] th e in equality (19) [( 20 )] is satisfied for all x ∈ C . Let i ∈ I be a row of the reco nstructed m atrix ˆ H . W e obtain i by p erform ing elementar y row o peration s in GF (2) on the rows o f the origin al H matrix. Th erefore it holds that ˆ H i,. x = 0 mod 2 for all x ∈ C . W e show th e pr oof for k i odd. When k i is even the proo f is analog ous. 1) L et k i be an odd nu mber . For x ∗ , since 0 < x ∗ j < 1 the left hand side of (19) is larger than the rig ht hand side thus x ∗ violates (19). 2)Supp ose k i is odd and x ∗ is th e op timal solution of RIPD. Our aim is to sh ow that (19) is satisfied by all co dew ords x ∈ C . First we define δ i ( x ) = X j ∈ ˆ N i x j . Next we rewrite (19) as X j ∈ ˆ N i a j x j ≤ k i − 1 wher e a j ∈ {− 1 , 1 } . (21) W e also d efine th e index sets S + = { j ∈ ˆ N i : a j = 1 } with   S +   = k i . S − = { j ∈ ˆ N i : a j = − 1 } with   S −   =    ˆ N i    − k i . Case 1 For any x ∈ C it holds that δ i ( x ) ≤ k i − 1 : X j ∈ ˆ N i a j x j ≤ k i − 1 is fulfilled. Case 2 a For any x ∈ C it holds that δ i ( x ) ≥ k i + 1 : At most k i of in dices j ∈ ˆ N i where x j = 1 can be in S + . Thus there is a t le ast one index j ∈ ˆ N i with x j = 1 in S − . Consequen tly X j ∈ ˆ N i a j x j ≤ k i − 1 . Case 2 b For any x ∈ C it ho lds that δ i ( x ) = k i : If there is at least o ne in dex j ∈ S − with x j = 1 then X j ∈ ˆ N i a j x j ≤ k i − 1 . Otherwise all j ∈ ˆ N i with x j = 1 are in S + . Then for row i , ˆ H i,. x = 1 mo d 2 since k i is o dd and th erefore the contradictio n x / ∈ C .  Note th at it is p ossible th at each row of ˆ H has at least two j ∈ J such that ˆ H i,j = 1 and x ∗ j is non-integral. I n this case no new fo rbidden set ineq uality can be found usin g C U T SUBMITTED TO IE EE TRANSA CTIONS ON INFORMA TION THEOR Y 7 G E N E R AT I O N A L G O R I T H M 2 . C U T G E N E R A T I O N A L G O R I T H M 2 Input : Optimum o f RIPD s.t. x ∗ non-in tegral, ˆ H . Output : New fo rbidden set inequality or error . 1 : Set i = 1 . 2 : If ther e is exactly on e j ∈ J such that ˆ H i,j = 1 and x ∗ j ∈ (0 , 1) , th en c alculate k i and go to 3 . Else g o to 4 . 3 : If k i is odd [ even ] con struct (19) [( 20 )] . T erminate. 4 : Set i = i + 1 . If i ≤ m then go to 2 . Else output error . The comp lexity of C U T G E N E R A T I O N A L G O R I T H M 2 is in O ( mn ) since in the worst case each entry of ˆ H ha s to be visited o nce . W e are now able to formulate o ur separatio n algo rithm. In the first itera tion, x ∗ can be foun d by hard decision decodin g. In all o f the following iterations RIPD d oes not necessarily have an optimal s olution with integral x ∗ . If the v ector ( x ∗ , z ∗ ) is integral then the optimal solution to I P D is foun d. I f x ∗ is integra l but z ∗ is n on-integral we apply C U T G E N E R A - T I O N A L G O R I T H M 1 to con struct Forbidd en Set In equalities. Although add ing any forbid den set ineq uality suffi ces to cut off the no n-integral solution ( x ∗ , z ∗ ) w e add all Forbidden Set Ineq ualities induced by all non- integral z i based o n the though t that th ey may be useful in future iterations. If x ∗ is non-in tegral we first employ th e C O N S T RU C T ˆ H A L G O R I T H M . Th en we che ck in C U T G E N E R A T I O N A L G O R I T H M 2 if ther e exists a row ˆ H i,. such that there exists e xactly one j ∈ J where ˆ H i,j = 1 and x ∗ j is non -integral. If su ch a row does not exist, then the C U T G E N E R A T I O N A L G O R I T H M 2 outputs an er ror . Otherwise we kn ow f rom Theore m 3.4 th at there exists a new forbid den set ineq uality wh ich c uts off x ∗ . In ˆ H th ere may exist several rows f rom which we can deriv e new Forb idden Set Ineq ualities. In this case we add all new Forbidden Set Inequa lities to the fo rmulation RIPD with th e same reason ing as before. Th e N E W S E PAR A T I O N A L G O R I T H M stop s if either ( x ∗ , z ∗ ) is integral which lea ds to an ML Codew ord or C U T G E N E R AT I O N A L G O R I T H M 2 returns an error wh ich mean s no f urther c uts c an b e f ound . N E W S E PA R A T I O N A L G O R I T H M . Input : Cost vecto r c , matrix H . Output : Current o ptimal solu tion x ∗ . 1 : Solve RIPD. 2 : If the optim al solution ( x ∗ , z ∗ ) is integral then go to 6 . O therwise g o to 3 . 3 : If x ∗ is integral, then call C U T G E N E R A T I O N A L - G O R I T H M 1 . Add the constrain ts to formulation RIPD, go to 1 . If x ∗ is non- integral go to 4 . 4 : Call C O N S T RU C T ˆ H A L G O R I T H M . Go to 5 . 5 : Call C U T G E N E R A T I O N A L G O R I T H M 2 . If the output is error then go to 6 . Otherwise add the ne w constraint to formu lation RIPD, g o to 1 . 6 : Output x ∗ and terminate. T wo strategies wh ich may be used in the implemen tation of the N E W S E PA R A T I O N A L G O R I T H M are: 1) Add all v alid cuts which can be obtained in one it eration. 2) Add only one o f the valid c uts which can b e obtained in on e iteratio n. There is a tr ade-off between Strategies 1 an d 2 , since strategy 1 m eans less iterations with large LP p roblems and Strategy 2 means more itera tions with smaller LP p roblems. W e em pirically tested Strategies 1 and 2 on the three codes described in the follwing section. F or all the three codes Strategy 1 outperfor med Strategy 2 in term s of run ning time and decoding succ ess. I V . N U M E R I C A L R E S U LT S W e compar e the com munication perfo rmance of o ur separ a- tion algorithm with the stand ard LP decod ing [10], BP decod- ing, an d the ref erence curve resu lting fro m ML decod ing. The latter results fr om mod eling and solving IPD using CPLEX 9.120 [6] as the IP solver . T hese fou r algorith ms, LP d ecoding (by Feldman et al. or T aghavi et al. ), BP , N E W S E PA R AT I O N A L G O R I T H M , and ML Decoding (IP , CPLEX) are tested o n two LD PC (one regular and on e irr egular) and on e BCH cod e considerin g tran smission over Additive White Gaussian Noise (A WGN) chan nels. Addition ally we pr esent for o ur separ ation algorithm the m in, max a nd av erage values for th e number of iterations, the number of generated Go mory cuts and the number of ge nerated RPC cuts in tables I, I I, I II. W e selected the (64 , 32) irregula r LDPC co de, T an ner’ s (155 , 64) gr oup structured LDPC c ode [20] and the (63 , 39) BCH code for our tests. The first LDPC co de is constru cted with Progr essi ve Edge Growth a lgorithm. T an ner’ s (155 , 64) LDPC code, which has min imum distanc e of 2 0 an d girth of 8, is constructed as described in [2 0]. Th e Frame Err or Rate (FER) against signal to n oise ratio (SNR) measured in E S / N 0 is shown in Figu res 1 to 3. W e used 2 00 iterations for BP d ecoding of (64 , 32) irregular LDPC and T ann er’ s (15 5 , 64) LDPC code. Figure 1 shows the results fo r the irr egular (64 , 32) LDPC code with d egree distribution 1 f [2 , 3 , 5 , 6] = [ f 2 = 1 2 , f 3 = 1 4 , f 5 = 1 8 , f 6 = 1 8 ] , g [6] = [1] . Ou r sepa ration algo rithm perfor ms by roughly 0 . 5 dB better than L P decoding f or this LDPC code. It is importan t to note that th e commu nication perfor mance of the N E W S E PA R A T I O N A L G O R I T H M is supe- rior to the BP algo rithm here . The results for the T annner ’ s (155 , 64) LDPC code are plotted in Fig ure 2. Performance of th e BP an d standard LP decod ing is very similar in this c ase whereas the N E W S E PAR A T I O N A L G O R I T H M g ains around 0 . 4 dB compare d to both. It is worth while mentionin g that BP decoding a nd our separation algorithm have a per formanc e degradation of > 0 . 8 dB compar ed to ML de coding for this group struc tured LDPC co de. LP deco ding v ia Forbidd en Set Inequ alities intr oduced in [10] canno t b e used fo r high d ensity codes sin ce the nu mber of constraints is expon ential in the ch eck node degree. This 1 Irregu lar LDP C codes are described by varia ble node degree distribu tion f i and check node degre e distrib ution g i , where f i and g i represent s the fracti on of varia ble nodes and check nodes with degre e i respecti vely . SUBMITTED TO IE EE TRANSA CTIONS ON INFORMA TION THEOR Y 8 causes a pro hibitive usage of m emory in the phase of building the LP mo del. The adap tiv e appro ach o f [13] overcomes this shortcomin g and yet perf orms as go od as LP decoding ( see Section III ). T herefor e we used th is metho d in the com parison of a lgorithms when deco ding a d ense (63 ,39) BCH code. Th e results fo r th is co de are shown in Figure 3 . I t shou ld also be noted that BP deco ding does not work for this typ e of codes d ue to the d ense structu re of their parity check matr ix. Our ap proach is on e of the first attemp ts (see [9]) to decod e dense codes using mathematical pro grammin g app roaches. Although the gap between M L decodin g and our separation algorithm increases to rou ghly 1 dB , the results obtained by our algorithm are su bstantially better (mor e th an 2 dB ) th an the results o btained by ada ptiv e LP decod ing. T o summarize, our separation algorithm improves LP decod- ing significantly for all th ree test setups. This improvement is due to new Forbidde n Set In equalities found b y C U T G E N E R - A T I O N A L G O R I T H M 2 . The con straints added by this algorithm are b ased o n the rows of the alternative rep resentations of the H matrix. T hese rows ca n also be interpreted a s red undan t parity checks. Consequently , the family Λ of inequalities we use inclu des a sub set of th e Forbidden Set Inequalities which can be d erived from redund ant p arity ch ecks and Λ is larger than the original family of Forbidden Set Ine qualities. Regarding the complexity of the N E W S E PA R AT I O N A L - G O R I T H M , we p resent the minimum , average, an d maximu m number of iterations, cuts introd uced by the C U T G E N E R A - T I O N A L G O R I T H M 1 (shown in Go mory cuts colu mn) and th e number of cuts in troduced by the C U T G E N E R AT I O N A L G O - R I T H M 2 ( shown in RPC cuts c olumn) in the tab les I, II, and III for the cod es (64 , 32) , (155 , 64) , an d (6 3 , 39) resp ectiv ely . Note that the number of iterations can b e c onsidered as the number of times we c all th e LP solver . 2 2.5 3 3.5 4 4.5 5 10 −4 10 −3 10 −2 10 −1 10 0 SNR [dB] Frame Error Rate (FER) LP Decoding (Feldman et al.) Belief Propagation New Separation Algorithm ML Decoding (IP, CPLEX) Fig. 1. Decoding performance of an irregular LDPC code (64,32). V . C O N C L U S I O N In th is pape r we p roposed a new IP form ulation and its LP relaxation. Instead o f solving the optimization prob lem, we 2 2.5 3 3.5 4 10 −7 10 −6 10 −5 10 −4 10 −3 10 −2 10 −1 10 0 SNR [dB] Frame Error Rate (FER) LP Decoding (Taghavi et al.) Belief Propagation New Separation Algorithm ML Decoding (IP, CPLEX) Fig. 2. Decoding performance of T anner’ s (155 , 64) L DPC code. 2.5 3 3.5 4 4.5 5 10 −4 10 −3 10 −2 10 −1 10 0 SNR [dB] Frame Error Rate (FER) LP Decoding (Taghavi et al.) New Separation Algorithm ML Decoding (IP, CPLEX) Fig. 3. Decoding performance of a BCH code (63,39). solve the sep aration pro blem. The indicator variables z yield an immediate recogn ition of parity violation s and ef ficient generation of cuts. W e used on one h and th e Forbid den Set Inequa lities of [1 0] which ar e a subset of all possible Gomor y cuts. On the oth er h and we showed h ow to genera te efficiently new cuts based on redun dant p arity checks. Note that th e rows in our ˆ H matr ix can be considered as redundan t parity checks. It is known that RPC cuts improve th e L P decod ing via tightening the fu ndamental polytope [10], [13]. Howe ver RPC g enerating app roaches known to us cann ot verify if th e particular RPC really introduce s a c ut or not. Another open question addr esses the co nfiguration S to be used fo r the RPC. In ou r ap proach, once we en sure th at ther e is only o ne j ∈ N i with non -integral x ∗ j in row ˆ H i,. , we can im mediately find the config uration S an d thu s th e new fo rbidden set ineq uality (19) or (20). Additiona lly , Theor em 3.4 states that the new SUBMITTED TO IE EE TRANSA CTIONS ON INFORMA TION THEOR Y 9 Number of LP s solved Number of Gomory cuts Number of RPC cuts SNR Min A vera ge Max Min A verage Max Min A vera ge Max 1.8 2 5.942 20 2 21.296 42 0 33.619 207 2.2 1 4. 896 21 0 19.187 41 0 21.465 227 2.6 1 4. 196 19 0 17.569 42 0 13.138 177 3.0 1 3.48 16 0 15.07 40 0 6.895 180 3.4 1 3. 005 19 0 13.228 39 0 2.917 145 3.8 1 2. 725 12 0 11.254 36 0 1.513 119 4.2 1 2. 446 11 0 9.738 31 0 0.428 111 4.6 1 2.297 10 0 8.195 32 0 0.27 52 5.0 1 2. 134 6 0 7.055 31 0 0.079 25 5.4 1 1. 977 6 0 5.585 23 0 0.014 6 5.8 1 1.872 6 0 4.448 18 0 0.012 12 T ABLE I I T E R AT I O N S A N D C U T S D E R I V E D F OR ( 6 4 , 3 2 ) L D P C C O D E . Number of LP s solved Number of Gomory cuts Number of RPC cuts SNR Min A vera ge Max Min A verage Max Min A vera ge Max 2.0 2 6. 093 20 20 60.235 94 0 74.161 594 2.2 2 5. 343 22 19 57.148 100 0 48.667 595 2.4 2 4. 828 21 19 54.013 94 0 31.713 640 2.6 2 4.363 23 14 50.817 92 0 20.254 549 2.8 2 3. 954 18 15 47.265 96 0 12.65 468 3.0 2 3. 798 26 16 45.324 98 0 10.776 632 3.2 2 3.47 17 16 42.2 79 0 4.211 431 3.4 2 3. 158 19 11 38.381 81 0 1.293 508 3.6 2 3. 13 13 6 36.478 76 0 1.122 228 3.8 2 2. 911 10 3 34.085 76 0 0.324 252 4.0 2 2.81 12 7 31.529 66 0 0.298 238 4.2 2 2. 725 9 7 29.576 68 0 0.146 78 T ABLE II I T E R AT I O N S A N D C U T S D E R I V E D F O R (155 , 64) T A N N E R C O D E . Number of LP s solved Number of Gomory cuts Number of RPC cuts SNR Min A vera ge Max Min A verage Max Min A vera ge Max 2.4 1 10.186 24 0 24.993 56 0 64.173 200 2.8 1 8.802 21 0 23.464 57 0 50.382 175 3.2 1 7.649 22 0 22.083 53 0 39.76 180 3.6 1 5.911 22 0 19.401 63 0 25.184 175 4.0 1 4.967 21 0 17.743 54 0 17.729 179 4.4 1 4.111 20 0 15.379 60 0 11.612 176 4.8 1 3.249 18 0 12.941 59 0 6.508 177 5.2 1 2.703 18 0 10.944 43 0 4.002 143 T ABLE III I T E R AT I O N S A N D C U T S D ER I V E D F O R ( 6 3 , 3 9 ) BC H C O D E . forbid den set inequality is a valid in equality which cuts off the fraction al optimal solutio n ( x ∗ , z ∗ ) . These theoretical improvements ar e sup ported with em pir- ical evidence. Comp ared to state of the art (adaptiv e) LP decodin g our algorithm is superio r in terms of fra me error ra te for all th e codes we have tested. Mo reover , it is competitive to the results obtained by BP decodin g. In contra st to the latter , our a pproach is ap plicable to codes with dense parity -check matrix and offers a po ssibility to de code such codes. One futu re research direction is to find n ew cut families when C U T G E N E R A T I O N A L G O R I T H M 2 stops. The polyh e- dral structure o f th e ML deco ding will be fu rther inv estigated. This w ill yield a b ranch-a nd-cut alg orithm which we exp ect to further extend the applicability of our a pproach . A C K N OW L E D G M E N T W e would like to th ank Pascal O. V ontobel for his con- structive comments & suggestions and our colleagu e Daniel Schmidt f or his initial work related to I PD fo rmulation pre- sented in this paper . W e gratefully ackn owledge par tial fin an- cial supp ort by the Center of Mathematical and Com putational Modeling of th e University of Kaiserslautern. R E F E R E N C E S [1] E. Berleka mp, R. McEliece and H. van Tilbor g, On the inhere nt in- tract ability of certain coding probl ems, IEEE Transact ions on Information Theory , 954-972, 1978. [2] D. Burshtein Iterati ve appr oximate linear pr ogra mming decoding of LDPC codes with linear complex ity , Proc. IE EE Intern. Symp. Inform. Theory , T oronto, Canada, pp. 1498-1502, July 2008. [3] M. Chertk ov and V . Y . Chernya k, Loop Calculus Helps to Impro ve Belief Pr opagat ion and L inear P r ogr amming Decodings of Low-Density -P arity- Chec k Codes, in Allerton Conference on Communicati ons, Control and Computing, Monticell o, IL, September 2006. [4] M. Chertko v , Reducing the Erro r Floor , http:/ /www .citebase.or g/abstract?id=oai:arXi v .org:0706.2926 , 2007. [5] M. Chertk ov and M. Stepanov , Pseudo-code word Landscape, ISIT 2007, Nice, June 2007. [6] ILOG CP LEX 9.0 Users Manual ILOG SA, France, 2003. SUBMITTED TO IE EE TRANSA CTIONS ON INFORMA TION THEOR Y 10 [7] C. Daskalakis, A. G. Dimakis, R. M. Karp and M. J. W ainwrigh t, Pr obabilist ic A nalysis of Linear Pro gramming Decoding, in Proceeding s of the 18th Annual Symposium on Discrete Algorithms (SODA), January 2007. [8] A. G. Dimakis, A. A. Gohari and M. J. W ainwright , Guessing F acets: P olytope Structur e and Impr ove d LP Decoding, in Internation al Sympo- sium on Information Theory , Seattle, W A. July 2006. [9] S. C. Draper , J. S. Y edidia and Y . W ang, ML decoding via mixed-int e ger adaptiv e linear pro gramming , Proc . IEEE Intern. Symp. on Inform. Theory , June 2007. [10] J. Feldman, M. J. W ainwright and D. R. Karger , Using linear pr ogram- ming to Decode Binary linear codes, IE EE Transact ions on Information Theory , 51:954-972, March 2005. [11] J. Feldman, T . Malkin, R. A. Servedi o, C. Stein, and M. J. W ainwright LP decoding corre cts a constant fraction of err ors, IEE E Tra ns. Inform. Theory , vol. 53, no. 1, pp. 82-89, Jan. 2007. [12] G. L . Nemhauser and L. A. W olsey , Inte ger and Combinatorial Opti- mization, Wile y-Interscience series in discrete mathemati cs and optimi za- tion, John Wil ey & Sons, 1988. [13] M. H. T aghavi and P . H. Siegel , Adaptive Linear P r ogramming Decoding , in IEEE Int. Symposium on Information Theory , Seattl e, W A, July 2006. [14] P . O. V ontobel and R. K oetter , T owar ds Low-Complexi ty Linear- Pr ogra mming Decoding, in Proc. Int. Conf. on T urbo Codes and Related T opics, Munich, Germany , April 2006. [15] P . O. V ontobel and R. K oetter , Graph-Cov ers and iterati ve Decoding of finite length codes, in Proc. 3rd Internat ional Symp. on Turbo Codes, September 2003. [16] P . O. V ontobel and R. Koet ter , Lower bounds on the minimum pseudo- weight of linear codes, in Proc. IEEE Intern. Symp. on Inform. Theory , (Chicag o, IL, USA), 2004. [17] P . O. V ontobel and R. Koett er , On the relat ionship betwee n linear pr ogramming deco ding and min-sum algorithm decodi ng, Proc. ISIT A 2004, Parma, Italy , p. 991-996, October 10-13, 2004. [18] K. Y ang, J. Feldman and X. W ang Nonline ar Pro gramming Approac hes to Decoding Low-Density P arity-Che ck Codes, Select ed Areas in Com- municati ons, IE EE Journal on , v ol.24, no.8, pp. 1603-161 3, A ugust 2006. [19] K. Y ang, X. W ang and J. Fel dman A Ne w L inear Pr ogramming Appr oach to Decoding Linear Block Codes, Informati on Theory , IEEE Transacti ons on Information Theory , March 2008. [20] R. M. T anner , D. Srkdhara and T . Fuja, A class of gr oup-structure d LDPC codes, Proc. of IST A 2001.