Irregular turbo code design for the binary erasure channel

Irregular turbo c ode d esign fo r th e binary erasure channe l Ghassa n M. Kraidy , V alentin S avin CEA-LETI, 1 7 rue des Martyrs, 38054 Grenoble, France { ghassan .kraidy ,valentin.savin } @cea.fr Abstract — In this p aper , the design of irregular turbo codes fo r the b inary erasure channel is inve stigated. An analytic expression of the erasure probability of punctured r ecursive systematic con vo lutional codes is derived. This exact expression will be used to track the densit y evolution of turb o codes over the erasure chann el, that wil l allo w for th e design of capacity- approaching irregular turbo codes. Next, we propose a graph- optimal interleav er f or irregular turbo codes. Simulation results fo r d ifferent coding rates is shown at the end . I . I N T RO D U C T I O N The per forman ce of e rror corr ecting codes over the bin ary erasure ch annel (BEC) can be analy zed precisely , a nd a flurry of resear ch pap ers have already ad dressed this issue. For small to mediu m codeword length, Maximum-Distance Sep arable (MDS) cod es achieve the capac ity of the BEC. Howe ver , f or large blo ck lengths, th eir decod ing b ecomes un tractable, and thus iteratively decoded g raph- based codes presen t the main alternative. Low-density parity- check (LDPC) cod es [1] [2] [3] and rep eat-accumu late ( RA) cod es [4] with message -passing decodin g proved to perform very close to the ch annel capacity with re asonable comp lexity . Moreover , “rateless” codes [ 5] [6] that are capable o f ge nerating an infinite sequ ence of p arity symbols were prop osed f or the BEC. Howe ver , conv olutional- based co des, that are widely used for Gaussian ch annels, are less investigated on the BEC. Amon g the few paper s that deal with convolutional a nd tur bo co des over the BEC are [7] [8] [9] [10]. I n this pap er , we p ropose irr egular turb o codes that app roach the cap acity of the BEC for med ium to large block le ngth. T his is accomplished th rough p recise a symptotic analysis of the co des together with a grap h-op timal interleaver . The paper is organized as follows: in Section II we describe the model of the irregular turbo code. Section III gives the exact erasure prob ability at the outpu t of a punctur ed R SC code. Th e asymptotic design of irregular tur bo co des is then discussed in Section I V, while Section V presents an op timal graph -based interleaver for su ch codes. Section VI shows th e perfo rmance of these cod es and Section VII gives the con cluding remarks. I I . I R R E G U L A R T U R B O C O D E S A parallel turb o co de [11] ge nerally consists of a co ncate- nation o f two r ecursive systematic co n volutional (RSC) c odes. An inf ormation sequ ence b is encode d b y th e first RSC code to generate a first par ity bit sequenc e; the same sequenc e is then scramb led by an inte rleaver Π an d en coded by a secon d RSC cod e to gene rate a second par ity b it seque nce. In most cases, th e two constituent RSC en coders of a parallel tur bo code are identical. For th is reason, the autho rs in [ 12] [ 13] propo sed a “self-co ncatenated ” tu rbo enco der in wh ich every informa tion bit is repea ted twice, inter leav ed, and fe d to an RSC cod e of doub le the size, as shown in Fig. 1. In this Fig. 1. Self-conc atena ted turbo encoder . new representation , each inform ation bit is co nnected to the code trellis via two edg es in th e propa gation tree of Fig. 2. Therefo re, we say that th e degr ee of th e info rmation b its is d = 2 , and that the tu rbo c ode is regular . Using this structur e, one can create irr egularity b y repeatin g a certain fractio n f d of the bits d tim es, providing bits that are more protec ted th an in the regular c ase. Like for LDPC and RA co des [2] [14], irregularity can boost the perfo rmance of turbo cod es fo r large block leng ths. Irregular tu rbo cod es were first introdu ced in [15]. In [12] [13], in a slightly different d esign, a fr action o f the info rmation bits is rep eated d time s with d > 2 , wh ile the pa rity bits remained of degree 1. In order to maintain th e same coding rate, a fractio n φ p of the parity bits is pu nctured . W e will use this representation to design irregular turbo codes for the bin ary erasure channel. The enco der o f an irregular turbo cod e is similar to that of Fig. 1, with the difference that the repetition is non -unifo rm. Th e in formatio n bits are th us divided into d classes with d = 2 , . . . , d max , wh ere d max is the maximum b it-node degree. Th e nu mber of bits in a class d is a frac tion f d of th e total num ber of infor mation bits at the turbo enco der input, whe re bits in class d are repeated d times. Finally , th e outp ut of the no n-unif orm rep eater is interleaved and fed to th e RSC con stituent code, of which (1 − φ p ) of the parity b its ar e tr ansmitted. Now let K deno te the length of the informa tion sequ ence, N th e interleaver size, ρ 0 and ρ the initial a nd th e final (pun ctured) rate of the RSC co nstituent code respectively , and R c the rate of the turb o co de. W e can write the fo llowing: d max X d =2 f d = 1 , d max X d =2 d.f d = d, N = K d max X d =2 d.f d = K. d (1) R c = K K + N ρ − N = 1 1 + “ 1 ρ − 1 ” d (2) ρ = 1 1 + (1 − φ p ) “ 1 ρ 0 − 1 ” (3) Fig. 2. Propagat ion tree of an irre gular turbo code For a degre e p rofile { f 2 , f 3 , ..., f d max } an d using the above equations, one can com pute the p uncturin g frac tion φ p corre- sponding to a target rate R c . The perf orman ce of an irregular turbo code will stron gly depend on the d egree p rofile and the punctur ing fraction or , mo re sp ecifically , o n the corresp onding punctur ing pattern. In the f ollowing sections we will consider the design o f capacity-a pproac hing irregular turbo codes over the BEC. T o do so, we will first comp ute the analytic expres- sion of th e extrinsic erasure probab ility at the output o f the punctur ed RSC deco der th at r epresents the key too l f or the density evolution of irregular turbo codes. I I I . E R A S U R E P R O BA B I L I T Y O F P U N C T U R E D R S C C O D E S In this section, we will derive th e exact erasure p robab ility of bin ary RSC code s, taking in to accoun t th e pu ncturing of parity bits. T o do so, we will follow the steps of the method propo sed in [7] used to comp ute the erasu re probab ility at the output unpu nctured RSC cod es. For th e sake o f simplicity , we only co nsider ha lf-rate co des with co nstraint leng th L = ν + 1 , where ν is the memo ry of the code. Th e same method applies to RSC co des with different rates. W e consider the fo llowing comm unication schem e: a uni- formly distrib uted sequen ce of bits b of le ngth K is fed to a binary RSC encode r tha t generates a sequence c of parity bits o f length N (1 − φ p ) . During transmission 1 , a bit b i (respectively c j ) is either erased with probab ility p (respectively q ), or p erfectly received with probab ility 1 − p (respectively 1 − q ). Let b ′ and c ′ be the received sequences at th e decoder . An RSC code has S = 2 ν states. Con sidering the “Forward-Back ward” [16] deco ding algor ithm, let F n ( s ) and B n ( s ) be the p robab ilities of being in state s = 1 , . . . , S computed in the fo rward and in the b ackward directions, at the left and at the right side of the n th trellis step respectively . Let l ( e ) and r ( e ) be the states to wh ich an edge e is con nected 1 W e consider dif ferent erasure probabiliti es on information and parity bits, in order to be able to distinguish between the extr insic (correspondin g to information bits) and the communication (correspond ing to parity bits) channe ls for the density e volut ion computatio n on the left and o n the r ight resp ectiv ely . The informa tion b it b ( e ) and parity b it c ( e ) are associated to ed ge e . As shown in [ 7], the extrinsic pr obability of an informatio n bit b n at the output o f the de coder is written as: P ext ( b n ) = P “ b n | b ′ n − 1 −∞ , b ′ ∞ n +1 , c ′ ” ∝ X e : b ( e )= b n F n ( l ( e )) · P ( c ( e )) · B n ( r ( e )) (4) Now the let Σ F = { σ 1 f , . . . , σ | Σ F | f } and Σ B = { σ 1 b , . . . , σ | Σ B | b } b e the sets from which F n and B n take values. The cardin ality o f the sets Σ F and Σ B is compu ted as: | Σ F | = | Σ B | = ν X α =0 “ 2 α 2 ν ” (5) Howe ver , as an RSC code is linea r , we assum e the all-zero s codeword is tra nsmitted without losing gen erality . This gives smaller state distribution sets Σ ∗ F and Σ ∗ B with card inality: | Σ ∗ F | = | Σ ∗ B | = ν X α =0 “ 2 α − 1 2 ν − 1 ” (6) A f our-state RSC co de ( L = 3 ) h as f or in stance: Σ ∗ F = Σ ∗ B = { (1 , 0 , 0 , 0) , (1 / 2 , 1 / 2 , 0 , 0) , (1 / 2 , 0 , 1 / 2 , 0) , (1 / 2 , 0 , 0 , 1 / 2) , (1 / 4 , 1 / 4 , 1 / 4 , 1 / 4) } (7) A. Computation of the Erasur e Pr obability The trellis of a co n volutional code forms two first-order S -state Markov ch ains correspon ding to th e f orward and backward recu rsions. This allows to co mpute the steady-state distributions of the Markov processes that will be used to com- pute the bit er asure pro bability at th e output o f the decoder . The distributions π F ( p, q ) and π B ( p, q ) are the no rmalized solutions o f th e following equ ations: π F ( p, q ) = π F ( p, q ) · M F ( p, q ); π B ( p, q ) = π B ( p, q ) · M B ( p, q ) (8) where the ( i, j ) th entry o f matrix M F is th e pro bability of the transition f rom state distribution F n = σ i f to state distribution F n +1 = σ j f . Similarly , the matrix M B represents the tra nsition probab ilities in th e b ackward direction . In other word s, the distributions π F and π B are the stationary distributions to which th e Mar kovian process co nv erges, as: lim δ →∞ M δ F = 1 ⊗ π F ; lim δ →∞ M δ B = 1 ⊗ π B (9) where 1 is a column vector o f ones. As an exam ple, we will consider th e fou r-state RSC (1 , 5 / 7) 8 code with L = 3 . Let p be th e erasur e prob ability on the informatio n bits, and q be the erasur e probab ility on parity bits. Assuming the all-zeros codeword h as been tr ansmitted, we hav e that | Σ ∗ F | = | Σ ∗ B | = 5 . The 5 × 5 Markov state transition matrix M F for the forward recursion o f th is co de is given b y: M F ( p, q ) = 2 6 6 6 4 1 − pq 0 pq 0 0 0 0 1 0 0 1 + pq − p − q p − pq 0 q − pq pq 1 + pq − p − q q − p q 0 p − p q pq 0 0 1 + pq − p − q 0 p + q − pq 3 7 7 7 5 and th e matr ix M B for the back ward recursion is giv en by: M B ( p, q ) = 2 6 6 6 4 1 − p q pq 0 0 0 1 + p q − p − q 0 p − pq q − pq pq 0 1 0 0 0 1 + p q − p − q 0 q − pq p − p q pq 0 1 + p q − p − q 0 0 p + q − pq 3 7 7 7 5 Once M F ( p, q ) and M B ( p, q ) are com puted, we can so lve for π F ( p, q ) and π B ( p, q ) . W e next consider the matrix T ( q ) whose ( i, j ) th entry repr esents the pro bability of an ou tput erasure co nditione d o n the le ft an d righ t state distributions σ i f and σ j b , knowing that parity bits are er ased with pro bability q : T i,j ( q ) = P “ P ext ( b n ) = 1 / 2 | F n = σ i f , B n = σ j b ” (10) The m atrix T ( q ) fo r th e RSC (1 , 5 / 7) 8 code is given by: T ( q ) = 2 6 6 6 4 0 0 q 0 0 q q q q q 0 1 q 0 1 0 0 q 1 1 q 1 q 1 1 3 7 7 7 5 Finally the extrinsic erasure pr obability is com puted as: P ext ( p, q ) = π F ( p, q ) · T ( q ) · π B ( p, q ) t (11) where the op erator ( . ) t denotes the tran spose o perator . B. Computation of the Erasur e Pr obability with pu ncturing Now suppo se a fraction φ p of the p arity bits of the code are punctur ed. If the punctur ed par ity bits were rando mly chosen at e ach tr ansmission, we cou ld consider that the f raction φ p of punc tured bits is a part o f the chann el, as if the decoder receives bits with prob ability of er asure o n p arity bits given by: q ′ = 1 − (1 − q ) (1 − φ p ) = φ p + q − q · φ p (12) Howe ver , if the punctu ring p attern is fixed, the extrinsic erasure pr obability computed using (1 1) by rep lacing q with q ′ from (12) is ina ccurate. T he g oal is the n to a nalytically compute the extrinsic erasure probability a t the output o f the d ecoder knowing that parity bits ar e p uncture d using a predefined pattern. For this purpo se, we define a pun cturing pattern X = [ x 1 , x 2 , ..., x Γ ] , x γ ∈ { 0 , 1 } , where a 0 in p osition γ means that the parity bit in the cor respond ing trellis step is punctur ed. The parity bits of the constituent R SC code are then punctur ed using a p eriodic punctu ring pa ttern with period X . W e consider a window of size Γ in the trellis of the cod e, and let M F, X ( p, q ) the matrix whose ( i, j ) th entry is th e prob ability of the tran sition from the state d istribution F n = σ i f at th e left side of the window to the state distribution F n +Γ = σ j f at the righ t side of the window . Similarly , the matrix M B , X represents “thro ugho ut-the-win dow” transition pro babilities in the backward direction . W e have the following: M F , X ( p, q ) = Γ Y γ =1 M F ( p, q x γ ); M B, X ( p, q ) = Γ Y γ =1 M B ( p, q x Γ+1 − γ ) (13) This mean s that M F, X ( p, q ) is obtained by multiply ing matrices M F ( p, 1) and M F ( p, q ) accordin g to whether th e correspo nding p arity bit is puncture d ( x γ = 0 ) or not ( x γ = 1 ). A similar assertion h olds for the backward matrix M B , X ( p, q ) . Let π F, X ( p, q ) and π B , X ( p, q ) be the cor respond ing steady- state d istributions, meaning tha t: lim δ →∞ M δ F , X = 1 ⊗ π F , X ; lim δ →∞ M δ B, X = 1 ⊗ π B, X (14) where 1 is a column vector of ones. Th ese expressions represent the state p robab ility d istributions in th e forward and backward d irections, at the left and at th e right side of the window r espectively . The distributions π F, γ ( p, q ) on the left side and π B ,γ ( p, q ) on th e righ t side of a window step γ can be rec ursively comp uted as: π F , 1 ( p, q ) = π F , X ( p, q ) , π F ,γ ( p, q ) = π F ,γ − 1 ( p, q ) · M F ( p, q x γ ) , γ = 2 , . . . , Γ (15) π B, Γ ( p, q ) = π B, X ( p, q ) , π B,γ ( p, q ) = π B,γ +1 ( p, q ) · M B ( p, q x γ ) , γ = Γ − 1 , . . . , 1 (16) Next, the extrinsic erasure prob ability of the info rmation bit in po sition γ can be computed as: P ext ,γ ( p, q ) = π F ,γ ( p, q ) · T ( q x γ ) · π B,γ ( p, q ) t (17) Finally , the extrin sic erasure pro bability at the outpu t of th e decoder corresp onding to the pu ncturin g patter n X is giv en by: P ext , X ( p, q ) = 1 Γ Γ X γ =0 P ext ,γ ( p, q ) (18) As an examp le, suppose we want to co nstruct a h alf-rate parallel tur bo cod e u sing half -rate (1 , 5 / 7) 8 RSC cod es. I n order to raise the rate of the co nstituent co des fr om 1 / 2 to 2 / 3 , we pun cture h alf of their parity b its using the p attern X = [1 , 0] . The expression o f th e exact probability of th is code can then be written as: P ext , X ( p, q ) = 1 2 ˆ P ext , 1 ( p, q ) + P ext , 2 ( p, q ) ˜ (19) where P ext , 1 = π F , X ( p, q ) · T ( q ) · ˆ π B, X ( p, q ) · M B ( p, 1) ˜ t (20) P ext , 2 = ˆ π F , X ( p, q ) · M F ( p, q ) ˜ · T (1) · π B, X ( p, q ) t (21) π F , X ( p, q ) = 1 5 · 1 t · lim δ →∞ M δ F , X ( p, q ) , and M F , X ( p, q ) = M F ( p, q ) · M F ( p, 1) (22) π B, X ( p, q ) = 1 5 · 1 t · lim δ →∞ M δ B, X ( p, q ) , and M B, X ( p, q ) = M B ( p, 1) · M B ( p, q ) (23) The exact expression of the erasu re proba bility in (18) is the key tool for designing irregular turb o cod es for th e BEC, as will be d iscussed in the fo llowing section. In fact, for the same punctur ing fractio n φ p , it is capable of de termining which p attern X gives the lowest P ext , X . Mo reover , it allows to detect a catastro phic punctu ring scen ario that lead s to infinite error ev ents and thu s har ms th e correction ca pacity of th e code. As an e xample, puncturing the RSC (1 , 5 / 7) 8 code using X = [1 , 0 , 0] g iv es: P ext , X (0 , q ) > 0 (24) This means that this punctu ring pattern is catastrophic, as a sing le bit error at the input o f the deco der generates an infinite err or event. If we h av e φ p = 2 / 3 , we would rather use X = [1 , 0 , 0 , 0 , 1 , 0 ] for instance. Although this example can be directly observed o n th e tr ellis of the (1 , 5 / 7) 8 code, (18) points ou t the phen omeno n for any S -state co de (on any channel! ), where tr ellis an alysis becomes more tedio us as S increases. I V . I R R E G U L A R T U R B O C O D E D E S I G N The an alytic expr ession o f th e erasur e pr obability of punc- tured RSC codes in the previous section a llows us to analyze the iterative decod ing of tu rbo co des over the BEC. As dis- cussed in Section II, a parallel turb o code co nsists o f a pa rallel concatenatio n of two RSC codes. The iterati ve dec oding of such codes can be an alyzed through EXIT charts [1 7], th at are n on-lin ear function s relating the o utput to the input of the RSC dec oders of the infinite-length turbo cod e. This tech nique giv es in sight on the iterative process in the sense that the decodin g is su ccessful for a certain cha nnel qu ality if the two curves correspo nding to the two decod ers do not intersect. The thresho ld of the cod e is th e worst value o f the chann el quality at which th e tunnel b etween the two cu rves is ope n. I n the case where the two con stituent co des are identical, the decodin g converges if the cur ve of the RSC d ecoder does not intersect with the for ty-five degree lin e. Over the BEC, and with th e difference o f Gaussian ch annels in gener al, an EXIT chart de scribing the iter ativ e d ecoding proc ess gives the exact density ev olution of erasure pro babilities, as we can compute ana lytic expressions of the output as a function of the input of the decod er . Altho ugh wid ely used for LDPC codes, this prope rty was first exploited in [18] to com pute exact thresholds for regular unpun ctured turbo codes. For the sake of infinite-length a nalysis, we rep resent a turb o co de u sing the tree struc ture as shown in Fig. 2, in which an info rmation bit of degree d is con nected to d trellises. For the r egular parallel turbo code, th e er asure pr obability at iteration ℓ + 1 is given by: P ℓ +1 = p 0 · P ext , X ( P ℓ , p 0 ) (25) where p 0 is the chann el erasure prob ability . This expression determines the density ev olution of the iterativ e decoding process, as it relates th e prob ability at an iteration to that of the previous iteration. Using (25), the th reshold prob ability p th of th e R c = 1 / 3 p arallel tu rbo co de built from r ate-half RSC (1 , 5 / 7 ) 8 constituent codes is computed as p th = 0 . 6428 , knowing that the cap acity of th e BEC is C = 1 − p 0 . Again, the pun ctured R c = 1 / 2 turbo code built f rom the same constituent co des has p th = 0 . 4729 . In order to tighten the ga p to th e capacity of th e BEC at a g iv en r ate, we consider the desig n of irr egular turbo codes. The erasure p robability at itera tion ℓ + 1 o f a b it of degre e d can be expre ssed as a f unction of the erasur e probab ility a t iteration ℓ as: P ℓ +1 ( d ) = p 0 · d · f d d · P ext , X ( P ℓ , p 0 ) d − 1 (26) Let λ d = d · f d d and λ ( X ) = d max X d =1 λ d X d − 1 (this d efinition will be mad e cleare r in Section V where we w ill intr oduce the factor graph of the tur bo cod e). W e can then write: P ℓ +1 ( d ) = p 0 · λ d · P ext , X ( P ℓ , p 0 ) d − 1 (27) A veraging over a ll p ossible bit degrees, we g et: P ℓ +1 = p 0 · λ ◦ P ext , X ( P ℓ , p 0 ) (28) Follo wing this e quation, the irregular turbo code can recover from a chan nel erasure prob ability p 0 if and only if p 0 · λ ◦ P ext , X ( x, p 0 ) ≤ x, ∀ x ∈ [0 , p 0 ] (29) The co de th reshold is defined as: p th ( λ, X ) = max { p 0 | p 0 · λ ◦ P ext , X ( x, p 0 ) ≤ x, ∀ x ∈ [0 , p 0 ] } (30) and it de pends on b oth degree distribution and pu ncturing pattern. The design o f capacity approa ching irregular turbo codes re duces to the o ptimization o f the fun ction ( λ, X ) 7→ p th ( λ, X ) . For instance, this can b e carried out u sing the differential ev olution algorithm [19]. In gener al, a un iform punctur ing p attern lead s to th e b est thr eshold, provide d the pattern is n ot catastrophic (which leads to a th reshold equal to zero!). Th erefore , in orde r to r educe the space o f p arameters of the optimizatio n fu nction, for each degree distribution λ , we compute the punctu ring fr action φ p accordin g to the target rate R c , and cho se the punctur ing p attern X as uniform as possible according to φ p . As an example, using the half-rate RSC (1 , 5 / 7) 8 code and by setting d max = 1 2 , we obtained the degre e profiles in T able I. T ABLE I D E G R E E P R O FI L E O F I R R E G U L A R T U R B O C O D E S OV E R T H E B E C R c f 2 f 4 f 5 f 7 f 8 f 9 f 12 d φ p p th 1 / 2 0 . 801 0 . 101 0 . 046 0 . 052 2 . 998 0 . 666 0 . 490 1 / 3 0 . 838 0 . 034 0 . 041 0 . 042 0 . 045 2 . 873 0 . 304 0 . 665 1 / 4 0 . 837 0 . 055 0 . 054 0 . 054 3 . 033 0 . 011 0 . 743 Note that the half -rate irregular turbo code designed thro ugh differential e volution has 2 parity bits out o f 3 p unctur ed, and the o ptimization algor ithm av oided ca tastrophic pun cturing while co mputing (18), as explained at the end of Sec tion I II. V . P E G - B A S E D I N T E R L E A V E R F O R T U R B O C O D E S W e investigate n ow the design of graph- based inter leavers for irregu lar turbo codes based on the pr ogressive-edge growth (PEG) algo rithm [20]. T o do so, we define the factor graph of an irregular turbo cod e, in a man ner similar to tha t o f [21], [22]. As shown in Fig. 3 the factor graph consists o f: • bit nodes, rep resented by simp le circles (info rmation bits are represented on the top , wh ile par ity bits are represented on the botto m); • state nodes, rep resented by dou ble circles; • trellis step nodes, also called transition nodes, repr esented by sq uares. If ρ 0 = k /n is the rate of the con stituent RSC codes, then each transition n ode is co nnected to k inform ation bits and n − k parity b its. Usin g this re presentation , the pr eviously d efined λ d = d · f d d is equal to the fraction of edges emanating from informa tion bit nodes of d egree d , and λ = ( λ 2 , . . . , λ d max ) is called th e e dge perspective d egree distribution. Fig. 3. Fact or graph of irregul ar turbo codes. The dec oding of tur bo co des can be perfo rmed on the factor graph, by iterativ ely pro pagating extrinsic messages fr om each graph no de to its neighb or n odes. As d iscussed in [2 3][22] [24], small cycles must b e av oided in the facto r grap h of a tur bo code so that it looks locally tr ee-like, and thu s the messages ar e more independ ent. An upper-boun d o n the gir th (minimal cycle length) o f an irregular factor graph c an be derived using a straigh tforward variation of the approa ch in [25] (see also Lemm a 2 in [20]): g ≤ 4 0 @ 6 6 6 4 log h ( N − 1) k k +2 + 1 i log( k + 1) 7 7 7 5 + 1 1 A (31) Thus, a graph -optimal inter leaving algorithm for irregular turbo codes would yield factor graphs with girths that gr ow as the logarithm o f the in terleaver size. Grap hs with large girths have been already used for the construction o f regular turbo codes [22] and LDPC codes [20]. The Pr ogressive Ed ge Gr owth alg orithm prop osed in [20] is b ased on a simple but very efficient id ea: it p rogressively establishes “best- effort” co nnection s in the graph, where a b est-effort co nnection correspo nds to an edg e m aximizing the gr aph girth. I n what follows, we extend this con struction to the ca se o f irregula r turbo codes. Th e c orrespon ding interleaver will be called PEG interleaver . The algorith m is sub mitted with th e set B = { b 1 , · · · , b K } of in formatio n bit nodes, the set T = { t 1 , · · · , t N } of tra nsi- tion no des, and a desired informatio n b it degree d istribution. According to th e submitted distribution, we can wr ite B as a disjoint union B = ∪ d max d =2 B d , where B d is be the set o f informa tion bits with submitted degree d . The algo rithm star ts with a factor g raph comp rising th e set B of inf ormation bit nodes, th e set T of tr ansition n odes and the co rrespon ding set of state n odes, ea ch tr ansition node b eing connected to its lef t and rig ht state nod es. At this mom ent there is n o connectio n be tween inform ation bit and transition node s. W e then p rogr essi vely add e dges emanatin g from bits in th e set B 2 , un til all the se bits r each th e submitted d egree 2 . W e next progr essi vely con nect th e bits from th e sets B 3 , . . . , B d max . It is impo rtant to notice that n o bit o f B d is co nnected , as long as there a re bits in B d − 1 that do n ot re ach th e subm itted degree ( d − 1 ). This is done in o rder to protect the bits of small degree in the following sen s: when inform ation bits of small degree ( e.g. d = 2 ) are connected , th e grap h girth is relatively large; th is will help to avoid sho rt cycles that co ntain only informa tion bit nodes of small degree. Adding mo re edges in th e graph , the g irth will decre ase. Whe n in formatio n bit nodes of higher degree are connected , we get smaller cycles, but the se cycles are better connected to oth er cycles in the graph. T o connect bit no des in B d we proceed as follows: Progr essi ve Edge Growth for B d f or i = 1 , . . . , d f or eac h b ∈ B d if i = 1 Choose a transi tion node t of lo w est deg ree in the curre nt graph and conne ct b to t else Expand the cur rent gra ph as a tree roote d at b , unt il al l transition nodes are in the tree. Iden tify tr ansition nodes tha t are conne cted to at most k − 1 information bits in the current graph. Among these transiti on nodes, identify those of maximal depth in the tree. Among these last identified transitio n nodes, choose a transiti on node t of lowe st degree in the current graph and connec t b to t end end end Note th at, we first add a n edge for each bit in B d , then a second edge for each bit, and so on, u ntil all b its r each the degree d . In case that the pu ncturing pattern is kn own at the time of the interleaver co nstruction, we can use a slightly modified version of the above algorithm as f ollows: first, each bit b ∈ B is co nnected to an unp unctured transition node of lowest degree in the cur rent grap h (we say that a transition node is un punctu red if its parity b it is unpun ctured) . I n case that the nu mber of u npun ctured tran sition node s is less than the number of in forma tion bit nod es, we co nnect the in formatio n bits of smallest subm itted degree. Finally , the interleaver is constructed by runnin g the above PEG algor ithm, but startin g from this grap h. In th is way , we make sur e th at if the n umber of unpun ctured transition no des is greater than the n umber of informa tion bit nodes, then a ny inform ation bit is con nected to at least an unpu nctured transition node. Now that we have co nstructed the PEG- based interleaver , we can derive a lower -bound o f the girth of the correspon ding factor graph . Sim ilar to the proof o f Theo rem 1 in [2 0], we can easily show that: g ≥ 2 0 @ 6 6 6 4 log h N ( k + 2) “ 1 − 1 d max ” − N + 1 i log [( d max − 1)( k + 1)] − 1 7 7 7 5 + 2 1 A (32) Otherwise for mulated, the girth o f a n irregular PEG in ter- leav ed facto r graph increases with the loga rithm of th e in- terleaver size, which is optimal acco rding to (31). Over the BEC, the p erform ance under iterative decod ing is determined by the m inimum stopping set [1] [9] of the factor graph of th e code, th e size of which is u pper-bound ed by its minimum distanc e. As this minimum stopping set is in g eneral related to the girth of a factor gr aph [26], we would expect its size increases with the gir th. Moreover , the m inimum distance of a tu rbo code g rows loga rithmically with the interleaver size [24]. The r ate o f growth of the min imum stop ping set of turbo codes u nder PEG- based in terleaving would thu s be optimal. V I . S I M U L AT I O N R E S U LT S In this sectio n, frame erro r rate per forman ce of ir regular turbo codes over the BEC is shown. Althoug h the analysis so far is based on the BCJR algorithm tha t supposes sof t informa tion exchange, it was shown in [8] that a har d-inpu t hard-o utput (HI HO) d ecoding algor ithm ( namely the V iterbi algorithm [2 7]) f or convolutional c odes is o ptimal in terms of bit err or prob ability over the BEC. For this reason, we will use a HIHO decodin g algorithm for irregular turbo cod es inspired by the algorithm in [28] f or LDPC codes, in that it propagates in the trellis o f the tur bo co de by rem oving transition s in the same way edges are r emoved in a bip artite gr aph under message-passing decoding [29]. This decoding scheme ensures a decoding complexity linear in the interleaver size. Codes from T able I , althou gh h aving very high thresholds, suffer from high er ror floors ( > 10 − 2 ). Howe ver , they are we ll suited for applicatio ns fo r which th e quality criterion is the average inefficiency [29]. Fig. 4 shows the perfor mance of irregular turbo codes having a g ood threshold- error floor trade off. In order to avoid high error floors, the 8 -state RSC (1 , 15 / 1 3) 8 code was used as the constituen t code o f the ir regular turbo code. At a target frame error rate of abo ut 1 0 − 3 , irregular turbo codes are within 0 . 018 ≤ ∆ p ≤ 0 . 02 8 from capacity for various c oding r ates. 10 -3 10 -2 10 -1 10 0 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 FER p 0 (channel erasure probability) R c = 1/2 R c = 1/3 R c = 1/4 RSC(1,5/7), K = 10000 RSC(1,5/7), K = 50000 RSC(1,15/13), K = 10000 RSC(1,15/13), K = 50000 capacity Fig. 4. Performance of irregu lar turbo codes over the BEC. V I I . C O N C L U S I O N S W e propo sed irregular turbo cod es that perfo rm close to capacity for th e binar y erasure chan nel. The cod es oper ate for various coding rates, an d they provide low erro r floors with a PEG-based inter leaver that m aximizes the cycles in the graph ical rep resentation of th e cod e. I mplemente d with an “o n-the- fly” hard-input har d-outp ut decoding algorithm, we believe that these codes are suited f or software implementation in upper-layer forward err or corr ection (UL- FEC) con texts. R E F E R E N C E S [1] C. Di, D. Proietti, E T ela tar , T Richard son, and R Urbanke, “Fini te- length analysis of lo w-den sity parity -check codes on the binar y erasure channe l, ” IEEE T rans. Inf . Theory , vo l. 48, no. 6, pp. 1570–1579, 200 2. [2] T . Richard son, A. Shokrollah i, and R. Urbanke, “Design of capacity- approac hing irregul ar low-de nsity parity-chec k code s, ” IEEE T rans. Inf. Theory , vol. 47, pp. 619–637, 2001. [3] M.G. L uby , M. Mitzenmache r , M.A. Shokrolla hi, and D.A. Spielman, “Ef ficient erasure correc ting codes, ” IEEE T rans. Inf . Theory , vol. 47, no. 2, pp. 569–584, 2001. [4] H.D. Pfister , I. Sason, and R. Urbanke, “Capa city-a chie ving ensembles for the binary erasur e channel with bounded comple xit y, ” IEEE Tr ans. Inf. Theory , vol. 51, no. 7, pp. 2352–2379, 2005. [5] M. L uby , “L T codes, ” Pr oc. ACM Symp. F ou nd. Comp. Sci. , 2002. [6] A. Shokrollahi, “Raptor codes, ” IEEE/ACM T rans. Networki ng (TON) , vol. 14, pp. 2551–2567, 2006. [7] B.M. Kurkoski , P .H. Siege l, and J. K. W olf, “Exact probability of erasure and a decodi ng alg orithm for con vo lutiona l codes on the binary erasure channe l, ” IEEE GLOBECOM , 2003. [8] B.M. Kurkoski, P .H. Siegel , and J.K. W olf, “ Analysis of con volutiona l codes on the erasure channel , ” IEEE Int. Symp. Inf . Theory , 2004. [9] E. Rosnes and O. Ytrehus, “T urbo decoding on the binary erasure channe l: Finite-le ngth analysis and turbo stopping sets, ” IEEE T rans. Inf. Theory , vol. 53, pp. 4059–4075, 2007. [10] J eong W . Lee, R. Urbank e, and R.E. Blahut, “On the performance of turbo codes ov er the binary erasure channel , ” IEEE Comm. Lett. , vol. 11, pp. 67–69, 2007. [11] C. Berrou and A. Gla vieux, “Near opti mum error co rrectin g coding and decodin g: turbo-codes, ” IEEE T rans. Comm. , vol. 44, pp. 1261–1271, 1996. [12] J .J. Boutros, “Asymptotic beha vior study of irregu lar turbo codes, ” 7th Int. W ork . on DSP T ec h. for Space Comm. , 2001. [13] J . Bout ros, G. Cai re, E. V iterbo, H. Sawaya , and S. V ialle , “T urbo code at 0.03 dB from capacity limit , ” IEE E Int. Symp. Inf. Theory , 2002. [14] M. G. Luby , M. Mitzenmac her , M.A. Shokrolla hi, and D. A. Spielman , “Improv ed low-de nsity parity-che ck codes using irregul ar graphs, ” IEEE T rans. Inf . Theory , vol. 47, no. 2, pp. 585–598, 2001. [15] B.J . Fre y and D. MacKay , “Irregular turbo code s, ” IEEE Int. Symp. Inf. Theory , 2000. [16] L . Bahl , J. Cocke , F . Jelinek, and J. Ra vi v , “Optimal decod ing of linear codes for minimiz ing symbol error rate , ” IEEE T rans. Inf. Theory , vo l. 20, no. 2, pp. 284–287, 1974. [17] S. ten Brink, “Con ver gence behavi or of iterati vely decoded paralle l concat enate d codes, ” IEE E T rans. Comm. , vol. 49, no. 10, pp. 1727– 1737, 2001. [18] C. Measson and R. Urbanke, “Further analyt ic propertie s of exit-li ke curve s and appl icati ons, ” IEEE Int. Symp. Inf . Theory , 2003. [19] R. Storn and K. Price, “Dif ferent ial e vol ution: a simple and ef ficient adapti ve scheme for globa l optimizatio n ove r continuo us spaces, ” Jou r- nal of Global Optimi zation , vol . 11, no. 4, pp. 341–359, 1997. [20] X-Y . Hu, E. Eleftheriou, and D.M. Arnold, “Re gula r and irre gular progressi v e edge-gro w th tanner graphs, ” IEE E T rans. Inf. Theory , vol . 51-1, pp. 386–398, 2005. [21] N. W iber g, Codes and Decoding on gener al graphs , Ph.D. thesis, Link ¨ oping Uni v ersity , Sweden, 1996. [22] P .O. V ontobel , “On the co nstructio n of turbo code int erlea ve rs based on graphs with large girth, ” IEEE ICC , 2002. [23] J . Boutros and O. Pothier , “Con ve rgenc e analysi s of turbo decoding, ” Canadian W ork. Inf. Theory , 1997. [24] M. Breiling, “ A logarith mic upper bound on the minimum dista nce of turbo codes, ” IEE E T rans. Inf. Theory , vol. 50, pp. 1692–1710, 2004. [25] R.G . Gallager , Low density parity chec k codes , MIT Press, 1963. [26] A. Orlitsky , R. Urbanke , K. Vi swanat han, and J. Zhang, “Stopping sets and the girth of T anne r graphs, ” IEEE Int. Symp. Inf. Theory , 2002. [27] A. V iterbi, “Error bounds for con volut ional codes and an asymptotical ly optimum decoding algorithm, ” IEEE T rans. Inf. Theory , vol. 13, pp. 260–269, 1967. [28] M. G. Luby , M. Mitzenmac her , M.A. Shokrolla hi, D.A. Spielman, and V . Stemann, “Practica l loss-resilient codes, ” Proc. 29th Annual ACM Symp. T heory Comp. , 1997. [29] G. M. Kraidy and V . Savi n, “Minimum-delay decoding of turbo-codes for upper -laye r fec, ” submitt ed to SP A WC 2008, ava ilabl e on arXiv .org.