Erasure Codes with a Banded Structure for Hybrid Iterative-ML Decoding

Erasure Codes with a Banded Structure for Hybrid Iterati v e-ML Decoding Alexandre Soro 1 , Mathieu Cunche 2 , J ´ er ˆ ome Lacan 1 and V incent Roca 2 1 Univ . of T oulouse, ISAE/DMIA, 10 a venue Edouard Belin, BP 54032 - 31055 T oulouse cedex 4 - FRANCE 2 INRIA Rhone-Alpes, Plan ` ete team, Inov all ´ ee, 655 av . de l’Europe, Montbonnot, 38334 St Ismier cedex - FRANCE Email: { alexandre.soro,jerome.lacan } @isae.fr , { mathieu.cunche,vincent.roca } @inria.fr Abstract —This paper presents new FEC codes for the erasure channel, LDPC-Band, that hav e been designed so as to optimize a hybrid iterative-Maximum Likelihood (ML) decoding. Indeed, these codes feature simultaneously a sparse parity check matrix, which allows an efficient use of iterative LDPC decoding, and a generator matrix with a band structure, which allows fast ML decoding on the erasure channel. The combination of these two decoding algorithms leads to erasure codes achieving a very good trade-off between complexity and erasure correction capability . I . I N T RO D U C T I O N A N D R E L A T E D W O R K S For the transmission of data packets on erasure channels, linear binary FEC codes often offer the best compromise between fast encoding/decoding operations and a good level of erasure recovery capability . For example, random binary codes hav e a correction capability very close to channel capacity [1]. Unfortunately , the decoding complexity of random codes is often prohibitiv e because the decoding algorithm, named Maximum-Likelihood (ML) decoding, basically consists in solving a linear system, which can be done by in verting the associated matrix. This matrix represents the relations between the set of receiv ed symbols and the set of missing symbols, in our representation a row contains the coefficient used to build an encoding symbol. If k denotes the dimension of the code the in version has a complexity of O ( k 2 ) ro w operations. In the remaining of the paper the complexity will be ev aluated in terms of ro w operation. A row operation includes the sum of two rows of the matrix and the sum of the corresponding symbols. It is importance to notice that in pratical applications the size of the symbols can be up to several hundred bytes. T o reduce this complexity , Studholme and Blake [2] showed that similar erasure capability can be obtained when the non zero entries of the generator matrix are located in a band of length 2 √ k and where each column contains 2 log( k ) nonzero elements. W ith this improvement, the complexity of the decoding is reduced to O ( k 3 / 2 ) row operations. The class of Fountain codes, like L T [3] or Raptor [4] codes can also obtain very good performance in terms of erasure correction capability . An estimation of the performance achiev ed by Raptor codes with ML decoding is provided in [5]. Howe ver , despite the description of decoding algorithm This work was supported by the French ANR grant No 2006 TCOM 019 (CAPRI-FEC project). provided in [6], the complexity needed to achiev e this lev el of performance is not very clear . LDPC codes are another class of binary codes providing good lev el of decoding performance with extremely fast encoding/decoding algorithms [7][8]. Indeed, the classical iterativ e decoding algorithm for the erasure channel, based on the work of Zyablov [9], has a linear decoding complexity . The drawback of this algorithm is that it does not reach the performance of ML decoding. Moreover , the sparsity of the matrices also reduces the ML performance compared to full random matrices. Recently , two independent works [10][11] proposed a hy- brid iterative-ML decoding algorithm, where ML decoding is only used when the iterati ve decoding does not succeed to decode a receiv ed codew ord. The ne w codes we introduce in this paper also rely on a hybrid decoding. Our goal is to build these codes in such a w ay that both the parity check matrix and the associated generator matrix hav e good properties for the iterative and ML decoding schemes. More precisely , the parity check matrix should be sparse in order to yield good performance with the iterative decoding. At the same time, the generator matrix should hav e a banded structure in order to reduce the computational complexity of the ML decoding scheme, as explained in [2]. Thanks to a polynomial representation of the rows and the columns of the generator and parity check matrices, we introduce a method that enabled us to build such codes. The idea of windo w based encoding in LDPC codes has been proposed by Hak en, Luby and al. in patent [12]. Howe ver , the goal of this patent is only to minimize memory access dur- ing encoding, by localizing the access in a window that slides ov er the input file. Independantly of whether our proposal falls into the scope of this patent or not, we see that, from a purely scientific point of vie w , the goal of [12] completely departs from the approach discussed in the current paper , as well as the theoretic tools we introduce to achiev e our goals. The paper is organized as follows: we detail the polynomial approach for the construction of our LDPC-Band codes in Section II. In Section III, we analyse the erasure correction capability of the proposed code and we provide simulation results. Finally , in Section IV we ev aluate the complexity of the hybrid decoding. Then we conclude. I I . C O N S T RU C T I O N O F T H E C O D E A. General structure of the code Let G be the systematic binary generator matrix of the banded code: G = ( I d | M ) Let H be its associated parity check matrix: H = ( A | U ) Let k be the dimension code and n the length of the code. It follows that G is a k × n matrix, A is a ( n − k ) × k matrix and U is a ( n − k ) × ( n − k ) binary matrix. Let us define U as a lo wer triangular T oeplitz matrix defined by its first column (1 , u 1 , . . . , u n − k − 1 ) . Since all the diagonal elements are equal to 1 , U is a full rank matrix. In addition, let us consider the associated polynomial u ( x ) = 1 + P n − k − 1 k =1 u k x k . The coefficient of U are the following: { u i,j } =    u i − j if i > j 0 if i < j 1 if i = j U =                1 0 0 . . . 0 0 u 1 1 0 u 2 u 1 1 . . . . . . . . . u 2 u 1 . . . u i . . . u 2 . . . 0 . . . u i . . . 1 0 u n − k − 1 . . . u i u 1 1                The relations between the generator matrix G and its asso- ciated parity check matrix H : G × H T = 0 give: M = ( U − 1 A ) T U M T = A Let B be the band width of the band matrix M studied here, which will hav e the following form: M =    m 0 , 0 m 0 , 1 . . . m 0 ,B − 1 0 . . . 0 0 m 1 , 0 m 1 , 1 . . . m 1 ,B − 1 0 . . . . . . 0 . . . . . . . . . . . . 0    For i = 0 , . . . , k − 1 , let us define the polynomial m i ( x ) = P B − 1 j =0 m i,j x j corresponding to the i th row of M . Since A = U M T , it can be shown that, for i = 0 , . . . , k − 1 , the i th column of A is characterized by the polynomial a i ( x ) such that a i ( x ) = u ( x ) m i ( x ) (see Figure 1). This result is important as it allo ws to construct banded generator matrices from particular distributions in the parity check matrix. Let us define the Hamming weight of a polynomial as the number of monomials. Then, the number of non-zero elements of a column of A is the weight of a i ( x ) , noted W ( a i ) . H =                a 0 , 0 u 0 a 0 , 1 a 1 , 0 u 1 u 0 a 0 , 2 a 1 , 1 a 2 , 0 u 2 u 1 u 0 a 0 , 3 a 1 , 2 a 2 , 1 u 3 u 2 u 1 . . . a 1 , 3 a 2 , 2 . . . . . . u 3 u 2 . . . . . . a 2 , 3 . . . u 3 . . . . . .                Fig. 1. The parity check matrix of the code with a band structure B. Design of the matrices In order to build a code supporting hybrid iterati ve/ML decoding, a first constraint is to optimize the iterative decoding on the parity check matrix. This decoding is very sensitive upon the degree of the symbol and check nodes, i.e. the weight of the columns and the rows of the parity check matrix. Thanks to the polynomial approach, these parameters can be controlled. Indeed the weight of the columns and rows of U is defined by the number of monomials in u ( x ) , and the weight of the column of A is defined by the number of monomials in the different a i ( x ) . W e will see later ho w the ro w weight can be controlled. The second constraint concerns the ML decoding. As explained before, the generator matrix of this code must be band of width B in order to reduce the ML decoding complexity . W e do not impose any other constraint on the generator matrix as we assume that the matrix is sufficiently random to allow good correction capabilities under ML decoding. In order to support an efficient iterative decoding, the columns of the parity check matrix must have a small weight. In particular we must choose a small weight polynomial for u ( x ) , the polynomial defining U the right side of the parity check matrix. The main point of the process is to find polynomials m i ( x ) of degree close or equal to B , such that a i ( x ) ha ve few monomials, where a i ( x ) = m i ( x ) u ( x ) . Let us call these polynomials candidate polynomials . Finding these candidate polynomials can be achiev ed with an exhausti ve search, in advance, once the v arious code parameters (n, k, B) are known. The details of how this search is done does not impact the decoding efficiency and therefore are not described in the reminder of this paper . Once a polynomial u ( x ) and a set of candidate polynomials hav e been selected, the weight of the columns of parity check matrix is fix ed. But the weight of the rows can still be modified to fit a specific value. Indeed by permuting two polynomial in M , or by e xchanging a polynomial with another candidate polynomial of same weight, we can modify the weight of sev eral rows without changing the weight of the columns. C. Optimizations W e have now defined a way to implement banded codes for both decoding techniques. Ho wever this implementation is limited to a number of encoding symbols of exactly n = 2 k + B . Furthermore, these codes turn out to be quite inef ficient for the ML decoding in terms of erasure recovery capability , because the first non-systematic symbols only protect a small part of the source symbols. This is a negati ve side ef fect of the approach. W e no w explain how to solve this problem. Giv en a polynomial u ( x ) and a bandwidth B , the set of candidate polynomials usualy contains several elements. Therefore it is possible to choose different polynomials and to interlace them to build the matrix M , and the matrix A inherits from a corresponding interlacement in its polynomial. This technique has two main benefits. The first one is an im- prov ement of the decoding performances for the ML decoding, with limited impacts on the iterative decoding. Indeed, many polynomials interlaced produce more v ariety in the generator matrix and av oid diagonals and regularities that damage the performances of the ML decoding. The second benefit is the suppression of the side effect mentionned abov e. The idea is to suppress the B 2 first and last columns of M . As it breaks the polynomial relations between A , U and M because of the first and last B 2 lines of M that are now truncated, we replace these lines with candidate polynomials which are of degree B 2 at maximum, in order to keep the matrix banded, and keep the constraints upon A and U . As a consequence, the encoding packets at the edge of the non-systematic part of the generator matrix are now connected to more source symbols. This also suppress the influence of the band size on the code rate, as the number of symbols produced is now n = 2 k , meaning a code rate strictly equal to 1 / 2 . Fig. 2. Design of the banded matrix of the non-systematic part with side effect remo ved The last point is to handle different code rates. T o that goal, we define the f parameter which corresponds to the column shift between consecutive lines of M . For instance, for a ratio of r = 1 / 2 , each successiv e line is shifted by 1 position from the previous one, i.e. f = 1 . In order to handle other ratios, we need to take f = 1 /r − 1 . Moreov er , in this case, the first and last B 2 f lines are of degree B 2 . If f is not an integer , we should use a family of integer offsets { f i } such as the av erage value of this set is equal to f . T o summarize, in this section we have defined a way to create erasure band codes that handle efficiently (both in terms of erasure recovery capability and algorithmic comple xity) both iterati ve (thanks to the sparseness of the parity check matrix) and ML decoding (thanks to its banded structure), for different code rates. I I I . C O R R E C T I O N C A PA B I L I T Y A N A L Y S I S A. Theoretical Analysis Influence of the band on the ML performances: For the ML performances on the generator matrix G , the crucial point is to determine a band size that ensures both a quick decoding and good correction capabilities. W ith the first constraint, the band size used should be as narrow as possible. Howe ver , this size should also be sufficient in order to avoid an explosion of the decoding overhead. It is important to notice that the band size corresponds to the maximum degree of the associated binary polynomials m i ( x ) . Studholme and Blake [2][1] hav e studied the influence of the window size on binary random matrices constrainted in a windo w . The conclusion of their work is that for random binary matrices constrained in a window , its size has to be kept close to 2 √ k . Moreov er , in this window , the number of non-zero elements must be at least 2 log k . These values guarantees that the probability of being full rank is close to the one of pure random matrices. As our approach uses specific configurations of banded matrices, this value should be considered as a minimum. The band size in our case has to be kept as lo w as possible and for long dimensions, as close as possible to this value, because it will have a great impact on decoding speed. The size of the band will then be a tradeof f between a good ML decoding speed and a strong ML decoding performance. Density evolution analysis: A density ev olution approach [13] for the design of this code has been studied, but hav e faced many obstacles. If we choose to consider a infinite matrix with an infinite band width, the density e volution will give the same distributions as for a standard Repeat- Accumulate code. But building a finite length matrix having a such distrib ution can be difficult as the rows distrib ution strongly depends on the choice of the polynomials and their permutations. Furthermore as the band is small, a strong irregularity in the joint degree distrib ution [14] may appear . On the other hand if we choose an infinite matrix with a finite band width, the hypothesis of infinite length cycle does not hold anymore. B. Simulation Results In this section, we compare the correction capability of our codes with standard LDPC-Staircase codes. First of all, it is worth noticing that we will use band codes with multiple polynomials interlaced. Indeed, experiments ha ve illustrated the fact that using only one polynomial degrades greatly the decoding performances and will not be discussed here. The crucial point is now to select a set of polynomials that will respect the conditions in the band, but also that hav e great performances in both decoding. In this study , we hav e tried to keep as close as possible of iterativ e adapted distributions. Howe ver , because of the additional conditions of banded matrices, an evolution density approach is not straightforward. It is also worth noticing that the chosen set of polynomials is always a compromise between performance of iterativ e decoding and ML decoding. This allows the user to adapt the polynomials to the environment of the channel. Experiments hav e also shown that the way the polynomials are interlaced and their position has only a little impact on the ML decoding performances, ev en for a random interlacing. The results presented in T able I and T able II are based on a regular interlacing for B = 100 and B = 200 for a code rate of 1 / 2 . W e study here, the average overhead required in order to successfully decode the receiv ed symbols. Decoding scheme Iterativ e Maximum-Likelihood LDPC-Staircase N 1 = 5 14.24% 1.21% LDPC-Band - B = 100 18.39% 2.97% LDPC-Band - B = 200 14.75% 1.24% T ABLE I A V E RA GE OV ER H E A D W . R .T DE C O D IN G S C HE M E , K = 1 00 0 C O DE R A T E = 1 /2 Decoding scheme Iterativ e Maximum-Likelihood LDPC-Staircase N 1 = 5 13.95% 1.15% LDPC-Band - B = 200 16.23% 1.19% T ABLE II A V E RA GE O VE R H EA D W . R .T D E C OD I N G S C H EM E , K = 20 0 0 C O D E R A T E = 1 /2 As expected, the band size is a crucial factor for the decoding performances. A band size of 100 is less performant but will ha ve a greater decoding speed. In order to have efficient systems, we can see that the band size has to be large compared to the W indowed Erasure Codes. These results also show that LDPC-Band codes are really close to standard LDPC-Staircase codes, while being more constrained. I V . C O M P L E X I T Y A NA LY S I S A. Theoretical Analysis A ke y point in data transmission is the encoding and decoding speed of the code. In this study we do not include the creation of the code in the complexity analysis. Indeed, the generation of the generator and parity check matrices is straightforward and can be done out of line. On the decoder side, both decoding algorithms apply . The first one is an iterativ e decoding on the parity check matrix. This decoding is fast and has a linear complexity on the dimension of the code O ( k ) . When the iterativ e decoding fails, a ML decoding is used on the generator matrix. The complexity of this algorithm is in the general case O ( k 2 ) row operations. Howe ver the decoding of LDPC-Band codes benefits from the band structure. Thanks to this structure, an optimized LU decomposition of the matrix [15] leads to a complexity of O ( kB ) ro w operations. This means that for k = 2000 source symbols and a band size of B = 200 symbols, the theoretical speed gain compared to a classical ML decoding will be about 10 100 1000 10000 30 35 40 45 50 average/min/max Bitrate (Mbps) Loss probability (%) LDPC-Band, band width=200 LDPC-Staircase Windowed Erasure, window size=89 Fig. 3. Bitrate W .R.T loss probability , k=2000 code rate=1/2 and symbol size=1024 bytes one order of magnitude. The complexity obtained here has to be compared with the complexity of O ( k 3 / 2 ) ro w operations of the Studholme and Blake approach. Howe ver , in practice, the size of the matrices that need to be in verted is lower than k . There are three reasons: (1) whenev er a source symbol is receiv ed, the corresponding row in the generator matrix is remov ed. Then (2), only the columns of the non-systematic part that are recei ved are used. It means that for a code rate r the system that has to be in verted is only in the order of (1 − r ) k × (1 − r ) k . Finally (3) the iterativ e decoding may hav e rebuilt some missing symbols, thereby reducing the system size. B. Simulation Results W e carried out several tests to assess the computation benefits of our proposal. These tests were obtained on a 4xIntel Xeon5120 @ 1.86GHz/4 GB RAM/Linux PC. W e compared the LDPC-Band codes with a band width of 200 with two other codes: 1) the LDPC-Staircase codes, regular repeat accumulate codes standardized by the IETF [8]. Follo wing the opti- misation for the hybrid iterative/ML decoding proposed in [16], the degree of the source symbol nodes is set to 5. 2) the random windowed codes proposed in [2]. These codes can be seen as a non systematic LDGM codes with 2 l og ( k ) elements per column. Because it is a non-systematic code, iterativ e decoding cannot be used (unlike standard LDGM codes). Figure 3 shows the average decoding bitrate as a function of the loss probability of the channel for the various code. The LDPC-Band and the LDPC-Staircase performances can be divided into two parts. In the region where the loss probability is low , the iterati ve decoder is sufficient to recover the losses and the bitrate is high. In the region of high loss probability the ML decoder is needed and the bitrate is reduced. On the opposite, the decoding speed of Windo wed code does not depend on the loss probability , since it can only be decoded with a ML decoder . W e see that when the iterativ e decoding is used, the LDPC- Staircase and the LDPC-Band present comparable decoding speed (with a slight adv antage to LDPC-Staircase codes), and they are both significantly faster than Windo wed codes (that need ML decoding). When ML decoding is needed, LDPC-Band codes outper- form the other two codes: they are roughly 2 times faster than the W indowed codes, and roughly 4 times faster than LDPC- Staircase codes. Let us remind that the ML decoding of the LDPC-Band and the LDPC-Staircase is equi v alent to solve a linear system of size (1 − r ) k × (1 − r ) k , whereas the ML decoding of the windowed erasure codes requires to solv e a system of size k × k (being non systematic, no source symbols is received and the system still has k unkno wn variables). T able III present the av erage ML decoding time for the three compared codes as a function of the number of source symbols k . The LDPC-Band is always faster than the two other codes. Furthermore the ML decoding speed falls more quickly with the two other codes than with the LDPC-Band. The decoding speed with an iterativ e decoder are sho wn in T able IV. The LDPC-Band is slower than the LDPC-Staircase but remains very close to it under iterativ e decoding. k 1,000 2,000 4,000 LDPC-Band 326 Mbps 235 Mbps 150 Mbps LDPC-Staircase 125 Mbps 60 Mbps 30 Mbps W indowed Erasure 220 Mbps 120 Mbps 68 Mbps T ABLE III A V E RA GE M L D E CO D I N G T I ME A S A F U N C TI O N O F T H E O B JE C T S I Z E , F O R A C O D E R A T E =1 / 2 A N D S Y M B OL S I Z E = 1 0 24 B Y T ES . k 1,000 2,000 4,000 LDPC-Band 1100 Mbps 1050 Mbps 900 Mbps LDPC-Staircase 1300 Mbps 1200 Mbps 1000 Mbps T ABLE IV A V E RA GE I T E R A T I VE D E C O DI N G T I M E A S A F U N CT I O N O F T H E O B JE C T S I ZE , F O R A C O DE R A T E = 1 / 2 A N D S Y M B OL S I Z E = 1 0 2 4 B Y T ES . V . C O N C L U S I O N In this paper we proposed a flexible scheme that allows the construction of generator and parity check matrices and their efficient hybrid decoding. The iterativ e decoding on the parity check matrix provides a fast way to recover the source sym- bols, as decoding complexity is linear . When the en vironment is too harsh and the loss rate is close to the recovery capability of the codes, a Maximum-Likelihood decoding is applied on the structured generator matrix. Thanks to the band structure of this matrix, the ML decoding complexity is reduced to O ( k B ) row operations. This property allo ws the hybrid decoder to sustain high decoding speeds, ev en in high loss en vironments. Thanks to the polynomial representation of the matrices, we presented a practical way of building the matrix of such codes. Our results shows that our LDPC-band codes match the performances of LDPC-staircase codes in terms of iterati ve decoding speed, while being much faster than these codes when ML decoding is required. Furthermore the LDPC-band codes exhibit erasure recov ery capabilities close to standard repeat-accumulate codes using iterativ e decoding, and are close to the channel capacity when decoded with a ML scheme. In future works, a density e volution study should enable us to further improve the erasure correction capabilites of the iterativ e decoding scheme, without degrading the ML decoding capability . R E F E R E N C E S [1] C. Studholme and I. Blake, “Random matrices and codes for the erasure channel, ” Algorithmica , April 2008. [2] ——, “Windo wed erasure codes, ” Information Theory , 2006 IEEE International Symposium on , pp. 509–513, July 2006. [3] M. Luby , “ LT codes, ” in Proceedings of the 43rd Annual IEEE Symposium on F oundations of Computer Science, FOCS’2002 , 2002, pp. 271–280. [4] A. Shokrollahi, “Raptor codes, ” IEEE Tr ansactions on Information Theory , vol. 52, no. 6, pp. 2551–2567, 2006. [5] M. Luby , T . Gasiba, T . Stockhammer, and M. W atson, “Reliable multi- media download delivery in cellular broadcast networks, ” Br oadcasting, IEEE T ransactions on , vol. 53, no. 1, pp. 235–246, March 2007. [6] M. Luby , A. Shokrollahi, M. W atson, and T . Stockhammer, “Raptor Forward Error Correction Scheme for Object Delivery , ” Internet Engi- neering T ask Force (IETF), T ech. Rep. RFC 5053 (Proposed Standard), Oct. 2007. [7] C. Di, D. Proietti, T . Richardson, E. T elatar , and R. Urbanke, “Finite length analysis of low-density parity-check codes on the binary erasure channel, ” IEEE Transactions on Information Theory , vol. 48, pp. 1570– 1579, 2002. [8] V . Roca, C. Neumann, and D. Furodet, “Low Density Parity Check (LDPC) Staircase and T riangle Forward Error Correction (FEC) Schemes, ” Internet Engineering T ask Force (IETF), RFC RFC 5170 (Proposed Standard), june 2008. [9] V . V . Zyablov and M. S. Pinsker , “Decoding complexity of low-density codes for transmission in a channel with erasures, ” Pr obl. P eredac hi Inf. , vol. 48, pp. 18–28, 1974. [10] M. Cunche and V . Roca, “Improving the decoding of ldpc codes for the packet erasure channel with a hybrid zyablov iterati ve decoding/gaussian elimination scheme, ” INRIA Research Report RR-6473, T ech. Rep., march 2008. [11] E. Paolini, G. Liv a, B. Matuz, and M. Chiani, “Generalized ira erasure correcting codes for hybrid iterative/maximum likelihood decoding, ” Communications Letters, IEEE , vol. 12, no. 6, pp. 450–452, June 2008. [12] A. Haken, M. Luby , G. Horn, J. Byers, J. Persch, and M. Mitzenmacher , “On demand encoding with a window, ” US Patent 6486803, Nov . 2002. [13] T . J. Richardson, M. A. Shokrollahi, and R. L. Urbanke, “Design of capacity approaching irregular low density parity check codes, ” IEEE T rans. Inform. Theory , vol. 47, no. 2, pp. 619–637, 2001. [14] K. Kasai, T . Shibuya, and K. Sakaniwa, “Detailed representation of ir- regular ldpc code ensembles and density ev olution, ” Information Theory , 2003. Proceedings. IEEE International Symposium on , pp. 121–, June-4 July 2003. [15] G. H. Golub and C. F . v an Loan, Matrix Computations , 3rd ed. Johns Hopkins Univ ersity Press, 1996. [16] M. Cunche and V . Roca, “Optimizing the error recovery capabilities of ldpc-staircase codes featuring a gaussian elimination decoding scheme, ” in 10th IEEE International W orkshop on Signal Processing for Space Communications (SPSC’08) , october 2008.