Woven Graph Codes: Asymptotic Performances and Examples

1 W o v en Graph Codes: Asymptotic Performances and Examples Irina E. Bocharo va and Boris D. Kudryashov Dept. of Information Syst ems, St. Petersb urg Univ . of Information T echnologies, Mechanics and Opti cs St. Petersb urg 197101, Russia Email: { iri na, boris } @eit.lt h.se Rolf J ohannesson Dept. of Electro and Information T echnology , Lun d University P . O. Box 118, SE-22100 Lund, Sweden Email: rolf@eit.lth. se V ictor V . Zyabl ov Inst. for Information Transm. Problems, Russian Academy of Sciences Moscow 101447, Russia Email: zyablov@iitp.ru Abstract Constructions of woven grap h codes based on co nstituent block and co n volutional codes ar e studied. It is shown that within th e ran dom ensemble of such cod es based on s -partite, s -un iform hypergrap hs, where s depend s only on the code rate, there exist cod es satisfying the V ar shamov-Gilbert (VG) and the Costello lower bou nd on the min imum distance and the free distance, respectively . A connec tion between regular bipartite graphs a nd tailbiting codes is shown. Som e examples of woven graph c odes are presented. Among them an example of a rate R wg = 1 / 3 woven graph code with d free = 32 based o n Heawood’ s bipartite graph and containin g n = 7 constituen t rate R c = 2 / 3 conv olutional cod es with overall con straint lengths ν c = 5 is gi ven. An encoding proc edure f or woven graph codes with co mplexity propo rtional to the numb er of constituen t codes an d the ir overall con straint length ν c is p resented. Index terms —Con voluti onal codes, girth, graphs, graph codes, hyper graphs, LDPC codes, tailbiting codes, w oven codes. I . I N T RO D U C T I O N W ov en graph cod es can be c onsidered a s a ge neralization of low-density parity-check (LDPC) bloc k cod es [1]. Their structure a s graph code s ma kes them suitable for iterativ e dec oding. Moreover , the LDP C block co des are known as co des with low- complexity dec oding an d they ca n be considered a s competitors to the turbo code s [2] wh ich are sometimes called pa rallel conc atenated codes. As me ntioned in [3], the u nderlying graph defines a permutation of the information sy mbols which resembles the interleaving in turbo coding sch emes. On the other h and, similarly to the LDPC c odes, g raph c odes u sually hav e minimum dis tances e ssentially smaller than tho se of the b est kn own l inear c odes of the sa me parameters. At a fi rst g lance, the minimum distance of a graph code does no t play an important role in iterativ e deco ding sinc e the error-correcting c apability of this s uboptimal procedure is often less than that guaran teed by the minimum distance . Ho we ver , in g eneral, the belief-propaga tion decoding algo rithms work better if the girth o f the underlying g raph is large, that is, if the minimum distance of the graph code is large [4]. In the sequ el we distinguish betwee n graph , graph-ba sed, and wo ven graph c odes. W e sa y tha t a graph code is a block co de wh ose pa rity-check ma trix co incides with the incide nce matrix of the corresp onding graph. Grap h-based codes con stitute a clas s o f concaten ated c odes with cons tituent block codes con catenated with a graph code (see , for example, [3]). Ea ch vertex in the underlying graph corresp onds to a constituent block code. The main feature of these codes is that the block length of the ir constituent block codes coincides with the degree of the unde rlying graph. 2 W e introduc e woven graph codes wh ich a re, in fact, graph -based c odes with co nstituent bloc k c odes wh ose block length is a multiple of the graph degree c , that is , the ir block len gth is l c , where l is an integer . In particular , when l tends to infinity we obtain co n v olutional con stituent code s. Distance properties of bipa rtite graph-based c odes with co nstituent b lock codes we re studied in [3]. It was shown tha t if the minimum distan ce of the c onstituent b lock code s is lar ger than o r eq ual to 3 , then there exist asymptotically g ood code s with fixed constituent codes amon g these graph-base d codes. Also it was shown in [3] that for some range of rates, random graph-ba sed cod es with bloc k constituent codes satisfy the VG bo und whe n the bloc k leng th of the c onstituent code s tends to infinity . One d isadvantage of graph-bas ed c odes tha t beco mes apparent in the a symptotic analysis is that good performances can only be achieved when the bloc k length of the constituent block codes (which in this case coincides with the graph degree c ) tends to infinity . In practice this leads to rathe r long graph-ba sed codes with not only rather high decoding complexity of the iterativ e dec oding procedures b ut also high encoding c omplexity . In this p aper , we con sider a class of the generalized graph-bas ed codes which we call woven graph codes with constituent block and con volutional c odes. They are ba sed on s -partite, s -uniform hyper graphs . Notice that grap h- based code s with cons tituent block codes b ased on hype r graphs were c onsidered in [5], [6 ]. It is mentioned in [5 ] that Ga llager’ s LDPC c odes are graph c odes over hypergraphs. W e cons ider first woven graph code s with c onstituent ( l c, lb ) block codes . A produc t-type lower bound on the minimum distance of such codes is deri ved. In order to analyze their asymptotic performances we mod ify the approach used in [3] to s -partite, s -un iform hy pergraphs and c onstituent ( l c, l b ) block co des 1 . It is shown that whe n l grows to infinity in the random ensemb le of w oven graph c odes with binary constituent block codes we can find s ≥ 2 such tha t there exist codes s atisfying the VG lower bound o n the minimum distance for any rate. In order to gen eralize the asymptotic a nalysis to wov en g raph codes with constituent conv olutional cod es we assume that the binary constituent b lock code is chosen a s a zero-tail (ZT) termi nated conv olutional c ode and consider a seq uence of ZT con v olutional code s of increasing block length l . It is shown that when the overall constraint length of the woven grap h c ode tends to infinity in the rando m ens emble of suc h c on v olutional c odes we can find s ≥ 2 suc h tha t there exist codes satisfying the Costello lo wer bound on the free distance for a ny rate. W e also des cribe the cons tituent con volutional code s as block c odes over the fie ld of b inary L aurent se ries [8]. This desc ription as we ll as the notion of block Hamming distance [9] of con volutional codes is us ed to deriv e a product-type lower bou nd on the free distance of woven graph codes with con stituent con voluti onal codes and to construct examples of such wov en code s with rate R wg = 1 / 3 . For a giv en hypergraph the free distance of the wov en graph code dep ends on the numbe ring of code symbols asso ciating to the hypergraph vertices. By a search over a ll possible permutations of the constituent code we found an example o f a rate R wg = 1 / 3 woven graph code with overall constraint length ν = 64 and free dis tance d free = 32 . The rate R wg = 1 / 3 woven graph co de is based on Heawood’ s bipartit e grap h [10] , [11] and contains constituent con voluti onal codes with overall con straint length ν c = 5 and free distan ce d c free = 6 . W e co nsider also the en coding prob lem for graph and woven grap h codes. The traditional enc oding technique for graph cod es has c omplexity O ( N 2 ) , where N is the blockleng th. W e show b y examp les that so me regular block graph code s are quasi-cyclic a nd thereby ca n be interpreted as tailbiting (TB) co des (see, for example, [12], [13]). It is kno wn that t he encoding complexity of su ch code s is proportional to the overall con straint length o f the parent con voluti onal c ode. By using a TB representation for the graph code we ca n construct an e xample of an enco der for a wov en graph code that is also represented in the form of a TB code but with overall con straint length less than or equal to 2 nν c , where n is the nu mber of constituent con v olutional co des with overall constraint length ν c each. In Section II, we c onsider some properties of s -partite, s -uniform, c -regular hypergraphs. W e define woven g raph codes with constituent block codes as well as with constituent con volutional codes and obtain product-type lower bounds on their minimum and free distances. Then, in Section II I, we deri ve a lo wer bound on the free dis tance o f the random ens emble o f woven graph code s. In Section IV , examp les o f woven graph code s a re given. W e c onclude the paper by conside ring encoding techniqu es for g raph co des and woven graph codes in Sec tion V . 1 When we were prep aring this paper we were informed that the possibility of achie ving the VG bou nd b y considering hyp ergraphs was kno wn to A. Barg [7]. 3 I I . P R E L I M I N A R I E S A hype r graph is a generalization of a g raph in which the edges are subse ts of vertices and may c onnec t (co ntain) any n umber of vertices. The se e dges are ca lled hypered ges. A hyp ergraph is called s -uniform if every hy peredge has ca rdinality s or , in other words, c onnects s vertices. If s = 2 the hypergraph is simply a grap h. The d e gr ee of a vertex in a hyp ergr aph is the n umber of h yperedge s that are connecte d to (contain) it. If all vertices h av e the same d egree we say that this is the degr ee of the hype r graph . The hy pergraph is c -r e gular if every vertex has the same d egree c . Let the set V of verti ces of a n s -uniform hype r graph be partitioned into t d isjoint subs ets V j , j = 1 , 2 , . . . , t . A hypergraph is s aid to be t -pa rtite if no edge conta ins two vertices from the same s et V j , j = 1 , 2 , . . . , t . In the sequel we conside r s -pa rtite, s -uniform, c -re gular hypergraphs. Such a hyper graph is a union of s disjoint subsets of vertices. Ea ch v ertex has no c onnections in its o wn set an d is connected wit h s − 1 vertices in the o ther subsets . In Fig. 1 a 3 -partite, 3 -uniform, 4 -re gular hypergraph is s hown. It con tains three sets of verti ces. They a re 0 1 2 3 4 5 6 7 10 9 11 8 Fig. 1. A 3-partite, 3-uniform, 4-regu lar hypergraph. shown by triangles, re ctangles, an d ovals, r espe cti vely . There are no edges conne cting vertices inside any of these three sets. The vertices are c onnected b y hyp eredges each of which conne cts three vertices. A c ycle of length L in the hype r graph is a n alternating sequ ence of L + 1 vertices and L hype redges where all vertices a re dis tinct e xcept t he initi al and the final vertex, which coincide, and all edges are distinct. The girth of a hypergraph is the length of its shortest cycle. In Fig. 2 we s how a subgrap h that contains the sh ortest cycle of the 3 -partite, 3 -uniform, 4 -re gular hyper graph in Fig. 1. It cons ists of the vertices 5 , 1 0 , and 5 a nd ha s girth equal to 2. W e introduce the notion of a c ompact ( ≥ d ) -connected su bgraph in the hy pergraph. It is a conn ected subgraph in which ea ch vertex is incident with at least d hy peredges . W e call the length (number of hypered ges) of t he shortest compact s ubgraph its ( s , d )-girth . In Fig. 2 the h yperedge s belonging to the sho rtest ( ≥ 2) -compact subgrap h are marked by circles. It is easy to see that ( 3 , 2 )-girth is 6 . A 2 -part ite, 2 -uniform hypergraph is a bipa rtite graph. For such a hypergraph the ( 2 , 2 )-gir th is equa l to the gir th and a compact subgraph is a cycle. He awood’ s bipartite graph [10], [11] wi th 1 4 vertices and 2 1 edges is shown in Fig. 3. This grap h contains a s et of n = 7 black and a set of n = 7 white vertices. Each vertex has no c onnections within its own set a nd is conn ected with c = 3 vertices from the other set. The girth of the Heawood graph is 6 . A. Graph-bas ed codes and graph codes In order to illustrate the structure of a binary graph-ba sed block c ode with constituent block codes we represent the Heawood bipartit e graph us ing a so-ca lled T an ner graph [15] a s sh own in Fig. 4. W e introdu ce a set of nc = 21 (variable) vertices which c orrespond to the code symbols. Ea ch of the 2 n = 14 (constraint) vertices on the ri ght- an d left-hand sides c orresponds to one of 14 parity checks . The c = 3 e dges 4 0 1 2 5 6 7 10 9 8 Fig. 2. A shortest compact subgraph . 2 3 4 5 6 7 8 9 1 0 11 12 13 0 1 Fig. 3. Heaw ood’ s bipa rtite graph. leaving one c onstraint vertex correspo nd to a codeword of the constituent ( c, b ) b lock co de of rate R c = b/c . The parity-check matrix of the co rresponding graph -based code with bina ry cons tituent block codes is H gb =  H 1 H 2  (1) where the parity-chec k matrix H 1 of s ize n × nc = 7 × 21 has the form H 1 =      H c 0 0 0 0 0 0 0 H c 0 0 0 0 0 . . . . . . . . . . . . . . . . . . . . . 0 0 0 0 0 0 H c      where H c is a size ( c − b ) × c = (3 − b ) × 3 parity-check matrix of the c onstituent block code, and H 2 is a size n × nc = 7 × 21 parity-chec k matrix which is the permutation of the c olumns o f H 1 determined by the graph. Notice that in general by c hoosing b < c and assigning con stituent block c odes of d if ferent rates R c = b/c to the same graph we ca n ob tain graph-base d code s of d if ferent rates. In general, since in an s -partite, s -uniform, c -regular hyper graph the total number of parity checks is equal to sn ( c − b ) , the code rate R gb of the graph-ba sed code is R gb ≥ n ( c − s ( c − b )) nc = s ( R c − 1) + 1 (2) with eq uality if and o nly if all parity-checks are linearly independ ent. If s = 2 , then we g et R gb ≥ 2 R c − 1 . 5 0 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 1 3 11 9 7 5 3 1 1 2 1 0 8 6 4 2 Fig. 4. A T anner graph ( c = 3 , n = 7 ) representation of Heaw ood’ s b ipartite graph. The simplest example of a Heawood graph-base d code ca n be obtaine d by choo sing as constituent block c odes a single-parity-check code of rate R c = 1 / 3 . Th en the parity-check matrix H c has the form H c =  1 1 1  and the parity-check matrix of the graph-bas ed code is H gb = H g = 0 B B B B B B B B B B B B B B B B B B B B B B B @ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 6 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 12 1 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 3 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 1 0 0 0 5 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 1 7 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 9 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 11 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 13 1 C C C C C C C C C C C C C C C C C C C C C C C A . (3) In this case the graph-base d code coincide s with the grap h code sinc e (3) is the inc idence matrix of the He awood graph. In [4] it is proved that the minimum dis tance of the bipartite grap h-based cod e with s ingle-parity-check constituent c odes is d min = g , wh ere g is the girth of the corresp onding graph . Notice t hat for the T anne r g raph we have d min = g / 2 . The parity-check matri x (3) is a 14 × 21 parity-check ma trix. T aking into acc ount that one check is linea rly de penden t on the othe r , we ob tain a (21 , 8) binary block co de. Its minimum distanc e is d min = g = 6 . 6 Consider the hy pergraph sho wn in Fig. 1. Its incidence matrix has the form H hg = H hgb = 0 B B B B B B B B B B B B B B B B B B B @ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 3 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 4 0 1 0 0 0 0 1 0 0 0 0 1 1 0 0 0 5 0 0 1 0 0 0 0 1 1 0 0 0 0 1 0 0 6 0 0 0 1 1 0 0 0 0 1 0 0 0 0 1 0 7 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 8 0 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 9 0 0 1 0 0 0 1 0 0 0 0 1 0 0 0 1 10 0 0 0 1 0 0 0 1 0 0 1 0 0 0 1 0 11 1 C C C C C C C C C C C C C C C C C C C A (4) and is a 12 × 16 parity-check ma trix o f a hypergraph-based code which coincides with the pa rity-check matrix of the hypergraph c ode. Ea ch c olumn rep resents a h yperedge an d ea ch row represents a vertex of this hypergraph. For example, the first four ro ws repres ent the vertices 1, 2, 3, a nd 4 (triangles), the next four rows he vertices 5 , 6 , 7, and 8 (rectangles), an d the last four ro ws the vertices 9, 10, 11 , and 12 (ov als). The first c olumn repres ents the hyperedg e which c onnects the vertices 1, 5, and 9, the second column the hyperedge connecting v ertices 1, 6, and 10 e tc . The rows of (4) are linearly depe ndent. By removing two parity che cks we obtain a (16 , 6) linear block code with the minimum distance d min = g 3 , 2 = 6 , where g 3 , 2 is the ( 3 , 2 )-girt h of the hype r graph. The rate of this hypergraph code is R hg = 3 / 8 , which satisfies inequ ality (2), R hg ≥ 3  3 4 − 1  + 1 = 1 4 . The T an ner version of this h ypergraph is shown in Fig. 5. 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 5 4 3 2 1 0 6 7 9 8 10 11 Fig. 5. A T anner graph representation of the ( 16 , 6) hypergraph-based code. 7 For an s -partite, s -uniform, c -r egular hyper graph-bas ed cod e with constituent bloc k cod es we ha ve the following theorem. Theorem 1: The minimum distance of a hyp ergraph-based co de based on a n s -partite, s -uniform, c -regular hypergraph with ( s , d c min )-girth g s,d c min and co ntaining cons tituent b lock cod es with minimum distance d c min ≥ 2 is d min = g s,d c min . Pr oof . An y nonzero code word in an s -partite, s -uniform, c -regular hyper graph-base d code alw ays corresponds to a connec ted ( ≥ d c min ) -subgraph or a set of disjoint connected subgraphs. Thes e subgraphs are called active [ 14], [4]. All hyperedges a nd vertices in an activ e subgraph a re also called a ctive . The number o f hyperedges i n the shortest connec ted subgraph is e qual to g s,d c min . Any non zero symbo l in a codeword corresponds to an ac ti ve hyperedge in the graph. By using the arguments given above, we con clude that for any co dew ord v , w H ( v ) ≥ g s,d c min where w H ( v ) is the Hamming weight of v . Minimizing over v co mpletes the proof. B. W oven grap h codes with cons tituent block codes Now as sume tha t the co nstituent c ode assign ed to the hypergraph vertices is a binary ( l c, l b ) linear block code determined b y a parity-check ma trix H c =      H c 11 H c 12 . . . H c 1 ,c H c 21 H c 22 . . . H c 2 ,c . . . . . . . . . . . . H c ( c − b ) , 1 H c ( c − b ) , 2 . . . H c ( c − b ) ,c      (5) where H c ij ∈ B l × l is a size l × l matrix, B l × l is the set of a ll pos sible binary matrices of size l × l . Let C 2 ( H c ) denote such a binary ( l c, l b ) constituent bloc k c ode d etermined by the matrix (5). W e call the correspond ing hy pergraph-based co de with C 2 ( H c ) as constituent codes a w oven graph code wi th constituent bloc k codes. Consider an example o f a w oven graph cod e based o n the bipartite graph wit h girth g = 4 shown in Fig. 6. The T a nner version of this the so-called “utility” bipa rtite g raph is shown in Fig. 7. 5 4 3 2 1 0 Fig. 6. Utility bipartite graph. The inc idence matrix of this g raph is H g =         1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 1 0 0 0 1 0 0 1 1 0 0 0 1 0 0 1 0 0 0 1 1 0 0         . (6) 8                Fig. 7. A T anner graph representation of the util ity bipartite graph. W e use a c onstituent ( 4 × 3 , 4 × 2 ) linear block code with d c min = 3 de termined by the p arity-check matrix H c = ( H c 1 H c 2 H c 3 ) =     1 0 0 0 1 1 1 0 1 1 0 0 0 1 0 0 0 1 1 1 0 1 1 0 0 0 1 0 1 0 1 1 0 0 1 1 0 0 0 1 1 1 0 1 1 0 0 1     . By sea rching over all possible permutations of the matrices H c 1 , H c 2 , and H c 3 we found the followi ng pa rity-check matrix of the wov en graph code with the best minimum distance H wg = 0 B B B B B @ H c 1 H c 2 H c 3 0 0 0 0 0 0 0 0 0 H c 1 H c 2 H c 3 0 0 0 0 0 0 0 0 0 H c 1 H c 2 H c 3 H c 2 0 0 0 H c 3 0 0 0 H c 1 0 0 H c 1 H c 2 0 0 0 H c 3 0 0 H c 3 0 0 0 H c 1 H c 2 0 0 1 C C C C C A . (7) The ma trix (7) desc ribes a (36 , 12) linear block code with d min = 10 . Any codeword v c of the ( l c, l b ) co nstituent block code c an be represe nted as a seq uence o f c blocks of length l , that is, v c = ( v c 1 , v c 2 , . . . , v c c ) , where v c i = ( v c i 1 , v c i 2 , . . . , v c il ) , i = 1 , 2 , . . . , c . W e define the minimum Hamming block distance betwe en the codewords v c and ˜ v c of the con stituent block code as d c block = min v c 6 = ˜ v c { w block ( v c − ˜ v c ) } where w c block ( v c ) = #( v c i 6 = 0 ) , i = 1 , 2 , . . . , c . Next we will p rove the followi ng theorem. Theorem 2: The minimum distance o f wov en g raph codes based on s -partite, s -uniform, c -regular h ypergraphs with ( s , d c block )-girth g s,d c block and co ntaining cons tituent block co des with minimum distance d c min and minimum block d istance d c block ≥ 2 can be lower -bounded b y d min ≥ max n g s,d c block c , s o d c min . Pr oof . Any nonze ro codeword c orresponds to a n a ctiv e c onnected s ubgraph or a set of disjoint con nected subgraph s and the nu mber of hyp eredges in the shortest subgraph is g s,d c block . Any nonzero symbol in a co dew ord activ ates a hyperedg e in the graph, tha t is, not less than s constituent subcodes c orrespond to a codeword. Since at most c hy peredges are conne cted with any hypergraph vertex then the number of a cti ve c onstituent subc odes ca n be lower -bounded b y g s,d c block c . T a king into account that any c odeword of block weight greater than or equa l to d c block in the constituent b lock co de has a weight at le ast equ al to d c min we obtain the follo wing ine quality w H ( v ) ≥ max n g s,d c block c , s o d c min for any codeword v and the proof is complete. From Th eorem 2 for the woven graph cod e determined b y (7) we obtain tha t d min ≥ max  4 4 , 3  3 = 9 . 9 C. W oven graph codes with con stituent con volutional code s W oven graph c odes with cons tituent c on v olutional c odes can be con sidered as a straightforward gene ralization of the woven graph code with constituent block co des. Assume that the C 2 ( H c ) code is chosen as a ze ro-tail terminated (ZT) c on v olutional co de and consider a sequ ence o f ZT con volutional cod es with increasing l . It is evident that when l tends to infinity the ( l c, lb ) co nstituent c ode C 2 ( H c ) can be chosen as a rate R c = b/c binary con v olutional code with cons traint leng th ν c . Then the correspo nding woven grap h code has rate R = s ( R c − 1) + 1 and its constraint leng th is at mos t s n ν c . Another de scription of woven graph code s with cons tituent con v olutional cod es follo ws from the rep resentation of the cons tituent co n v olutional c ode in polynomial form. Le t G c ( D ) be a minimal encod ing matrix [8] of a rate R c = b/c , memory m c con voluti onal c ode, given in polynomial form, tha t is, G c ( D ) =    g c 11 ( D ) . . . g c 1 c ( D ) . . . . . . . . . g c b 1 ( D ) . . . g c bc ( D )    (8) where g c ij ( D ) = g c (0) ij + g c (1) ij D + g c (2) ij D 2 + · · · + g c ( m ) ij D m , i = 1 , 2 , . . . , b , j = 1 , 2 , . . . , c , are binary polynomials such that m c = m ax i,j { deg g c ij ( D ) } . The overall constraint length is ν c = P i max j { deg g c ij ( D ) } . The binary information se quenc e u c ( D ) = ( u c 1 ( D ) , u c 2 ( D ) , . . . , u c b ( D )) is enco ded as v c ( D ) = u c ( D ) G c ( D ) where v c ( D ) = ( v c 1 ( D ) , v c 2 ( D ) , . . . , v c c ( D )) is a binary code sequen ce. Let H c ( D ) denote a parity-check matrix for the same code, H c ( D ) =    h c 11 ( D ) . . . h c 1 c ( D ) . . . . . . . . . h c r 1 ( D ) . . . h c r c ( D )    (9) where r = c − b is the redun dancy of the constituent c ode. W e deno te b y F 2 (( D )) the field o f binary La urent series a nd regard a rate R c = b/ c constituent co n v olutional code as a rate R c = b/c block co de C c over the field o f binary La urent series enco ded by G c ( D ) . Then its c odewords v c ( D ) are e lements of F 2 (( D )) c , which is the c -dimensional vector spa ce over the field of b inary Laurent s eries [8]. The minimum Ha mming block distance between the codewords v j ( D ) a nd v k ( D ) is d efined [9] as d block = min v j ( D ) 6 = v k ( D ) { w block ( v j ( D ) − v k ( D )) } where w block ( v ( D )) = #( v i ( D ) 6 = 0) is the Hamming (block) weight of v ( D ) = ( v 1 ( D ) , v 2 ( D ) , . . . , v c ( D )) . Represen ting a c on v olutional cod e as a block c ode over the fie ld o f binary Laurent series we c an o btain a wo ven graph code with cons tituent con v olutional codes as a generaliza tion of a graph -based code with binary con stituent block code s. For example, a parity-check ma trix H wg ( D ) of the rate R wg = 4 / 3 − 1 = 1 / 3 He awood’ s graph-based code with R c = 2 / 3 co nstituent con voluti onal c odes ha s the form H wg ( D ) = 0 B B B B B B B B B B B B B B B B B B B B B @ h c 1 h c 2 h c 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 h c 1 h c 2 h c 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 h c 1 h c 2 h c 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 h c 1 h c 2 h c 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 h c 1 h c 2 h c 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 h c 1 h c 2 h c 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 h c 1 h c 2 h c 3 t c 1 0 0 0 t c 3 0 0 0 0 0 0 t c 2 0 0 0 0 0 0 0 0 0 0 0 0 t c 1 0 0 0 t c 3 0 0 0 0 0 0 t c 2 0 0 0 0 0 0 0 0 0 0 0 0 t c 1 0 0 0 t c 3 0 0 0 0 0 0 t c 2 0 0 0 0 0 0 0 0 0 0 0 0 t c 1 0 0 0 t c 3 0 0 0 0 0 0 t c 2 0 0 t c 2 0 0 0 0 0 0 0 0 0 t c 1 0 0 0 t c 3 0 0 0 0 0 0 0 0 0 t c 2 0 0 0 0 0 0 0 0 0 t c 1 0 0 0 t c 3 0 0 t c 3 0 0 0 0 0 0 t c 2 0 0 0 0 0 0 0 0 0 t c 1 0 0 1 C C C C C C C C C C C C C C C C C C C C C A (10) 10 where h c i and t c i are short-hand for h c i ( D ) and t c i ( D ) , respectively , a nd H c ( D ) = ( h c 1 ( D ) h c 2 ( D ) h c 3 ( D )) is a parity-check matrix of the rate R c = 2 / 3 c onstituent conv olutional code and ( t c 1 ( D ) , t c 2 ( D ) , t c 3 ( D )) is o ne of six possible pe rmutations of h c 1 ( D ) , h c 2 ( D ) , h c 3 ( D ) . Exploiting the above defi nitions we can interpret this b ipartite wov en g raph-based code with constituent con- voluti onal c odes as follo ws. The left column o f vertices in Fig. 4 repres ents n parity chec ks each of which determines on e of n con stituent fixed an d iden tical conv olutional code s and the ir nc branche s represent the e lements v c L ij ( D ) ∈ F 2 (( D )) , i ev en, 0 ≤ i ≤ 2 n − 2 , 1 ≤ j ≤ c . Similarly , the right c olumn of vertices represen ts sa me con voluti onal cod es and the ir nc branches represe nt the elements v c R ij ( D ) ∈ F 2 (( D )) , i odd, 1 ≤ i ≤ 2 n − 1 , 1 ≤ j ≤ c , w here the set { v c R ij ( D ) } is a random p ermutation o f the set { v c L ij ( D ) } determined by the graph. W e can also regard the n left constituent con volutional codes as a warp with nc threa ds. Eac h of the n right constituent con v olutional codes a re tacked o n c of the threads in the warp such that each thread of the warp is tacked on exactly o nce. T hus, our con struction is a special ca se of a w oven code [17] and we call this graph-base d code a woven graph c ode. Theorem 3: The free distan ce of a woven graph code based on a n s -partite, s -uniform, c -regular hypergraph with the ( s , d c block )-girth g s,d c block and containing c onstituent c on v olutional c odes with free distanc e d c free and minimum block d istance d block ≥ 2 can be lower -bounded b y d free ≥ max n g s,d c block c , s o d c free . Pr oof . S ince w oven graph code s with cons tituent conv olutional co des can be cons idered as a gene ralization of wov en graph codes with cons tituent block codes, the theorem follows from Theorem 2 wh en l tend s to infinity . For a woven graph code based on a bipa rtite g raph with girth g an d con taining con stituent conv olutional codes with minimum block dis tance d c block = 2 and free distance d c free by a straightforward generaliza tion o f the approach of [4] we obtain the following tighter b ound on the free distan ce d free ≥ max n g 2 , 2 o d c free . (11) I I I . A S Y M P T OT I C B O U N D S O N T H E M I N I M U M D I S T A N C E O F WOV E N G R A P H C O D E S W e will show that the ense mble of rand om woven graph cod es based on rando m s -partite, s -uniform, c -regular hypergraphs wit h a fixed degree c and with a fixed number o f vertices n in each subgraph contains asymptotically good c odes. In order to prove this we will modify the approa ch in [3]. A. W oven grap h codes with cons tituent block codes First we conside r the ensemble of random woven graph codes w ith rate R c = b/c c onstituent block c odes determined by the ed ges of a random s -pa rtite, s -uniform, c -regular hypergraph corresponding to the time-v arying random pa rity-check matrix H wg =      ˜ H 1 ˜ H 2 . . . ˜ H s      =      π 1 ( H 1 ) π 2 ( H 2 ) . . . π s ( H s )      (12) where ˜ H i = π i ( H i ) , i = 1 , 2 , . . . , s , is a b lock matrix of size nc (1 − R c ) × nc (or a binary matrix of size n ( c − b ) l × n cl ) and π i denotes a random permutation o f the columns of H i , H i =       H c (1) i 0 . . . 0 0 H c (2) i 0 . . . . . . . . . . . . . . . 0 . . . 0 H c ( n ) i       (13) where H c ( t ) i , t = 1 , . . . , n , den otes the ran dom pa rity-check ma trix (5) which dete rmines the ( l c, l b ) constituent block c ode and n is the number o f co nstituent cod es in each sub graph. 11 Remark : In [3] a more r estricted ensemble of rando m code s is studied in which all matrices are identical random matrices. In the proof of Theore m 1 we need that the synd rome compone nts are independ ent random variables in the produc t probability sp ace of ran dom matrices and random permutations. The follo wing simple example shows that this is not always the cas e if all matrices are identical. Consider n = 1 con stituent block code s of block length c = 2 with b = 1 information s ymbols. This example is rather artificial sinc e the rate o f the cons tituent block co de R c = 1 / 2 and therefore the rate of the grap h-based code with s = 2 is R wg = s ( R c − 1) + 1 = 2 R c − 1 = 0 . In this c ase the parity-check matrix of the code h as the form H wg =  H 1 π ( H 2 )  where π is a random permutation of c elements. First ass ume tha t all matrices are identical, that is, H 1 = H 2 . There a re only 8 eq uiprobable eleme nts in the produ ct spa ce, namely , { H wg } =  0 0 0 0  ,  0 0 0 0  ,  0 1 0 1  ,  0 1 1 0  ,  1 0 1 0  ,  1 0 0 1  ,  1 1 1 1  ,  1 1 1 1  . For any vector x of weight 1 we have the following se t of random eq uiprobable syn dromes: { x H T wg } =  0 0  ,  0 0  ,  0 0  ,  0 1  ,  1 1  ,  1 0  ,  1 1  ,  1 1  . Therefore, P ( x H T wg = 0 | w H ( x ) = 1) = 3 8 > 1 4 . If H 1 and H 2 are both random a nd inde penden t this probability is equ al to 1/4. Although this rema rk co ntradicts the p roof o f Th eorem 3 i n [3], there exists a nother (co mbinatorial) way to prove the same statement for ide ntical H i [16]. Next we prove the following theorem. Theorem 4: (V arshamov-Gilbert lower b ound) F or any ǫ > 0 , some l 0 > 0 , some integer s > 0 and for a ll l > l 0 in the random ensemble of le ngth ncl woven graph codes with ( lc, l b ) binary block constituent cod es of rate R c = b/c there exist cod es of rate R wg = s ( R c − 1) + 1 such that t heir relati ve minimum distanc e δ wg = d min /ncl satisfies the inequ alities δ wg ≥  δ ( R wg ) − ǫ , if R wg > 1 + s log 2 (1 − δ VG ( R wg )) δ VG ( R wg ) − ǫ , if R wg ≤ 1 + s log 2 (1 − δ VG ( R wg )) (14) where δ ( R wg ) is a roo t of the equation (1 − s ) h ( δ ) − δ s log 2  2 − ( R wg − 1) /s − 1  = 0 and δ VG ( R wg ) is the solution of h ( δ ) + R wg − 1 = 0 , and h ( · ) deno tes the binary en tropy function. Pr oof. Le t w b e the Hamming weight of the c odeword v o f the random binary woven graph code C 2 ( H wg ) . W e are going to fin d a parameter d suc h that the proba bility P ( v H T wg = 0 | w ) tend s to 0 for all w < d . W e can rewri te P( v H T wg = 0 | w ) a s P( v H T wg = 0 | w ) = X j P( v H T wg = 0 | w , j )P( j | w ) (15) where j = ( j 1 , j 2 , . . . , j s ) and j i denotes the numb er of n onzero c onstituent co dew ords in the i th subg raph correspond ing to the codeword of weight w . In the e nsemble of random parity-chec k matrices H c ( t ) , t = 1 , 2 , . . . , n , of size l c (1 − R c ) × l c the probability that a nonz ero vector v c is a codeword of the co rresponding cons tituent ran dom b inary code C 2 ( H c ) is equal to 2 − ( c − b ) l since the s yndromes of the c onstituent code s are eq uiprobable sequ ences of length ( c − b ) l . T aking into 12 accoun t that in the i th su bgraph we have j i nonzero cons tituent codew ords the probability P( v H T wg = 0 | w , j ) can be up per- bound ed by P( v H T wg = 0 | w , j ) ≤  ncl w  s Y i =1 2 − j i cl (1 − R c ) . (16) In orde r to estimate the prob ability P( j | w ) we prove the following lemma. Lemma 1 : For the ense mble of binary wov en g raph c odes w ith c onstituent b lock c odes d escribed in Theorem 14, the probability P( j | w ) that a codeword of weight w contains j = ( j 1 , j 2 , . . . , j s ) n onzero cons tituent codewords in the s subgraph s can be upper-bounded by P ( j | w ) ≤ s Y i =1  n j i  cl w /j i  j i  w − 1 j i − 1   ncl w  . (17) Pr oof. T a king into account that in the i th subg raph the number of nonzero co mponent cod ew ords is equal to j i and that the su bgraphs are ran dom and indepen dent we c an rewrite the p robability P ( j | w ) as P ( j | w ) = s Y i =1 P ( j i | w ) . The prob ability P ( j i | w ) c an be upper-bounded as P ( j i | w ) ≤ |H i ( v , w, j i ) |  ncl w  where H i ( v , w , j i ) = { H i | v H T i = 0 , w , j i } . The cardinality of H i ( v , w , j i ) ca n be upper-bounded as |H i ( v , w , j i ) | = X w k ≥ 1 , P w k = w  n j i  j i Y k =1  cl w k  ≤  n j i  cl w/j i  j i  w − 1 j i − 1  (18) where the sum is u pper-bounded by the maximal term times the n umber of terms  w − 1 j i − 1  . Notice that in the above de ri vati ons we ignored the fact that w /j i can be noninteger s ince we consider the asymptotic b ehaviour of (15). It follows from Lemma 1 that P( v H T wg = 0 | w ) ≤ X j  nl c w  1 − s s Y i =1 2 − j i cl (1 − R c )  n j i  cl w/j i  j i  w − 1 j i − 1  ≤ ( n + 1) s  nl c w  1 − s max j s Y i =1 2 − j i cl (1 − R c )  n j i  cl w/j i  j i  w − 1 j i − 1  = ( n + 1) s  nl c w  1 − s s Y i =1 max j i 2 − j i cl (1 − R c )  n j i  cl w/j i  j i  w − 1 j i − 1  = ( n + 1) s  nl c w  1 − s max j 2 − j cl (1 − R c )  n j  cl w/j  j  w − 1 j − 1  ! s . (19) Consider the asymp totic beh aviour o f (15) when m te nds to infinity . Introduce the notations γ = j /n and δ = w/ ( ncl ) a nd the function F ( δ ) = lim l →∞ log 2 P( v H T = 0 | w ) nl c . 13 After s imple deriv ations we obtain F ( δ ) ≤ ˆ F ( δ ) ,  max γ ∈ (0 , 1] (1 − s ) h ( δ ) − (1 − R wg ) γ + sγ h  δ γ  (20) where R wg = s ( R c − 1) + 1 is the rate o f bina ry woven graph code . Maximizing (20) over 0 < γ ≤ 1 gives γ opt = min  1 , δ 1 − 2 ( R wg − 1) /s  . Inserting γ opt < 1 an d γ opt = 1 into (20 ) we obtain ˆ F ( δ ) =  h ( δ ) + R wg − 1 , if 0 < δ ≤ 1 − 2 ( R wg − 1) /s (1 − s ) h ( δ ) − δ s log 2  2 − ( R wg − 1) /s − 1  , if δ ≥ 1 − 2 ( R wg − 1) /s (21) which co incides with (9) and (10) in [3] for s = 2 , that is, if the graph is bipartite. For any R wg and δ from ˆ F ( δ ) < 0 , it follo ws that there e xist codes of rate R wg with relati ve minimum distance δ wg = δ . Let δ ( R wg ) de note the solution of the equation ˆ F ( δ ) = 0 (22) for 0 < δ ≤ 1 − 2 ( R wg − 1) /s and let δ V G ( R wg ) be the so lution of h ( δ ) + R wg − 1 = 0 . Solving (22) for γ opt < 1 and γ opt = 1 we obtain that there exist woven graph co des of rate R wg with the relati ve minimum distanc e δ wg satisfying the inequ alities: δ wg ≥  δ ( R wg ) − ǫ , if R wg > 1 + s log 2 (1 − δ VG ( R wg )) δ VG ( R wg ) − ǫ , if R wg ≤ 1 + s log 2 (1 − δ VG ( R wg )) . (23) 0 0.2 0.4 0.6 0.8 1 0 0.1 0.2 0.3 0.4 0.5 R wg s= ∞ (VG bound) s=2 s=3 δ (R wg ) Fig. 8. The relativ e minimum distance as a function of the code rate for the ensemble of binary wove n graph codes with block constituent codes. In Fig. 8 the lo wer bound (14) on the relati ve mi nimum distance for the ensemb le o f binary wov en graph codes with block constituent co des as a function of the code rate is shown. It is easy to s ee that wh en s g rows the ensemble of binary wov en graph code s contains co des meeting the VG bou nd for almost a ll rates 0 ≤ R wg ≤ 1 . Fig. 9 demo nstrates the gap R VG − R wg between the VG bo und and the code rate as a function of the relative minimum d istance δ wg for different values of s . It follows from Fig. 9 that for s ≥ 3 the diff erence in code rate compared to the VG bound is negligible. 14 0 0.05 0.1 0.15 0.2 0.25 0 0.02 0.04 0.06 0.08 0.1 δ s=2 s=3 s=4 s=5 R VG ( δ )−R wg ( δ ) Fig. 9. The gap between the V G bound and t he code rate as a function of the relative minimum distance. B. Asymp totic bound o n the fr ee d istance of woven grap h codes with cons tituent con volutional codes Consider a ZT conv olutional woven graph co de with constituent ZT con voluti onal c odes of rate R c = b/c . The length o f a ZT woven graph co dew ord in n c -tuples is equal to l + m wg where l is the numb er of nc -tuples influenced by information symbo ls a nd m wg is the me mory of the woven graph code of rate R wg = s ( R c − 1) + 1 . Den ote by d wg free the free distance o f the corresp onding wov en graph code. Now we ca n prove the following Theorem 5: (Costello lower boun d) For any ǫ > 0 , some m 0 > 0 , some integer s ≥ 2 , and for a ll m wg > m 0 in the random e nsemble of rate R wg = s ( R c − 1) + 1 woven graph codes ov er s -partite, s -uniform, c -regular hypergraphs with constituent co n v olutional cod es of rate R c = b/c there exists a cod e with memory m wg such that its relative free distanc e δ wg free = d wg free /ncm wg satisfies the Costello lo wer bound [8], δ wg free ≥ − R wg log 2 (2 1 − R wg − 1) − ǫ. (24) Pr oof : Analogou sly to the deri vations in the proof of Th eorem 4 let j = ( j 1 , j 2 , . . . , j s ) where j i denotes the number of nonze ro constituent cod ew ords in the i th subgraph corresp onding to the cod ew ord of weight w , j i ∈ { 1 , ..., n } . In o rder to evaluate the nu mber o f no nzero c onstituent codewords among the n constituent codewords, notice that the s et of su ch c odewords is a union of se ts of no nzero con stituent codewords belonging to e ach of the s subgrap hs. The cardinality of the union is at least j max = max i { j i } . Therefore the all-zero “tail” required to force t he encoder into the zero state ha s length at least j max cm wg . T he total number of redundant s ymbols c onsists of two parts: the numb er P s i =1 j i cl (1 − R c ) of parity-chec k symbols for the n onzero constituent codewords in the s subgraph s and at lea st j max cm wg redundan t sy mbols required for zero-tail terminating of the wov en graph code. Thus, formula (16) can be rewritt en as P( v H T wg = 0 | w , j ) ≤  nc ( l + m wg ) w  s Y i =1 2 − j i cl (1 − R c ) ! 2 − j max cm wg . (25) The s tatement of the Lemma 1 is c hanged in a follo wing way P ( j | w ) ≤ s Y i =1  n j i  c ( l + m wg ) w /j i  j i  w − 1 j i − 1   nc ( l + m wg ) w  . (26) 15 Instead o f (19) we now hav e P( v H T wg = 0 | w ) ≤ ( n + 1) s  nc ( l + m wg ) w  1 − s × max j ( 2 − j cl (1 − R c )  n j  c ( l + m wg ) w/j  j  w − 1 j − 1  ! s 2 − j cm wg ) . (27) By introducing the notations δ = w ncm wg , µ = l m wg , γ = j n we obtain from (25)–(27) that F ( δ ) = lim m wg →∞ log 2 P( v H T = 0 | w ) ncm wg ≤ max γ ∈ (0 . 1]  (1 − s )(1 + µ ) h  δ 1 + µ  − γ (1 + µ − µR wg ) + γ (1 + µ ) sh  δ γ (1 + µ )  . (28) Maximizing (28) over 0 < γ ≤ 1 , we obtain γ opt = min  1 , δ (1 + µ )(1 − 2 − x )  (29) where x = 1 + µ (1 − R wg ) s (1 + µ ) . If s is large enough , then γ opt = 1 . It follo ws from (28) that F ( δ ) ≤ (1 + µ ) h  δ 1 + µ  − 1 − µ + µR wg , (30) Maximization of F ( δ ) over µ gives F opt ( δ ) ≤ − δ log 2  2 1 − R wg − 1  − R wg (31) where µ opt = δ 1 − 2 R wg − 1 − 1 . W e can find a bou nd on δ wg free by s olving F opt ( δ ) = 0 . Thu s, we can conc lude tha t for any ǫ > 0 we ca n find a wov en graph code su ch that (24) holds I V . E X A M P L E W e s tart with cons idering a graph code de termined by the parity-chec k matrix (3). As men tioned before, the matrix (3) can be conside red as a 14 × 21 p arity-check matrix. Since the parity che cks define d by the graph are linearly depende nt (the sum of the rows of ( 3) is equa l to zero) it turned out that by ignoring one parity chec k we obtain a parity-check matrix of a (21 , 8) linear block code. For simplicity we consider the rate R g = 1 / 3 code t hat is o btained by ignoring the eigh th information symbol which yields a (21 , 7) sub code of this cod e. It is eas y to see tha t renumbering the graph vertices by adding to each vertex n umber some fixed n umber modulo the t otal number of v ertices p reserves both the incidence and adjace ncy matrices of the graph. For example, in Fig. 3, by adding 2 modulo 14 we will get exactly the same graph. When we have a similar prope rty for l inear code s we call such codes quasi-cyclic code s an d the se b lock codes ca n be described as tailbiting (TB) con volutional codes. 16 Renumbering the vertices c orresponds to permuting the rows of (3). By row permutations, (3) can be reduced to the form 0 B B B B B B B B B B B B B B B B B B B B B @ 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 1 C C C C C C C C C C C C C C C C C C C C C A . (32) It follo ws from (32) that the graph shown in Fig. 3 correspo nds to a (21 , 7) TB co de with a paren t conv olutional code de termined by the p arity-check matrix H con v ( D ) =  1 1 1 1 D D 3  . (33) It means that this code is “tail-bitten” at the 7th le vel of the trellis diagram. A corresponding polynomial generator matrix of the parent co n v olutional cod e has the form G con v ( D ) =  D + D 2 1 + D + D 2 1  . (34) The minimum dis tance of the (21 , 7) TB code is eq ual to the grap h girth, that is, d min = g = 6 . Notice that many regular bipartite graphs look v ery simil ar to the Heawood g raph in the sense that by manipulating the incidenc e (pa rity-check) ma trices and trunc ating lengths we can obtain infinite famili es of graphs. Some properties o f thes e g raphs can be eas ily predicted from the prope rties of the corresponding parent co n v olutional codes. Consider the parity-check matrix (10) of the woven graph code base d on the He awood bipartite graph with constituent c on v olutional co des of rate R c = 2 / 3 . This woven graph c ode h as the rate R wg = 2 / 3 · 2 − 1 = 1 / 3 . Let the rate R c = 2 / 3 constituent con volutional cod e of memory m = 3 a nd overall constraint length ν c = 5 with d c free = 6 be g i ven by the gen erator matrix G c ( D ) =  1 + D 2 D 2 1 + D + D 2 D + D 2 + D 3 1 1 + D 2  . (35) A c orresponding parity-check matrix H c ( D ) is H c ( D ) =   1 + D + D 4 1 + D + D 3 + D 4 + D 5 1 + D 2 + D 3 + D 4 + D 5   T . (36) Notice that the constituent c ode C c considered as a block co de over F (( D )) represents a (3 , 2) bloc k code with the minimum distanc e d c block = 2 . By using the product-type lo wer b ound (11) we ob tain d wg free ≥ ( g / 2) d c free = 3 × 6 = 18 . On the other hand , it was verified by computer s earch that a ny codeword of the wov en graph co de determined by (10) c onsists o f a t least three nonz ero codewords o f the comp onent co de C c described by (36). Moreover , it was found by computer se arch that each of these non zero codewords of C c has the minimum block weight d c block = 2 . Note that the codewords of the block co de over F (( D )) with block weight d c block = 2 c orresponds to the c odewords of the c on v olutional code be longing to its subcod es of rate R c = 1 / 2 . Th ese three subcode s ha ve generator matrices G c 1 ( D ) =  g c 1 ( D ) g c 2 ( D )  G c 2 ( D ) =  g c 3 ( D ) g c 2 ( D )  17 G c 3 ( D ) =  g c 1 ( D ) g c 3 ( D )  where g c 1 ( D ) = 1 + D + D 3 + D 4 + D 5 , g c 2 ( D ) = 1 + D + D 4 , a nd g c 3 ( D ) = 1 + D 2 + D 3 + D 4 + D 5 . The minimum free distance over all thes e subcod es of rate R c = 1 / 2 is equal to 8. T aking into acco unt that a ll other cod ew ords o f the woven grap h code con tain at least four nonzero c odewords of C c of block weigh t d c block = 3 we ob tain an improved lo wer b ound on the free distance of the woven graph c ode as d free ≥ min { 3 × 8 , 4 × 6 } = 24 . In order to obtain an upp er bou nd on the free distanc e o f the woven graph code we conside r the parity-check matrix (10 ) in more detail. It also describes a quasi-cyclic code and can by ro w permutations be redu ced to a parity-check matrix of a two-dimensional c ode, a TB (block) code in o ne dimens ion a nd a c on v olutional code in the other , H wg ( D ) = 0 B B B B B B B B B B B B B B B B B B B B B @ h c 1 h c 2 h c 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 t c 1 0 0 0 t c 2 0 0 0 0 0 0 t c 3 0 0 0 0 0 0 0 0 0 0 0 0 h c 1 h c 2 h c 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 t c 1 0 0 0 t c 2 0 0 0 0 0 0 t c 3 0 0 0 0 0 0 0 0 0 0 0 0 h c 1 h c 2 h c 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 t c 1 0 0 0 t c 2 0 0 0 0 0 0 t c 3 0 0 0 0 0 0 0 0 0 0 0 0 h c 1 h c 2 h c 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 t c 1 0 0 0 t c 2 0 0 0 0 0 0 t c 3 0 0 0 0 0 0 0 0 0 0 0 0 h c 1 h c 2 h c 3 0 0 0 0 0 0 0 0 t c 3 0 0 0 0 0 0 0 0 0 t c 1 0 0 0 t c 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 h c 1 h c 2 h c 3 0 0 0 0 0 0 0 0 t c 2 0 0 0 0 0 0 0 0 0 t c 3 0 0 0 t c 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 h c 1 h c 2 h c 3 0 t c 1 0 0 0 0 0 0 t c 2 0 0 0 0 0 0 0 0 0 t c 3 0 0 1 C C C C C C C C C C C C C C C C C C C C C A (37) A p arity-check matrix of the parent con vol utional co de for the TB code (37) gi ven in s ymbolic form is H ( D , Z ) =  h c 1 ( D ) h c 2 ( D ) h c 3 ( D ) t c 1 ( D ) t c 2 ( D ) Z t c 3 ( D ) Z 3  (38) where Z and D are formal vari ables. The matrix (38) can b e considered as a parity-check matrix of a two- dimensional con voluti onal code . The v ariable Z c orresponds to the pare nt con v olutional code of the Heawood graph c ode (33), the vari able D is used for the cons tituent con v olutional cod e (36). A g enerator matrix of the two-dimensional con v olutional co de with the pa rity-check matrix (38) h as the form G ( D , Z ) =  g c e ( D ) Z + g c c ( D ) Z 3 g c a ( D ) + g c d ( D ) Z 3 g c b ( D ) + g c f ( D ) Z  (39) where g c a ( D ) = h c 3 ( D ) t c 1 ( D ) , g c b ( D ) = h c 2 ( D ) t c 1 ( D ) , g c c ( D ) = h c 2 ( D ) t c 3 ( D ) , g c d ( D ) = h c 1 ( D ) t c 3 ( D ) , g c e ( D ) = h c 3 ( D ) t c 2 ( D ) , a nd g c f ( D ) = h c 1 ( D ) t c 2 ( D ) . The ge nerator matrix (39) tail-bitt en over v ariable Z at length 21 yields the generator matrix G ( D ) of the code (37), G wg ( D ) = 0 B B B B B B B B @ g c c g c d 0 0 0 0 g c e 0 g c f 0 g c a g c b 0 0 0 0 0 0 0 0 0 0 0 0 g c c g c d 0 0 0 0 g c e 0 g c f 0 g c a g c b 0 0 0 0 0 0 0 0 0 0 0 0 g c c g c d 0 0 0 0 g c e 0 g c f 0 g c a g c b 0 0 0 0 0 0 0 0 0 0 0 0 g c c g c d 0 0 0 0 g c e 0 g c f 0 g c a g c b 0 g c a g c b 0 0 0 0 0 0 0 0 0 g c c g c d 0 0 0 0 g c e 0 g c f g c e 0 g c f 0 g c a g c b 0 0 0 0 0 0 0 0 0 g c c g c d 0 0 0 0 0 0 0 g c e 0 g c f 0 g c a g c b 0 0 0 0 0 0 0 0 0 g c c g c d 0 1 C C C C C C C C A (40) where g c i is s hort-hand for g c i ( D ) . Notice tha t any of the s ix permutations of the columns h c i ( D ) , i = 1 , 2 , 3 , gene rates a woven graph c ode. Th e permutation t c 1 ( D ) = h c 1 ( D ) , t c 2 ( D ) = h c 3 ( D ) , a nd t c 3 ( D ) = h c 2 ( D ) d escribes the woven graph code w ith the lar gest free distance. The overall con straint length of this generator matrix is equal to 70 but the matrix is not in minimal form. A minimal-basic gen erator matrix [8] has the overall constraint length equ al to 64 and dif fers from (40) by one row which can replac e any of the ro ws o f G ( D ) and has the form  G 0 ( D ) G 0 ( D ) G 0 ( D ) G 0 ( D ) G 0 ( D ) G 0 ( D ) G 0 ( D )  where G 0 ( D ) =  g c p ( D ) g c q ( D ) g c q ( D )  where g c p ( D ) = D + D 2 and g c q ( D ) = 1 + D + D 4 . 18 The matrix (40) is a generator matrix of a con volutional code of rate R wg = 7 / 21 . By applying the BEAST algorithm [19] to the minimal-basic gen erator matrices corresponding to the dif ferent permutations of the columns h c i ( D ) , i = 1 , 2 , 3 , we ob tained the free distance and a f ew spectrum coefficients o f the corresponding wov en graph codes. T he pa rameters of the be st obtained woven graph codes are prese nted in T able 1. T AB LE I S P E C T R A A N D O V E R A L L C O N S T R A I N T L E N G T H S O F R A T E R wg = 1 / 3 W O V E N G R A P H C O D E S P erm utation ν d free Sp ectrum h c 1 ( D ) , h c 3 ( D ) , h c 2 ( D ) 64 32 7 , 0 , 0 , 0 , 0 , 0 , 7 , 0 , 7 , 0 . . . h c 2 ( D ) , h c 1 ( D ) , h c 3 ( D ) 65 32 7 , 0 , 0 , 0 , 7 , 0 , 0 , 0 , 21 , 0 . . . h c 2 ( D ) , h c 3 ( D ) , h c 1 ( D ) 66 30 7 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 7 , 0 . . . V . E N C O D I N G Generally spe aking, enc oding o f graph-ba sed block cod es has complexity O ( N 2 ) , where N is the blockleng th. This technique implies that we find a generator matrix corresponding to the gi ven parity-check matrix and then multiply the information sequen ce by the o btained gen erator matrix. H owe ver , we showed by examples that some regular graph code s as well as woven graph codes are quas i-cyclic co des and the reby they can be inte rpreted a s TB codes. For this cla ss of codes the complexity of the enc oding is proportional to the constraint length o f the pa rent con voluti onal c ode. In this section we are going to illustrate by an exa mple an en coder of a woven grap h co de with c onstituent con voluti onal code s having encoding complexity proporti onal to the overall constraint length of the c orresponding wov en graph code ν wg ≤ n s ν c . Consider again the woven graph cod e in ou r examp le. It is b ased o n the Heawood graph an d us es constituent con voluti onal c odes of rate R c = 2 / 3 an d overall constraint length ν c = 5 . T aking into a ccoun t the repres entation (40) of the woven g raph code as a rate R wg = 7 / 2 1 two-dimensional co de, a TB (block) code in one dimen sion and a con volutional code in the other , we c an draw its enco der as sh own in Fig. 10. 3 v 2 v 1 v 7 u 6 u 5 u 4 u 3 u 2 u 1 u Fig. 10. An en coder of the two-dimension al w ov en graph code. The input symbols u 1 , u 2 , . . . , u 7 enter the encoder o nce per each cycle of duration s ev en time instants . At each time mo ment the c ontents of each register is rewritt en into the next (modu lo seven) register a nd the three o utput 19 symbols v 1 , v 2 , v 3 are gene rated. In othe r words, e ach of the registers correspo nding to the constituent cod e can be considered as an enlarged de lay eleme nt of the encod er of the “TB-dimens ion” c ode determined by the g raph. The sequen ce u 1 , u 2 , . . . , u 7 determines a transition b etween the states o f this encoder . After a cycle of sev en time instants we return to the starting s tate of the enlarged encoder a nd a TB-codew ord (or a w ord from one of its cosets) of length 21 has been generated. Then the following s even input sy mbols u 8 , u 9 , . . . , u 14 enter and after seven time insta nts ano ther word of length 21 has been g enerated, etc. V I . C O N C L U S I O N The as ymptotic behavior of the woven graph codes with block as well as with conv olutional c onstituent cod es has been studied. It was shown tha t in the rando m ensemble of suc h cod es based on s -partite, s -uniform, c -re gular hypergraphs we ca n fi nd a value s ≥ 2 such that for any code rate there exist codes meeting the VG a nd the Costello lo wer bound on the minimum distanc e and free d istance, respectively . Product-type lower boun ds on the minimum distan ce of graph-based and w oven graph co des have b een d eri ved. Example of a rate R wg = 1 / 3 wo ven graph code with fr ee distance above the product bound i s presen ted. It is shown, by an example, tha t wov en grap h codes c an be encoded w ith a complexity prop ortional to the constraint length of the constituent con volutional code. A C K N O W L E D G M E N T This work was su pported in part by the Royal Swe dish Acad emy of Sciences in c ooperation with the Russ ian Academy o f Scien ces and in part by the Swed ish Res earch Council un der Grant 621-2007 -6281. R E F E R E N C E S [1] R. G. Gallager, Low-density P arity-Check Codes , C ambridge, MA: MIT P ress, 1963. [2] C. Berrou, A. Glavieux , and P . Thitimajshima, “Near Shannon limit error-correcting coding and decoding: T urbo codes, ” in Pr oc . IEEE Int. Conf. Commun. (ICC) , Genev a, Switzerland, pp. 1064–1070, 1993. [3] A. Barg and G. Zemor , “Distance properties of expand er codes, ” IEEE T rans. Inf. Theory , vol. 52, no.1, pp. 78–90, Jan. 2006. [4] X.-Y . Hu and M. F . Fossorier “On the com putation of the minimu m distance of lo w-density parity-check codes, ” Int. Confer ence on Communications (ICC’04) , Paris, June 2004. [5] G. Schmidt, V . V . Zyablov , and M. Bos sert, “On ex pander codes based on h ypergrap hs, ” ISIT 2003 , Y okohama, Japan, June 29–July 4, 2003. [6] S. Hoory and Y . Bilu, “The codes from hypergraphs, ” Eur opea n Journa l of C ombinatorics vol. 25, Issue 3, pp. 339–354 , April, 2004. [7] A. Barg, Priv ate communication, May 2007. [8] R. Johannesson and K. Sh. Zigangirov , Fundamentals of Con volutional Coding , IEEE Press, P iscatawa y , NJ, 1999. [9] M. B ossert, C. Medina, and V . Sidorenko , “Encoding and distance estimation of product con v olutional codes, ” in Pro c. IEEE Int. Symp. Information Theory (ISIT’05) , Adelaide, Australi a, Sept. 4-9, pp. 10 63–1067 , 2005. [10] J. A. Bondy and U. S. R. Murty , G raph theory with Applications , N.Y .: North Holland , pp. 236 and 244, 1976. [11] F . Harary , Graph theory , Reading, MA.:Addison-W esley , p. 173, 1994. [12] G. Solomon and H. C. A. van Tilborg , “ A connection between block and con v olutional codes, ” SIAM J. Appl. Math. , vol. 37, pp. 358–36 9, 1 979. [13] J. H. Ma and J. K. W olf, “On tail-biting con v olutional codes, ” IEEE T ran s. Commun. , vol. COM-34, pp. 104–111, 1986. [14] R. M. T anner , “Minimum-distance bounds by graph analysis, ” IEEE Tr ans. Inf. Theory , vol. 47, no. 2, pp. 808–820, Feb . 2001. [15] R. M. T anner , “ A Recursiv e Approach to Lo w Complex ity Codes, ” IEEE T ran s. Inf. Theory , vol. 27, no. 5, pp. 533–547, S ep. 1981. [16] L. Bassalygo, priv ate communication, May 2007. [17] S. H ¨ o st, R. Johan nesson, and V . V . Zya blov , “W oven con v olutional codes I: Encode r properties, ” IEEE T rans. Inf. Theo ry , v ol. 48 , no. 1, pp. 149–161, Jan . 2002. [18] R. M. T anner , D. Sridhara, A. Sridharan, T . E. Fuja, and D. J. Costello, Jr ., “LDPC Block and Con v olutional Codes Based on Ci rculant Matrices, ” IEEE Tr ans. Inf. T heory , vol. 50, no. 12, pp. 2966–298 4, Dec. 2004. [19] I. E. Bocharov a, M. Handlery , R. Johannes son, and B.D. Kudryasho v , “ A BEAST for prowling in trees, ” IEEE T r ans. Inf. Theory , vol. 50, no. 6, pp. 1 295–1302 , June 2004. [20] I. E. Bocharov a, B . D. Kudryasov , R. Johannesson, and P . Stahl, “T ailbiti ng codes: bounds and search results, ” IEEE Tr ans. Inf. Theory , vol. 48, no. 1, pp. 137–148 , Jan. 2002.