List Decoding Tensor Products and Interleaved Codes

LIST DECODING TENSOR PR ODUCTS AND INTERLEA VED CODES P ARIKSHIT GOP ALAN, VENKA TESAN GUR USW AMI, AND PRASAD RAGHA VEN D RA Abstra ct. W e design the ﬁrst eﬃcien t algorithms and prov e new combinatorial b ounds for list deco ding t ensor pro ducts of co des and interlea ved codes. • W e sho w that for every code, the ratio of its lis t deco ding radius to its minimum dis tance sta ys unchanged under the tensor product op eration (rather than squaring, as one might exp ect). This giv es the ﬁrst eﬃcien t list deco ders and new com binatorial b ounds for some natural co des including multiv ariate p olynomials where the degree in eac h v ariable is b ound ed. • W e show t h at for every co de, its list decod ing radius remains unchanged under m - wise interl ea ving for an integer m . This generali zes a recent result of Din ur et al. [ 6 ], who prov ed suc h a result for interlea ved Hadamard co des (equiv alently , linear t ransforma- tions). • Using the notion of generalized Hamming w eights, we give b etter list size b ounds for b oth tensoring and interlea ving of binary linear co des. By analyzing th e weig ht distribution of these codes, w e redu ce the task of b ounding the list size to b ounding the num b er of close-b y low-rank co dew ords. F or deco ding linear transformations, u sing rank-red u ction together with other ideas, we obtain list size b oun ds that are tigh t o ver small ﬁelds. Our results giv e bett er boun ds on the list decoding radius than what is obt ained from the Johnson b ound , and yield rather general families of codes deco dable b eyond th e Johnson b ound. 1. Introduction The d eco din g pr oblem f or error-correcting co des consists of ﬁnding the original message giv en a corrupted version of the codeword en co d ing it. When the num b er of errors in the co dew ord is to o large, and in particular could exceed half the minim u m distance of the co de, unambiguous reco ve ry of the original co dew ord is no longer alwa ys p ossible. The notion of list de c o ding of error-co rrecting co des, introdu ced b y Elias [ 10 ] and W ozencraft [ 35 ], pro vides one a v enue for err or-correction in such high noise r egimes. The goal of a list deco ding alg orithm is to eﬃcien tly reco v er a list of all co dewo rds within a s p eciﬁed Hamming r adius of an inpu t string. The cen tral problem of list deco ding is to iden tify the radius up to whic h this goal is tractable, b oth com b inatorially (i n terms o f t he output list b eing guaran teed to be smal l, regardless of the input) and algorithmically (in terms of b eing able to ﬁnd the list eﬃcien tly). The classical Johnson b oun d s h o ws that at lea st combinato rially , list decod ing alw a ys allo ws one to correct errors b ey ond h alf the minimum d istance. It states that ev ery co de of distance δ o ve r F q is list-decodable up to the Johnson radius J q ( δ ) wh ic h lies in the r ange ( δ / 2 , δ ]. Ho wev er, the Johnson b ound is oblivious to the structure of the cod e; it only d ep ends on its Researc h was supp orted in part by NS F CCF-03436 72 and a D a vid and Lucile Pac k ard F ellow ship. 2 P . GOP ALAN, V. GUR USW AMI, AND P . RAGHA VENDRA minim um d istance. P otenti ally , a cod e might b e list-deco dable at larger error-radii than what is guaran teed by the Johnson boun d. The question of iden tifying the precise radiu s up to whic h list deco ding is tractable for a family of co d es is a c hallenging problem. Despite m u ch progress in designing list deco ding algorithms ov er t he last decade, th is problem is still op en ev en for w ell-studied cod es s uc h as Reed-Solomon and Ree d-Muller co des. On the algorithmic side, follo wing the breakthrough r esults of Goldreic h -Levin [ 11 ] and Su dan [ 28 ] whic h ga v e list decoder s for Hadamard codes and Reed-Solomon co des resp ectiv ely , there has b een tremendous progress in devising list deco ders for v arious co des (see the surv eys [ 15 , 16 , 29 ]). This stud y has h ad substantia l impact on other area s su c h as co mplexit y theory [ 30 , 32 ], cryptograph y [ 11 , 1 ] and computational learnin g [ 19 , 21 ]. Examples of codes whic h are kno wn to b e list-deco dable b eyond the Johnson b ound ha ve been rare: Extractor cod es [ 31 , 14 ], folded Reed-Solomon co des [ 24 , 17 ], group homomorphism s [ 6 ] and Reed-Muller co des o ver sm all ﬁelds [ 12 ] are the few examples kno wn to us. A natural wa y to design new error-correcting co des from old o nes is via v arious pr o duct op- erations on these co d es. In this w ork, w e study th e eﬀect of tw o basic pr o duct op erations, tensoring and interlea ving, on list-deco d abilit y . In what follo ws, [ q ] stands for an alph ab et of size q , f or examp le { 0 , 1 , . . . , q − 1 } . Deﬁnition 1.1. (T ensor Pr o duct) Give n t wo linear codes C 1 ⊆ [ q ] n 1 and C 2 ⊆ [ q ] n 2 , their tensor pro duct C 2 ⊗ C 1 consists of all matrices in [ q ] n 2 × n 1 whose ro ws b elong to C 1 and columns b elong to C 2 . F or a co de C ⊆ [ q ] n , its m -wise tensor p ro duct for m > 1 is a co de of length n m deﬁned inductiv ely as C ⊗ 1 = C and C ⊗ m = C ⊗ ( m − 1) ⊗ C for m > 1. F or example, Reed-Muller co d es in m v ariables where the degree in eac h v ariable is restricted to d can b e view ed as th e m -wise tensor of Reed-S olomon co d es. Our algorithm do es not require the C i s to be linear, but we mak e the assumption since the tensor of t wo non-linear co des migh t b e empt y . Using δ ( C ) and R ( C ) to denote th e normalized distance and r ate of C resp ectiv ely , it follo ws that δ ( C ⊗ m ) = δ ( C ) m and R ( C ⊗ m ) = R ( C ) m . Hence for tensor pro du cts, we are primarily in terested in the setting where m is either constan t or a slo wly gro win g fun ction of the blo c k length. Deﬁnition 1.2. (Interle ave d Co des) The m -wise Interle aving (or in terlea v ed pro du ct) C ⊙ m of the co d e C ⊆ [ q ] n consists of n × m matrices o ver [ q ] whose co lumns are co dew ords in C . Eac h row is treated as a single sym b ol, th us C ⊙ m ⊆ [ q m ] n . In other w ords, und er m -w ise interlea ving, m indep end ent messages are enco ded using C , and the m sym b ols in eac h p osition are juxtap osed toget her into a single sym b ol o v er a large r alphab et. F or instance, linear transformations from F n 2 → F m 2 can b e view ed as interlea v ed Hadamard co d es. It is easy to see that δ ( C ⊙ m ) = δ ( C ), R ( C ⊙ m ) = R ( C ) but the alph ab et gro ws fr om [ q ] to m -dimensional v ectors o ver [ q ]. So unlik e for tensors, for in terlea vin g m could b e as large as p olynomial in the blo c k length; indeed our results hold for any m . 1.1. Tens or pr oducts and inte rlea ved codes: Prior w ork and m otiv a tion. 1.1.1. T ensoring. T ensor pro du cts o ccupy a cent ral place in codin g theory , so muc h so that tensor pro d uct co d es are t y p ically referred to as jus t prod uct co des. Produ ct co des provide LIST DECODING PR ODU CT A ND INTERLEA VED CODES 3 a con v enien t wa y to construct longer co d es fr om shorter co mp onent co des. Elias [ 9 ] u sed tensor pro d uct of Hamming co des to constru ct the ﬁrst explicit co des with p ositiv e rate for comm u n ication ov er a b inary sym metric channel. The s tructure of pro du ct cod es en ab les de- co ding them along columns and ro w s, and the column decod er can provide v aluable r eliabilit y information to the ro w decoder [ 2 6 ]. Pro duct co des ﬁn d man y u s es in pr actice ; for example, the prod uct of t w o Reed-Solomo n co des is used f or en cod ing data on D VDs. More recen tly , tensor pro ducts ha v e found applications in seve ral areas of th eoretical C S suc h as hardness of appro ximation [ 8 , 20 ] and constructions of lo cally testable cod es [ 22 ]. The eﬀect of tensoring on on lo cal testabilit y of co des has b een extensiv ely studied [ 2 , 7 , 34 , 4 ]. T ensor p r o ducts admit a natural tester (whic h chec ks a random ro w/column) that has a certain “robustn ess” pr op ert y [ 2 ]. Exploiting this, b y r ecursiv e tensoring one can obtain simple constru ctions o f locally testable c o des with non-trivial parameters, starting from any reasonable constan t-sized co de. But to our kno wledge, there s eems to n o pr ior w ork focus ing on the eﬀect o f te nsoring o n the list deco ding radius . In particular, the b est com binatorial b ound kn o wn for the list deco ding r adius seems to hav e b een the Johnson b ound , and we are una w are of an e ﬃcien t algorithm th at decodes C 2 ⊗ C 1 up to the J ohnson b ound, assuming that s uc h algorithms exists for eac h C i . A suﬃcient ly strong result on list d ecod ing tensor pro du cts could lea d to a simple, recursiv e construction of list-decod able co d es starting from an y small cod e of go o d d istance. Our results a re optimal in terms of the radiu s to wh ic h we deco de, but n ot strong enough in terms of output list-size guarante e to obtain suc h a result. 1.1.2. Interle aving. At a high lev el, in terlea vin g is a w ay to arr ange data in a non-con tiguous w a y in order to increase p erformance. In terlea vin g is u sed in practical cod in g sys tems to group together the sym b ols of s everal co dewo rds as a wa y to guard against burst errors. A b urst error could ca use to o man y errors within o ne co dew ord, making it unreco verable. Ho wev er, with int erlea ving, the errors get distrib u ted in to a small, correctable n um b er of err ors in eac h of the interlea v ed co dew ords. This is quite imp ortan t in practice; for example the data in a CD is p r otected usin g cross-int erlea ved Reed-Solomon c o ding (CIR C). Though co de in terlea ving has b een implicitly studied for its practica l imp ortance, o ur wo rk app ears to b e the ﬁrst to study it in generalit y as a formal pro d uct op er ation. W e describ e some r ecen t th eoretica l w ork on in terlea ved co des whic h set the stage for our in vestiga tion. The pr ob lem of decodin g inte rlea ved Reed-Solo mon cod es from a large num b er of r andom errors was tac kled in [ 5 , 3 ]. Th e folded Reed-Solomon co des constructed by Gu r uswa mi and Rudra [ 17 ] (whic h ac hieve list decoding capacit y), and their precursor P arv aresh-V ardy co des [ 24 ], are b oth s ub co des of in terlea v ed Reed-Solomo n codes, where the m in terlea v ed co dew ords are carefully chose n to ha v e dep endencies on eac h other. Dinur e t al. in their w ork on list deco ding group homomorphisms studied interlea v ed Hadamard co d es, wh ic h are essen tially all linear transformations from F n q → F m q [ 6 ]. Their wo rk raised the qu estion of how the in terlea ving operation aﬀects the li st deco ding radius of arbitrary co d es, and motiv ated our r esults. 1.2. Brief high-lev el description of key resul ts. W e start with some terminology . The distance dist( C ) of a code C ⊆ [ q ] n is the minim um Hamming distance b etw een t w o distinct co d ew ord s in C , and its r elative distanc e is deﬁned as dist( C ) /n . F or a cod e C o f blo c k length n and 0 < η < 1, the list-size for r adius η , denoted ℓ ( C , η ), is deﬁned as th e maximum 4 P . GOP ALAN, V. GUR USW AMI, AND P . RAGHA VENDRA n um b er of co dewords in a Hamming ball of radius ηn . Informally , the list de c o ding r adius ( LDR ) of C is the largest η such th at for eve ry co nstan t ε > 0, the list size for radius ( η − ε ) is b ou n ded by some function f ( q , ε ) indep endent of the blo c k length n . 1 Our main result on tensor pro d ucts is the follo w in g: if C h as relativ e distance δ and LDR η , then the list deco ding r ad iu s of the m -wise pr o duct C ⊗ m is η δ m − 1 . In other w ords, the rat io of LD R to relativ e distance is preserv ed und er tensoring. F or in terlea v ed co des, w e pr o ve that the LDR remains unc hanged irresp ectiv e o f t he n umber of interlea v es. In particula r, if C has relativ e distance δ , and C ⊙ m is its m -wise in terlea ving, then for ev ery η < δ one has ℓ ( C ⊙ m , η ) 6 A · ℓ ( C , η ) B where A, B are constan ts dep ending only on δ , η and indep endent of m . Organization. F ormal statemen t of all the resu lts o f this work app ear in Section 2 . These include the results describ ed ab ov e whic h apply to arb itrary co des, along with imp ro ved list size b ounds f or sp ecial case s lik e b inary linear codes and transformations. W e pr esent our b ound s for interlea v ed cod es in Section 3 , and our deco din g algorithm for tensor pro ducts together with a sk etc h of its analysis in Section 4 . The form al analysis of the tensor deco ding algorithm app ears in Section 5 . In Section 6 , w e use the notion of generalized Hamming w eigh ts to d eriv e improv ed list-size b oun ds f or tensor pro ducts and in terlea vin gs of b in ary linear co des. Finally , in Section 7 , w e sho w list size b ound s for linear transformations that are tigh t o v er small ﬁelds. 2. Our Resul ts 2.1. List decoding tens or products. Gi v en tw o codes C 1 ⊆ [ q ] n 1 and C 2 ⊆ [ q ] n 2 , their tensor pro duct C 2 ⊗ C 1 consists of all matrices in [ q ] n 2 × n 1 whose ro ws b elong to C 1 and columns b elong to C 2 . W e design a generic algorithm that list decodes C 2 ⊗ C 1 using list decoders for C 1 and C 2 as subroutines, and b ound the list-dec o ding rad iu s b y analyzing its p erformance. F u rther, if w e ha ve eﬃcien t list decod ers f or C 1 and C 2 , then w e get an eﬃcie n t alg orithm for list d ecod ing C 2 ⊗ C 1 . A brief o v erview of this alg orithm is gi v en b elow in Section 2.1.2 . Our main result on list deco din g tensor pr o ducts is the follo wing. Theorem 2.1. L et C 1 ⊆ [ q ] n 1 and C 2 ⊆ [ q ] n 2 b e c o des of r e lative distanc e δ 1 and δ 2 r esp e ctively, and 0 < η 1 , η 2 < 1 . Deﬁne η ∗ = min( δ 1 η 2 , δ 2 η 1 ) . Then ℓ ( C 2 ⊗ C 1 , η ∗ − ε ) 6 q O ε,δ 1 ,δ 2 (log ℓ ( C 1 ,η 1 ) log ℓ ( C 2 ,η 2 )) . In p articular, if the LDR of C 1 , C 2 ar e η 1 , η 2 r esp e ctively, then the LDR of C 2 ⊗ C 1 is at le ast η ∗ . The deco ding radius ac h iev ed in T heorem 2.1 is in fact tight: it is easily sho wn that assum- ing that C cannot b e li st deco ded b eyo nd radius η , C ⊗ 2 cannot b e list decoded b ey ond δ η (Lemma 4.1 ). As a corolla ry , the ratio of the list deco ding r adius to the r elativ e distance stays unc hanged under rep eated tensorin g. 1 Here it is implicitly implied that the codes C belong to an inﬁnite family of co des of increasing blo c k length. LIST DECODING PR ODU CT A ND INTERLEA VED CODES 5 Corollary 2.2. If C ha s r elative distanc e δ and list de c o ding r adius η , then the list d e c o ding of the m - wise tensor pr o duct C ⊗ m is η δ m − 1 . The b ounds that w e get on list-size for C ⊗ m are doubly exp onen tial in m . Improving this b ound to singly exp onent ial in m , sa y exp( O ( m )), could ha ve interesting applications, su ch as a simple construction o f list-decod able co des b y rep eated tensoring, with parameters strong enough for the man y complexit y-theoretic applications whic h currently rely crucially on Reed- Solomon list decoding. W e are able to obtain some im p ro vemen ts for the case of tensoring binary linear co des; this is describ ed in Th eorem 2.8 . Comparison to Johnson bound. E ven assuming that eac h of C 1 and C 2 are d ecod ab le only up to the Johns on bou n d, Theorem 2.1 giv es a b oun d that is signiﬁcan tly b etter th an the Johnson b ound, since by con vexit y J q ( δ 1 δ 2 ) < min( δ 1 J q ( δ 2 ) , δ 2 J q ( δ 1 )) 6 η ∗ . 2.1.1. Implic ations for natur al c o des. Theorem 2.1 giv es new b ounds on li st decoding radius for some natural families of codes, whic h w e d iscuss b elo w. Reed-Solomon tensors. Let RS [ n, k ] q denote the Reed S olomon cod e consisting of ev alua- tions of degree k p olynomials o ver a set S ⊆ F q of size n , with distance δ = 1 − k /n . S u c h co des are li st-deco dable up to the Johnson radiu s J ( δ ) = 1 − √ 1 − δ using the Gurusw ami-Sudan algorithm [ 18 ]. The m -wise tensor pro duct of such a RS co d e is a [ n m , k m ] q co de co nsisting of ev aluations on S m ⊆ F m q , of multiv ariate p olynomials in m v ariables with individ ual d egree of eac h v ariable b eing at most k . Pa rv aresh et al. considered the problem of deco ding these pro du ct cod es, and exte nded the Ree d-Solomon list d ecod er to this setting [ 23 ]. This yields relativ ely w eak b ounds, and a s they n ote, by reducing the pr oblem to deco ding Re ed-Muller co des of o rder mk , one can do b etter [ 25 ]. S till, these b ounds are wea k er than the Johnson radius J ( δ m ) o f t he m -wise p r o duct, a nd in fact become trivial w hen m > n/k . Our results giv e m uc h stronger b ounds, and enable deco ding b eyond the Joh n son b ound. Corollary 2.3. The m -wise tensor of the RS[ n, k ] q R e e d-Solomo n c o de is eﬃciently list- de c o dable up to a r adius δ m − 1 J ( δ ) − ε , w her e δ = 1 − k /n is the r elative distanc e of RS[ n, k ] q , and ε > 0 is arbitr ary. One can compare this with the John son b ound J ( δ m ) b y noting that δ m − 1 J ( δ ) = δ m  1 2 + 1 4 δ + 3 8 δ 2 + · · ·  , J ( δ m ) = δ m  1 2 + 1 4 δ m + 3 8 δ 2 m + · · ·  . Hadamard tensors. Let Had b e the [ q k , k ] q Hadamard co de, where a ∈ F k q is encod ed as the v ector { a · x } x ∈ F k q . C is list-dec o dable up to its relativ e d istance δ = 1 − 1 /q . The m -wise tensor pro d uct C ⊗ m consists of all “blo ck-li near” p olynomials. Sp eciﬁcally , eac h co d ew ord in C ⊗ m is a p olynomial P on m × k v ariables giv en b y x = ( x (1) , x (2) , . . . , x ( m ) ) where eac h x ( i ) = ( x ( i ) 1 , . . . , x ( i ) k ), such that for eac h i , P is a linear function in x ( i ) for eac h ﬁxing of t he other v ariables. Corollary 2.4. The m -wise tensor of [ q k , k ] q Hadamar d c o des has list de c o ding r adius e qual to its r elative distanc e  1 − 1 q  m . 6 P . GOP ALAN, V. GUR USW AMI, AND P . RAGHA VENDRA This result is in teresting in ligh t of a conjecture b y Gopalan e t al. stating that Reed-Muller co des of degree m ov er F q are list-decodable up t o the minim um d istance (they pro ve d this result for q = 2) [ 12 ]. Our result sho ws m -wise Hadamard tensors wh ich are a natur al sub cod e of ord er m Reed-Mulle r co des (with b etter distance but low er rate) are indeed list-deco d able up to the minimum distance. 2.1.2. T ensor de c o der overview. O u r algorithm for list deco ding C 2 ⊗ C 1 starts by p ic king small random sub sets S ⊂ [ n 2 ] an d T ⊂ [ n 1 ] of the ro ws and columns resp ectiv ely . W e assume that w e are giv en the co d ew ord restricted to S × T as advice. By alte rnately runnin g the r ow and column d eco ders , we impro v e the qualit y of the advice. W e sh o w that after four alternations, one can reco ver the codeword correctly with high probabilit y (ov er the c hoice of S and T ). An obstacle in deco ding tensor pr o duct codes is that some of the ro w s/columns could hav e ev ery high error-rates, and deco ding those r ows/co lumns of the receiv ed word giv es incorrect co dew ords. W e sho w that the advice string allo ws us to identi fy su c h ro ws/columns with go o d probabilit y , thus redu cing the problem to deco ding from (few) errors and (man y) erasures. The scheme of starting with a small advice string and reco vering the co dewo rd via a series of self-correction steps h as b een u sed for list deco d ing Hadamard co des and Reed-Muller cod es. Our w ork is the ﬁrst (to our kn o wledge) that applies it outside the setting of algebraic co des deﬁned o v er a v ector space. 2.2. Int erlea ve d Codes. Armed with a list d eco din g a lgorithm for C , a naiv e attempt a t list deco ding C ⊙ m w ould p ro ceed as follo ws: List deco de eac h col umn of the receiv ed w ord separately to obtain m diﬀeren t lists { L 1 , . . . , L m } , then iterate o ver all matrices w ith ﬁrst column from L 1 , s econd column from L 2 , etc., and output those clo se enough to the receiv ed w ord. The naiv e algorithm describ ed ab ov e yields the follo w in g simple pro du ct b oun d (2.1) ℓ ( C , η ) 6 ℓ ( C ⊙ m , η ) 6 ℓ ( C , η ) m This upp er b ound is unsatisfactory since even if ℓ ( C , η ) = 2, the upp er b ound on ℓ ( C ⊙ m , η ) is 2 m . Recen t work of Dinur et al. [ 6 ] ov ercame this naive pro du ct b oun d when the cod es b eing in terlea v ed arise from group homomorphisms. T o this end, they extensiv ely u sed prop er ties of certain set systems that arise in the con text of group homomorphisms. Surp r isingly , w e sho w that the pro du ct b ound can b e su bstan tially impro ved for eve ry code C . In fact, the list size b ound w e o btain is ind ep endent of t he num b er of in terlea vings m (as in the ab ov e-men tioned results of Din ur et al. [ 6 ] ). Theorem 2.5. L et C b e a c o de of r elative distanc e an d let η < δ . Deﬁne b = ⌈ η δ − η ⌉ and r = ⌈ log δ δ − η ⌉ . Then, for al l inte gers m > 1 , we have (2.2) ℓ ( C ⊙ m , η ) 6  b + r r  ℓ ( C , η ) r . The imp lies that if C is list-dec o dable up to radius η , then so is C ⊙ m . Th e condition η < δ in Theorem 2.5 is necessary , as it is easily sh o wn that ℓ ( C ⊙ m , δ ) > 2 m (unless C is trivial and h as only one co deword). LIST DECODING PR ODU CT A ND INTERLEA VED CODES 7 2.2.1. P r o of te chnique. The pro of of Theorem 2.5 relies on a simple observ ation which we outline b elow. Assume that w e list deco de the receiv ed wo rd corresp onding to the ﬁrst column, to get a list of candidate co dewords for that column and pic k one co deword from this list. Ro ws where the ﬁr st column of the receiv ed w ord diﬀers from this codewo rd corresp ond to errors, hence w e can rep lace those ro ws b y erasures. Thus for the second column, some of the error locations are erased, which mak es the decodin g easier. Of course, if the co dewo rd is close to the receiv ed word, then th er e ma y b e ve ry few (or no) erasures in tro du ced. But w e sho w th ere a re only a few co dewords in the list that are v ery close to the receiv ed w ord. Extending this intuitio n, w e co nstruct a tree of p ossible cod ewords for eac h column and sho w that the tree is either sh allo w or it do es not b r anc h too muc h. 2.3. Bet ter list-s ize bounds using gener alized Hamming weights. F or the case of binary linear co des, we are able to impro ve the list size u pp er b ounds for b oth tensoring and in terlea ving (Theorems 2.1 and 2.5 ab ov e) using a co mmon te c h nique. W e no w describ e the underlying idea and the results it yields. 2.3.1. M etho d overview. Co dewords of b oth interlea v ed and 2-wise tensor pro du cts are natu- rally viewed as matrices. W e bring the rank of these matrices in to pla y , and argue that if the rank of a co dewo rd is large, then its Hamming w eigh t is su bstan tially higher than the d istance of the code. It turn s out that this phenomenon is ca ptured exactl y b y a w ell-studied notion in coding theory called g e ner alize d H amming weights (see the s u rv ey [ 33 ]) that is also closely related to list deco ding from erasures [ 13 ]. The precise connection is that if a cod ew ord of C ⊙ m has rank r , then its rela tiv e Hamming weig h t is at least the r th generalized Hamming w eigh t δ r ( C ) of C . Similarly , rank r co dewords in C ⊗ C ha ve w eigh t at least δ r ( C ) δ ( C ). F or b in ary cod es, for r large enough, δ r ( C ) approac h es 2 δ . Th e John son radius of 2 δ exceeds δ . Therefore, for r = r ( δ, η ) large enough, the n umb er of codewords in a Hamming ball of radius η < δ whose pairwise diﬀerences all hav e rank > r can be b ounded from ab o ve using the John s on b ound. Using the deleti on argumen t f rom [ 12 ], th e task of b ounding the list-siz e for radius η now reduces to b ou n ding the n umb er of rank 6 r co dewords within r adius η . W e accomplish this task, for b oth interlea v ed and prod uct codes, using additional combinato rial ideas. W e remark that our use of the deletio n argumen t is more sophistica ted than in [ 12 ], since m ost of th e work goes in to b ounding the list-size for the low-rank ca se. W e note that the reason the abov e a pproac h does n ot wo rk for non-binary alphab ets is that the ge neralized Hamming w eight δ r ( C ) ma y not b e larger than q q − 1 δ for q -ary co des. 2.3.2. R esults for interle ave d c o des. T heorem 2.5 sho wed that for an y co de C of distance δ and for any η < δ , the list-size for the m -wise inte rlea ved co de C ⊙ m is b ou n ded by ℓ ( C , η ) ⌈ log δ δ − η ⌉ . Note that for η → δ , the exp onen t gro ws without boun ds. F or b inary linear co des, usin g the ab o ve generalized Hamming w eigh ts b ased approac h , w e can impro ve this b ound to a ﬁ xed p olynomial in ℓ ( C , η ), remo vin g the dep endence on log(1 / ( δ − η )) in the exp onent. Theorem 2.6. F or any b inary line ar c o de w ith dista nc e δ , we have: ℓ ( C ⊙ m , η ) 6 4 δ 4  2 ℓ ( C , η ) δ 2 ( δ − η )  ⌈ log 2 δ 2 ⌉ . (2.3) 8 P . GOP ALAN, V. GUR USW AMI, AND P . RAGHA VENDRA Giv en a bin ary linear error-correcting co de C , the Johns on b oun d states that the li st size at radius J 2 ( δ ) − ε is b ound ed b y O ( ε − 2 ). W e can sho w that essentia lly th e same list-size b ound holds f or C ⊙ m , pro vid ed the distance δ is b ounded a w a y from 1 2 . Theorem 2.7. F or every δ < 1 2 , ther e exists a c onstant c δ such that for every binary line ar c o de C of r elative distanc e δ , ℓ ( C ⊙ m , J 2 ( δ ) − ε ) 6 c δ ε − 2 . 2.3.3. R esult for tensor pr o duct. App lying the ab o ve approac h to the tensor pro duct of t w o co des, w e prov e the follo w ing (Th eorem 6.15 ). Note that the list size is at most a ﬁx ed p olynomial in the list sizes ℓ ( C 1 , η 1 ) and ℓ ( C 2 , η 2 ) o f the orig inal cod es, instead of the quasi- p olynomial dep endence in Th eorem 2.1 . Theorem 2.8. Supp ose C 1 ⊆ F n 1 2 and C 2 ⊆ F n 2 2 ar e binary line ar c o des of r elative distanc e δ 1 and δ 2 r esp e ctively. L et η 1 6 δ 1 and η 2 6 δ 2 . D e ﬁne η ∗ = min( δ 1 η 2 , δ 2 η 1 ) and r = ⌈ log ( 2 δ 1 δ 2 ) ⌉ . Then ℓ ( C 2 ⊗ C 1 , η ∗ − ε ) 6 2 O ( r 2 ) ℓ ( C 1 , η 1 ) r ℓ ( C 2 , η 2 ) r ε − 2 r . 2.4. List decoding l inear tran sforma tions . Let Lin( F q , k , m ) denote the set all linear transformations L : F k q → F m q . The co d e Lin( F q , k , m ) is n othin g bu t the m -wise inter- lea ving Had ⊙ m ( F q , k ) of the Hadamard cod e Had( F q , k ). Let ℓ (Lin( F q , k , m ) , η ) den ote the maxim u m list size for the co de Lin( F q , k , m ) at a distance η . Din u r et al. [ 6 ] show that ℓ (Lin( F 2 , k , m ) , 1 / 2 − ε ) 6 O ( ε − 4 ) and ℓ (Lin( F q , k , m ) , 1 − 1 /q − ε ) 6 O ( ε − C ) for some con- stan t C for general q . The b est lo wer-boun d kno wn for an y ﬁeld is Ω( ε − 2 ). Being a general result for all codes, Theorem 2.5 only giv es a qu asip olynomial b ound for the sp ecial case of linear transformations. By s p ecializing the ab o ve generalize d Hammin g w eights approac h to the case of linear transformations, and u sing more sophisticated arguments based on deco ding from erasures for the lo w-rank ca se, w e pro ve the f ollo wing stronger b ounds for list decodin g linear transform ations o ver F 2 . Theorem 2.9. Ther e is a c onstant C su ch that for al l p ositive inte g ers k , m , ℓ (Lin( F 2 , k , m ) , 1 2 − ε ) 6 C ε − 2 . F or arbitrary ﬁelds F q , we pro ve the follo wing b ounds, the ﬁrst is asymptotically tight for small ﬁ elds while the second is indep endent of q and impro v es on the b ound of Din ur et a l. . Theorem 2.10 . Ther e is an absolute c onstant C ′ such that for every ﬁnite ﬁeld F q , ℓ (Lin( F q , k , m ) , 1 − 1 q − ε ) 6 C ′ min( q 6 ε − 2 , ε − 5 ) . 3. Interlea v ed Codes In this sectio n, C ⊂ [ q ] n will be a n a rbitrary code (possib ly non-linear) o v er an alphab et [ q ]. W e will use ℓ ( η ) and ℓ ⊙ m ( η ) for ℓ ( C , η ) and ℓ ( C ⊙ m , η ) resp ectiv ely . L et d q ( c 1 , c 2 ) denote the Hamming distance b et wee n strings in [ q ] n and ∆ q ( c 1 , c 2 ) = d q ( c 1 , c 2 ) /n denote the normalized Hamming distance. W e drop the subscript q wh en the a lphab et is clear f rom con text. F or LIST DECODING PR ODU CT A ND INTERLEA VED CODES 9 r ∈ [ q ] n , B( r , η ) ⊂ [ q ] n denotes the Hammin g ball cen tered at r of radiu s η n . W e use C for co dew ords of C ⊙ m and c for codewo rds of C . W e will in terc hangeably view C as a matrix in [ q ] n × m and a v ector in [ q m ] n . F or a k × m matrix A , a 1 , . . . , a m will denote its columns, a [1] , . . . , a [ k ] will denote the r ows, and A 6 i will d enote the k × i matrix( a 1 , . . . , a i ). Giv en an algorithm DecodeC that can list d ecod e C up to r ad iu s η , it is easy to giv e an algorithm DecodeC ⊙ m that uses DecodeC as a sub routine and runs in time p olynomial in the list- size and m ; we p resen t this algo rithm in Section 3.1 . Thus it suﬃces to b ound the list- size to p ro ve Theorem 2.5 . W e do this b y giving a (p ossibly ineﬃcien t) algorithm, whic h iden tiﬁes ro ws where err ors ha ve o ccurred and erases them. Erasing a set S ⊂ [ n ] of co-ordinates is equiv alen t to pun ctur ing the co d e by remo ving those indices. Given r ∈ [ q ] n , w e u se r − S to denote its pro jection on to the co-ordinates in [ n ] \ S . Deﬁnition 3.1. (Er asing a subset) Giv en a co de C ⊆ [ q ] n , erasing the indices corresp on d ing to S ⊆ [ n ] giv es th e co d e C − S = { c − S : c ∈ C } ⊆ [ q ] n −| S | . Let | S | = µn . W e will only consider the case that µ < δ . It is e asy to see that the resulting co de C − S has distance d ( C − S ) > ( δ − µ ) n . T h ere is a 1-1 corresp ondence b et wee n co dew ords in C and their pro jections in C − S . F or the co de C − S , it will b e con v enien t to consider standard Hamming distance, to a v oid normalizing by 1 − µ . F or η < 1 − µ , let ℓ − S ( η ) b e the maxim um n um b er of co dewords of C − S that lie in a Hamming ball of radius ηn in [ q m ] n (1 − µ ) . Lemma 3.2. F or any η < 1 − µ , ℓ − S ( η ) 6 ℓ ( η + µ ) . Pr o of. T ak e a receiv ed w ord r − S ∈ [ q ] n (1 − µ ) so that there are L co dewords c − S 1 , . . . , c − S L satisfying d ( r − S , c − S i ) 6 η n . Deﬁn e r ∈ [ q ] n b y ﬁ xing v alues at the set S arb itrarily . By the triangle inequalit y , d ( r , c i ) 6 ( η + µ ) n , sho wing that ℓ ( η + µ ) > L .  Assume w e ha v e a pro cedur e List -Decode that ta k es as inpu ts set S ⊆ [ n ], r ∈ [ q ] n , an error parameter e and return s al l co d ewords c ∈ C so that d ( c − S , r − S ) 6 e (it need not b e eﬃcien t). W e use it to giv e an alg orithm for list deco ding C ⊙ m , whic h iden tiﬁes rows where errors hav e o ccur red and erases them. Assum e w e ha ve ﬁxed C 6 i = ( c 1 , . . . , c i ). W e erase the set of p ositions S where C 6 i 6 = R 6 i and then run a list decod er for C − S on r i +1 . The crucial observ ation is that sin ce the erased p ositions all corresp ond to errors, the n u m b er of errors drops b y | S | . Th e distance migh t also drop b y | S | , but since η < δ to b egin with, this tradeoﬀ w orks in our fa vor. Algorithm 1. Eras e-Decode Input: R ∈ [ q m ] n , η . Output: List L of all C ∈ C ⊙ m so that ∆ q m ( R, C ) 6 η . Set S 1 = φ, µ 1 = 0 . For i = 1 , . . . , m 1. Set L i = List-Decode ( S i , r i , ( η − µ i ) n ) . 2. Choose c i ← L i . 3. Set S i +1 = { j ∈ [ n ] s .t. C 6 i [ j ] 6 = R 6 i [ j ] } ; µ i +1 = S i +1 /n . Return C = ( c 1 , . . . , c m ) . 10 P . GOP ALAN, V. GUR USW AMI, AND P . RAGHA VENDRA In Step 2, we non-deterministic al ly try all p ossible c hoices for c i ; th e list L is obtained by taking all p ossib le C s that might b e retur ned b y this algorithm. Also, c i is a co deword in C − S i but w e can think of it as a co dew ord in C by the 1-1 corresp ondence. Diﬀeren t c hoices for c i lead to diﬀerent sets S i +1 , a nd hence to diﬀeren t li sts L i +1 . So the execution of Erase-Decode is b est view ed as a tree, w e formalize this b elo w. F or a receiv ed w ord R , T ree( R ) is a tree with m + 1 level s. The ro ot is at lev el 0. A no de v at level i is lab eled b y C ( v ) = ( c 1 , . . . , c i ). It is asso ciated with a s et S ( v ) ⊆ [ n ] of erasures accum ulated so far which has size µ ( v ) n . The resulting co de C − S ( v ) has minimum distance δ ( v ) n > ( δ − µ ( v )) n . W e ﬁ nd all co dew ords in C − S ( v ) that are within d istance ( η − µ ( v )) n of the receiv ed w ord r − S ( v ) i +1 , call this list L ( v ). By L emma 3 .2 , L ( v ) con tains at most ℓ ( η ) co dew ords. Eac h edge lea ving v is lab elled b y a distinct co deword c i +1 from L ( v ); it is assigned a weigh t w ( c i +1 ) = d ( c − S ( v ) i +1 , r − S ( v ) i +1 ) /n . T he w eigh t w ( c ) ∈ [0 , 1] of an edge indicates how man y new erasures that edge con tributes. Thus µ ( v ) = w ( c 1 ) + · · · + w ( c i ). Th e lea v es at lev el m corresp ond to co dew ords in the list L . There migh t b e no out-edge s f rom v if the list L ( v ) is empt y . This could result in a leaf no de at a lev el i < m which does not corresp ond to co dew ords. Thus the n umber of lea v es in T ree( R ) is an upp er b ound on the list-size for R . In order to b ound the n um b er of lea v es, we assign colors to th e v arious ed ges b ased on their w eigh ts. L et c be an edge lea ving the verte x v . W e co lor it White if w ( c ) < δ − η , Blue if w ( c ) > δ − η bu t w ( c ) < δ ( v ) 2 and Red if w ( c ) > δ ( v ) 2 . Wh ite edges corresp ond to codewo rds that are v ery close to the receiv ed w ord, Blue edges to co dew ords th at are within the unique- deco ding radius, and Red edges to co dewords b ey ond the unique decodin g r ad iu s. W e b egin by observing that White edges can only occur if the list is of size 1. Lemma 3.3. If a vertex v has a White out-e dge, then it has no other out-e dges. Pr o of. Assu me that the edge lab elled w ith c ∈ L ( v ) is colored White , so that d ( c, r − S ( v ) i +1 ) < ( δ − η ) n . L et c ′ b e another codeword in L ( v ), so that d ( c ′ , r − S ( v ) i +1 ) 6 ( η − µ ( v )) n . Then b y the triangle in equ alit y , d ( c, c ′ ) < ( δ − η ) n + ( η − µ ( v )) n = ( δ − µ ( v )) n 6 δ ( v ) n But this is a con tradiction since d ( c, c ′ ) > δ ( v ) n .  W e observe that Blue edge s do n ot cause m u c h branching and cannot result in v ery deep paths. Lemma 3.4. A vertex c an have at most o ne Bl ue e dge le aving it. A p ath fr om the r o ot to a le af c an h ave no mor e than ⌈ η δ − η ⌉ Blue e dges. Pr o of. T h e ﬁrs t part holds as there can b e at most one codeword within the unique deco din g radius. Eac h Blue edges results in at least ( δ − η ) n e rasures. T herefore, after ⌈ η δ − η ⌉ Blue edges, all η n errors ha v e b een identiﬁed, s o all remaining edges ha ve to b e White .  Lastly , w e show that Red edges do not give d eep paths either, although a v ertex can h a ve up to ℓ ( η ) Red edges lea ving it. LIST DECODING PR ODU CT A ND INTERLEA VED CODES 11 Lemma 3.5. A p ath fr om the r o ot to a le af c an ha ve no mor e than ⌈ log ( δ δ − η ) ⌉ Red e dges. Pr o of. W e claim that every Red edge lea v in g v ertex v has w eigh t at least ( δ − µ ( v )) / 2. Indeed, since c is b eyond the unique-deco ding radius of C − S ( v ) , w ( c ) > δ ( v ) 2 , and the r elativ e distance δ ( v ) of the code C − S ( v ) at nod e v satisﬁes δ ( v ) > ( δ − µ ( v )) n . Assume no w f or con tradiction that some path from the ro ot to a leaf con tains k red edges for k > ⌈ log ( δ δ − η ) ⌉ . Supp ose that the edges h a ve we igh ts ρ 1 , . . . , ρ k resp ectiv ely . Contract the Blue an d White edges b et ween su ccessiv e Re d edges in to a s ingle edge, whose weig h t is the sum of we igh ts o f the co nt racted edge s. W e al so do this for the edge s b efore the ﬁrst Red edge and those a fter the last Red edge. Th is give s a path con tains 2 k + 1 edges, where th e ev en edges are Re d , and the weig h t of th e ed ges along the path are β 1 , ρ 1 , β 2 , . . . , ρ k , β k +1 resp ectiv ely . Let v i b e the paren t v ertex of the i th Red edge for i ∈ [ k ]. Then w e ha ve µ ( v 1 ) = β 1 and µ ( v i ) = β i + ρ i − 1 + µ ( v i − 1 ) for j > 1. But since ρ i − 1 > ( δ − µ ( v i − 1 )) / 2 a nd β i > 0, w e get µ ( v i ) > δ + µ ( v i − 1 ) 2 No w a simple induction o n i prov es that µ ( v i ) > δ (1 − 2 1 − i ). If w e tak e i = ⌈ lo g ( δ δ − η ) ⌉ + 1, then µ ( v i ) > δ  1 − δ − η δ  = η . So when w e d ecod e at ve rtex v i , all the error lo cations h a ve b een iden tiﬁed and erased. Hence w e are n o w deco din g from η < δ erasures and no err ors , so the d eco din g is unique and error- free. So v ertex v i will ha ve a sin gle White edge leaving it and n o Red edges, whic h is a con tradiction.  Theorem 3.6. Assume η < δ and let b = ⌈ η δ − η ⌉ , r = ⌈ log δ δ − η ⌉ . Then T ree( R ) has at most  b + r r  ℓ ( η ) r le aves (and henc e ℓ ⊙ m ( η ) 6  b + r r  ℓ ( η ) r ). Pr o of. W e ﬁrst cont ract the White edges, since they are the only out-edge s lea ving their paren t no des. This giv es a tree with only Red and Blue edges. Let t ( b, r ) denote th e maxim u m n um b er of lea v es in a tree where eac h path has at most b Blue and r R ed edges, and eac h n o de ha ve ha v e at most one Blue edge and ℓ ( η ) R ed edges lea ving it. So w e ha v e the recursion t ( b, r ) 6 t ( b − 1 , r ) + ℓ ( η ) t ( b, r − 1) with the base case t ( b, 0) = 1. It is easy to c hec k that t ( b, r ) 6  b + r r  ℓ ( η ) r .  W e conclude this section b y pro ving that the condition η < δ is necessary . Lemma 3.7. If c o de C has r elative dista nc e exactly δ , then ℓ ⊙ m ( δ ) > 2 m . Pr o of. T ak e c 1 , c 2 ∈ C where ∆ q ( c 1 , c 2 ) = δ . No w tak e the receiv ed word in C ⊙ m to b e R = ( c 1 , . . . , c 1 ). F or ev ery T ⊆ [ m ], let C T to b e the co d eword wher e the i th column equ als c 1 if i ∈ T and c 2 otherwise. It is easy to sho w that ev ery such codeword is at distance at most δ from R , sho win g that ℓ ⊙ m ( δ ) > 2 m .  12 P . GOP ALAN, V. GUR USW AMI, AND P . RAGHA VENDRA 3.1. A Efficient Decoding Al gorithm for C ⊙ m . Giv en a receiv ed w ord R = ( r 1 , . . . , r m ), for i 6 m let R 6 i = ( r 1 , . . . , r i ). Algorithm 2. Deco deC ⊙ m Input: R ∈ [ q m ] n , η . Output: List of all C ∈ C ⊙ m so that ∆ q m ( R, C ) 6 η . 1. For i = 1 , . . . , m Set L i = DecodeC ( r i , η ) . 2. Set L 6 1 = L 1 . 3. For i = 2 , . . . , m For C ∈ L 6 i − 1 × L i , Add C to L 6 i if ∆ q m ( C, R 6 i ) 6 η . 4. Return L 6 m . Claim 3.8 . Assume that Decode C ( r , η ) runs in time T . Then DecodeC ⊙ m ( R, η ) r eturns a list of c o dewor ds within distanc e η of R in time O ( mT + m 2 nℓ ( η ) ℓ ⊙ m ( η )) . Pr o of. F or an y C suc h th at ∆ ( C , R ) 6 η , it m ust hold that for ev er y i , ∆ q i ( C 6 i , R 6 i ) 6 η . An easy ind uction on i sho ws that C 6 i ∈ L 6 i , whic h prov es the correctness of the algorithm. It is clear th at Step 1 tak es time O ( mT ). T o b ound S tep 3, w e u se the follo wing simple observ ation for all i 6 m : |L 6 i | 6 ℓ ⊙ i ( η ) 6 ℓ ⊙ m ( η ) . Th us eac h iteration of the loop in Step 3 requires computing th e distance betw een R and at most ℓ ( η ) ℓ ⊙ m ( η ) candidates for C .  4. List Decoding T ensor Products In this s ection and the next, C 1 ⊂ F n 1 q and C 2 ⊂ F n 2 q will b e linear co des ov er F q . Give n a matrix A ∈ [ q ] n 2 × n 1 , and tw o subsets S ⊆ [ n 2 ] a nd T ⊆ [ n 1 ], w e shall use A [ S, T ] to denote the submatrix in dexed b y ro w s in S and columns in T . F ur ther, w e will write A [ S, ∗ ] instead of A [ S, [ n 1 ]]. Thus th e sym b ol “ ∗ ” when used as an in dex, denotes the sets [ n 1 ] or [ n 2 ]. Fix a receiv ed word R ∈ [ q ] n 2 × n 1 and a co dew ord C ∈ C 2 ⊗ C 1 so that δ ( R , C ) 6 η ∗ − 3 ε . The advice/guess A [ S, T ] t o the algorithm Tens orDecode consists of the v alues of C on a random submatrix S × T . Giv en the advice A , the Tensor Decode algorithm works in four phases, describ ed informally b elo w. This is follo w ed by a formal description of the alg orithm in Figure 1 . The reader might ﬁnd Figure 2 helpful to dev elop in tuition ab out the op eration of th e v arious ph ases. Phase 1: W e run the list decoder on eac h ro w s ∈ S to get a new advice strin g B [ S, ⋆ ]. If there is a c o dewo rd in the list th at agrees with the advice A [ s, T ], w e set B [ s, ⋆ ] to b e that co dew ord (ties are broken arbitrarily; or w e could declare an erasur e if there isn’t a un iqu e c hoice). If there is no suc h co deword, we s et ev ery sym b ol in B [ s, ⋆ ] to ⊥ (wh ic h denotes an LIST DECODING PR ODU CT A ND INTERLEA VED CODES 13 erasure). Note that the list for ro w s migh t conta in some codewo rd other than C [ s, ⋆ ] that agrees with C [ s, ⋆ ] on th e p ositions in T . In such a case B [ s, ⋆ ] could b e incorrect. As a resu lt, some ro ws in B [ S, ⋆ ] agree with th e codeword C , some of them could b e wr ong, and the rest are erasures. Claim 5.3 show that with high pr ob ab ility , (1 − δ 2 + ε ) | S | of the ro ws are correct, and n o more than ε | S | are in correct. Phase 2: Viewed columnwise, B [ S, ⋆ ] give s us advice strings for the co-ordinates S of ev ery column codeword. How ev er, the advice is noisy: it is correct on (1 − δ 2 + ε ) fraction of co- ordinates within S , wrong on an ε fraction, and ⊥ on the rest. But since any t wo cod ew ord s in the column co de C 2 are distance δ 2 apart, in exp ectatio n the advice string has more agreemen t with the correct co dew ord than any ot her; th us it is lik ely to identify the correct co dew ord from a small list of candidates. W e crea te a new advice string D [ ⋆, ⋆ ] b y list deco ding every column t ∈ [ n 1 ], and selecting from the list a co dew ord that disagrees with B [ S, t ] in le ss than ε fraction of co -ordinates. If n o suc h codeword exists w e set th e column to ⊥ . Claim 5.4 sho ws that at least (1 − δ 1 + 2 ε ) fraction of columns are correctly deco ded to the corresp ondin g columns of C and n o more than ε f r action are incorrectly deco ded. Phase 3: View ed ro w-wise, D giv es an advice string for ev ery ro w that is corr ect on at least (1 − δ 1 + ε ) n 1 co-ordinates, wrong o n at most εn 1 and blank on th e rest. The advice though noisy is sound: since the code C 1 has distance δ 1 n 1 , a simple app licatio n of the triangle inequalit y sho w s that th ere is a unique co deword which disagrees w ith D [ s, ⋆ ] on few er than εn 1 co-ordinates, and that is the r o w co deword C [ S, ⋆ ]. Th u s the advice uniqu ely identiﬁes the correct ro w cod ew ord . W e create a new receiv ed w ord E b y list deco ding eac h ro w and usin g D to iden tify the c orrect codewo rd in the list, and set ting the ro w to ⊥ if no su c h codeword exists. Claim 5.5 sho ws this step will ﬁn d the correct co dew ord on (1 − δ 2 + 3 ε ) fraction of the ro ws. Phase 4: When viewed column-wise, E giv es the correct v alue of C on 1 − δ 2 + 3 ε fraction of co -ordinates, a nd is blank on the rest. Crucially , it do es not ha ve an y incorrect sym b ols. So no w w e can reco v er C b y deco ding eac h column from erasures (note that one can uniquely deco de C 2 from less than a fraction δ 2 of erasures). It is easy to sho w that the list deco ding radius reac hed by T ensorDeco de is the correct one. Lemma 4.1. F or a line ar c o de C , ℓ ( C ⊗ 2 , δ η ) > ℓ ( η ) . Pr o of. Let r ∈ [ q ] n b e a receiv ed wo rd with co dewo rds c 1 , . . . , c ℓ ∈ C within radius η . T ak e c 0 to b e a co deword of m inim um wei gh t δ . Deﬁne the receiv ed w ord r ′ = c 0 ⊗ r . It is easy to see that the codewords c ′ i = c 0 ⊗ c i for i ∈ [ ℓ ] are all within distance δη from r ′ , w hic h pro v es the claim.  Th us, if list deco din g C b ey ond radius η is combinatoria lly in tractable, th en so is d eco din g C ⊗ 2 b ey ond radius δ η . 14 P . GOP ALAN, V. GUR USW AMI, AND P . RAGHA VENDRA Algorithm 3. Tens orDecode Setup : L et D ecode 1 , Deco de 2 denote list decod ing algo rithms f or C 1 and C 2 , up to error rates η 1 and η 2 resp ectiv ely . Let ℓ 1 ( η 1 ) and ℓ 2 ( η 2 ) b e the upp er b ounds on list size output b y Deco de 1 and Deco de 2 resp ectiv ely . Fix η ∗ = min( δ 1 η 2 , δ 2 η 1 ). Let ε > 0 b e a parameter to the algorithm. Input : A receiv ed word R such that δ ( R, C 2 ⊗ C 1 ) 6 η ∗ − 3 ε . Output : A list L of all cod ew ord s C ∈ C 2 ⊗ C 1 with δ ( C , R ) 6 η ∗ − 3 ε . • Pick subsets S ⊆ [ n 2 ] a nd T ⊆ [ n 1 ] u niformly at random among all subsets of size m 1 and m 2 respecti vely. • For each assignmen t A : S × T → [ q ] -- P hase 1 (Comp uting B : S × [ n 1 ] → [ q ]) For each s ∈ S , ∗ List decode the row R [ s, ∗ ] : L s = Deco de 1 ( R [ s, ∗ ]) . ∗ Set B [ s , ∗ ] to be an arbitra ry codeword c i n the list L s satisfyi ng c [ T ] = A [ s, T ] and B [ s, ∗ ] = ⊥ if no s uch codeword exists. Define S f ail = { s ∈ S | B [ s, ∗ ] = ⊥} a nd S succes s = S − S f ail . -- P hase 2 (Comp uting D : [ n 2 ] × [ n 1 ] → [ q ]) For each t ∈ [ n 1 ] , ∗ List decode the column R [ ∗ , t ] : L t = Deco de 2 ( R [ ∗ , t ]) ∗ Set D [ ∗ , t ] to be an y codeword c in the list L t satisfyi ng ∆( c [ S succes s ] , B [ S succes s , t ]) < ε | S | . F ix D [ ∗ , t ] = ⊥ if no such codeword exists. Define T f ail = { t ∈ [ n 1 ] | D [ ∗ , t ] = ⊥} a nd T succes s = [ n 1 ] − T f ail . -- P hase 3 (Comp uting E : [ n 2 ] × [ n 1 ] → [ q ]) For each s ∈ { n 2 } , ∗ List decode the row R [ s, ∗ ] : L s = Deco de 1 ( R [ s, ∗ ]) . ∗ Set E [ s, ∗ ] to be any cod eword c in the list L s satisfyi ng ∆( c [ T succes s ] , D [ s, T succes s ]) < εn 1 . Set E [ s, ∗ ] = ⊥ if no such codeword exists. Define U f ail = { s ∈ [ n 2 ] | E [ ∗ , t ] = ⊥} and U succes s = [ n 2 ] − U f ail . -- P hase 4 ( Computing C : [ n 2 ] × [ n 1 ] → [ q ] ) For each t ∈ { n 1 } , Unique decode the column E [ ∗ , t ] under er asures: C [ ∗ , t ] = UniqueDeco deErasures 2 ( E [ ∗ , t ]) -- Output C if δ ( C, R ) 6 η ∗ − 3 ε . Figure 1. List decodin g algorithm for tensor pr o duct LIST DECODING PR ODU CT A ND INTERLEA VED CODES 15 C 2 ( n 2 , δ 2 , η 2 ) E [ ∗ , ∗ ] Phase II I C 2 ( n 2 , δ 2 , η 2 ) A T S Guess A [ S, T ] C 1 ( n 1 , δ 1 , η 1 ) C 2 ( n 2 , δ 2 , η 2 ) T Phase I B [ S, ∗ ] S success T f ail T success U success U f ail C 1 ( n 1 , δ 1 , η 1 ) Phase II D [ ∗ , ∗ ] T w T r C 1 ( n 1 , δ 1 , η 1 ) S r S w S f ail Figure 2. Phases of the TensorDecode Alg orithm 5. Anal ysis of the Tensor Pr od uct D ecoder In this section, we will p ro ve the correctness, analyze the list size output and compute r unning time of the Tens orDecode algo rithm. T o wards this go al, we ﬁrst presen t a concen tration b ound concerning s ums of samples c h osen without r eplacemen t. Lemma 5.1. L et z 1 , z 2 , . . . , z n b e r e al numb ers b ounde d in [0 , 1] . L et S ⊆ [ n ] b e a uniformly r andom subset o f si ze m . Then, Pr h    1 | S | X s ∈ S z s − 1 n X i ∈ [ n ] z i    > γ i 6 p ( γ , | S | ) = 2 e − 2 γ 2 m The ab ov e concen tr ation b oun d e ssen tially a restatemen t o f Corollary 1 . 1 in Serﬂing’s w ork [ 27 ] on sums in samp ling without replacemen t. Henceforth, for the sak e of succinctness, we shall use the notation p ( γ , m ) to denote the upp er b oun d (2 e − 2 γ 2 m ) in the ab o v e lemma. Firstly , w e will show that for ev ery co deword C close enough to the receiv ed wo rd, the algorithm TensorDecode returns C w ith p robabilit y 1 / 4, giv en the r igh t advice s tr ing A = C [ S , T ]. 16 P . GOP ALAN, V. GUR USW AMI, AND P . RAGHA VENDRA Theorem 5.2. F or a c o dewor d C ∈ C 2 ⊗ C 1 within distanc e η ∗ − 3 ε of the r e c eive d wor d R , the algorithm Tenso rDecode with input R , and g uess A = C [ S, T ] r eturns C with pr ob ability a t le ast 1 − p ( ε, m 2 ) − ℓ 1 ( η 1 ) p ( δ 1 , m 1 ) /ε − ℓ 2 ( η 2 ) p ( ε, m 2 ) /ε . Pr o of. T h e argument is broken up in to fou r p arts (Claims 5.3 , 5 .4 , 5.5 , 5.6 ) eac h concerning a p hase of the TensorDecode algorithm.  5.1. Phase 1. In this phase, th e algorithm constru cted the advice string B [ S, ∗ ] starting with the guess A [ S, T ] and list deco ding eac h ro w in S . Of the set of ro ws S succes s on whic h th e deco ding succeeded, some of them are decod ed correctly to th e corresp onding ro w in C , w hile some are incorrect. Deﬁn e sets S r , S w ⊆ S succes s to b e the s ets of ro ws that are deco ded correctly and incorrectly resp ectiv ely . F ormally , S r = { s ∈ S | B [ s, ∗ ] = C [ s, ∗ ] } S w = S succes s − S r W e make the follo wing claim regarding the fractio n of ro w s deco ded correctly . Claim 5.3. With pr ob ability at le ast 1 − p ( ε, m 2 ) − ℓ 1 ( η 1 ) p ( δ 1 , m 1 ) /ε over the choic e of the sets S , T | S succes s | > (1 − δ 2 + 2 ε ) | S | | S r | > (1 − δ 2 + ε ) | S | | S w | 6 ε | S | . Pr o of. Let S 1 denote the set of rows in S with fewe r than a v erage n u m b er of errors. S p eciﬁcally , S 1 is deﬁned as S 1 = { s ∈ S | δ ( C [ s, ∗ ] , R [ s, ∗ ]) 6 η 1 } . Observe th at for eac h s ∈ S 1 , the cod ew ord C [ s, ∗ ] will b e part of the list L s , obtained b y de- co ding the ro w s . Consequen tly , for eac h s ∈ S 1 , B [ s, ∗ ] 6 = ⊥ ,i.e., S 1 ⊆ S succes s . W e s h all lo wer b ound the size of S succes s b y the siz e of S 1 . App ly Lemma 5.1 w ith { z i = δ ( C [ i, ∗ ] , R [ i, ∗ ]) } and the set S . S ince δ ( C, R ) 6 δ 2 η 1 − 3 ε , the a verag e of the z i is less than or equal to δ 2 η 1 − 3 ε . Th us w e get Pr S h δ ( C [ S, ∗ ] , R [ S, ∗ ]) > δ 2 η 1 − 2 ε i 6 p ( ε, m 2 ) . Let u s supp ose δ ( C [ S, ∗ ] , R [ S, ∗ ]) 6 δ 2 η 1 − 2 ε . By an av eraging argumen t, for at most δ 2 − 2 ε fraction of the ro w s { C [ s , ∗ ] | s ∈ S } , the distance δ ( C [ s, ∗ ] , R [ s, ∗ ]) > η 1 , i.e., | S 1 | > (1 − δ 2 + 2 ε ) | S | . This immediate ly implies the lo w er b oun d on size of S succes s . No w we shall up p er b ound the pr obabilit y that a ro w s ∈ S is deco ded to an incorrect co dew ord c 6 = C [ s, ∗ ]. Fix an s ∈ S and a co dew ord c ∈ L s other than C [ s, ∗ ]. The codeword c is chosen, if an d only if it agrees with the advice A [ S, T ] = C [ S, T ] on the columns in T , i.e., C [ s , T ] = c [ T ]. Applying Lemma 5. 1 on the set of rea l num b er s n z i = 1  C [ s , i ] 6 = c [ i ]  o and the c hoice of the set T , we can conclude Pr T h C [ s , T ] = c [ T ] i 6 p ( δ 1 , m 1 ) . By a un ion b oun d o v er all co dewo rds c ∈ L s , for an y ro w s ∈ S , the probabilit y of deco ding an incorrect co deword is upp er b oun ded by Pr T h C [ s , ∗ ] 6 = B [ s, ∗ ] ∧ B [ s, ∗ ] 6 = ⊥ i 6 ℓ 1 ( η 1 ) p ( δ 1 , m 1 ) . LIST DECODING PR ODU CT A ND INTERLEA VED CODES 17 Hence in expectation, at most ℓ 1 ( η 1 ) p ( δ 1 , m 1 ) fraction of ro ws in S are d ecod ed incorrectly , i.e E[ | S w | ] 6 ℓ 1 ( η 1 ) p ( δ 1 , m 1 ) | S | . Ap plying Mark o v’s inequalit y , w e get Pr T h   S w   > ε | S | i 6 ℓ 1 ( η 1 ) p ( δ 1 , m 1 ) /ε . Supp ose | S w | 6 ε | S | , then observ e that | S r | = | S succes s − S w | > (1 − δ 2 + ε ) | S | . Th us with p robabilit y a t least 1 − p ( ε, m 2 ) − ℓ 1 ( η 1 ) p ( δ 1 , m 1 ) /ε b oth of the assertions of the claim hold.  5.2. Phas e 2. In Phase 2, the algo rithm u ses the advice B [ S, ∗ ] generated in the ﬁrst phase in order to deco de the columns of the matrix. Among the columns T succes s that are deco ded s uc- cessfully , let T r , T w ⊆ T succes s denote the set of r o ws th at are d ecod ed correctly and in correctly resp ectiv ely . F ormally , deﬁne sets T r , T w as follo ws: T r = { t ∈ [ n 1 ] | D [ ∗ , t ] = C [ ∗ , t ] } T w = T succes s − T r Giv en th at the advice B [ S, ∗ ] generate d in the ﬁrst phase is correct on suﬃcien tly man y ro ws, the follo wing claim su ggests that at least 1 − δ 1 fraction of th e column s are deco ded successfully , of whic h most columns agree with co dewo rd C . In other wo rds, the a dvice D [ ∗ , ∗ ] generated b y this phase is a ne ar-sound advice string with almost all co lumns having either th e correct v alue or a failure sym b ol ⊥ . Claim 5.4. Con ditione d on the event that the assertions o f Claim 5.3 hold, With p r ob ability at le ast 1 − ℓ 2 ( η 2 ) p ( ε, m 2 ) /ε over the choic e of the sets S , T | T succes s | > (1 − δ 1 + 3 ε ) n 1 | T r | > (1 − δ 1 + 2 ε ) n 1 | T w | 6 εn 1 Pr o of. Along the lines of the pro of of Claim 5.3 , let T 1 b e th e set of column s with few er than a verag e fr action of errors. Deﬁne the set T 1 as follo ws: T 1 = { t ∈ [ n 1 ] | δ ( C [ ∗ , t ] , R [ ∗ , t ]) 6 η 2 } By an a v eraging argumen t, for at most δ 1 − 3 ε fr action of the c olumns { C [ ∗ , t ] | t ∈ [ n 1 ] } , the distance δ ( C [ ∗ , t ] , R [ ∗ , t ]) > η 2 , i.e., | T 1 | > (1 − δ 1 + 3 ε ) n 1 . Observe that for ea c h column t ∈ T 1 , the codeword C [ ∗ , t ] b elongs to the list L t . By Claim 5.3 , in Phase 1, at most ε | S | ro ws in S were decoded incorrectl y , i.e, | S w | 6 ε | S | . Hence the co dew ord C [ ∗ , t ] satisﬁes ∆( C [ S succes s , t ] , B [ S succes s , t ]) 6 ε | S | . Consequen tly , for eac h t ∈ T 1 , D [ ∗ , t ] 6 = ⊥ , i.e. T 1 ⊆ T succes s . T h us a lo w er b ound on the size of T succes s is gi v en b y | T 1 | > (1 − δ 1 + 3 ε ) n 1 . Fix a t ∈ [ n 1 ]. Let us supp ose D [ ∗ , t ] is neither equal to ⊥ or C [ ∗ , t ]. Thus D [ ∗ , t ] is a co dewo rd in C 2 suc h that ∆( D [ S succes s , t ] , C [ S succes s , t ]) 6 ε | S | By assertion of 5.3 , | S f ail | 6 | S | − | S succes s | 6 ( δ 2 − 2 ε ) | S | . F r om the ab o v e inequalities, w e can conclude δ ( D [ S, t ] , C [ S, t ]) 6 δ 2 − ε . 18 P . GOP ALAN, V. GUR USW AMI, AND P . RAGHA VENDRA Fix a co deword c ∈ L t . By the distance p rop erty of the co de, w e h a ve δ ( c, C [ ∗ , t ]) > δ 2 . Applying Lemma 5.1 on the set of r eal num b ers n z i = 1  c [ i ] 6 = C [ i , t ]  o and the s et S , we get Pr S h δ ( C [ S, t ] , c [ S ]) 6 δ 2 − ε i 6 p ( ε, m 2 ) . By a un ion b ound o ve r all cod ew ord s c ∈ L t , f or a column t ∈ T , Pr S h C [ ∗ , t ] 6 = D [ ∗ , t ] ∧ D [ ∗ , t ] 6 = ⊥ i 6 ℓ 2 ( η 2 ) p ( ε, m 2 ) . In other w ords , the exp ected size of T w is at most ℓ 2 ( η 2 ) p ( ε, m 2 ) | T | . Applying Marko v’s inequalit y , we get: Pr S h   T w   > εn 1 i 6 ℓ 2 ( η 2 ) p ( ε, m 2 ) /ε . T o ﬁnish the argument, observe that | T r | = | T succes s − T w | > (1 − δ 1 + 2 ε ) n 1 . Th us with pr obabilit y 1 − ℓ 2 ( η 2 ) p ( ε, m 2 ) /ε b oth of the assertions of th e claim hold.  5.3. Phas e 3. This p h ase con v erts a ne ar sound advice D [ ∗ , ∗ ] in to a p erfe ctly so und advice E [ ∗ , ∗ ] all of whose ro w s are either the correct co dewords or the fail s y mb ol ⊥ . The follo w ing claim is a formal statemen t of this fact. Claim 5.5. Conditione d on the ev e nt th at the assertio ns of Claim 5.4 hold , for e ach s ∈ U succes s , E [ s, ∗ ] = C [ s, ∗ ] and | U succes s | > (1 − δ 2 + 3 ε ) n 2 . Pr o of. F or e ac h s ∈ [ n 2 ], w e claim that C [ s, ∗ ] is the u nique co deword satisfying ∆( c [ T succes s ] , D [ s, T succes s ]) 6 εn 1 . Clearly , this implies that for eac h ro w s ∈ U succes s , we deco de the correct cod ew ord , i.e., E [ s, ∗ ] = C [ s, ∗ ]. F or the sake of contradicti on, let us su pp ose there exists c 6 = C [ s, ∗ ] satisfying ∆( c [ T succes s ] , D [ s, T succes s ]) 6 εn 1 . By tr iangle inequ ality , we can conclude ∆( c [ T succes s ] , C [ s, T succes s ]) 6 2 εn 1 . F u rther from claim 5.4 , | T f ail | 6 [ n 1 ] − | T succes s | 6 ( δ 1 − 3 ε ) n 1 . Th is implies ∆( c, C [ s, ∗ ]) 6 2 εn 1 + ( δ 1 − 3 ε ) n 1 < δ 1 n 1 . This is a contradicti on since c and C [ s, ∗ ] are t w o distinct co dewo rds of C 1 that are less than δ 1 apart. Let U 1 denote the set of ro ws w ith less than a v erage fraction of errors; formally , U 1 = n s ∈ [ n 2 ]    δ ( C [ s , ∗ ] , R [ s, ∗ ]) 6 η 1 o . By an a v er aging argumen t, for at m ost δ 2 − 3 ε fraction of th e r o ws of C , the distance δ ( C [ s , ∗ ] , R [ s, ∗ ]) > η 1 , i.e., | U 1 | > (1 − δ 2 + 3 ε ) n 2 . F or eac h ro w s ∈ U 1 , we hav e C [ s, ∗ ] ∈ L s . F r om Claim 5.4 , for at most εn 1 columns in T succes s , C [ ∗ , t ] 6 = D [ ∗ , t ]. Consequently , for eac h ro w s ∈ U 1 , E [ s, ∗ ] 6 = ⊥ , i.e., U 1 ⊆ U succes s . He nce w e get | U succes s | > | U 1 | > (1 − δ 2 + 3 ε ) n 2 .  LIST DECODING PR ODU CT A ND INTERLEA VED CODES 19 5.4. Phas e 4. This is a fairly simple p hase where a p erfe ctly sound a dvice E [ ∗ , ∗ ] is used to completely retriev e the cod ew ord C . Sp eciﬁcally , w e mak e the follo wing claim : Claim 5.6. Conditione d on th e event that the assertio ns of Claim 5.5 hold, the algo rithm TensorDecode outputs the c o dewor d C . Pr o of. By Claim 5.5 , we kno w E [ s, ∗ ] = C [ s , ∗ ] for at least (1 − δ 2 + 3 ε ) n 2 ro w s and E [ s, ∗ ] = ⊥ for the remaining r o ws. Hence f or eac h column t ∈ [ n 1 ], the U niqueDeco deErasures 2 algorithm returns the co deword C [ ∗ , t ]. Th us the algorithm TensorDecode returns the co dew ord C .  Theorem 5.7. Given two c o des C 1 , C 2 , for ev e ry ε > 0 , the numb er of c o dewo r ds of C 2 ⊗ C 1 within distanc e η ∗ = min( δ 1 η 2 , δ 2 η 1 ) − 3 ε of any r e c eive d wor d is b ounde d by ℓ ( C 2 ⊗ C 1 , η ∗ ) 6 4 q 1 4 δ 2 1 ε 2 ln 8 ℓ 1 ( η 1 ) ε ln 8 ℓ 2 ( η 2 ) ε . F urther, if C 1 and C 2 c an b e eﬃciently list de c o de d up to err or r ates η 1 , η 2 and C 2 is a line ar c o de, then C 2 ⊗ C 1 c an b e list de c o de d eﬃciently up to e rr or r ate η ∗ . Sp e ciﬁc al ly, if T denotes the time c omplexity of list de c o ding C 1 and C 2 , then the running time of the list de c o ding algorithm for C 2 ⊗ C 1 is O (4 q 1 4 δ 2 1 ε 2 ln 8 ℓ 1 ( η 1 ) ε ln 8 ℓ 2 ( η 2 ) ε × T n 1 n 2 ) Pr o of. Rewr iting the expression for the probabilit y in Theorem 5.2 using Lemma 5.1 , 1 − p ( ε, m 2 ) − ℓ 1 ( η 1 ) p ( δ 1 , m 1 ) ε − ℓ 2 ( η 2 ) p ( ε, m 2 ) ε = 1 − 2 e − 2 ε 2 m 2 − 2 ε  ℓ 1 ( η 1 ) e − 2 δ 2 1 m 1 + ℓ 2 ( η 2 ) e − 2 ε 2 m 2  . Set m 1 = 1 2 δ 2 1 ln 8 ℓ 1 ( η 1 ) ε and m 2 = 1 2 ε 2 ln 8 ℓ 2 ( η 2 ) ε . It is ea sy to see that the probabilit y of success is at least 1 4 with this choice o f parameters. In other w ords , with t his c hoice of parameters, an y co dewo rd C within distance η ∗ − 3 ε is output with p robabilit y at least 1 4 if the initial guess A i s consistent with C . Hence, the n umber of codewords within distance η ∗ − 3 ε from the rece iv ed wo rd R is ℓ ( C 2 ⊗ C 1 , η ∗ − 3 ε ) 6 4 q m 1 m 2 = 4 q 1 4 δ 2 1 ε 2 ln 8 ℓ 1 ( η 1 ) ε ln 8 ℓ 2 ( η 2 ) ε . It is easy to c hec k that the running time of the alg orithm is as claimed ab o v e.  Theorem 5.8. L et C b e a line ar c o de with dista nc e δ , list de c o dable up to an err or r ate η . F or every ε > 0 , the m -wise tensor pr o duct c o de C ⊗ m c an b e list de c o de d up to an err or r ate δ m − 1 η − ε with a list size exp  O ( ln ℓ ( η ) /ε ε 2 )  m ! . Pr o of. Ap p lying Theorem 5.7 with C 1 = C 2 = C , w e get list size b ound at error rate δ η − 3 ε for C ⊗ C . Applying theorem again on C ⊗ 2 , w e get list size b ounds at error rate δ 2 × ( δ η − 3 ε ) − 3 ε = δ 3 η − 3 δ 2 ε − 3 ε . In general, for C ⊗ 2 k let η 2 k denote t he error rate a t whic h we obtain a list size b ound. Then, η 2 k = δ 2 k − 1 η − 3 ε k X i =0 δ 2 k > δ 2 k − 1 η − 3 ε 1 − δ 2 20 P . GOP ALAN, V. GUR USW AMI, AND P . RAGHA VENDRA F or br evit y , let us denote by S k the list size ℓ ( C ⊗ 2 k , η 2 k ). Then from Theorem 5.7 , w e ha v e the follo wing recursive inequalit y: ln S k +1 6 ln q 4 δ 2 ε 2 ln 2 4 S k ε + ln 4 Rewriting the ab o ve inequalit y , ln 4 S k +1 ε 6 ln q 4 δ 2 ε 2 ln 2 4 S k ε + ln 4 + ln 4 ε 6 ln q 2 δ 2 ε 2 ln 2 4 S k ε Set s k = ln 4 S k /ε and a = ln q 2 δ 2 ε 2 . Th en we hav e the recurrence r elation : s k +1 6 a · s 2 k s 0 = ln 4 ℓ ( η ) ε Th us we g et s k 6 (ln 4 ℓ ( η ) /ε ) 2 k a 2 k − 1 < ( a ln 4 ℓ ( η ) /ε ) 2 k . Hence we obtain the follo wing list size b ound for m = 2 k . ℓ  C ⊗ m , δ m − 1 η − 3 ε 1 − δ 2  6 exp  ln q ln 4 ℓ ( η ) /ε 2 δ 2 ε 2  m ! Rewriting the ab o ve expression with ε in place of 3 ε 1 − δ 2 , ℓ ( C ⊗ m , δ m − 1 η − ε ) 6 exp  9 ln q ln 12 ℓ ( η ) /ε (1 − δ 2 ) 2 δ 2 (1 − δ 2 ) 2 ε 2  m ! .  6. Impro ve d List-s ize Bounds via Generalize d Ham ming Weights In this section, w e pro v e impro ved list- size b ound s on tensor pro ducts and interlea vings of binary linear codes. This is done b y making a connection b et ween the w eight- distributions of suc h co des and the cl assical coding theoreti c notion of G eneralized Hamming W eigh ts. This allo ws us to use the De letion tec hniqu e of [ 12 ] to reduce the problem of b oundin g list-sizes to the lo w-rank case. W e start b y introd ucing the version of the De letion Lemma that we need. It is a mildly stronger v ersion of the deletion lemma from [ 12 ], the graph theoretic view w as prop osed b y Impagliazzo. Lemma 6.1 (Deletio n lemma) . [ 12 ] L et C ⊂ F n q b e a line ar c o de over F q . L et C ′ ⊆ C b e a (p ossibly non-line ar) subset of c o dewor ds so that c ′ ∈ C ′ iﬀ − c ′ ∈ C ′ , and every c o dewor d c ∈ C \ C ′ has wt( c ) > µ . L et η = J q ( µ ) − γ for γ > 0 . Then ℓ ( C , η ) 6 γ − 2 ℓ ( C ′ , η ) . Pr o of. Let r ∈ F n q denote a receiv ed w ord , an d let L = { c 1 , . . . , c t } b e the list of all co d ew ord s in C so that ∆( r, c i ) 6 η . Construct an (und ir ected) grap h G w here V ( G ) = { c 1 , . . . , c t } and ( i, j ) is an ed ge if c i − c j ∈ C ′ . Ou r goal is to b ound | V ( G ) | . W e claim G do es not ha ve large ind ep endent sets. Let I = { c 1 , . . . , c s } b e an indep enden t set. This means that for e v ery i 6 = j ∈ [ s ], c i − c j 6∈ C ′ so ∆( c i , c j ) > µ . But ev ery codewo rd in I LIST DECODING PR ODU CT A ND INTERLEA VED CODES 21 lies within distance η o f r . W e now in vok e the Johns on b oun d w hic h states that in a cod e of distance µ , the list-size at radius J q ( µ ) − γ is b ound ed by γ − 2 . Th is shows that α ( G ) 6 γ − 2 . W e claim that the degree of G is b ounded by ℓ ( C ′ , η ). Supp ose that a verte x c has d neighb ors { c 1 , . . . , c d } . They can be written as c + c ′ 1 , . . . , c + c ′ d where c ′ ∈ C ′ . Sin ce ∆( c + c ′ i , r ) = ∆( c ′ i , r − c ) 6 η the co dew ords c ′ 1 , . . . , c ′ d giv e us a list of co dew ords in C ′ at distance η fr om th e receiv ed w ord r − c . He nce d 6 ℓ ( C ′ , η ). Th us G has degree d ( G ) 6 ℓ ( C ′ , η ) and the max ind ep endent set size α ( G ) 6 γ − 2 . Thus | G | 6 α ( G ) d ( G ) 6 γ − 2 ℓ ( C ′ , η ) .  The Dele tion lemma of [ 12 ] corresp ond s to taking C ′ to b e all co dewords for weigh t less than µ , and using ℓ ( C ′ , η ) 6 |C ′ | . Ho w ev er, in our applicatio ns |C ′ | w ill b e to o large for this to b e a useful b ound , thus w e essent ially use the Deletion lemma as a reduction to th e lo w-rank case. Generalized Hamming W eights (GHWs) arise natur ally in the c on text of list-deco ding from erasures [ 13 ]. F or a v ector v of length n , Supp( v ) ⊆ [ n ] d enotes the co-ordinates wh ere v i 6 = 0. F or a v ector space V , Supp( V ) = ∪ v ∈ V Supp( v ). Deﬁnition 6.2. The r th generalized Hamming w eight of a linear cod e C ⊆ F n 2 denoted by δ r ( C ) is deﬁn ed to b e | Su pp( V k ) | /n o v er all k -dimensional subspaces V k of the code C . Clearly , δ 1 ( C ) = δ ( C ) is just the minimum distance. The follo wing lo wer b ound on δ r ( C ) whic h is folklore [ 15 , 33 ], sa ys that as w e consider larger v alues of r , δ r ( C ) approac hes q q − 1 δ . Thus for binary linear co des, δ r ( C ) approac hes 2 δ ( C ) as r gro ws. Lemma 6.3. F or any line ar c o de C ⊆ F n q with minimum distanc e δ ( C ) a nd a ny r > 1 , δ r ( C ) > q q − 1 δ ( C )  1 − 1 q r  . Giv en a matrix C ∈ F n × m q , let Rank( C ) denote its rank, let Ro wSpan( C ) b e the space spann ed b y its ro ws and ColSpan( C ) b e the space spann ed by its columns. W e use the follo wing standard fact from linear algebra: F act 6.4. Given C ∈ F n × m q such that Rank( C ) = r , let h v 1 , . . . , v r i b e a b asis for RowSpan( C ) . Then we c an write C = r X s =1 u s ⊗ v s for some v e ctors { u 1 , . . . , u r } which form a b asis for ColSpan( C ) . 6.1. Int erlea ve d Codes. In this s u bsection C is a bin ary linear co de. W e ℓ ⊙ m ( η ) to denote ℓ ( C ⊙ m , η ). W e u se C ⊙ m r to denote the sub-co de of C ⊙ m consisting of codewords of rank at most r , and ℓ ⊙ m r ( η ) for ℓ ( C ⊙ m r , η ). The follo wing lemma relates th e rank of a cod ew ord to GHWs. Lemma 6.5. Given C ∈ C ⊙ m such that Rank ( C ) = r , wt( C ) > δ r ( C ) . 22 P . GOP ALAN, V. GUR USW AMI, AND P . RAGHA VENDRA The lemma holds since dim(ColSpan( C )) = r hence its sup p ort is a t least δ r ( C ). W e now apply the deleti on argumen t to reduce the problem of b ounding the l ist-size to the lo w-rank case. Lemma 6.6. L et C b e a binary line ar c o de and let r = ⌈ log 2 δ 2 ⌉ . Then for any η 6 δ , we have ℓ ⊙ m ( η ) 6 4 δ 4 ℓ ⊙ m r ( η ) . Pr o of. It is easy to chec k that δ r > 2 δ (1 − 2 − r ) > 2 δ − δ 3 , J ( δ r ) = 1 − p 1 − δ r > δ + δ 2 2 . The b ound on J ( δ r ) m a y b e v eriﬁed b y observing that J − 1 ( δ + δ 2 2 ) = 2 δ − δ 3 − δ 4 4 < δ r . Let C ′ consist of all co dew ords C where Rank( C ) 6 r so that w e can tak e µ = δ r . S in ce J ( µ ) > δ + δ 2 / 2 whereas η 6 δ , we hav e γ = J ( µ ) − η = ( δ − η ) + δ 2 2 > δ 2 2 . Applying th e Dele tion Lemma 6.1 , w e obtain the desired conclusion ℓ ⊙ m ( η ) 6 4 δ 4 ℓ ⊙ m r ( η ).  An immediate corollary of F act 6.4 is Corollary 6.7. Given a c o dewor d C ∈ C ⊙ m of r ank r , let { b [1] , . . . , b [ r ] } b e b asis for Ro w S pan( C ) . Then C c an b e writt en as C = P r s =1 c s ⊗ b [ s ] wh er e c s ∈ C for s ∈ [ r ] . Our goal is to reduce the lo w -rank problem for in terlea ved co d es to the case wh en m = r , b y ﬁ xing a basis for the ro w-space. The follo wing lemma narrows the c hoices for the basis elemen ts to ro ws that ha v e reasonably large weigh t. Lemma 6.8. L et ε > 0 and let η = δ − ε . L et C ∈ C ⊙ m b e a r ank r c o dewor d and R b e a r e c eive d w or d such that ∆( R, C ) 6 η . Ther e is a b asis { b [ 1] , . . . , b [ r ] } for Ro wSpan( C ) wher e wt( C, b [ s ]) > ε 2 1 − r for al l s ∈ [ r ] . Pr o of. Let S = { b ∈ Span( C ) | wt( R, b ) > ε 2 1 − r , b 6 = 0 m } . W e claim that S co n tains a basis for Ro wSp an( C ), or equiv alen tly dim( S ) = r . Assume fo r con tradiction that d im( S ) = r − 1, and that b [1] , . . . , b [ r − 1] is a b asis for it. Comp lete it to a b asis for Span( C ) by adding b [ r ], and let S ′ = b [ r ] + h b [1] , . . . , b [ r − 1] i . Note that S ′ is disjoin t from S . By Corollary 6.7 , we ca n w r ite C = P r t =1 c t ⊗ b [ t ]. If c r [ i ] 6 = 0, then C [ i ] ∈ S ′ . S ince wt( c r ) > δ n , it follo ws that δn of the ro ws of C lie in S ′ ; th at is wt( C, S ′ ) > δ . Since ∆( C, R ) 6 η , we ha v e wt( R, S ′ ) > ε . But since | S ′ | = 2 r − 1 , there m ust exist b ∈ S ′ so that wt( R, b ) > ε 2 1 − r . Since S ′ is disjoin t fr om S , this con tradicts the deﬁn ition of the set S .  LIST DECODING PR ODU CT A ND INTERLEA VED CODES 23 Lemma 6.9. L et ε > 0 and let η = δ − ε . Set r = ⌈ log 2 δ 2 ⌉ . We have ℓ ⊙ m ( η ) 6 2 r 2 +2 δ 4 ε r ℓ ⊙ r ( η ) . Pr o of. By Lemma 6.6 , it suﬃces to b ound ℓ ⊙ m r ( δ − ε ) wh ere r = ⌈ 2 δ 2 ⌉ . W e ﬁx the c hoice of basis { b [1] , . . . , b [ r ] } for Ro wSp an( C ). Lemma 6.8 s h o ws that there are at most 2 r 2 ε − r c hoices for the basis. W e then map R : F k 2 → F m 2 to a receiv ed word R ′ : F k 2 → { F r 2 ∪ ⋆ } as follo ws: R ′ ( x ) = ( ( λ 1 , . . . , λ r ) if R ( x ) = P i λ i b [ i ] ⋆ if R ( x ) 6∈ Span( C ) Ev ery C ′ satisfying ∆( R ′ , C ′ ) 6 η is in on-to-one corresp ond en ce with C so that ∆( R, C ) 6 η and S pan( C ) = h b [1] , . . . , b [ r ] i . S o the num b er of suc h co dewords is b oun ded by ℓ ⊙ r ( η ).  Clearly , ℓ ⊙ r ( η ) 6 ℓ ( η ) r . Plugging this into Lemma 6.9 give s Theorem 2.6 . F urther impro ve- men ts on this b ound are possib le using the analysis of Theorem 3.6 com b ined with b etter list-size b ounds f or deco ding binary codes from erasures; w e present them is Section 6.3 . The only step which needs q = 2 is Lemma 6.6 , where w e c ho ose r large enough so that J 2 ( δ r ) > δ . Th is d o es not ha ve an analogue o ver F q since δ r ma y o nly increase b y a factor q / ( q − 1), hence there ma y not b e an r s uc h that J q ( δ r ) > δ . But the s tep o f b ound ing the n um b er of small-rank co dewo rds w orks for any ﬁ eld. 6.2. Tens or Pr od ucts. In th is subsection, C 1 and C 2 are binary linear co des. W e use δ i,r to denote the r th generalized wei gh t of C i . W e u se ℓ i ( η ) for ℓ ( C i , η ) and ℓ ⊗ ( η ) for ℓ ( C 2 ⊗ C 1 , η ) and ℓ ⊗ r ( η ) for the list-size wh en we r estrict ourselv es to codewords of rank at most r . The follo wing lemma relates the w eight -distribution of tensor pro duct co d es to the generaliz ed Hamming w eight s of C 1 and C 2 . While the lemma is straigh tforw ard, w e ha ve not found a n explicit stat emen t in the lite rature. Lemma 6.10. Given C ∈ C 2 ⊗ C 1 such that Rank( C ) = r , wt( C ) > 2 δ 1 δ 2 (1 − 2 − r ) . Pr o of. T h e column rank of C is r , th us | S upp(ColSpan( C )) | > δ 2 ,r n 2 . E ac h of these indices corresp onds to a r ow from C 1 with weigh t δ 1 n 1 , thus o v erall the co dewo rd has weig h t at least δ 2 ,r δ 1 n 2 n 1 > δ 1 δ 2 (1 − 2 − r ) n 1 n 2 .  If we let wt r denote the min im u m weig h t of a r an k r codeword, we hav e wt r > 2 δ 1 δ 2 (1 − 2 − r ). W e now show a reduction to the lo w -rank case for tensor pro d ucts. Lemma 6.11. Set r = log ( 2 δ 1 δ 2 ) . Then for any η 6 δ 1 δ 2 , ℓ ⊗ ( η ) 6 4 δ 2 1 δ 2 2 ℓ ⊗ r ( η ) . Pr o of. W e clai m that for any δ 6 1 2 , J 2 (2 δ − δ 2 ) > δ + δ 2 / 2. T o pr o ve this, observ e that J − 1 2 ( δ ) = 2 δ (1 − δ ). He nce J − 1 2 ( δ + δ 2 / 2) = 2 δ + δ 2 − 2 δ 2 (1 + δ / 2) 2 < 2 δ − δ 2 . 24 P . GOP ALAN, V. GUR USW AMI, AND P . RAGHA VENDRA Lemma 6.10 sho w s that f or r = log( 2 δ 1 δ 2 ) an y co dewo rd C with Rank ( C ) = r h as w eigh t at least wt r = 2 δ 1 δ 2 (1 − 2 − r ) > 2 δ 1 δ 2 − δ 2 1 δ 2 2 . W e apply the Deletion lemma taking C ′ to b e al l co d ewords of rank a t most r , with µ = 2 δ 1 δ 2 − δ 2 1 δ 2 2 . S ince J ( µ ) > δ 1 δ 2 + δ 2 1 δ 2 2 / 2, whereas η 6 δ 1 δ 2 , w e can take γ = δ 2 1 δ 2 2 / 2, whic h giv es the desired b ound.  A corollary of F act 6 .4 for tensor pr o duct co d es is: Corollary 6.12 . L e t C ∈ C 2 ⊗ C 1 b e a c o dewor d of r ank r , and let h v 1 , . . . , v r i = Ro wS pan( C ) . Then C c an b e writt en as C = P r s =1 u s ⊗ v s wher e h u 1 , . . . , u r i = ColSpan( C ) . Fix a receiv ed w ord R , whic h w e wish to d ecod e from η ⋆ − ε fractio n of error w h ere η ⋆ = min( η 1 δ 2 , η 2 δ 1 ) and ε > 0. By deco ding e ac h ro w up to radius η 1 , we get lists L 1 , . . . , L n 2 of co dew ords from C 1 eac h of size at most ℓ 1 ( η 1 ). By deco din g eac h column up to r adius η 2 , w e get lists L ′ 1 , . . . , L ′ n 1 of cod ewords from C 2 eac h of size at most ℓ 2 ( η 2 ). The follo wing lemma giv es an analogue of Lemm a 6.8 , restricting the choice of basis v ectors to those that o ccurs relativ ely frequently among the lists. Lemma 6.13. L e t C ∈ C 2 ⊗ C 1 b e a r ank r c o dewor d such tha t ∆( R , C ) 6 η ⋆ − ε . Ther e is a b asis V = { v 1 , . . . , v r } for Ro wSpan( C ) wher e e ach v i o c curs in at le ast ε 2 1 − r n 2 of the lists L i . Pr o of. C on s ider the set of codewo rds S = { v ∈ C 1 } which o ccur in the row lists at least ε 2 1 − r n 2 times. W e claim that S con tains a basis for Ro wSpan( C ). Assume for con tradiction that i t only spans an r − 1 dimensional sub s pace. Cho ose { v 1 , . . . , v r − 1 } whic h form a basis for it and complete it to a basis by adding v r . Deﬁne the set S ′ = v r + h v 1 , . . . , v r − 1 i . No w b y Corollary 6.12 , we can write C in the form C = P r s =1 u s ⊗ v s for some u 1 , . . . , u r ∈ S whic h span C olSpan( C ). Note that wt( u r ) > δ 2 , hence at least δ 2 n 2 ro w s (c orresp ond ing to ind ices in the supp ort of u r ) come f r om the s et S ′ , call this set A ⊂ [ n 2 ]. Since the error rate is η ⋆ − ε 6 δ 2 η 1 − ε , it m ust b e the case that for some su bset B ⊆ A of r o ws wh ere | B | > εn 2 , the error r ate on those r o ws is less than η 1 ; else the ov erall error r ate is at least ( δ 2 − ε ) η 1 > δ 2 η 1 − ε > η ⋆ − ε . List decod ing ro ws in B u p to radius η 1 reco ve rs the corresp onding ro w vecto r from C . So a v ector from S ′ o ccurs in all lists for rows in B . Hence o ne of these v ectors has to o ccur with frequency at least ε 2 1 − r n 2 , but this cont radicts the fact that S and S ′ are disjoin t.  Similarly , let T d enote the set of v ector u ∈ C 2 whic h o ccur in in at least ε 2 1 − r n 1 of the lists L ′ i . One can sho w that T con tains a basis U = { u 1 , . . . , u r } for ColSp an( C ). Lemma 6.14. We ha ve ℓ ⊗ r ( η ⋆ − ε ) 6 2 4 r 2 ε − 2 r ℓ 1 ( η 1 ) r ℓ 2 ( η 2 ) r . Pr o of. Note that | S | 6 2 r − 1 ℓ 1 ( η 1 ) ε − 1 , and | T | 6 2 r − 1 ℓ 2 ( η 2 ) ε − 1 . W e c ho ose r basis vec tors from S and T resp ectiv ely as b ases for Ro wSpan( C ) and ColSpan( C ), for whic h there are at most | S | r | T | r c hoices. W e th en c ho ose r row v ectors { v 1 , . . . , v r } from Ro wSpan( C ) and r column vec tors { u 1 , . . . , u r } from ColSpan( C ) s o that C = P r s =1 u i ⊗ v i . This giv es 2 2 r 2 LIST DECODING PR ODU CT A ND INTERLEA VED CODES 25 additional c hoices. Thus ℓ ⊗ 2 ( η ⋆ − ε ) can b e b ound ed b y 2 2 r 2 | S | r | T | r , whic h give s the desired b ound .  Putting tog ether Lemmas 6.14 and 6.11 , we ha v e pro v ed the follo wing theorem. Theorem 6.15 . L et r = ⌈ log( 2 δ 1 δ 2 ) ⌉ , η 2 6 δ 1 , η 2 6 δ 2 and η ⋆ = min( η 1 δ 2 , η 2 δ 1 ) . Then ther e exist c onstants c 1 , c 2 so that for any ε > 0 . ℓ ⊗ ( η ⋆ − ε ) 6 c 1 2 c 2 r 2 ℓ 1 ( η 1 ) r ℓ 2 ( η 2 ) r ε − 2 r . 6.3. Fu r t her Imp ro veme nts for Interle a ved Codes. Theorem 2.6 was pro v ed b y ﬁrst reducing to the r an k r ca se for constan t r , then r ed ucing to m = r b y ﬁxing a basis, and using the trivial upp er b ound ℓ ⊙ r ( η ) 6 ℓ ( η ) r . One can impro ve on this last b ound using the analysis of T h eorem 3.6 combined with b etter list-size b ounds f or d ecod ing b inary cod es f rom erasures. Theorem 6.16. F or any binary line ar c o de C of r elative d istanc e δ , let r = ⌈ log 2 δ 2 ⌉ . F or any η < δ ℓ ⊙ m ( η ) 6 2 2 r 2 δ 4 ( δ − η ) r r − 1 Y k =0 ℓ  η − δ (1 − 2 − k ) 2  (6.1) F or a binary error correcting co d e with relativ e distance δ and µ 6 η < δ , w e let ℓ ′ ( µ, η − µ ) denote th e list-size for C with µ erasur es and η − µ errors. Lemma 6.17. F or any µ 6 η , we ha ve ℓ ′ ( µ, η − µ ) 6 2 · ℓ ( η − µ/ 2) . Pr o of. Let r ∈ { 0 , 1 , ⋆ } n b e a receiv ed w ord where wt( r , ⋆ ) > µ . Let L = { c 1 , · · · , c k } denote all co dewords of C that s atisfy ∆( c i , r ) 6 η . Set th e erased p ositions of r at random in { 0 , 1 } , call this rec eiv ed w ord r ′ . Then Pr[∆( r ′ , c i ) 6 η − µ/ 2] > 1 2 . Thus, in exp ectation o v er the settings of the erased bits, k/ 2 of the co d ew ord s from L satisfy ∆( r ′ , c i ) 6 η − µ/ 2. Fixing one s u c h c hoice of r : ℓ ′ ( µ, η − µ ) 2 6 ℓ ( η − µ/ 2) .  The follo wing lemma completes the p ro of of Theorem 6.16 . Lemma 6.18. L et C b e a binary c o de w ith distanc e δ . Then for any η < δ we have ℓ ⊙ r ( η ) 6 r 2 r r − 1 Y k =0 ℓ  η − δ (1 − 2 − k ) 2  . (6.2) Pr o of. W e r u n Alg orithm 1 on the receiv ed wo rd R and build a tree. W e mark eac h edge lea ving v Blue if it lies within the u n ique-deco ding radius (for the co de b eing decoded at 26 P . GOP ALAN, V. GUR USW AMI, AND P . RAGHA VENDRA v ) and Red ot herwise. A simp le induction sho ws that after k R ed edges, we ha v e at least δ (1 − 2 − k ) erasures. Thus, after k Re d edges, the degree of the tree drops to ℓ ′ ( η , δ (1 − 2 − k )) 6 2 ℓ  η − δ (1 − 2 − k ) 2  . Solving the recur sion for the n u m b er of lea v es shows that ℓ ⊙ r ( η ) 6 r 2 r r − 1 Y k =0 ℓ  η − δ (1 − 2 − k ) 2  .  Equation 6.2 allo ws us to replace ℓ ( η ) r term in Equation 2.3 with the p ro duct ℓ ( η ) ℓ ( η − δ / 4) ℓ ( η − 3 δ / 8) · · · . This adv antag e is pronounced if the list size decreases r apidly as the radius shrin ks; whic h happ ens if w e use the J oh n son b ound to b ou n d the list-size. W e ﬁnish the proof of Theorem 2.7 . Pr o of of The or em 2.7 . Set η = J 2 ( δ ) − ε . Note that δ − η > δ − J 2 ( δ ). Let r = ⌈ log 2 δ 2 ⌉ . Using Lemma 6.9 , w e get ℓ ⊙ m ( J 2 ( δ ) − ε ) 6 4 δ 4 2 r 2 ( δ − J 2 ( δ )) r ℓ ⊙ r ( J 2 ( δ ) − ε ) = c δ ℓ ⊙ r ( J 2 ( δ ) − ε ) , (6.3) for some constan t c δ that only dep ends on δ < 1 / 2. Applying L emm a 6.18 , w e conclude that ℓ ⊙ r ( J 2 ( δ ) − ε ) 6 r 2 r r − 1 Y k =0 ℓ  J 2 ( δ ) − γ k  , where γ k = δ (1 − 2 − k ) 2 + ε . The Johnson b ound states that ℓ ( J 2 ( δ ) − γ ) 6 γ − 2 . F or k > 1, we can low er b ound γ k b y δ (1 − 2 − k ) 2 . Hence w e ha ve ℓ ⊙ r ( J 2 ( δ ) − ε ) 6 r 2 r ε − 2 r − 1 Y k =1 4 δ 2 (1 − 2 − k ) 2 6 c ′ δ ε − 2 . (6.4) Com b ining Equations 6.3 and 6.4 giv es the desired r esu lt.  7. List D ecoding Linear Transforma tions In the p revious sections, w e h a ve dev elop ed tec hniques for deco din g generic in terlea ved codes based on generalized Hamming w eigh ts and decoding from erasures. T h ese c an b e applied to get sharp er b ou n ds for speciﬁc families of c o des. W e illustrate this in the case of linear transformations. The problem of list deco ding linear transf ormations is equiv alen t to list decod ing in terlea v ed Hadamard co des. Equiv alent ly , one can think of the message as a matrix M o ver F q of dimension k × m , en cod ed b y the v alues x t M for every x ∈ F k q . Thus the enco ding is a matrix C of dimension q k × m where eac h column is a co dew ord i n the Hadamard co de. Recall the w ell kno wn list size b ound ℓ (1 − 1 q − ε ) 6 O ( ε − 2 ) for Hadamard co des o ver F q . LIST DECODING PR ODU CT A ND INTERLEA VED CODES 27 Consider the space Lin( F q , k , m ) of all linear transform ations L : F k q → F m q . W e us e ℓ ⊙ m (Had q , η ) instead of to denote the d enote the list-size for Lin( F q , k , m ) at distance η . W e let Lin r ( F q , k , m ) denote the sp ace of all linear transformations of rank at most r in Lin( F q , k , m ) and let ℓ ⊙ m r (Had q , η ) d enote the list-size for Lin r ( F q , k , m ). 7.1. Linear Transf orma tions over F 2 . W e ﬁ rst u se the Deletio n lemma to reduce the task of p ro vin g list-size b ounds for Lin( F 2 , k , m ) to that of pr o ving b ound s for Lin 2 ( F 2 , k , m ). In this s u bsection, w e use ℓ ⊙ m ( η ) to denote ℓ ⊙ m (Had 2 , η ) and ℓ ⊙ m r ( η ) to denote ℓ ⊙ m r (Had 2 , η ). Lemma 7.1. F or any η < 1 / 2 , we have ℓ ⊙ m ( η ) 6 C ℓ ⊙ m 2 ( η ) wher e C is an absolute c onstant. Pr o of. W e take C ′ to consist of all linear transf ormations o f rank at most 2. Thus for an y co dew ord L 6∈ C ′ , Rank( L ) > 3, hence wt( L ) > 7 8 , so w e tak e µ = 7 8 . Sin ce J (7 / 8) > 0 . 65 and η < 1 2 , we ca n apply the Deletion lemma with γ = 0 . 15 to co nclude that ℓ ⊙ m ( η ) 6 C ℓ ⊙ m 2 ( η ).  T o b ou n d ℓ ⊙ m 2 ( η ), w e start by b ou n ding ℓ ⊙ m 1 ( η ), whic h is th e m axim um n um b er of rank 1 linear tr an s formations w ithin distance η of the receiv ed w ord R . W e need some facts ab out the structure of L ( x ) wh en L is a linear transform ation of rank r . Recall that w e u se l [1] , . . . , l [ k ] ∈ F m 2 for the ro ws of L , th ough t of as v ectors from F m 2 . Let Ro wSpan( L ) ⊆ F m 2 denote the r - dimensional space spanned by these v ectors. W e write RowSpan( L ) = h l ′ [1] , . . . , l ′ [ r ] i to denote the fact that l ′ [1] , . . . , l ′ [ r ] form a basis for RowSpan( L ). F or a function r : F k 2 → F m 2 and a ve ctor v ∈ F m 2 , we deﬁne wt( R , v ) = Pr x [ R ( x ) = v ]. Lemma 7.2. F or L ∈ Lin r ( F 2 , k , m ) a nd any v ∈ Ro wS p an( L ) , wt( L, v ) = 2 − r . Pr o of. T h e j th column of L deﬁnes a linear fu nction l j ( x ) = x t l j from F k 2 → F 2 . W e ha ve L ( x ) = ( l 1 ( x ) , . . . , l t ( x )). Let us pic k a basis for the columns, assu m e that this ba- sis is l 1 , . . . , l r . Let v 6 r denote the pro jection of v onto th e ﬁrst r co-ordinates. W e h a ve Pr x [( l 1 ( x ) , . . . , l r ( x )) = v 6 r ] = 2 − r , w hic h implies the claim.  7.1.1. Bounding r ank 1 line ar tr ansformations. The ab o v e le mma implies, in p articular, that if Rank( L ) = 1 and Ro w Span( L ) = h l ′ [1] i , then w t( L, l ′ [1]) = wt( L, 0 k ) = 1 2 . Corollary 7.3. L et L ∈ Lin 1 ( F 2 , k , m ) a nd R : F k 2 → F m 2 b e such that ∆( R, L ) 6 1 2 − ε . If Ro wS pan( L ) = h l ′ [1] i , then wt( R, l ′ [1]) > ε . This narrows th e c h oice of basis v ectors f or Ro wS pan( L ) to at most 1 ε v ectors wh er e wt( L, l ′ [1]) > ε . Once w e ﬁx l ′ [1], the p roblem reduces to Hadamard d ecod ing (or rather m = 1). Given R : F k 2 → F m 2 , we deﬁne r : F k 2 → { F 2 ∪ ⋆ } as follo ws: r ( x ) =      0 if R ( x ) = 0 m , 1 if R ( x ) = l ′ [1] , ⋆ otherwise . Setting r ( x ) = ⋆ denotes an erasure; since R ( x ) 6∈ Ro wS pan( L ), w e k n o w there is an error at index x . Giv en a Hadamard co dewo rd l : F k 2 → F 2 , if w e deﬁne L ∈ Lin 1 ( F 2 , k , m ) by rev ersing 28 P . GOP ALAN, V. GUR USW AMI, AND P . RAGHA VENDRA the substitution: L ( x ) = ( l ′ [1] if l ( x ) = 1 , 0 m if l ( x ) = 0 then it follo ws that ∆( R , L ) = ∆( r , l ). This p ro ves th e follo wing claim. Lemma 7.4. The line ar tr ansforma tions L ∈ Lin 1 ( F 2 , k , m ) so th at ∆( R, L ) < 1 2 − ε and Ro wS pan( L ) = h l ′ [1] i ar e in one-to-one c orr esp ondenc e with Hadamar d c o dewor ds l so that ∆( r , l ) < 1 2 − ε . Since there can b e at most O ( 1 ε 2 ) codewo rds of the Hadamard cod e within distance 1 2 − ε , and at most 1 ε c hoices for l ′ [1], this suﬃces to pro v e a b ound of O ( 1 ε 3 ). W e can impro ve this to O ( 1 ε 2 ) b y observing that if there are man y c h oices for l ′ [1], then eac h of them is lik ely to result in f ewer codewords. T his relies on the follo wing Lemma ab out Hadamard deco d ing with erasures. Lemma 7.5. Given r : F k 2 → { F 2 ∪ ⋆ } so tha t wt( R, ⋆ ) > η , the numb er of c o dewor ds l such that ∆( r , l ) < ε is b ounde d by 2 ( η +2 ε ) 2 . Pr o of. W e u s e the w ell kno wn fact that the num b er of co dewords so that ∆( r, l ) < 1 2 − ε is b ound ed b y 1 4 ε 2 . So assume there are s cod ew ord s l 1 , . . . , l s . Consid er setting eac h erasu re to a random v alue in F 2 . F or an y l , with probabilit y 1 2 w e ha ve ∆( r , l ) 6 1 2 − η 2 − ε . Thus in exp ectation, s 2 of the co dewo rds will no w satisfy ∆( l , r ) < 1 2 − η 2 − ε . Fix one suc h setting o f the erasures. But no w w e ha ve s 2 6 1 ( η + 2 ε ) 2 ⇒ s 6 2 ( η + 2 ε ) 2 .  W e can no w sho w an O ( ε − 2 ) b ound on rank 1 transformations. Lemma 7.6. F or any ε > 0 , we have ℓ ⊙ m 1 ( 1 2 − ε ) 6 1 2 ε 2 . Pr o of. Assu me that the condition wt( R, l ′ ) > ε holds for t non-zero ro w v ectors l ′ ∈ F m 2 . Fix one such c hoice of l ′ . W e then erase η > ( t − 1) ε p ositions, so by Lemma 7.5 the n um b er o f candidates is no m ore than 2 ( t +1) 2 ε 2 . Th us o verall the list-size is b ound ed b y 2 t ( t +1) 2 ε 2 6 1 2 ε 2 .  7.1.2. Bounding r ank 2 line ar tr ansformations. W e no w pro ceed to analyzing the list-size for rank 2 linear transformations. W e b egin with the analog of Corollary 7.3 for the r ank tw o case. Corollary 7.7. L et L ∈ Lin 2 ( F 2 , k , m ) and R : F k 2 → F m 2 b e such that ∆( R, L ) 6 1 2 − ε . Ther e is a b asis { u, v } for Ro wSp an( L ) such that wt( R, u ) a nd wt( R, v ) ar e b oth at le ast ε 2 . Pr o of. Note that wt( L, l [1]) , w t( L, l [2]) and wt( L, l [1] + l [2]) are all exac tly 1 4 . If t wo of the sym b ols { l [1] , l [2] , l [1] + l [2] } h a ve w eigh t less than ε 2 in R , then ∆( R , L ) > 2( 1 4 − ε 2 ) = 1 2 − ε .  LIST DECODING PR ODU CT A ND INTERLEA VED CODES 29 In fact, w e c an assume that wt( R, u ) > 1 12 and wt( R, v ) > ε 2 , bu t w e will not use this clai m. Once w e pic k u and v we redu ce to the case m = 2 b y deﬁning r : F k 2 → { F 2 2 ∪ ⋆ } as follo ws: r ( x ) = ( ( λ, µ ) if R ( x ) = λu + µv , ⋆ otherwise . W e hav e the follo wing analogue of Lemm a 7.4 . Lemma 7.8. The line ar tr ansformations L ∈ Lin 2 ( F 2 , k , m ) so that ∆( L, R ) < 1 2 − ε ar e in one-to-one c orr esp ondenc e with line ar tr ansforma tions l ∈ Lin 2 ( F 2 , k , 2) so that ∆( l, r ) < 1 2 − ε . Lemma 7.9. L et r : F k 2 → { F 2 2 ∪ ⋆ } b e such that w t( r, ⋆ ) > η . The numb er of line ar tr ansfor- mations l ∈ Lin 2 ( F 2 , k , 2) so that ∆( l, r ) < 1 2 − ε is b ounde d by 100 ( η +2 ε ) 2 . Pr o of. Let l = ( l 1 , l 2 ) where eac h l i : F k 2 → F 2 . W e pr o ceed as in Theorem 2.5 , erasing errors whenev er they are lo cated. W e r un Hadamard deco ding on the ﬁ rst column which gives a list L 1 of 2 ( η +2 ε ) 2 candidates for ℓ 1 . W e then c ho ose c ∈ L 1 , erase the p ositions where it diﬀers from R and list decod e the resu lting w ord to get candidates for l 2 . A t most one co deword c ∈ L 1 can ha v e error rate le ss than 1 4 . If w e c ho ose l 1 = c , w e ha v e no more than 2 ( η +2 ε ) 2 c hoices for the second column. The other co dew ords in L 1 whic h are b ey ond distance 1 4 result in η > 1 4 erasures, h ence a list- size of C 6 32 for the second column. So the o v erall list size is b ounded b y 1 · 2 ( η + 2 ε ) 2 + 2 ( η + 2 ε ) 2 · 32 6 100 ( η + 2 ε ) 2 .  W e are no w ready to b ound the n u m b er of rank 2 transformations. Lemma 7.10. Ther e is a c onstant C ′ so that for any ε > 0 , we have ℓ ⊙ m 2 ( 1 2 − ε ) 6 C ′ ε 2 . Pr o of. Assu me there are t > 2 non-zero v ectors v ∈ F m 2 so that wt( v ) > ε 2 . By Corollary 7.7 , eac h there is a basis { u, v } of L where wt( R, u ) and wt( R, v ) are b oth at least ε 2 . Once we ﬁx this basis, w e erase all p ositions con taining v ectors other than u, v and u + v . This results in η > ( t − 3) ε erasu res. T hus b y Lemma 7.9 , the num b er of L suc h that RowSpan( L ) = h u, v i is b ou n ded by 100 ( η + 2 ε ) 2 6 100 ( t − 1) 2 ε 2 . Since there are  t 2  c hoices for { u, v } , w e ge t an o v erall b ound of  t 2  · 100 ( t − 1) 2 ε 2 = t 2( t − 1) 100 ε 2 6 100 ε 2 If there are t < 2 nonzero ve ctors v ∈ F m 2 with wt( v ) > ε 2 , then it corresp onds to th e r ank one case. Thus, adding b ounds for rank 1 and rank 2 c ases, we ha v e ℓ ⊙ m 2 ( 1 2 − ε ) 6 100 ε 2 + 1 2 ε 2 = C ′ ε 2 .  30 P . GOP ALAN, V. GUR USW AMI, AND P . RAGHA VENDRA 7.1.3. Final list-size b ound. Combining Lemma 7.10 with Lemma 7.1 , w e conclude that for some abs olute constan t C > 0, ℓ ⊙ m ( 1 2 − ε ) 6 C ε 2 , th us completing the pro of of Theorem 2.9 . 7.2. Linear Transforma tions over F q . In this su bsection, w e use ℓ ⊙ m ( η ) to denote ℓ ⊙ m (Had q , η ) and ℓ ⊙ m r ( η ) to denote ℓ ⊙ m r (Had q , η ). One can r ep eat th e ab o ve pr o of of Theorem 2.9 from the previous subsection for any F q , and show th at for some absolute constan t c , ℓ ⊙ m (Had q , 1 − 1 /q − ε ) 6 cq 6 ε 2 . This giv es an asymp totical ly tigh t b ound for constant q . In this subsection, our g oal will b e to remov e the dep endence on q . W e will do so at the exp ense of a worse dep endence (of ε − 5 ) of th e b ound on ε . W e b egin with a deletion argument analogous to Lemma 7.1 . T he diﬀerence is that we only delete rank 1 transformations, and the multiplicat iv e ov erhead is O ( ε − 2 ) as opp osed to an absolute constan t. Lemma 7.11. F or η = q − 1 q − ε , w e h ave ℓ ⊙ m ( η ) 6 C ε 2 ℓ ⊙ m 1 ( η ) . Pr o of. W e tak e C ′ to consist of all linear transf ormations o f r ank at most 1. So we can ta k e µ = 1 − 1 q 2 and J ( µ ) = 1 − 1 q . Thus w e can apply the Deletion lemma with η = ε to conclude that ℓ ⊙ m ( η ) 6 1 ε 2 ℓ ⊙ m 1 ( η ).  Giv en v ∈ F m q w e no w deﬁn e wt( R, v ) = P µ ∈ F ⋆ q P r x ∈ F m q [ R ( x ) = µv ]. W e hav e the follo wing q -ary analogue of Corollary 7.7 . Corollary 7.12. L et L ∈ Lin 1 ( F q , k , m ) a nd R : F k q → F m q b e suc h that ∆( R, L ) 6 1 − 1 q − ε . If Ro wSpan( L ) = h v i then wt( R, v ) > ε . Fixing the vec tor v (from amongst t 6 1 ε c hoices), red uces the problem to Hadamard deco din g o ver F q . The Johnson boun d states that at radius 1 − 1 q − ε , the list-size is b ounded by 1 ε 2 . This gi v es the follo wing b oun d. Lemma 7.13. F or any ε > 0 we have ℓ ⊙ m 1  1 − 1 q − ε  6 1 ε 3 . Plugging this into Lemma 7.11 complete s the pro of of Th eorem 2.10 . Referen ces [1] A. A k avia, S. Goldw asser, a nd S. Safra. Pro ving h ard-core predicates using list d ecoding. In Pr o c. 44 th IEEE Symp osium on F oundations of Com puter Scienc e (FOCS’03) , pages 146–155, 2003. [2] E. Ben-Sasson and M. Sudan. Robust lo cally testable co des and p rod ucts of cod es. R andom Struct. Algo- rithms , 28(4):387–402, 2006. [3] D. Bleic henbac her, A . Kiayias, and M. Y ung. Decod ing interlea ved Reed-Solomon codes ov er noisy c han- nels. The or etic al Com puter Scienc e , 379(3):348– 360, 200 7. LIST DECODING PR ODU CT A ND INTERLEA VED CODES 31 [4] D. Copp ersmith and A. Rudra. On th e robust testabilit y of pro duct of co des. T echnical Rep ort (10 4), Electronic Colloqu ium on Computational Complexity (ECCC), 2005. [5] D. Copp ersmith and M. Sudan. Reconstructing curv es in three (and higher) dimensional space from noisy data. In Pr o c. 35 th ACM Symp osium on The ory of Computing (STOC’03) , pages 136–1 42, 2003. [6] I. Dinur, E. Grigorescu, S. Kopparty , and M. Sudan. Deco dability of group homomorphisms b eyond t he Johnson b ound . In Pr o c. 40 th ACM Symp osium on The ory of Computing (STOC’08) , pages 275–284, 2008. [7] I. Dinur, M. Sudan, and A. Wigderso n. Robust local testability of ten sor pro ducts of LDPC co des. In APPRO X-RANDOM’ 06 , pages 304–315, 2006. [8] I. Dumer, D. Micciancio, and M. Sudan. H ardness of a pproximating the minim um distance of a linear code. IEEE T r ansactions on Information The ory , p ages 22–37, (2003). [9] P . Elias. Error-free coding. IEEE T r ansactions on Information The ory , 4(4):29–37, 1954. [10] P . Elias. List decod ing for n oisy channels. T echnical Rep ort 335, R esearch Lab oratory of Electronics, MIT, 1957. [11] O. Goldreic h and L. Levin. A h ard - core p redicate for all one-wa y funct ions. In Pr o c. 21 st ACM Symp osium on the The ory of Computing (STOC’89) , pages 25–32, 1989. [12] P . Gopalan, A. Kliv ans, and D. Zuckerma n. List-decoding Reed-Muller codes ov er small ﬁelds. In Pr o c. 40 th ACM Symp osium on The ory of Computing (STOC’08) , pages 265–274, 2008. [13] V. Guruswami. List decoding from eras ures: b ounds and co de constructions. IEEE T r ansactions on In- formation The ory , 49(11):2826–2833 , 2003. [14] V. Guruswami. Better extractors for b etter co des? In Pr o c e e dings of the 36th Annual A CM Symp osium on The ory of Computing (STOC’04) , pages 436–44 4, 2004. [15] V. Gurusw ami. Li st De c o ding of Err or-Corr e cting Co des , volume 3282 of L e ctur e Notes in Computer Sci- enc e . Sp ringer, 2004. [16] V. Guruswami . Algorith mic R esults in List De c o di ng , vo lume 2 of F oundations and T r ends in The or etic al Computer Scienc e . N o w Publishers, 2006. [17] V. Guruswa mi and A. Rudra. Explicit codes ac hieving list deco ding capacit y: Error-correction with optimal redundancy . IEEE T r ansactions on I nformation The ory , 54(1):135–150, 2008. [18] V. Gu rusw ami and M. Su dan. Improv ed decod ing of Reed-Solomon and Algebraic-Geometric co des. IEEE T r ansactions on Information The ory , 45(6):175 7–1767, 1999. [19] J . Jackson. An eﬃcient membership- query algo rithm for learning DNF with respect to the uniform distri- bution. Journal of Computer and System Scienc es , 55:41 4–440, 1997. [20] S. Khot. Hardness of appro ximating the shortest vector problem in lattices. Journal of the ACM , 52(5):78 9– 808, 2005. [21] Y. Mansour. Learning Bo olean functions via the Fourier transform. The or etic al A dvanc es in Neur al Com- putation and L e arning , pages 391–424, 1994. [22] O. Meir. Com binatorial construction of lo cally testable co des. In Pr o c. 40 th ACM Symp osium on The ory of Computing (STOC’08) , pages 285–294, 2008. [23] F. Parv aresh, M. El-Khamy , M. Stepanov, D. A ugot, R . J. McEliece, and A . V ardy . Algebraic list deco ding of Reed-Solomon pro duct codes. In Pr o c e e di ngs of the T enth Int ernational Works hop on Algebr aic and Combinatorial Co ding The ory, Zvenigor o d, Russia , pages 210–213, September 2006. [24] F. P arv aresh and A. V ardy . Correcting errors beyond th e Guruswa mi-Sudan radius in p olyn omial time. In Pr o c e e dings of the 46th IEEE Symp. on F oundations of Computer Scienc e , pages 285–294, 2005. [25] R. Pelli k aan and X.-W. W u. List deco ding of q-ary Reed-Muller codes. I EEE T r ansactions on Information The ory , 50(4):679 –682, 2004. [26] S. Reddy and J. P . Rob in son. Rand om error and b urst correction by iterated code. IEEE T r ansactions on Information The ory , 18(1):18 2–185, 1972. [27] R.J. Serﬂing. Probability ineq ualities for t h e sum in sampling without replacement. Annals of Statistics , 2(1):39–48 , (1974). [28] M. S udan. Decoding of Reed-Solomon codes b eyond the error-correction b ound. Journal of Complexity , 13(1):180– 193, 1997. [29] M. Sudan. List deco ding: Algorithms and applications. SIGACT News , 31(1):16–27, 2000. [30] M. Su d an, L. T revisan, and S. P . V adhan. Pseudorandom generators without th e XOR lemma. J. Comput. Syst. Sci. , 62(2):236–266 , 20 01. [31] A. T a-S hma and D. Zu ckerman. Ext ractor co des. IEEE T r ansactions on Information The ory , 50(12):3015 – 3025, 2004. 32 P . GOP ALAN, V. GUR USW AMI, AND P . RAGHA VENDRA [32] L. T revisan. Lis t-deco ding using the XOR lemma . In Pr o c. 44 th IEEE Symp osium on F oundations of Computer Scienc e (FOCS’03) , pages 126–135, 2003. [33] M. A . Tsfasman and S . G. Vl˘ adut. Goemetric approac h to h igher weig hts. IEEE T r ansactions on Infor- mation The ory , 41:1564–1588 , 1995. [34] P . V aliant. The ten sor p rodu ct of tw o co des is not necessarily robustly testable. In APPRO X-RANDOM’05 , pages 472–481, 2005. [35] J . W ozencraft. List deco ding. T ec hnical Rep ort 48:90-95, Quarterly Progress Rep ort, Research Lab oratory of Electronics, MIT, 1958. Micros oft Research, Silicon V alley. Some of this wo rk w as d one when the author w as a postdoc a t the Unive rsity of W ashin gton visiting Princeton Unive rsity. E-mail addr ess : parik @microsoft. com Dep ar tment of Computer Scie nce and Engineering, University of W ashing ton. Currentl y vis- iting the Compu ter Science Dep t., Carnegie Mello n University. Some of this work w as done when the author w as a me mber in the School of Ma thema tics, Institute for Adv anced Study. E-mail addr ess : venka t@cs.washin gton.edu Dep ar tment of Computer Science and Enginee ring, Uni versity of W ashington. Some of this w ork w as done when the author w as vi siting Princeton University. E-mail addr ess : prasa d@cs.washin gton.edu

List Decoding Tensor Products and Interleaved Codes

Original Paper

Comments & Academic Discussion

Leave a Comment

Original Paper

Related Papers

Comments & Academic Discussion

Leave a Comment