On Identity Testing of Tensors, Low-rank Recovery and Compressed Sensing

On Iden tit y T esting of T en sors, Lo w-rank R eco v ery and Compressed Sensing Mic hael A. F o rb es ∗ Amir Shpilk a † Octob er 26, 20 18 Abstract W e study the problem of obtaining efficien t, deterministic, black-b ox p olynomial identity testing algorithms for depth-3 set-multilinear circuits (o ver arbitrar y fields). This class of circuits has an efficient, deterministic, white-b ox p olynomia l identit y testing algorithm (due to Raz and Shpilk a [ RS05 ]), but has no known such black-box alg o rithm. W e recas t this pr oblem as a question of finding a lo w-dimensional subspa ce H , spanned b y r a nk 1 tensors, such that any non- zero tensor in the dual space ker( H ) has high rank. W e obtain explicit co nstructions of essent ially optimal-size hitting sets for tensors o f degree 2 (matrices ), and obtain quas i-p olynomial sized hitting sets for arbitr ary tensors (but this second hitting set is les s explicit). W e also show connections to the task o f perfor ming low-r ank r e c overy o f matrices , which is studied in the field o f co mpressed se ns ing. Low-rank recovery ask s (say , ov er R ) to recover a ma trix M from few measurements, under the promise that M is rank ≤ r . In this work, we re s trict our attention to rec overing matrices that a re exactly r ank ≤ r using deterministic, non-adaptive, linea r measurements, that ar e free from noise. O ver R , we provide a set (of size 4 nr ) of such measurements, fro m which M can b e rec overed in O ( rn 2 + r 3 n ) field op eratio ns, and the num b er of mea s urements is essentially optimal. F urther, the measure ments can be taken to b e all rank-1 matrices, or all sparse matric e s. T o the b est of our kno wledg e no explicit constructions with those prop er ties were kno wn prior to this w ork. W e also give a more forma l connection betw een low-rank r ecov er y and the task of sp arse (ve ctor) r e c overy : an y sparse - recov ery algorithm that exa c tly recovers v ec tors of length n and sparsity 2 r , using m non-adaptive measurements, yields a low-rank recov ery sch eme for exactly recov e ring n × n matrices of rank ≤ r , making 2 nm non-a daptive m easurements. F urthermo re, if the spa rse-r e cov ery algo r ithm runs in time τ , then the low-rank recovery alg orithm runs in time O ( r n 2 + nτ ). W e obtain this reductio n using linear-a lg ebraic techniques, and not using conv ex optimization, which is more commonly seen in co mpr essed sensing alg orithms. Finally , we also mak e a connection to r ank-metric c o des , as studied in c o ding theory . These are co des with co dewords co nsisting of matr ices (or tensor s) where the distance o f matrices A and B is rank( A − B ), as opp osed to the usual hamming metric. W e obtain essen tially optimal- rate co des ov er matrices, and pr ovide an efficient deco ding alg o rithm. W e obtain co des ov er tensors a s well, with po orer rate, but still with efficient deco ding. ∗ Email: miforbes@mit.edu , Departmen t of El ectrical Engineering a nd Computer Science, MIT CSAIL, 32 V assar St., Cam bridge, MA 02139 , Supp orted by NS F grant 6919791, MIT CSAIL and a Sieb el Schola rship. † F aculty of Computer Science, T echnion — Israel Institute of T echnology , Haifa, Israel, shpilka@cs .technion .ac.il . The research leading to these results has received funding from the Eu rop ean Comm u nity’s S eventh F ramework Pro gramme (FP7/200 7-2013) u nder g rant agreement n umber 257575. 1 Con ten ts 1 In t ro duction 1 1.1 P olynomial Id en tity T esting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Lo w-Rank Reco very and Compressed S ensing . . . . . . . . . . . . . . . . . . . . . . 2 1.3 Rank-Met ric Co des . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.4 Reco n struction of Arithmetic Circuits . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.5 Our Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.6 Proof O v erview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2 Notation 10 3 Preliminaries 10 3.1 P ap er Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 4 Impro ved Construction of Rank-preserving Matrices 15 5 Iden t it y T esting for Matrices 17 5.1 V ariable Redu ction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 5.2 The Hitting Set f or Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 5.3 An Alternate Construction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 6 Iden t it y T esting for T ensors 23 6.1 V ariable Redu ction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 6.2 The Hitting Set f or T ensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 6.3 Identit y T esting for T ensors o v er S m all Fields . . . . . . . . . . . . . . . . . . . . . . 29 7 Explicit Lo w Rank Reco v ery of Matrices 32 7.1 Prony’s Method and S yndrome Decod ing of Dual Reed-Solomon Co des . . . . . . . . 34 7.2 Lo w Rank R eco v ery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 8 Rank-Metric T ensor co de s 46 9 Discussion 48 A Cauc h y-Binet F orm ula 53 1 In tro duction W e start with a motiv ating example. Let x and y b e v ectors of n v ariables eac h. Let M b e an n × n matrix (o ve r some field, sa y R ), and define the qu adratic form f M ( x , y ) def = x † M y . Supp ose n ow that we are giv en an oracle to f M , that can ev aluate f M on inputs ( x , y ) that w e supply . The t yp e of question w e consider is: how many (deterministically c h osen) ev aluations of f M m u st w e mak e in ord er to determine whether A is n on-zero? It is not hard to sh o w th at n 2 ev aluations to f M are necessary and sufficien t to d etermin e whether A is non-zero. The qu estion b ecomes more int eresting when w e are promised that r ank( M ) ≤ r . That is, giv en that rank( M ) ≤ r , can we (deterministically) determine whether M = 0 us in g ≪ n 2 ev aluations of f M ? It is not hard to sh o w that there (non-explicitly) exist ≈ 2 nr ev aluations to determine whether M = 0, and one of th e n ew results in this pap er is to giv e an explicit constru ction of 2 nr such ev aluations (o ver R ). W e also consider v arious generaliza tions of this pr oblem. The first generalizatio n is to mo ve from matrices (whic h are in a sense 2 dimensional) to the more general notion of tensors (w h ic h are in a sense d -dimensional). That is, a tensor is a map T : [ n ] d → F and lik e a m atrix we can define a p ol ynomial f T ( x 1 , 1 , . . . , x 1 ,n , . . . , x d, 1 , . . . , x d,n ) def = X i 1 ,...,i d ∈ [ n ] T ( i 1 , . . . , i d ) d Y j =1 x j,i j . As with matrices, tensors ha ve a notion of r ank (defined later), and w e can ask: giv en that rank( T ) ≤ r ho w man y (deterministically chosen) ev aluations of f T are n eeded to d etermine whether T = 0. As T = 0 iff f T = 0, we see that this pr oblem is an instance of p olynomia l identity testing , w hic h asks: giv en oracle access to a p olynomial f that is someho w “simp le”, how man y (deterministically c hosen) queries to f are needed to determine w h ether f = 0? The ab o v e questions ask w hether a certain matrix or tensor is zero. How ev er, we can also ask for more, and seek to reconstruct this matrix/tensor fully . That is, ho w many (deterministically c hosen) ev aluations to f M are needed to determine M ? T his question can b e seen to b e related to compressed s ensing and sp arse reco v ery , w here th e goal is to reconstruct a “simple” ob jec t from “few” measur emen ts. In this case, “simple” refers to the m atrix b eing low-rank, as opp osed to a vecto r b eing sparse. As ab ov e, it is n ot h ard to show that there exist ≈ 4 nr ev aluations that determine M , and this pap er giv es an explicit construction of 4 nr su c h ev aluations, as well as an efficien t algorithm to reconstruct M f rom these ev aluations. W e will no w place this wo rk in a broader con text b y providing bac kground on p olynomia l iden tity testing, compr essed sensing and lo w-rank reco very , and the theory of rank-metric co des. 1.1 P olynomial Iden tity T esting P olynomial id en tity testing (PIT) is the problem of deciding whether a p olynomial (sp ecified b y an arithmetic circuit) computes the identi cally zero p olynomial. Th e ob vious deterministic algorithm that completely expands the p olynomia l u nfortunately tak es exp onentia l time. This is in con trast to the f act that there are sev eral (qu ite simple) randomized algorithms that solv e this problem qu ite efficien tly . F urther, some of these randomized algorithms tr eat the p olynomial as a black-b ox , so that th ey only u se the arithmetic circuit to ev aluate the p olynomial on chosen p oints, as opp osed 1 to a white-b ox algorithm whic h can examine the in ternal structure of the circuit. Ev en in the white-b o x mo d el, no efficien t deterministic algorithms are kno wn for general circuits. Understanding th e deterministic complexit y of PIT h as come to b e an imp ortant problem in theoretical computer s cience. Starting with the w ork of Kabanets and Impagliazzo [ KI04 ], it has b een sho wn that the existence of efficien t deterministic (white-b o x ) algorithms for PIT has a tight connection with the existence of explicit fu nctions with large circuit complexit y . As p ro ving lo wer b ound s on circuit complexity is one of the m a jor goals of th eoretical computer science, th is has led to m uc h researc h into PIT. Stronger connections are k n o wn when the deterministic algorithms are blac k-b o x. F or, any suc h algorithm corresp onds to a hitting set , whic h is a set of ev aluation p oints suc h that any small arithmetic circuit computin g a non-zero p olynomial must ev aluate to non-zero on at lea s t one point in the set. Heintz an d Sc h n orr [ HS80 ], as well as Agra wa l [ Agr05 ], sho w ed th at an y deterministic blac k-b o x PIT algorithm ve ry easily yields explicit p olynomials that ha ve large arithm etic circuit complexit y . Moreo v er, Agra w al and Vina y [ A V08 ] sh o wed that a d eterministic constru ction of a p olynomial size hitting set f or arithmetic circuits of dep th-4 giv es rise to a qu asi-p olynomial sized hitting set for general arithm etic circuits. Th u s, the black- b o x deterministic complexit y of PIT b ecomes in teresting ev en for constan t-depth circuits. How ev er, currently no p olynomial size h itting sets are known for general depth-3 circuits. Muc h of r ecen t work on blac k-b ox deterministic PIT has iden tified certain sub classes of circuits for whic h sm all hitting sets can b e constru cted, and this w ork fits into that paradigm. See [ S Y10 ] for a su r v ey of recen t results on PIT. One sub class of depth-3 circuits is the mo d el of set-multiline ar depth-3 circuits, first introd uced b y Nisan and Wigderson [ NW96 ]. Raz and S hpilk a [ RS05 ] ga v e a p olynomial-time white-b o x PIT algorithm for non-comm u tativ e arithmetic form ulas, whic h con tains set-m ultilinear depth-3 circuits as a sub cl ass. How ev er, n o p ol ynomial-time blac k-b o x deterministic PIT algorithm is kno wn for set- m u ltilinear d epth-3 circuits. Th e b est kn own blac k-b ox PIT r esults for the class of set-m ultilinear circuits, with top fan-in ≤ r and degree d , are hitting sets of size m in( n d , p oly (( nd ) r )), where the first part of b o und comes from a s imple argument (presented in Lemma 3.11 ), and the second part of the b oun d ignores that w e hav e set-m ultilinear p olynomials, and simply uses the b est kno w n hitting sets for so-called ΣΠΣ( k ) circuits as established b y Saxena and Seshadhr i [ SS11 ]. F or non- constan t d and r , these b ound s are sup er-p olynomial . Impro vin g th e size of these hitting sets is the pr imary motiv ation for this wo r k. T o conn ect PIT for set-m ultilinear depth-3 circuits w ith the ab o ve qu estions on matrices and tensors, w e no w note that any suc h circuit of top fan-in ≤ r , d egree d , on dn v ariables (and thus size ≤ dnr ), computes a p olynomial f T , where T is an [ n ] d tensor of rank ≤ r . Conv ersely , an y suc h f T can b e computed by su c h a circuit. Th us, constructing b etter h itting sets for this class of circuits is exactly the question of finding sm aller sets of (d etermin istically chosen) ev aluations to f T to determine wh ether T = 0. 1.2 Lo w-Rank Reco very and Compressed Sensing Lo w-rank Reco v ery (LRR) asks (for matrices) to reco ver an n × n matrix M from f ew me asur ements of M . Here, a measurement is some in ner p ro duct h M , H i , wh er e H is an n × n matrix and the inner pr o duct h· , ·i is the natural in n er pro duct on n 2 long v ectors. This can b e seen as the n atural generalizat ion of the sp arse r e c overy problem, wh ich asks to reco ve r sparse v ectors from few linear measuremen ts. F or, o ver matrices, our notion of sparsit y is simp ly that of b eing low-rank. Sparse reco v ery and compressed sensing are activ e area s of researc h, see for example [ CSw ]. Muc h of this area fo cuses on constru cting distributions of measuremen ts suc h that the un k n o wn sparse ve ctor can b e reco v ered efficiently , with h igh probabilit y . Also, it is often assumed that the 2 sequence of measurements will not dep end on any of the measur emen t results, and this is known as non-ada ptive sp arse r e c overy . W e n ote that Indyk, Price and W o o dr uff [ IPW11 ] sho wed that adaptive sp arse r e c overy can outp erform non-adaptive measurement s in certain regimes. Much of the existing work also fo cuses on efficiency concerns, and v arious algorithms coming from conv ex programming h a ve b een u sed. As such, these algorithms tend to b e stable u nder n oise, and can reco v er appro x im ations to the spars e vecto r (and can ev en d o s o only if the original v ector wa s appro ximately spars e). One of th e initial ac hiev emen ts in this field is an efficien t algorithm for reco v ery of a k -sp arse 1 appro ximation of n -entry v ector in O ( k log ( n/k )) measuremen ts [ CR T05 ]. Analogous questions f or lo w-rank reco v ery h av e also b een exp lored (for example, see [ lrr ] and references there in). Initial w ork (such as [ CT09 , CP09 ]) ask ed the question of lo w-rank matrix c ompletion , w here en tries of a low-rank m atrix M are revea led ind ividually (as opp osed measuring linear com binations of matrix en tries). It w as shown in these w orks that for an n × n rank ≤ r matrix that O ( nr p olylog n ) noisy samples su ffice f or nucle ar-norm minimization to complete the m atrix efficien tly . F ur ther works (suc h as [ ENP11 ]) prov e that a randomly c hosen set of measurements (with appropriate parameters) giv es enough information for low-rank r eco v er y , other works (such as [ CP11 , RFP10 ]) giving explicit conditions on the measuremen ts that guaran tee that the nuclear norm minimization algorithm w ork s , and finally other w orks seek alte rnativ e algorithms for certain ensem b les of measurements (suc h as 2 [ K OH11 ]). As in the s parse reco ve ry ca se, most of these w ork seek stable algo rithms that can deal with noisy measurements as w ell as matrices that are only app ro ximately lo w -rank. Finally , we note th at some applications (suc h as quantum state tomograph y) hav e additional requirement s for their measurement s (for example, they should b e easy to pr epare as quantum s tates) and some w ork has gone in to this as w ell [ GLF + 10 , Gro09 ]. W e no w mak e a cru cial observ ation wh ic h sho w s that blac k-b ox PIT for the quadr atic form f M is actually v ery closely related to lo w-rank reco very of M . That is, note that f M ( x , y ) = x † M y = h M , x † y i . Th at is, an ev aluation of f M corresp onds to a measuremen t of M , and in particular this measuremen t is realized as a rank-1 matrix. Th u s, w e see that an y lo w-rank-r eco v ery algorithm that only uses rank-1 measur emen t can also determine if M is non-zero, and th u s also p erforms PIT for qu adratic forms. Con v ersely , su pp o se we ha ve a blac k-b o x PIT algorithm for rank ≤ 2 r quadratic forms. No te then that for any M , N with rank ≤ r , M − N has rank ≤ 2 r . Th us, if M 6 = N then f M − N will e v aluate to non-ze ro o n some p oin t in the h itting set. As f M − N = f M − f N , it follo ws that a hitting set f or rank ≤ 2 r matrices will distin gu ish M and N . In particular, this sho w s that information-theoretically an y hitting set for rank ≤ 2 r matrices is also an LRR set. Th us, in addition to constructing h itting sets for the qu ad r atic forms f M , this pap er will also use those hitting s ets as LRR sets, and also give efficien t LRR algorithms for these constru ctions. 1.3 Rank-Metric Co des Most existing w ork on LRR has fo c used on random measur emen ts, whereas the in teresting asp ec t of PIT is to deve lop deterministic ev aluations of p olynomials. As the main motiv ation for this pap er is to d ev elop new PIT algorithms, w e w ill seek d etermin istic LRR sc h emes. F urther, we will w ant results that are field indep e ndent, a n d so this wo rk will fo cus on noiseless measurements (and matrices that are exactly of rank ≤ r ). In suc h a setting, LRR constru ctions are ve ry r elated to r ank- metric c o des . These co des (related to arr ay c o des ), are error-correcting co des w here the messages are matrices (or tensors) and the norm al notion of distance (the Hamming metric) is replaced by the rank metric (that is, the distance of matrices M and N is rank( M − N )). Ov er matrices, these 1 A v ector is k -sparse if it has a t mos t k non-zero entries. 2 Interestingly , [ KOH11 ] use what they call subsp ac e exp anders a notion that was stu died b efore in a different context in theoretical computer scie nce and mathematics under the name of dimension exp anders [ LZ08 , DS08 ]. 3 co des w ere originally introdu ced ind ep endently b y Gabidulin , Delsarte and Roth [ GK72 , Gab85b , Gab85a , De l78 , Rot91 ]. They show ed, usin g ideas from BCH codes, ho w to get optimal (that is, meeting an analog ue of the Singleton b oun d ) rank-metric cod es o ver matrices, as w ell as h o w to decode these co des efficien tly . A later result by Meshulam [ Mes95 ] constructed rank-metric co des where ev ery co dew ord is a Hanke l matrix. Roth [ Rot91 ] also sho wed ho w to constru ct rank- metric co d es from any hammin g-metric cod e, bu t did n ot p ro vide a deco din g algorithm. Later, Roth [ Rot96 ] considered rank-metric co des ov er tensors a nd ga ve deco ding algorithms for a constant n u m b er of errors. Roth also discussed analog u es to the Gilb ert-V arshamov and Singleton b ounds in this r egime. This alternate metric is motiv ated b y crisscr oss err ors in data storage scenarios, w here corruption can o ccur in b ursts along a r ow or column of a matrix (and are thus r ank-1 errors). W e no w explain ho w r ank-metric co d es are related to L RR. Su pp o se w e hav e a set of matrice s H whic h form a set of (non-adaptiv e, deterministically c hosen) L RR measurement s that can reco v er rank ≤ r m atrices. Define the co de C as the set of matrices orthogonal to eac h matrix in H . Thus, C is a linear co de. F urther, giv en some M ∈ C and E such that rank( E ) ≤ r , it follo ws th at H ( M + E ) = H E (where w e abuse notation and treat M and E as n 2 -long vec tors, and H as an |H| × n 2 matrix). That H is an LRR set means that E can b e r eco v ered from the m easuremen ts H E . Thus the co de C can correct r errors (and has minim um distance ≥ 2 r + 1, by a standard co ding theory argument, as en capsulated in Lemma 8.4 ). Similarly , giv en a rank-metric co d e C that can correct up to rank ≤ r errors, the parit y c hec ks of this co de d efine an LRR scheme. T hus, a small LR R set is equiv alen t to a rank-metric co de with go o d rate. The previous subsection sh o wed the tight connection b et w een LRR and PIT. Via the ab o ve paragraph, we see that hitting sets for qu adratic f orm s are equiv alen t to rank-metric co des, w h en the parity chec k constr aints are restricted to b e rank 1 matrices. 1.4 Reconstruction of Arithmetic Circuits Ev en more general th an the PIT and LR R problems, w e can consider the p roblem of reconstruction of general arithmetic circuits only giv en oracle access to the ev aluation of that circuit. This is the arithmetic analog of the pr oblem of learning a fun ction using mem b ership queries. F or more bac kground on reconstruction of arithmetic circuits w e r efer the reader to [ SY10 ]. Just as w ith the PIT and LRR co nnection, PI T for a sp ecific circuit class giv es information-theoretic reconstruction for that circuit class. As w e consider the PIT question for tensors, we can also consider the reconstruction pr oblem. The general reconstruction problem f or tensors of degree d and rank r wa s considered b efore in the literature [ BBV96 , BBB + 00 , KS 06 ] where learning algorithms were giv en for an y v alue of r . Ho wev er, those algorithms are inherently rand omized. Also of n ote is th at the algorithms of [ BBB + 00 , KS06 ] output a multiplicity automata , whic h in the con text of arithmetic circuits can b e though t of as an arithmetic br anching pr o gr am . In con trast, the most natural form of the reconstruction question w ould b e to outp u t a degree d tensor. 1.5 Our Results In this subs ection we informally summarize ou r r esults. W e again stress that our r esults h an d le matrices of exactly rank ≤ r , and we consider non-adaptive , deterministic measuremen ts. The cul- minating result of this work is the connection s h o win g that lo w-rank r eco v ery redu ces to p erforming sparse-reco ve ry , and that w e can u s e d ual Reed-Solomon co d es to instan tiate the s p arse-reco v ery oracle to ac hiev e a lo w-rank reco very set that only requires rank-1 (or ev en sparse) measuremen ts. W e find the fact t hat we can transform a n algorithm for a combinatorial prop er ty (reco v ering sparse 4 signals) to an algorithm for an alge braic pr op ert y (reco vering lo w-rank matrices) quite in teresting. Hitting Sets for Mat rices and T ensors W e b egin with constructions of hitting sets f or matri- ces, so as to g et blac k b o x PIT for quadratic forms. By impro ving a construction of rank-preserving matrices from Gabizon-Raz [ GR08 ], we are able to sho w the follo wing result, whic h we can then lev erage to constru ct hitting sets. Theorem (Theorem 5.1 ) . L et n ≥ r ≥ 1 . L et F b e a “lar ge” field, and let g ∈ F have “ lar ge” multiplic ative or der. L et M b e an n × n matrix of r ank ≤ r over F . L et ˆ f M ( x, y ) = x † M y b e the bivariate p olynomia l define d by the ve ctors x ∈ F n and y ∈ F n such that 3 ( x ) i = x i and ( y ) i = y i . Then M is non-zer o iff one of the univariate p olynomials ˆ f M ( x, x ) , ˆ f M ( x, g x ) , . . . , ˆ f M ( x, g r − 1 x ) is non-zer o. In tuitiv ely this says th at we can test if the qu ad r atic form f M is zero by testing whether eac h of r un iv ariate p olynomials are zero. As these un iv ariate p olynomials are of degree < 2 n , it follo ws that w e can in terp ola te them fully using 2 n ev aluations. As suc h a univ ariate p olynomial is zero iff all of th ese ev aluations are zero, this yields a 2 nr sized hitting set. While this only works for “large” fields, w e can com bine this with results on sim ulation of large fields (see Section 6.3 ) to deriv e r esults ov er any field with some loss. Th is is encapsu lated in the next results for blac k-b o x PIT, where th e log factors are unn ecessary ov er large fields. Theorem (Corollaries 6.13 and 6.17 ) . L et n ≥ r ≥ 1 . L et F b e any field, then ther e is a p oly ( n ) - explicit 4 hitting set for n × n matric es of r ank ≤ r , of size O ( nr lg 2 n ) . Theorem (Coroll ary 6.18 ) . L et n, r ≥ 1 and d ≥ 2 . L et F b e any field, then ther e is a p oly (( nd ) d , r lg d ) -explicit hitting set for [ n ] d tensors of r ank ≤ r , of size O ( dnr lg d · ( d lg ( nd )) d ) . If F is large enough th en th e O (( d lg ( nd )) d ) term is unnecessary . In such a situation, this is a quasi-p olynomial sized h itting set, imp ro vin g on the min( n d , p oly (( nd ) r )) sized hitting set ac hiev able b y in voki ng the b est kno w n r esults for ΣΠΣ( k ) circuits [ SS11 ]. Ho w eve r, this h itting set is not as explicit as the constru ction of [ SS11 ] since it tak es at least n d time to compute, as opp osed to p oly ( n, d, r ). Neve rtheless, although it tak es p oly (( nd ) d , r lg d ) time to construct the set, the fact th at it is of quasi-p olynomial size is qu ite inte resting an d n o vel . Ind eed, in general it is not clear at all ho w to construct a quasi-p olynomial sized hitting set for general circuits (or just for depth-3 circuits), when one is allo w ed ev en an exp ( nd ) construction time (where n is th e num b er of v ariables, an d d is th e degree of the outpu t p olynomial). W e note that this result imp ro ves on the t wo ob vious hitting sets seen in Lemmas 3.11 and 3.13 . The fir st giv es n d tensors in the hitting set and is p olyl og ( n, d, r )-explicit wh ile the second giv es a set of size ≈ dnr while not b eing explicit at all. T he ab o v e result non-trivially interpolates b et ween these t wo results. Finally , w e mention that in Remark 6.9 w e exp lain how one ca n achiev e (roughly) a p oly ( r ( dn ) √ d )-constructible hitting set of th e same s ize. As th is is a somewhat mild improv emen t (this is still not the explicitness that w e we re lo oking for) w e only briefly sk etc h the argumen t. Lo w- Rank Reco very As mentioned in the previous section, blac k-b o x PIT resu lts imp ly LRR constructions in an information theoretic s ense. Thus, the ab o ve h itting s ets imply LRR con- structions bu t the algorithm for reco v ery is not implied b y the ab ov e resu lt. T o yield algorithmic 3 In this paper, vec tors and matrices are indexed from zero, so x = (1 , x , x 2 , . . . , x n − 1 ) † . 4 A n × n matrix is t - ex plicit if each entry can b e (deterministically) computed in t steps, where field op erations are considered unit cost. 5 results, we actually establish a stronger claim. T hat is, w e first sho w that the ab o v e h itting sets em b ed a natural sp arse-reco v er y set arising fr om the dual Reed-Solomon cod e. Then w e develo p an algorithm that sho w s that any s p arse-reco v ery set gives r ise to a lo w-ran k -r eco v ery set, and that r eco v er y can b e p erformed efficien tly giv en an oracle for s p arse reco v ery . This connection (in the context that an y error-correcting co de in the hamming metric yields an error-correcting co d e in the rank-metric) w as in dep end en tly made by Roth [ Rot91 ] (see Theorem 3), who did not giv e a reco v ery pro cedu re for the resulting LRR sc heme. T h e next theorem, whic h is the main r esult of the pap er, shows th is connection is also efficient with resp ec t to reco very . Theorem (Theorem 7.19 ) . L e t n ≥ r ≥ 1 . L et V b e a set of (non-adaptive) me asur e ments for 2 r -sp arse-r e c overy for n -long ve ctors. Then ther e is a p oly ( n ) -explicit set H , which is a (non- adaptive) r ank ≤ r low-r ank-r e c overy set for n × n matric es, with a r e c overy algorithm running in time O ( r n 2 + nτ ) , wher e τ is the amount of time ne e de d to do sp arse-r e c overy fr om V . F urther, |H| = 2 n |V | , and e ach matrix in H is n - sp arse. This result shows that sparse-reco very and lo w -rank reco v ery (at least in the exact case) are v ery closely connected. In terestingly , this shows that sparse-reco very (whic h can b e regarded as a com binatorial prop erty) and lo w-rank reco v ery (which can b e regarded as an algebraic prop ert y) are tigh tly connected. Man y fruitfu l connections ha ve tak en this form, su c h as in sp ectral graph theory , and p erhaps the connection presented here will yield y et further results. Also, the algorithm used in the ab o ve result is purely linear-algebraic, in con trast to the con v ex optimization approac hes th at man y compressed sensing w orks use. Ho wev er, we d o n ot kno w if the ab o ve result is stable to noise, and regard th is issu e as an imp ortan t qu estion left op en b y this w ork. When the ab ov e result is com b ined with our hitting s et results, we ac hieve the follo win g LRR sc heme for matrices (and an LRR sc h eme for tensors, with parameters similar to Corollary 6.18 men tioned ab ov e, and Corollary 8.6 mentioned b elo w, is d eriv ed in Corollary 8.2 ). Theorem (Corollary 7.26 ) . L et n ≥ r ≥ 1 . Over any field F , ther e is an p oly ( n ) - explicit set H , of O ( r n lg 2 n ) size, such that me asur ements against H al low r e c overy of n × n matric es of r ank ≤ r in time p o ly ( n ) . F urther, the matric es in H c an b e chosen to b e al l r ank 1, or al l n - sp arse. W e note again that o ver large fi elds these logarithmic factors are seen to b e un needed. Some prior work [ GK72 , Gab85b , Gab85a , Del78 , Rot91 ] on LRR fo cused on fin ite fi elds , and as suc h b ased their results on BCH co d es. The ab o v e result is based on (dual) Reed-Solomon co des, and as su c h w orks o ve r an y field (when com bined with results allo wing simulation of large fields b y small fields). Other p rior wo rk [ RFP10 ] on exact LRR p ermitte d randomized measur ements, while w e ac hieve deterministic measurement s. F ur ther, we are able to d o LRR with measurements that are either all n -sparse, or all rank- 1. As Roth [ Rot91 ] indep end en tly observ ed, th e n -sparse LRR measuremen ts can arise from an y (hamming-metric) error-correcting co de (but he did not p ro vid e decodin g). T an, Balzano and Drap er [ TBD11 ] show ed that random ( n lg n )-sparse m easur emen ts provide essen tially the same lo w-rank reco v ery prop erties as rand om measuremen ts. Thus, our results essentia lly ac hieve this deterministically . W e further observe that a sp ecific co de (the d ual Reed-Solomon co de) allo ws a c hange of basis for the measur emen ts, and in this new basis the measuremen ts are all rank 1. Rec ht et al. [ RFP10 ] ask ed whether lo w-rank r eco v ery was p ossible when the measurements w ere rank 1 (or “fact ored”), as suc h measuremen ts could b e more practical as t hey are simpler to generate and store in memory . Th us, our construction an s w ers this qu estion in the p ositiv e direction, at least for exact LRR. 6 Rank-Metric Co des App ealing to the connection b et w een LRR and rank-metric co des, we ac hiev e the follo win g constru ctions of rank-metric co des. Theorem (Corollary 8.5 ) . L et F b e any field, n ≥ 1 and 1 ≤ r ≤ n/ 2 . Then ther e ar e p oly ( n ) - explicit r ank-metric c o des with p oly ( n ) -time de c o ding for up to r err ors, with p ar ameters [[ n ] 2 , ( n − 2 r ) 2 · O (lg 2 n ) , 2 r + 1] F , and the p arity che cks on this c o de c an b e chosen to b e al l r ank-1 matric es, or al l n -sp arse matric es. Earlier work on rank-metric codes o v er finite fields [ GK72 , Gab85b , Gab85a , Del78 , Rot91 ] ac hiev ed [[ n ] 2 , n ( n − 2 r ) , 2 r + 1] F q rank-metric cod es, w ith efficient deco din g algorithms. T hese are optimal (meeting th e analogue of the Singleton b oun d for rank-metric co des). How ev er, these constructions only work o ver fin ite fields. While our co de achiev es a w orse rate, its construction w orks o v er any field, and ov er infi n ite fields the O (lg 2 n ) term is un n eeded. F ur ther, Roth [ Rot91 ] observ ed that the resulting [[ n ] 2 , ( n − 2 r ) 2 , 2 r + 1] cod e is optimal (see d iscussion of his Theorem 3) o ver algebraically closed fields (wh ic h are infin ite). W e are also able to giv e rank-metric cod es o v er tensors, which can correct err ors u p to rank ≈ n d/ lg d (out of a maximum n d − 1 ), while s till ac hieving constan t rate. The ran k -metric cod e arising from the naiv e lo w-rank r eco v er y of Lemma 3.11 nev er ac hiev es constan t rate, and prior w ork b y Roth [ Rot96 ] only gav e deco ding against a constan t num b er of errors. Theorem (Corollary 8.6 ) . L et F b e any field, n, r ≥ 1 and d ≥ 2 . Then ther e ar e p oly (( nd ) d , r lg d ) - explicit r ank-metric c o des with p oly (( nd ) d , r lg d ) -time de c o ding for up to r err ors, with p ar ameters [[ n ] d , n d − O ( d 2 nr lg d lg( dn )) , 2 r + 1] F . W e note h ere that our decod ing algorithm will return the entir e tensor, whic h is of size n d . T r ivially , an y algorithm returning th e enti re tensor m u s t tak e at least n d time. In this case, the lev el of explicitness of the co d e w e ac hieve is reasonable. Ho we v er, a more desirable result would b e for th e algorithm to r eturn a rank ≤ r r epresen tation of the tensor, and th u s the n d lo we r b ound would not apply so that one could hop e for faster deco din g algorithms. Un fortunately , ev en for d = 3 an efficien t algo rithm to d o so would imply P = NP . Th at is, if an algorithm (even one which is not a rank-metric deco d ing or lo w-rank reco very algorithm) could pro d uce a rank ≤ r decomp osition for a n y rank ≤ r t ensor, then one could compute te nsor-rank b y as it is th e minimum r suc h that the r esulting rank ≤ r decomp osition actually computes the desired tensor (this can b e c heck ed in p oly ( n d ) time). How ev er, H ˚ astad [ H ˚ as90 ] sh o wed that tensor-rank (o ver finite fields) is NP -hard for any fixed d ≥ 3. It follo ws that for an y (fixed) d ≥ 3, if one could reco ve r (ev en in p oly ( n d )-time) a rank ≤ r tensor in to its r ank ≤ r decomp osition, then P = NP . Th us, w e on ly discuss reco ve ry of a tensor by r epro du cing its en tire list of entries, as opp osed to its m ore concise represent ation. Finally , we remark that in [ Rot96 ] Rot h discussed the question of deco din g rank-metric co des of degree d = 3, ga v e deco din g algorithms for errors o f rank 1 and 2 , and wrote that “Since compu ting tensor rank is an in tractable p roblem, it is un likely that we will ha ve an efficien t deco ding algorithm . . . otherwise, w e could use the d eco d er to compute the rank of any tensor. Hence, if there is an y efficien t deco ding algorithm, then w e exp ect suc h an algorithm to reco v er the error tensor without necessarily obtaining its r ank. S uc h an algorithm, that can hand le any p rescrib ed n um b er of errors, is not yet kno w n.” Th us, our work giv es the first su c h algorithm for tensors of d egree d > 2. 1.6 Pro of Ov erview In this section w e giv e pr o of outlines of th e results mentioned so far. 7 Hitting Sets for Matrices The main id ea f or our hitting set construction is to reduce the question of hitting (n on -zero) n × n matrices to a question of hitting (non -zero) r × r matrices. Once this r eduction is p erformed, w e can then run the naive hitting set of Lemma 3.11 , which queries all r 2 en tries. T h is can loosely b e seen in analogy with the k er n elization pro cess in fixed- parameter tractabilit y , wh ere a problem d ep ending on the inp ut size, n , and some p arameter, k , can b e solv ed by first redu cing to an ins tance of size f ( k ), and then br ute-forcing this instance. T o p erform this kerneliza tion, we first note that an y n × n matrix M of rank exactly r can b e written as M = P Q † , where P and Q are n × r matrices of rank exactly r . T o red uce M to an r × r matrix, it th us suffi ces to reduce P and Q eac h to r × r matrices, denoted P ′ and Q ′ . As this red uction must preserve the fact th at M is non-zero, w e n eed that P ′ Q ′ 6 = 0. W e enforce this requirement by insisting that P ′ and Q ′ are also rank exactly r , so that M ′ = P ′ Q ′ is also n on-zero. T o ac hiev e this rank-preserv ation, we tu rn to a lemma of Gabizon-Raz [ GR08 ] (we n ote that this lemma h as b een used b efore f or blac k-b o x PIT [ KS08 , SS 11 ]) . They ga ve an explicit family of O ( nr 2 )-man y r × n -matrices { A ℓ } ℓ , suc h th at for any P and Q of rank exactly r , at least one matrix A ℓ from the family is suc h that r ank( A ℓ P ) = rank( A ℓ Q ) = r . T ranslating this result into our problem, it follo ws that one of the r × r matrices A ℓ M A † ℓ is full-rank. The ( i, j )-th ent ry of A ℓ M A † ℓ is h M , ( A ℓ ) i ( A ℓ ) † j i , where ( A ℓ ) i is the i -th ro w of A ℓ . It follo ws that q u erying eac h ent ry in these r × r matrices corresp onds to a rank 1 measuremen t of M , and thus make u p a h itting set. As there were O ( nr 2 ) c hoices of ℓ and r 2 c hoices of ( i, j ), th is giv es a O ( nr 4 )-sized hitting set. T o ac hieve a smaller hitting s et, w e u se the follo w ing sequence of ideas. First, we observ e that in the ab ov e, w e can alwa ys assu me i = 0. Lo osely , this is b ecause A ℓ M A † ℓ is alw a ys full-rank, or zero. Thus, only the first ro w of A ℓ M A † ℓ needs to b e queried to d etermin e this. Second, we imp ro ve up on the Gabizon-Raz lemma, and pro vid e an explicit family of rank-pr eservin g matrices with size O ( nr ). This follo ws from mo difying th eir construction so the d egree of a certain d etermin ant is smaller. T o ensure that the determinant is a non-zero p olynomia l, we s ho w that it has a u nique monomial that ac hiev es m aximal degree, and that the term achieving maximal d egree has a non- zero coefficient as a V anderm on d e determinant (formed from p ow ers of an elemen t g , wh ic h has large multiplic ativ e order) is n on-zero. Finally , we observe th at the hitting set constraints can b e view ed as a constraints regarding p olynomial interp olation. This view sho ws that some of th e constrain ts are linearly-dep enden t, and th u s can b e remo v ed. Eac h of the ab ov e ob s erv ations sa ves a factor of r in the size of th e hitting set, and th us pro duces an O ( nr )-sized hitting set. Lo w- Rank Reco v e ry Ha ving constructed hitting sets, Lemma 3.10 imp lies th at the same con- struction yields lo w-r an k -r eco v ery sets. As this lemma do es not provi de a reco ve ry algorithm, we pro v id e one. T o do so, we must fi rst c hange the b asis of our hitting set. That is, th e h itting set B yields a set of constrain ts on a matrix M , and we are free to c ho o se another b asis for these constrain ts, whic h we call D . T he vir tu e of this new basis is that eac h constrain t is non-zero only on some k -diagonal (the en tries ( i, j ) such that i + j = k ). It turns out that these constr aints are th e parity c h ec ks of a d ual Reed-Solomon co de with distance Θ ( r ). This co d e can b e deco ded efficien tly u sing what is kno wn as Prony’s metho d [ dP95 ], w h ic h w as dev elop e d in 1795. W e give an exp osition in Section 7.1 , w here we show how to synd rome-decod e this co de up to half its m inim u m distance, coun ting erasures as half-errors. Th u s, giv en a Θ( r )-sp ars e vecto r (wh ich can b e thou ght of as errors from the ve ctor 0 ) these parit y c h ecks imp ose constrain ts from wh ic h the sparse v ector can b e reco v ered. Put another wa y , our low-rank-reco v ery set naturally em b eds a sp arse-reco v ery set along eac h k -d iagonal. Th us, in d esigning a reco ve r y algorithm for our low-rank reco very s et, w e do more and show ho w to reco v er f rom any set of measurements which em b ed a sparse-reco very set along eac h k -diagonal. 8 In term s of err or-correcting co d es, this sho ws th at an y hamm in g-metric co de yields a ran k -metric co de o ver matrices, and that deco ding the rank-metric co de efficient ly reduces to deco ding the hamming-metric co de. T o p erform reco very , we introdu ce the notion of a m atrix b eing in ( < k )-up p er-ec helon form. Lo osely , this sa ys that M ( d ≥ 2 . Then ther e is a hitting set for J n K d tensors of r ank ≤ r , of size ≤ dnr / log q ( q /d ) + 1 ≈ dnr . F urther, ther e is an r -low-r ank r e c overy set of size ≤ 2 dnr / log q ( q /d ) + 2 . Pr o of. F or any n on -zero tensor T : J n K d → F , f T has degree d , and thus by the S ch wa r tz-Zipp el Lemma, for a random a ∈ F n q , f T ( a ) = 0 with probabilit y at most d/q . There are at most q dnr suc h non -zero tenors. By a union b oun d, it follo w s that k rand om p oin ts are not a h itting set for rank ≤ r tensors with probability at most q dnr ( d/q ) k , whic h is < 1 if k > dnr / log q ( q /d ). The lo w-rank-reco very set follo w s fr om Lemma 3.10 . W e n o w briefly remark on the tigh tness of the ab ov e result. Th e general case of tensors is n ot w ell u ndersto o d, as it is not we ll-un dersto o d h o w many tensors there are of a giv en r ank. F or matrices, the situation is muc h more clear. In p articular, Roth [ Rot91 ] show ed (us ing the language of rank-metric cod es) that ov er finite fields the b est (improp er) h itting set for n × n matrices of rank ≤ r is of s ize nr , and o ver algebraically closed fi elds the b est (improp er) h itting set is of s ize (2 n − r ) r . As w e will aim to b e field indep en den t, th e second b oun d is more relev an t, and we indeed matc h this b ound (as seem in Theorem 5.10 ) with a prop er hitting set. Clearly , the ab o v e lemma is non-explicit. Ho wev er, it yields a muc h smaller hitting set than the n d result giv en in Lemma 3.11 . Note that p revious work (eve n for d = 2) on LRR and rank-metric co des did not fo cus on requiring that the measuremen ts are rank-1 tensors, and th us cannot b e 13 used for PIT. Giv en this lac k of kn o wledge, this pap er seeks to construct prop er hitting sets, and lo w-rank-reco very sets, that are b ot h explicit and small. W e remark that any explicit hitting set naturally leads to tensor r ank lo w er b oun ds 5 . The follo w- ing lemma, w hic h can b e seen as a sp ec ial case of th e more general results of Heintz -Sc hn orr [ HS80 ] and Agra wal [ Agr05 ], shows th is connection m ore concretely . Lemma 3.14. L et H b e a hitting set for J n K d tensors of r ank ≤ r , such that |H | < n d . Then ther e is a p oly ( n d , |H| ) -explicit tensor of r ank > r . Pr o of. C onsider the constraint s imp osed on a tensor T by the system of equations h T , H i = 0 . There are |H| constraint s and n d v ariables. It follo ws that there is a non-zero T solving this system. By the definition of a hitting set, it follo ws that rank( T ) 6≤ r . T hat T is explicit follo ws from Gaussian Elimination. F or d = 2, the ab o v e is less interesting, as m atrix rank is w ell und ersto o d and we kn ow man y matrices of high rank. F or d ≥ 3, tensor ran k is f ar less u n dersto o d . F or d = 3, the b est kno wn lo we r b oun ds for the rank of explicit tensors, o v er arbitrary fields, due to Alexeev, F orb es, an d Tsimerman [ AFT11 ], are 3 n − O (lg n ) (o ver F 2 , a lo we r b ou n d of 3 . 52 n is kno wn, essen tially due to Bro wn and Dobkin [ BD80 ]). More generally , for an y fixed d , no exp licit tensors are kno wn with tensor rank ω ( n ⌊ d/ 2 ⌋ ). The ab ov e lemma sho w s that constru cting hitting sets is at least as hard as getting a lo w er b oun d on an y sp ecific tensor. In p articular, constructing a h itting set for J n K d tensors of rank ≤ r of size O ( d nr k ) w ith k < 2 w ould yield new tensor ran k lo w er b ounds for o d d d , in particular d = 3. Such lo w er b ounds would imply new circuit lo wer b o unds, using the results of Strassen [ Str73 ] and Raz [ Raz10 ]. Ou r results give a hitting set with k ≈ lg d , and w e lea v e op en whether fur ther impro v ements are p ossib le. W e will mentio n the definitions an d preliminaries of rank-metric co d es in Section 8 . 3.1 P ap er Outline W e briefly outline the rest of the pap er. In S ection 4 we giv e our improv ed constru ction of rank- preserving matrices, wh ic h w ere firs t constructed by Gabizon-Raz [ GR08 ]. In S ection 5 we th en use this constru ction to giv e our reduction from biv ariate iden tit y testing to univ ariate identi t y testing (Section 5.1 ), whic h then readily yields our hitting s et for matrices (S ection 5.2 ). In Section 5.3 we sho w an equiv alen t hitting set, which is more us eful for lo w-rank-reco v ery . Section 6 extends the ab ov e results to tensors, where Section 6.1 redu ces d -v ariate identit y testing to univ ariate id en tity testing, and Section 6.2 uses this r eduction to construction h itting sets for tensors. Finally , Section 6.3 sho w s ho w to extend these resu lts to any field. Lo w-rank reco ve ry of matrices is d iscu ssed in Section 7 . I t is split in to t wo parts. Section 7.1 sho w s how to deco de dual Reed-Solomon co des, wh ic h we u se as a sparse-reco ve ry oracle. Sec- tion 7.2 sho ws ho w to, giv en an y suc h sparse-reco ve ry oracle, p erf orm low-rank-reco very of matrices. Instan tiating the oracle with dual Reed-Solomon co des give s our lo w -rank-reco ve ry constru ction. Section 8 shows h o w to extend our LRR algorithms to tensors, and ho w to use these resu lts to construct rank-metric co des. Finally , Section 9 discusses some problems left op en by this wo rk. 5 This conn ection, along with the conn ection to rank- metric cod es mentioned earlier, can b e put in a more broad setting: hitting sets (and thus low er-b ounds) for circuits from some class C are in a sense equiv alent to C - metric linear codes. That is, co des where dist( x, y ) is defined as the size of the smallest circuit whose truth table is th e string x − y . W e do not pursue th is idea furt h er in t h is work. 14 4 Impro v ed Construction of Rank-preserving Matrices In this section w e will giv e an imp ro ved v ers ion of the Gabizon-Raz lemma [ GR08 ] on the con- struction of rank-pr eservin g matrices. The goal is to transform an r -d im en sional subspace living in an n -dimensional am bien t space, to an r -dimensional su bspace living in an r -dimensional am bi- en t sp ace. W e will later sho w (see Theorem 5.1 ) ho w to use such a tran s formation to red u ce the problem of PIT for n × m matrices of r ank ≤ r to the pr oblem of P IT for r × r matrices of rank ≤ r . W e first presen t the Gabizon-Raz lemma ([ GR08 ], Lemma 6.1), stated in the language of this pap er. Lemma (Gabizon-Raz ([ GR08 ], Lemma 6.1)) . L et 1 ≤ r ≤ n . L et M ∈ F n × r b e of r ank r . Define A α ∈ F r × n by ( A α ) i,j = α ij . Then ther e ar e ≤ nr 2 values α ∈ F such that rank( A α M ) < r . Our v er s ion of this lemma giv es a set of matrices parameterized by α where there are only n r v alues of α that lead to rank( A α M ) < r . Th is extra factor of r allo ws us to ac h iev e an O (( n + m ) r )- sized hitting set f or matrices in stead of a O (( n + m ) r 2 )-sized hitting set. W e commen t more on the necessit y of this impr o vemen t in Remark 5.3 . W e now state our version of this lemma. Ou r pro o f is v ery similar to that of Gabizon-Raz. Theorem 4.1. L et 1 ≤ r ≤ n . L et M ∈ F n × r b e of r ank r . L et K b e a field extending F , and let g ∈ K b e an element of or der ≥ n . De fine A α ∈ K r × n by ( A α ) i,j = ( g i α ) j . Then ther e ar e ≤ nr −  r +1 2  < nr values α ∈ K su ch that r ank( A α M ) < r . Pr o of. W e w ill no w treat α as a v ariable, and th us refer to A α simply as A . The matrix AM is an r × r matrix, and th u s the claim will f ollo w from sho wing that det( AM ) is a non-zero p olynomial in α of degree ≤ nr −  r +1 2  . As r ≥ 1, nr −  r +1 2  < nr . T o analyze this determinant , we in vo k e the Cauc hy-Binet formula. Lemma (Cauch y-Binet F orm u la, Lemma A.1 ) . L et m ≥ n ≥ 1 . L et A ∈ F n × m , B ∈ F m × n . F or S ⊆ J m K , let A S b e the n × | S | matrix forme d fr om A by taking the c olumns with indic es in S . L et B S b e define d analo gously, but with r ows. Then det( AB ) = X S ∈ ( J m K n ) det( A S ) det( B S ) so that det( AM ) = X S ∈ ( J n K r ) det( A S ) det( M S ) F or S = { k 1 , . . . , k r } , det( A S ) =          ( α ) k 1 · · · ( α ) k r ( g α ) k 1 · · · ( g α ) k r . . . . . . . . . ( g r − 1 α ) k 1 · · · ( g r − 1 α ) k r          =          1 · · · 1 g k 1 · · · g k r . . . . . . . . . ( g k 1 ) r − 1 · · · ( g k r ) r − 1          · α P r ℓ =1 k ℓ = α P r ℓ =1 k ℓ Y 1 ≤ i | S 2 | , then there is some v ∈ S 1 \ S 2 suc h that S 2 ∪ { v } is linearly in d ep end ent. Thus, if S 1 , S 2 are b oth sets of linearly indep enden t v ectors and | S 1 | = | S 2 | then f or an y w ∈ S 2 \ S 1 there is a v ector v ∈ S 1 \ S 2 suc h that ( S 2 \ { w } ) ∪ { v } is linearly in dep end en t. No w supp ose (for con tradiction) th at there are t wo different sets S 1 , S 2 ⊆ J m K that maximize w ( S ) o ver the sets suc h that det( M S ) 6 = 0, so that | S 1 | = | S 2 | = n . Pick the smallest index k in the (non-empt y) symmetric difference ( S 2 \ S 1 ) ∪ ( S 1 \ S 2 ). Without loss of generalit y supp ose k ∈ S 2 \ S 1 . It follo w s that there is an index l ∈ S 1 \ S 2 suc h that the column s in S 3 def = ( S 2 \ { k } ) ∪ { l } are linearly in dep end en t (by the S teinitz Exc h an ge Lemma), and thus det( M S 3 ) 6 = 0 as | S 3 | = n by construction. By c hoice of k and construction of l , k 6 = l and th us k < l . Thus, w ( S 3 ) = w ( S 2 ) + l − k > w ( S 2 ). Ho we v er, this cont radicts that S 2 w as a maximizer to w ( S ) sub ject to det( M S ) 6 = 0. Th u s, the assumption of non-u nique maximizers is false; there m u st b e a unique maximizer. Th us det( AM ) is a non-zero p olynomial of degree ≤ n r −  r +1 2  in α , so there are at most that man y v alues such th at det( AM ) = 0. W e remark that Lemma 4.2 can b e seen as a s p ecial case of a more general r esult ab out matroids, whic h states that if eac h element in the ground set h as a unique (p o sitiv e) we igh t, then there is a u nique ind ep enden t set with maximal weig h t. Ho wev er, as we index matrix columns starting at 0 this general fact do es not immediately apply . Rather, w e implicitly use that all bases in v ector matroids hav e the same num b er of v ectors. In suc h a case, the w eight fu n ction can b e shifted b y an additiv e constant without affecting th e prop ert y of havi ng a un ique maximizer. W e no w extend th e ab o v e result to the case when the ran k of the n × r matrix may b e less than r . This will b e useful when study in g hitting sets for rank ≤ r matrices, for then w e do n ot know the true r ank of the unknown matrix, and only hav e the b oun d of “ ≤ r ”. Corollary 4.3. L et 1 ≤ s ≤ r ≤ n . L e t M ∈ F n × r ′ b e of r ank s , for r ′ ≥ s . L et K b e a field extending F , and let g ∈ K b e an element of or der ≥ n . De fine A α ∈ K r × n by ( A α ) i,j = ( g i α ) j . Then ther e ar e ≤ nr −  r +1 2  < nr values α ∈ K su ch that the first s r ows of A α M have r ank < s . Pr o of. C onsider M ′ ∈ F n × s to b e a matrix formed from s basis columns of M . It follo w s, fr om Theorem 4.1 , that there are at most ns −  s +1 2  v alues of α such that the s × n matrix A ′ α has rank( A ′ α M ′ ) < s . As r an k( AM ′ ) = r ank( AM ) holds for an y A , there are at most ns −  s +1 2  man y v alues of α such that r ank( A ′ α M ) < s . Also, as ns −  s +1 2  ≤ nr −  r +1 2  for s ≤ r ≤ n , it also holds that there are ≤ n r −  r +1 2  v alues of α suc h that rank( A ′ α M ) < s . Final ly the claim follo ws by observing that, b y construction, A ′ α M is exactly th e fi rst s r o ws of A α M . 16 5 Iden tit y T esting for Matrices The previous section sh ow ed how w e can map an r -dimensional sub space of an n -dimensional am bient s p ace to an r -dimen sional subs pace of an r -dimensional am bient sp ace. In this section, w e w ill use this map to reduce the PIT problem for ran k ≤ r matrices of size n × m to the PI T problem from rank ≤ r matrices of size r × r . T his will b e d one b y app lying the d im en sion reduction t wice, once to the rows and once to th e columns. F urther, the r × r v ersion can b e solv ed in r 2 ev aluations, using the n aiv e approac h of Lemma 3.11 in querying eac h en try in the matrix. When phrased this wa y , one can s h o w that this give s a Θ(( n + m ) p oly ( r ))-sized hitting set. This reduction idea is an alogous to the k ern elization tec hniqu e used in fi xed-parameter tractabilit y , b ut we d o not dev elop this connection fur ther. While this idea demonstrates the feasibilit y of the rough b ound cited ab ov e, we actually ac h ieve a Θ(( n + m ) r )-sized hitting set via tighte r analysis. 5.1 V ariable Reduction Before giving the hitting set constru ction and its analysis, we firs t p resen t the main theorem used in the analysis. While its statemen t seems unrelated to the intuitio n presen ted ab o ve, the pr o of will exploit this in tuition. When interpreting the result, recall th at we index en tries in matrices (and v ectors) starting at 0, as w ell as recalling the defi nition of ˆ f T from a tensor T . Theorem 5.1. L et m ≥ n ≥ r ≥ 1 . L et K b e an extension of F such that g ∈ K has or der ≥ m . L et M b e an n × m matrix of r ank ≤ r over F . Then M is non-zer o (over F ) iff one of the univariate p olynomials ˆ f M ( x, x ) , ˆ f M ( x, g x ) , . . . , ˆ f M ( x, g r − 1 x ) is non-zer o (over K ). Pr o of. ( ⇐ = ) : If M is zero then so must all ˆ f M ( x, g i x ) b e as well. T aking th e con trap osit iv e yields this direction. ( = ⇒ ) : Sa y r ank( M ) = s . By assumption 0 < s ≤ r . Recall that putting M into r educed ro w-ec helon form yields a decomp o sition M = P Q † , suc h that P ∈ F n × s and Q ∈ F m × s suc h that rank( P ) = r an k ( Q ) = s . W e remark th at it is crucial for our pr o of that w e ha ve “rank( P ) = rank( Q ) = s ” here. Inv oking the b ound “rank( P ) , rank( Q ) ≤ s ”, w h ic h one gets directly via the definition of rank of M , is insu fficien t. W e now exploit the kerneliza tion idea menti oned ab o ve. Consid er the matrices A α ∈ K r × n and B α ∈ K r × m defined by ( A α ) i,j = ( g i α ) j and ( B α ) i,j = ( g i α ) j . No w consid er A α P and B α Q , wh ic h ha ve sizes r × s eac h. W rite them in blo c k n otation as  P ′ α P ′′ α  and  Q ′ α Q ′′ α  suc h that P ′ α and Q ′ α are b oth s × s matrices. By our r efinemen t of the Gabizon-Raz lemma [ GR08 ], our Corollary 4.3 , it follo ws that there are < nr v alues of α such that rank( P ′ α ) < s and < m r v alues of α suc h that rank( Q ′ α ) < s . By the un ion b ound, th ere are < ( n + m ) r v alues such that rank ( P ′ α ) < s or rank( Q ′ α ) < s . Let H b e an extension field of K , su c h that | H | ≥ ( n + m ) r . It follo ws that there is some α ∈ H su c h that rank( P ′ α ) = s and rank( Q ′ α ) = s . Fix this as the v alue of α , and w e now d rop α from our n otation. Via blo c k multiplicat ion we see th at AM B † = AP ( B Q ) † =  P ′ P ′′   Q ′ Q ′′  =  P ′ Q ′ P ′ Q ′′ P ′′ Q ′ P ′′ Q ′′  As rank( P ′ ) = s and rank( Q ′ ) = s , it f ollo ws that rank( P ′ Q ′ ) = s . W e remark that it is h ere wher e the n aiv e b ound “rank( P ) , ran k ( Q ) ≤ s ” is insufficient , an d w e crucially use that “rank( P ) = rank( Q ) = s ”. 17 As rank( P ′ Q ′ ) = s , and P ′ Q ′ is an s × s matrix, it follo ws th at some en tr y in its first ro w (whic h has index 0, by our notation) is non-zero. As P ′ Q ′ is a p rincipal minor of AM B † , it follo ws that some entry in th e fi rst ro w of AM B † is non-zero. Denote r o w i of A as A i , and ro w j of B as B j . As the fi rst ro w of AM B † is A 0 M B † , it follo ws th en there is s ome 0 ≤ ℓ ≤ r − 1 su c h th at A 0 M B † ℓ 6 = 0. Expand ing this ev aluation out, we see that A 0 M B † ℓ = h M , A 0 B † ℓ i = n − 1 ,m − 1 X i =0 ,j =0 M i,j · ( A 0 ) i ( B ℓ ) j = n − 1 ,m − 1 X i =0 ,j =0 M i,j A 0 ,i B ℓ,j = n − 1 ,m − 1 X i =0 ,j =0 M i,j ( g 0 α ) i ( g ℓ α ) j = ˆ f M ( α, g ℓ α ) Th us, we see th at ˆ f M ( x, g ℓ x ) has a n on-zero p oint ov er the field H . It follo ws that it is a non-zero p olynomial ov er H . As it has co efficien ts o ve r K , ˆ f M ( x, g ℓ x ) is non -zero o ve r K as well. R e mark 5.2 . W e n o w remark on h o w to implement th e ke rnelization idea, men tioned in the in- tro duction to this section, in a more straigh t-forwa rd sense. One can see that rank( P ′ Q ′ ) = s sho w s that AM B † 6 = 0. As AM B † is of size r × r , w e can then run the naiv e r 2 -size hitting set of Lemma 3.11 for r × r -sized matrices, wh ic h chec ks eac h individual ent ry . Noting that the ( i, j )-th en try of AM B † is equal to h M , A i B † j i w e s ee that we can implemen t this naiv e hitting set as a hitting set f or n × m matrices. Th us, for eac h α th ere are r 2 rank-1 matrices to test, and w e need at most ( n + m ) r c h oices of α (where h ere we assume K is at least this big). It f ollo ws that there exists an explicit h itting set of size ( n + m ) r 3 . R e mark 5.3 . W e br iefly discuss the n ecessit y of our version of the Gabizon-Raz lemma for the ab o v e pro of. The ab o v e pr o of do es not in v oke our version of th e lemma in the fu llest, in th e sense that the nr b ound on the num b er of “bad” α w as only used in the sense that it w as a fin ite b ound. Th us, giv en that our v ersion of the lemma “only” impr o ves the nr 2 b ound of Gabizon-Raz to n r , it ma y b e u n clear wh y our ve rsion is needed here. The crucial use of our v ersion of the lemma is k eeping the degree lo w. T h at is, if one inv ok ed the original Gabizon-Raz lemma, one would result in “ M is n on-zero iff one of the u niv ariate p olynomials ˆ f M ( x, x ) , ˆ f M ( x, x 2 ) , . . . , ˆ f M ( x, x r ) is non-zero”. While this is correct, it will lead to a larger hitting set as one needs to inte rp olate r p olynomials, eac h of degree ≈ r n , wh ic h will giv e a Θ( nr 2 )-sized set instead of the Θ ( nr )-sized set we are able to ac hieve . W e also state an equiv alen t version of this result, wh ic h w ill b e usefu l for h igher-degree tensors. Corollary 5.4. L et m ≥ n ≥ r ≥ 1 . Over the field F , c onsider the bivariate p olynomial f ( x, y ) = P r i =1 p i ( x ) q i ( y ) such that deg( p i ) < n and deg ( q i ) < m for al l i . L et K b e an extension of F such that g ∈ K has or der ≥ m . Then f is non-zer o (over F ) iff one of the univ ariate p olynomials f ( x, x ) , f ( x, g x ) , . . . , f ( x, g r − 1 x ) is non-zer o (over K ). 18 5.2 The Hitting Set for Matrices In this sub s ection w e use the theorem of the last subsection to construct h itting sets for matri- ces. First, recall our n otion of a h itting set for matrices, as giv en in Definition 3.8 . Now recall that Th eorem 5.1 shows that for an y M of rank ≤ r , M is non-zero iff one of the p olynomials in { ˆ f M ( x, g ℓ x ) } 0 ≤ ℓ k + 1 ≥ ( n + m ) − r , D ′ r,n,m tak es r − (( n + m ) − ( k + 1)) few er matrices then D r,n,m . It f ollo ws that |D ′ r,n,m | = |D r,n,m | − r ( r − 1) = ( n + m − 1) r − r ( r − 1) = ( n + m − r ) r . W e sket c h another p ro of of this r esult in Remark 7.24 . The ab ov e r esults also imply th at ran k( B r,n,m ) = ( n + m − r ) r , which is b etter than the analysis giv en in Theorem 5.6 . This immediately give s that there are explicit ( n + m − r ) r -sized (pr op er) hitting sets for n × m matrices of r ank ≤ r , as we can (in p oly ( m ) steps) fi nd a b asis for B r,n,m . Th is basis will consist of rank-1 matrices, an d also b e the desired hitting set. Ho w ev er, in the in terest of b eing more explicit, we p resen t the follo win g constr u ction. Construction 5.9. L et m ≥ n ≥ r ≥ 1 . L et K b e an extension of F such that g ∈ K is of or der ≥ m and α 0 , . . . , α n + m − 2 ∈ K ar e distinct. L et B ′ k ,ℓ ∈ K n × m to b e the r ank-1 matrix define d by ( B ′ k ,ℓ ) i,j = α i k ( g ℓ α k ) j , and let B ′ r,n,m def = { B k ,ℓ } 0 ≤ ℓ r ). W e will sho w th at the inner-pr o ducts h M , B ′ r,n,m i determine the inner-pr o ducts h M , D r,n,m i . Then we sho w that this implies the claim. Recall that the in ner-pro d uct of a matrix D ∈ D r,n,m is simply a co efficien t C x k ( ˆ f M ( x, g i x )) for some 0 ≤ k ≤ n + m − 2 and 0 ≤ i < r . So to p ro ve the claim we will sp eak of these co efficient s determining other suc h co efficien ts. No w observe that for any k ∈ { 0 , . . . , r − 1 } , th e coefficients C x k ( ˆ f M ( x, x )), C x k ( ˆ f M ( x, g x )), . . . , C x k ( ˆ f M ( x, g r − 1 x )) are linear com binations of the k + 1 ≤ r elemen ts in { M i,j } i + j = k . Ju s t as in the analysis of D ′ r,n,m in Th eorem 5.8 , the fi rst k + 1 of these lin ear com bin ations are ro w s of a V andermond e matrix ov er d istinct num b ers, and thus these linear com b inations sp an all 22 v ectors. Thus, it follo ws that the coefficients { C x k ˆ f M ( x, g i x ) } 0 ≤ i 1: W e will first reshap e f in to a biv ariate p olynomial, an d app eal to the d = 1 case. W e will then u n-reshap e this p olynomial into a 2 d − 1 -v ariate p olynomial, and then app eal to ind uction. By induction on Lemma 6.2 (and app ealing to Lemma 6.1 to s ee that Lemma 6.2 applies to an y t wo v ariables, not ju st the fi rst) we see that f ( h x j i 2 d − 1 j =0 ) = 0 iff f ( h x ( n 2 b ) j 0 , x ( n 2 b ) j 1 i 2 d − 1 − 1 j =0 ) = 0 (1) (where so far w e only need that b ≥ 1). W e split the rest of the p ro of in to t wo claims. The first claim sho ws how we can, using the biv ariate case, test ident it y of the righ t-hand -side of Eq u ation ( 1 ) b y testing identit y of a set of r p olynomials, eac h of 2 d − 1 v ariables. The s econd claim sho ws how testing iden tit y of these new p olynomials can b e reduced to testing id en tity of univ ariate p ol ynomials, where we use the ind uction h y p othesis. Claim 6.7. f ( h x ( n 2 b ) j 0 , x ( n 2 b ) j 1 i 2 d − 1 − 1 j =0 ) = 0 iff { f ( h x j , g i 1 ( n 2 b ) j x j i 2 d − 1 − 1 j =0 ) } 0 ≤ i 1 d eg x 0 f ′ , deg x 1 f ′ . Using that 2 b ≥ 2 d − 1 ≥ 2, we can undu e the v ariable su b stitutions x j 7→ x ( n 2 b ) j 0 . That is, applying L emm a 6.2 in rev erse, we s ee that the ab o ve set of p olynomials is zero iff { f ( h x j , g i 1 ( n 2 b ) j x j i 2 d − 1 − 1 j =0 ) } 0 ≤ i 1 i 0 . Th u s, w e no w expand out the i 0 -th index of the ab ov e su mmation ( X i c i v ( i ) ) j i 0 = X i j . I t is then th at ( LL ′ ) i,j = X k ∈ J n K L i,k L ′ k ,j = X i ≥ k ≥ j L i,k L ′ k ,j = L i,i L ′ i,j + X i>k >j L i,k L ′ k ,j + L i,j L ′ j,j = 1 · L ′ i,j + X i>k >j L i,k L ′ k ,j + L i,j · 1 Observe that as i > j and j / ∈ J , L ′ i,j = L ′ k ,j = L i,j = 0 (for an y k > j ). Th u s, the ab ov e sum is zero. Hence, the desired en tries ( i, j ) with i > j and j / ∈ J are zero, pro ving the claim. W e now u se these lemmas to analyze Algorithm 2 , wh ich giv es a wa y to transf orm a matrix in ( < k )-upp er-ec helon into one whic h is ( ≤ k )-upp er-ec h elon, and d o es so efficiently . Algorithm 2 T ransform a ( < k )-upp er-ec helon matrix in to ( ≤ k )- upp er-e c helon form 1: pro cedure Mak eUpperEch elon ( M , n , m , k ) 2: L ← I n 3: for all ( i, j ) ∈ LNE( M ( 0: Using that the inv arian ts held at k − 1, w e now establish th em at k . As P ( n / 2 w e h a ve that |D ′ 2 r,n,m | ≥ nm (one cannot use th e form u la “ |D ′ 2 r,n,m | = 2( n + m − 2 r ) r ” here, b u t the b ound |D ′ 2 r,n,m | ≤ |D 2 r,n,m | = 2( n + m − 1) r is still v alid). Thus, for r > n/ 2 there is no gain from u s ing D ′ 2 r,n,m o ve r the obvio u s nm low-rank-reco v ery set that qu eries eac h entry in the matrix. R e mark 7.24 . On e can also use Algorithm 3 to reprov e Th eorem 5.8 , that is, to reprov e th at D r,n,m is a h itting set (note that we use r and n ot 2 r here). T o do so, note that Lemma 7.15 sh o ws that for a rank ≤ r matrix M , if M ( 2 case. Theorem 8.1. L et n, r ≥ 1 and d ≥ 2 . Then B d,n, 2 r , as define d in Construction 6.10 , has 1. |B d,n, 2 r | ≤ O ( dn (2 r ) O (lg d ) ) 2. B d,n, 2 r is an r - low-r ank-r e c overy set, a nd r e c overy c an b e p erforme d i n time p oly ((2 dn ) d , (2 r ) O (lg d ) ) 46 Pr o of. ( 1 ): This is b y construction. ( 2 ): The hitting set allo ws us to interpolate the p olynomials stated in the h yp othesis of The- orem 6.6 . Once we ha ve the co efficien ts of th is p olynomial, w e can un do the reductions u sed in the pro of of Theorem 6.6 . That is, that pro of u ses Lemmas 6.1 and 6.2 to resh ap e p olynomials b y merging their v ariables. This is clearly efficien tly rev ersib le. More crucially , the p ro of uses th e biv ariate v ariable reduction of Theorem 5.1 for rank ≤ r matrices, but when w e tak e 2 r distinct p o w ers of g . How ev er, Corollary 7.22 shows that one can r eco v er ˆ f M ( x, y ) from the p olynomials { ˆ f M ( x, g i x ) } i ∈ J 2 r K in p oly (deg x ( ˆ f M ) , deg y ( ˆ f M ) , r ) steps. As the degrees inv olv ed in Theorem 6.6 are only up to (2 dn ) d , this is within the stated time b oun ds. Thus, w e can also reverse the biv ari- ate v ariable redu ction steps used in T heorem 6.6 . Com b ining these steps sho w s that we can fully reco v er the entire p olynomial ˆ f T ( x 1 , . . . , x d ), which gives the tensor T . W e n ext observe that, just as w ith Corollary 7.26 , we can p erform this lo w-rank-r eco v ery ov er small fields, w hen incurring a loss. Corollary 8.2. L et n, r ≥ 1 and d ≥ 2 . Over any field F , ther e is an p oly ((2 nd ) d , r O (lg d ) ) -explicit r -low-r ank-r e c overy set for J n K d tensors, which has si ze O ( dn (2 r ) O (lg d ) · ( d lg 2 dn ) d ) and is such that e ach r e c overy tensor i s r ank 1. F urther, ther e is an p oly ((2 nd ) d , r O (lg d ) ) -explicit r -low -r ank-r e c overy set for J n K d tensors, which has size O ( dn (2 r ) O (lg d ) · d lg 2 dn ) . F urther, r e c overy fr om either of these low-r ank-r e c overy sets c an b e p erforme d in p o ly ((2 nd ) d , r O (lg d ) ) time. Pr o of. Like Corollary 7.26 , we apply Pr op ositions 6.16 and 6.12 to a lo w-rank-reco v ery set, wh ere here we use the ab ov e set from T heorem 8.1 . As Prop ositions 6.16 and 6.12 , as well as Theorem 8.1 , are efficien tly imp lemen table, so are th e resulting lo w-rank-reco ve ry sets. W e no w apply these results to create error correcting co d es o ve r the r ank-metric, which we no w define. W e will r estrict our atten tion to linear co d es in this work. Definition 8.3. A [ J n K d , k , r ] F r ank-metric c o de C is a k - dimensional subsp ac e of F J n K d (the sp ac e of J n K d tensors) such that for al l T 1 6 = T 2 ∈ C , rank( T 1 − T 2 ) ≥ r . Denote r as the distanc e of the c o de. An algorithm Dec c o rr e cts e err ors against C if for any T ∈ C and E ∈ F J n K d with rank( E ) ≤ e it is such that Dec ( T + E ) = T . Th us this is the natural defin ition for error-correcting co d es when we use th e rank-metric (notice that rank-distance is in fact a metric) as the notion of distance. As w e are interested in linear co des T 1 − T 2 ∈ C also, so an equiv alen t defin ition to the abov e would sa y that r ≤ r ank( T ) for all 0 6 = T ∈ C . Just as with the Hamming-metric, if w e h a ve a d istance 2 r + 1 co d e C th en it is information theoretically p ossib le to deco de up to r errors. The con ve rse is sho wn b elo w . Lemma 8.4. L et C b e a [ J n K d , k , r ′ ] F r ank- metric c o de that c an c orr e ct up to r err ors. Then r ′ ≥ 2 r + 1 . Pr o of. S upp ose n ot for con tradiction. Then there are t wo tensors T 1 6 = T 2 ∈ C suc h that rank( T 2 − T 1 ) ≤ 2 r . But then T 2 − T 1 = S 1 + · · · + S 2 r , where these S i are all rank-1 (or rank-0) tensors. Then it follo ws that T 1 + S 1 + · · · + S r is r -close to b oth T 1 and T 2 , which is imp ossible as the correctness of th e d eco ding pro cedure indicates that there should b e a unique tens or that T 1 + S 1 + · · · + S r is r -close to. Corollary 8.5. L et F b e a field, m ≥ n ≥ r ≥ 1 . Then ther e ar e p o ly ( m ) -explicit r ank-metric c o des with p oly ( m ) -time de c o ding for u p to r err ors, with p ar ameters: 47 1. [ J n K × J m K , nm − 2( n + m − 2 r ) r, 2 r + 1] F , if | F | > m , and the p arity che ck s on this c o de c an b e either al l r ank-1 matric es, or al l O ( n ) -sp arse matric es. 2. [ J n K × J m K , nm − 2( n + m − 2 r ) r · O (l g m ) , 2 r + 1] F , any F , and the p arity che cks on this c o de ar e al l O ( n ) -sp arse matric es. 3. [ J n K × J m K , nm − 2( n + m − 2 r ) r · O (lg 2 m ) , 2 r + 1] F , any F , and the p arity che cks on this c o de ar e al l r ank-1 matric es. Pr o of. W e first generically sho w how to define an [ nm, nm − |H| , 2 r + 1] F rank-metric co de C f rom an r -lo w -rank-reco ve ry set H and how to us e the lo w-rank-reco v ery algorithm f or H to deco de C up to r errors. Th e corolla ry is then immediate by u sing the results of Corollaries 7.25 , 7.22 , 7.26 , and inv oking the efficiency of their lo w-rank-reco very . Define C to b e the matrices in the n ullsp ace of H . That is, C = { M : h M , C i = 0 } . It is clear that C is a subs p ace (and assuming that the matrices in H are linearly indep enden t, whic h is tru e for the low-rank-reco v ery sets D ′ 2 r,n,m and B ′ 2 r,n,m ) and has dimension nm − |H| . No w consider some T ∈ C and matrix E with rank( E ) ≤ r . Abusing notation, consider T and E as nm -long v ectors, and H as a |H | × nm matrix. It follo ws that H ( T + E ) = H E as T ∈ C . As H is an r -lo w-ran k -r eco v ery set, it f ollo ws that we can reco ver E from H E , and thus can reco ver T , p erformin g successful d eco d ing of up to r errors. By Lemm a 8.4 w e s ee that the min im u m d istance of this co d e is ≥ 2 r + 1. W e n o w sep arately state the result for tensors, whic h is prov ed exactly as th e ab o v e corollary , but usin g the relev ant lo w -rank-reco ve ry r esu lts for tensors. Corollary 8.6. L et F b e a field, n, r ≥ 1 and d ≥ 2 . Then ther e ar e p oly ((2 nd ) d , (2 r ) O (lg d ) ) -explicit r ank- metric c o des with p oly ((2 nd ) d , (2 r ) O (lg d ) ) -time de c o ding for up to r err ors, with p ar ameters: 1. [ J n K d , n d − dn (2 r ) ⌈ lg d ⌉ , 2 r + 1] F , if | F | > (2 nd ) d , and the p arity c he cks on this c o de ar e al l r ank- 1 tensors, 2. [ J n K d , n d − dnr ⌈ lg d ⌉ · O ( d lg (2 dn )) , 2 r + 1] F , any F , 3. [ J n K d , n d − dnr ⌈ lg d ⌉ · O (( d lg (2 dn )) d ) , 2 r + 1] F , any F , and the p arity che cks on this c o de ar e al l r ank-1 tensors. 9 Discussion W e briefly discu s s some directions for further researc h. Reducing Noisy Lo w-Rank Reco v ery to Noisy Sparse Reco very W e sh ow ed in Th eo- rem 7.19 that low-rank-reco v ery of matrices can b e done using an y sp ars e-reco v ery oracle. Th is reduction w as for non-adaptive measuremen ts, and was done in the presence of no noise. As m uc h of the compressed sensing communit y is inte rested in the noisy case (so M is only close to ran k ≤ r ) the main op en question of th is w ork is wh ether the reduction extends to the noisy case. 48 Smaller Hit ting Sets While the observ ations of Roth [ Rot91 ] sho w th at our h itting set for matrices is op timal o v er algebraica lly closed fields, our results (Corollary 6.18 ) o ver tensors w ith d > 2 are muc h larger than the existenti al b ound s of Lemm a 3.13 . Can th ese hitting sets b e impro v ed to size O ( p oly ( d ) nr k ) for k = O (1)? As ment ioned in the preliminaries (Lemma 3.14 ), an y suc h hitting set with k < 2 wo u ld yield impr o ve d tensor-rank lo wer b o unds (and th us circuit lo we r b o unds) for o dd d such as d = 3. Ho wev er, as the b est tensor-rank lo wer b ounds for d = 3 are Θ( n ) and our hitting set (o v er infinite fields) yields this b ound (with a smaller constan t), ev en improving our hitting s et for d = 3 by constant factors could yield interesting n ew resu lts. Sp ecifically , for d = 3 can one construct (sa y o v er infinite fields) a hitting set of size ≤ nr 2 / 10 f or J n K 3 tensors of rank ≤ r ? Better V ariable Reduction Th eorem 5.1 sho ws th at a biv ariate p olynomial with b ounded individual degrees can b e identi t y tested b y iden tit y testing a collection of univ ariate p olynomials, where th e size of this colle ction dep en ds on the rank of biv ariate p olynomial. Th is naturally led to our h itting sets for matrices. W e generalized th is to d -v ariate p o lynomials in Theorem 6.6 , b ut th e collect ion of un iv ariate p olynomials has a size with a muc h w orse dep endence on the tensor-rank of the d -v ariate p o lynomial and is muc h less explicit. Can the s ize of the collectio n b e r educed, or can th e explicitness of this set b e only p olynomially larger than its size? W e note that according to Lemma 3.14 a more exp licit hitting set will yield lo w er b ound s on tensor rank , how ev er f or tensors of high d egrees suc h lo w er b o unds are known [ NW96 ]. Large Field Sim ulation The results of Section 6.3 sho w that hitting sets (and LRR sets) that in volv e tensors o ver an extension field imply h itting sets (and lo w-rank r eco v er y sets) o v er the base field. While Pr op ositio n 6.16 sh ows that w e can preserve the r ank-1 p rop erty of these tensors while doing so, it in tro duces an exp( d ) factor in the size of the hitting set. Can this b e impr ov ed? Ac kno wledgemen ts W e would lik e to thank Olgica Milenk o vic f or p oin ting us to the low-rank r eco v ery problem, and Madh u Sud an for some helpfu l commen ts regarding deco din g d ual Reed-Solomon co des. P art of this work w as done wh ile the fir st author was visiting Stanford Un iv ersity , as well as when the second author was visiting the Bern ou lli cen ter at EPFL. 49 References [AFT11] B. Alexeev, M. F orb es, and J. Tsimerman. T ensor r ank: some lo wer and upp er b ounds. In IEEE Confer enc e on Computational Complexity , pages 283–2 91. I EEE Computer So ciet y , F eb 2011. 14 [Agr05] M. Agra wal. Pro ving lo w er b ounds via pseud o-random generators. In Pr o c e e dings of the 25th FSTTCS , vo lume 3821 of LNCS , pages 92–105, 2005. 2 , 14 [A V08] M. Agraw al and V. Vina y . Arithmetic circuits: A chasm at depth four. In Pr o c e e dings of the 49th A nnual FOCS , p ages 67–75 , 2008. 2 [BBB + 00] A. Beimel, F. Bergadano, N. H. Bshouty , E. Kushilevitz, and S. V arr icc hio. Learning functions repr esen ted as m ultiplicit y automata. J. ACM , 47(3) :506– 530, 2000. 4 [BBV96] F. Bergadano, N. H. Bshouty , and S. V arricc hio. Learning m ultiv ariate p ol ynomials from sub s titution and equiv alence qu eries. E CCC , 3(8), 1996. 4 [BD80] M. R. Brown and D. P . Dobkin. An imp r o ved low er b oun d on p o lynomial multiplica tion. IEEE T r ans. Computers , 29(5):33 7–340, 1980. 14 [CP09] E. J. Candes and Y. Plan. Matrix Comp letion With Noise. ArXiv e-prints , March 2009. 3 [CP11] E. J. Cand ´ es and Y. Plan. Tigh t oracle inequalities f or lo w-ran k matrix reco v ery from a minimal num b er of noisy rand om measuremen ts. IEEE T r ans. Inform. The- ory , 57(4):234 2–235 9, 2011. 3 [CR T 05] E. Candes, J. Rom b erg, and T. T ao. Stable Signal Reco v ery from Incomplete and Inaccurate Measurement s. ArXiv M athematics e-prints , March 2005. 3 [CSw] h ttp://dsp .rice.edu/cs. 2 [CT09] E . J. Candes and T. T ao. The P ow er of Con vex Relaxation: Near-Optimal Matrix Completion. ArXiv e- prints , Marc h 2009. 3 [Del78] Ph. Delsarte. Bilinea r form s o ver a finite field, with applications to co d in g theory . J. Combin. The ory Ser. A , 25(3):2 26–24 1, 1978. 4 , 6 , 7 , 32 [dP95] G. C. F. M. Ric h e de Pr on y . Es sai ´ exp er im ental et analytique: sur les lois de la dilata- bilit ´ e de fluid es ´ elasti que et sur celles de la f orce expansiv e de la v ap e ur de l’alk o ol, ` a diff ´ eren tes temp´ eratures. Journal de l’´ ec ole Polyte chnique , 1:24–76, 1795. 8 , 34 [DS08] Z. Dvir and A. Shpilk a. T o w ard s d imension expanders ov er fin ite fields. In Pr o c e e dings of the 23r d Annual IEEE Confer enc e on Computational Complexity (CCC) , pages 304–310 , 2008. 3 [ENP11] Y. C. Eld ar, D. Needell, and Y. Plan. Unicit y conditions for low-rank matrix reco v ery . arXiv:1103.54 79 , 2011. 3 [Gab85a] E. M. Gabidulin. Optimal arra y error-correcting co des. Pr obl. Per e dach. Inform. , 21(2): 102–1 06, 1985. 4 , 6 , 7 , 32 50 [Gab85b] E. M. Gabidulin. Th eory of cod es with maxim um rank distance. P r obl. Inform. T r ansm. , 21(1): 1–12, 1985. 4 , 6 , 7 , 32 [GK72] E. M. Gabid ulin and V. I. Korzhik. Co des correcting lattice -pattern er r ors. Zzvestiya VUZ. R adio elektr onika , 1972 . 4 , 6 , 7 , 32 [GLF + 10] Da v id Gross, Yi-Kai Liu, Steven T. Flammia, Stephen Bec k er, and Jens Eisert. Quantum state tomograph y via compressed sensing. Phys. R ev. L ett. , 105:15 0401, Oct 2010. 3 [GR08] A. Gabizon and R. Raz. Deterministic extractors for affine sour ces o ver large fields. Combinatoric a , 28(4):415 –440, 2008 . 5 , 8 , 14 , 15 , 17 [Gro09] D. Gross. Reco v er in g lo w-rank matrices from few co efficient s in an y basis. ArXiv e- prints , Octob er 2009. 3 [H ˚ as90] J. H ˚ astad. T ensor rank is n p-complete. J. Algorithms , 11(4) :644–6 54, 1990. 7 [HS80] J. Hein tz and C. P . Sc hnorr. T esting p ol ynomials w hic h are easy to compu te (extended abstract). In P r o c e e dings of the 12th annual STOC , pages 262–272 , 1980. 2 , 14 [IPW11] P . Ind y k , E. Price, and D. P . W o o dru ff. On the Po w er of Ad aptivit y in Sparse Reco very. F oundations of Computer Scienc e , Octob er 2011. 3 , 32 [KI04] V. Kaban ets and R. I m pagliazzo . Derand omizing p olynomial ident it y tests means prov- ing circuit lo we r b ounds. Computational Complexity , 13(1-2):1– 46, 2004. 2 [K OH11] A. Kha jehnejad, S. Oymak, and B. Hassibi. Su bspace expand ers an d matrix rank minimization. arXiv:1102.3947 v1 , 2011. 3 [KS06] A. Kliv ans and A. S hpilk a. Learning r estricted mo dels of arithmetic circuits. The ory of c omputing , 2(10):185 –206, 2006. 4 [KS08] Z. S. Karnin and A. Shpilk a. Dete rministic blac k b o x p o lynomial id entit y testing of depth-3 arithmetic circuits with b ounded top fan-in. In Pr o c e e dings of the 23r d Annual CCC , pages 280–291, 2008. 8 [lrr] h ttp://p erce ption.csl.uiuc.edu/matrix-rank/. 3 [LZ08] A. Lub otzky and Y. Zelmanov. Dimension expanders. J. Algebr a , 319(2):7 30–73 8, 2008. 3 [Mes95] R. Meshulam. Spaces of Hank el matrices o ver fin ite fi elds. Line ar Algebr a Appl. , 218:73– 76, 1995. 4 [NW96] N. Nisan and A. Wigderson. Lo we r b ound on arithm etic circu its via p artial deriv ativ es. Computation al Complexity , 6:217–234 , 1996. 2 , 49 [PCM88] G.M. Pitstic k, J.R. Cruz, and R.J. Mu lholland. A n o vel in terpr etation of Pron y’s metho d. Pr o c e e dings of the IEEE , 76(8):1 052 –1053, aug 1988. 34 [Raz10] R. Raz. T ensor-rank and lo w er b ounds for arithmetic form u las. In Pr o c e e dings of the 42nd Annual STO C , pages 659–666 , 2010. 14 [RFP10] B. Rech t, M. F azel, and P . Parrilo. Guaran teed min im u m-rank solutions of linear matrix equations via nuclear norm minim ization. SIAM R evi ew , 52(3):471 –501, 2010. 3 , 6 , 32 51 [Rot91] R. M. Roth. Maximum-rank array co des and th eir application to crisscross error cor- rection. IEEE T r ansactions on Information The ory , 37(2):32 8–336 , 1991. 4 , 6 , 7 , 13 , 32 , 49 [Rot96] R. M. Roth. T ensor co d es for th e rank metric. IEEE T r ansactions on Information The ory , 42(6):214 6–215 7 , 1996. 4 , 7 , 32 [RS05] R. Raz and A. Shpilk a. Deterministic p olynomial iden tit y testing in non comm utativ e mo dels. Computational Complexity , 14(1):1– 19, 2005. 1 , 2 , 12 [Sc h80] J. T . Sc hw artz. F ast p robabilistic algorithms for v erification of p olynomial iden tities. J. ACM , 27(4):701 –717, 1980. 13 [SS11] N. Saxena and C. Sesh ad h ri. Blac kb o x identit y testing for b ound ed top fanin depth- 3 circuits: the field do esn’t matter. In Pr o c e e dings of the 43r d A nnual STOC , p ages 431–4 40, 2011. 2 , 5 , 8 [Str73] V. Strassen. V erm eidung v on divisionen. J. of R eine Angew. Math. , 264:182–20 2, 1973. 14 [SY10] A. Sh pilk a and A. Y ehuda y off. Arithmetic circuits: A survey of recen t resu lts an d op en questions. F oundations and T r ends in The or etic al Computer Scienc e , 5(3-4):207 –388, 2010. 2 , 4 , 11 [TBD11] V. Y. F. T an, L. Balzano, and S. C. Drap er. Rank minimization o v er finite fields: F un damen tal limits and co ding-theoretic in terp retations. arXiv:1104.43 02v2 , 2011. 6 [Zip79] R . Zipp el. Probabilistic algo rithms for sparse p olynomials. In Symb olic and algebr aic c omputatio n , pages 216–226. 1979. 13 52 A Cauc h y-Binet F orm ula F or completeness w e give the pr o of of the Cauc hy-Binet formula here. Lemma A.1 (Cauc hy-Binet F orm u la) . L et m ≥ n ≥ 1 . L et A ∈ F n × m , B ∈ F m × n . F or S ⊆ J m K , let A S b e the n × | S | matrix forme d fr om A by taking the c olumns with indic es in S . L et B S b e define d analo gously, bu t with r ows. Then det( AB ) = X S ∈ ( J m K n ) det( A S ) det( B S ) Pr o of. Let C b e an m × m diagonal matrix with the v ariables x 1 , . . . , x m on the diagonal. Define the p olynomial f ( x 1 , . . . , x m ) def = det( AC B ), so that f (1 , . . . , 1) = det( AB ). Every en tr y of AC B is a h omogeneous linear fu nction in x 1 , . . . , x m , w hic h imp lies (as the determinan t is homogeneous of degree n ) that f is homogeneous of degree n , or zero. Let S ∈  J m K n  and consider all monomials only cont aining v ariables in { x i | i ∈ S } . Note th at also consider monomials w ith in d ividual degrees ab o ve 1. Eac h monomial of d egree n (and thus eac h monomial with non-zero co efficien t in f ) m u st b e asso ciated with some su ch S . Define ρ S to b e the v ector of v ariables when the subs titution x i 7→ 0 is p erformed for i / ∈ S . It follo ws then that f ( ρ S ) = d et( A S C S B S ) = d et( A S ) det( B S ) · Q i ∈ S x i , wh er e the last equalit y follo ws as A S , B S and C S are all n × n matrices. By the ab o v e r easoning, this implies that the only monomials with n on-zero coefficient s in f are monomials of the form Q i ∈ S x i and suc h monomials ha ve co efficient det( A S ) det( B S ). Th us f = P S ∈ ( J m K n ) det( A S ) det( B S ) Q i ∈ S x i , and s o d et( AB ) = f (1 , . . . , 1) = P S ∈ ( J m K n ) det( A S ) det( B S ), yielding th e claim. 53