Arithmetic Circuits and the Hadamard Product of Polynomials

Arithmetic Cir cuits and the Hadamard Produ ct of Polyn omials V . Arvind , Pushkar S. Joglekar , Srikanth Sriniv asan Institute of Mathematical Sciences C.I.T Campus,Chennai 600 113, India { arvind,pushkar, srikanth } @imsc.r es.in Abstract. Moti v ated by the Hadamard product of matrices we define the Hadamard product of multi v ariate polyno mials and study its arithmetic circuit and bran ching program co mplexity . W e also gi v e application s and con nections to polynomial identity testing. Our main results are the following. • W e sho w that non commutati ve polynom ial identity testing for algebraic branching p rograms ov er rationals is complete f or the logspace counting class C = L , and ove r fields o f characteristic p the pro blem is in Mod p L / poly. • W e sho w an exponential lower bound for expressing the Raz-Y ehudayof f polynomial as the Hadamard product of two monoto ne multilinear polyno- mials. In contrast the Permanent can be expres sed as the Hadamard product of two monoton e multi linear formulas of quadratic size. 1 Intr oduction In this p aper we defin e the Hadamar d pr oduct of tw o polyn omials f and g in F h X i and study its expressi ve power and applications to the complexity of arithm etic circuits and algebraic branchin g progr ams. W e also apply it to gi ve a fairly tight characterization of polyno mial identity test ing for algebraic branching progr ams over the field of rationals. Suppose X = { x 1 , x 2 , · · · , x n } is a s et of n non commutin g variables. T he free monoid X ∗ consists of all words over these variables. For a field F let F h x 1 , x 2 , · · · , x n i den ote the free noncommutative polyn omial ring over F ge nerated by t he variables in X . Thus, the polyno mials in this ring are F -linear com binations of words over X . For a given polyno mial f ∈ F h X i , let mon ( f ) = { m ∈ X ∗ | m is a nonze ro monomial in f } . If X = { x 1 , x 2 , · · · , x n } is a set of n co mmuting variables then F [ X ] de notes th e commutative polynom ial ring with coefficients from F . Motiv ated by the well-known Had amard prod uct o f matrices (see e. g. [Bh97]) we define the Hadamard produ ct of polyn omials. Definition 1. Let f , g ∈ F h X i where X = { x 1 , x 2 , · · · , x n } . The Hadamard p roduct of f and g , denoted f ◦ g , is the polyn omial f ◦ g = P m a m b m m , wh er e f = P m a m m and g = P m b m m , wher e the sums index over monomials m . Complexity t heory preliminaries W e r ecall some definition s of log space countin g classes from [A O9 6]. Let Ł deno te the class of language s accepted by deterministic logspace machines. GapL is th e class of function s f : Σ ∗ → Z , fo r wh ich there is a logspa ce bo unded NDTM M such tha t for each input x ∈ Σ ∗ , we have f ( x ) = acc M ( x ) − r ej M ( x ) , where acc M ( x ) and rej M ( x ) are the nu mber of acceptin g and rejectin g paths of M on input x , respectively . A lang uage L is in C = L if there exists a fu nction f ∈ GapL such that x ∈ L if and only if f ( x ) = 0 . For a p rime p , a language L is in the complexity class Mod p L if there exists a function f ∈ Gap L such that x ∈ L if and on ly if f ( x ) = 0( mod p ) . It is shown in [A O96] that checkin g if an integer m atrix is singu lar is com plete fo r C = L with respect to lo gspace many-on e reductio ns. The same prob lem is known to be complete for Mod p L over a field of cha racteristic p . It is useful to recall that both C = L and Mod p L are contained in TC 1 (which, in turn, is contained in NC 2 ). An Algebraic Branching P r o gram (ABP) [N9 1,RS05] over a field F and variables x 1 , x 2 , · · · , x n is a layered directed acyclic graph with one sour ce vertex of in degree zero and one s ink vertex o f outd egree zero. Let t he l ayers be number ed 0 , 1 , · · · , d . The source and sink are the uniqu e layer 0 an d layer d vertices, respecti vely . Edges only go from layer i to i + 1 for each i . Each edge in the ABP is labeled with a linear form over F in the input variables. E ach so urce to sink path in the ABP co mputes the pr oduct o f the linear forms labelling the edg es on the path, and the sum of these polynomials over all source to sink paths is the polyno mial c omputed by the ABP . Th e size of the ABP is the number of vertices. Main results. W e s how that th e n oncommuta tive b ranching p rogram com plexity of the Hadamard p roduct f ◦ g is upper boun ded by the product of the branching progr am sizes for f an d g .This up per boun d is natur al because we know from Nisan’ s seminal work [N91] that the alg ebraic branch ing program (ABP) co mplexity B ( f ) is well ch aracter- ized by the ranks of its “commu nication” matrices M k ( f ) , and the rank of Had amard produ ct A ◦ B o f tw o matrices A an d B is upper boun ded by the prod uct of their rank s. Our proof is constructiv e: we give a deterministic logspace algor ithm for compu ting an ABP for f ◦ g . W e the n apply this result to polyn omial iden tity testing. It is shown by Raz and Sh- pilka [RS05] tha t poly nomial identity testing of nonco mmutative ABPs can be done in deterministic polyn omial time. A simple divide an d conq uer alg orithm can b e easily designed to show tha t the prob lem is in deterministic NC 3 . What th en is the prec ise complexity of polyno mial identity testing for no ncommu tativ e AB Ps? For nonco mmu- tati ve ABPs over rationals we g i ve a tight characterizatio n by sho wing that t he problem is C = L -complete . W e p rove this result using the re sult on Had amard p roduct of ABPs explained above. 2 For non commuta ti ve ABPs over a finite field of cha racteristic p , we show that iden tity testing is in th e n onunif orm class Mod p L / poly ( more precisely , in randomized Mod p L). Furthermo re, the problem turns out to be hard (w . r . t. logspace many-one reduction s) for both NL and Mod p L. Hen ce, it is not likely to be easy to improve this uppe r b ound uncon ditionally to Mod p L (it would im ply that NL is contained in Mo d p L). Howev er, under a hardness assumption we c an app ly standard arguments [ARZ9 9,KvM02] to derand omize this algorith m and put the problem in Mod p L. In Section 4 we con sider the Hadamard p roduc t for commu tativ e poly nomials. W e sh ow an expon ential lower bound for expressing the Raz-Y ehud ayoff polynomial [R Y0 8] a s the Hadamard prod uct of two mono tone multilinear po lynomials. In contrast th e Perma- nent can be expressed as the Had amard prod uct of two mon otone multilinear fo rmulas of quadra tic s ize. 2 The Hadamard Prod uct Let f , g ∈ F h X i wh ere X = { x 1 , x 2 , · · · , x n } . Clear ly , mon ( f ◦ g ) = mon ( f ) ∩ mon ( g ) . T hus, the Hadamard pro duct can be seen as an alg ebraic version of th e inter- section of formal languag es. Our definition of the Hadamard product of polynom ials is actually motiv ated by the well-kn own Hadamard p roduct A ◦ B of two m × n matr i- ces A and B . W e reca ll the fo llowing well-kn own bound for the rank o f the Hada mard produ ct. Proposition 1. Let A and B b e m × n matrices over a fi eld F . The n rank ( A ◦ B ) ≤ rank ( A ) rank ( B ) . It is k nown f rom Nisan’ s work [N9 1] that the ABP complexity B ( f ) of a po lynomial f ∈ F h X i is clo sely co nnected with th e ran ks of th e com munication matrices M k ( f ) , where M k ( f ) has its r ows indexed by degree k m onomials and columns by degree d − k mon omials a nd the ( m, m ′ ) th entry of M k ( f ) is the c oefficient of mm ′ in f . Nisan showed th at B ( f ) = P k rank ( M k ( f )) . Ind eed, Nisan’ s result and th e above propo sition easily imply the following bound on the ABP complexity of f ◦ g . Lemma 1. F or f , g ∈ F h X i we h ave B ( f ◦ g ) ≤ B ( f ) B ( g ) . Pr oof . By Nisan’ s result B ( f ◦ g ) = P k rank ( M k ( f ◦ g )) . The a bove p roposition implies X k rank ( M k ( f ◦ g )) ≤ X k rank ( M k ( f )) rank ( M k ( g )) ≤ ( X k rank ( M k ( f ))( X k rank ( M k ( g ))) , and the claim follows. 3 W e now s h ow an algorithmic v ersion of this upper bound. Theorem 1. Let P and Q be two give n ABP’s c omputing poly nomials f and g in F h x 1 , x 2 , . . . , x n i , r e spectively . Then there is a deterministic po lynomial-time algo- rithm tha t will o utput an ABP R for the polynomia l f ◦ g such tha t the size of R is a constant multiple of the pr odu ct of the sizes of P an d Q . (Ind eed, R can b e computed in deterministic logspace.) Pr oof . Let f i and g i denote the i th homog eneous parts of f and g respectively . Then f = P d i =0 f i and g = P d i =0 g i . Since the Had amard prod uct is d istributi ve over addi- tion and f i ◦ g j = 0 for i 6 = j we have f ◦ g = P d i =0 f i ◦ g i . Thus, we can assume that both P and Q are homogene ous ABP’ s of degree d . Otherwise, we can easily construct an ABP to compu te f i ◦ g i separately for each i and put them together . Note that we can easily compute ABPs for f i and g i in logspace given as inp ut the ABPs for f and g . By allowing parallel edges between no des of P and Q we c an assume that th e labels associated with each edg e in an ABP is either 0 or αx i for some variable x i and scalar α ∈ F . Let s 1 and s 2 bound the numb er of nodes in each layer o f P an d Q respectively . Denote the j th node in layer i by h i, j i f or ABPs P and Q . Now we describe the construction of the ABP R fo r comp uting th e polynomial f ◦ g . Each layer i , 1 ≤ i ≤ d of R will have s 1 · s 2 nodes, with node labeled h i, a, b i corr espondin g to the node h i, a i of P an d the node h i, b i of Q . W e can assume there is an edge from every node in layer i to e very node in layer i + 1 for both ABPs. For , if there is no such ed ge we can always include it with label 0 . In the ne w ABP R we pu t an edge fr om h i, a, b i to h i + 1 , c, e i w ith label αβ x t if and only if there is an edge fr om node h i, a i to h i + 1 , c i with label αx t in P and an edge from h i, b i to h i + 1 , e i with label β x t in ABP Q . Let h 0 , a, b i and h d, c, e i deno te the so urce an d the sink n odes of ABP R , wh ere h 0 , a i , h 0 , b i ar e the source n odes of P and Q , and h d, c i , h d, e i ar e the sink nodes of P and Q resp ectiv ely . It is easy to see that ABP R can be computed in deterministic logspace. Let h h i,a,b i denote the polyno mial computed at n ode h i, a, b i of ABP R . Similarly , let f h i,a i and g h i,b i denote the poly nomials co mputed at nod e h i, a i o f P and node h i , b i of Q . W e can easily check that h h i,a,b i = f h i,a i ◦ g h i,b i by an induction argument o n the number o f layers in th e ABPs. It follows from this in ductive argumen t that the ABP R com putes the polyno mial f ◦ g at its sink no de. The bound on the size of R also follows easily . Applying the above theor em we can gi ve a tight complexity theo retic upp er bou nd for identity testing of nonco mmutative ABPs over rationals. Theorem 2. The pr ob lem of polyno mial identity testing for noncommutative algebraic branching pr ograms over Q is in NC 2 . Mor e p r ecisely , it comp lete for the logsp ace counting class C = L un der lo gspace r educ tions. 4 Pr oof . Let P be the given ABP c omputing f ∈ Q h X i . W e apply the constru ction of Theorem 1 to compute a polynom ial sized ABP R for the Hadamard product f ◦ f (i.e. of f with itself). Notice that f ◦ f is nonzero iff f is nonzero. Now , we crucially use the fact that f ◦ f is a polynomial whose nonzer o coefficients are all positive . Hence, f ◦ f is n onzero iff it evaluates to nonze ro on the all 1 ’ s inp ut. The prob lem thus b oils down to checking if R evaluates to non zero on the all 1 ’ s input. By Th eorem 1, the ABP R for polyno mial f ◦ f is comp utable in d eterministic log space, giv en as input an ABP for f . Fur thermore , ev alua ting the ABP R on the all 1 ’ s in put can be e asily conv erted to iterated in teger matrix multiplicatio n (one matrix for each layer of th e ABP), a nd c hecking if R ev alu ates to n onzero can b e done by c hecking if a specific entry of the produ ct matrix is non zero. I t is well known that chec king if a specific entry of an iterated integer matrix produ ct is zer o is in the logspace coun ting class C = L (e.g. see [A O96,ABO99]). Howe ver , C = L is contained in NC 2 , in fact in TC 1 . W e n ow argue the hardness o f this problem for C = L . T he pro blem of checkin g if an integer matrix A is singular is well k nown to be com plete for C = L un der deterministic logspace reduction s. The standard GapL algorithm for computing det( A ) [T 91] can be conv e rted to an ABP P A which will compute det( A ) . 1 Hence the ABP P A computes the identically zero polynom ial iff A is singular . Putting it all together, it follows that identity testing of nonco mmutative ABPs over ration als is complete for the class C = L . An iterative mat rix product problem Suppose B is a non commutative ABP com- puting a hom ogeneo us polynomia l in F h X i of degree d , where each edge of the ABP is labeled by a homog eneous linear for m in v ariable s from X . Let n ℓ denote the n umber of nod es o f B in lay er ℓ , 0 ≤ ℓ ≤ d . For each x i and lay er ℓ , we associate an n ℓ × n ℓ +1 matrix A i,ℓ where the ( k , j ) th entry o f matr ix A i,ℓ is the coefficient of x i in th e lin ear form associated with the ( v k , u j ) edge in the ABP B . Here v k is the k th node in layer ℓ an d u j the j th node in the layer ℓ + 1 . The following claim is easy to see and relates these matrices to the ABP B . Claim. The co efficient of any d egree d mo nomial x i 1 x i 2 · · · x i d in th e poly nomial com- puted by the ABP B is th e matrix pr oduct A i 1 , 0 A i 2 , 1 · · · A i d ,d − 1 (which is a scalar since A i 1 , 0 is a row and A i d ,d − 1 is a column ). Let i and j be any two nodes in the ABP B . W e den ote by B ( i, j ) the algebraic branch- ing pr ogram o btained fro m th e ABP B b y designa ting n ode i in B as th e source node and no de j as the sink node. Clearly , B ( i, j ) computes a h omoge neous polyn omial of degree b − a if i a ppears in layer a and j in layer b . 1 Notice that the polyn omial co mputed by the ABP P A is a c onstant since P A has only con st ants and no v ari ables. 5 For layers a, b , 0 ≤ a < b ≤ d let t = b − a and P ( a, b ) = { A s 1 ,a A s 2 ,a +1 . . . A s t ,b − 1 | 1 ≤ s j ≤ n, for 1 ≤ j ≤ t } . P ( a, b ) consists of n a × n b matrices. Thu s the dime nsion of th e linear space span ned by P ( a, b ) is bou nded b y n a n b . It follows f rom Claim 2 that the lin ear span of P ( a, b ) is th e zero space iff the polyno mial comp uted by ABP B ( i, j ) is identically zero fo r every 1 ≤ i ≤ n a and 1 ≤ j ≤ n b . Thus, it suffices to comp ute a basis for the space spanned by matr ices in P (0 , d ) to check whether the polyn omial computed by B is identically zero . W e can easily g iv e a deterministic NC 3 algorithm for this pr oblem over any field F : First r ecursively compute bases M 1 and M 2 for the spac e spanned b y matrices in P (0 , d/ 2) and P ( d / 2 + 1 , d ) respectively . From bases M 1 and M 2 we can co mpute in determin istic NC 2 a basis M for space span ned by matrices in P (0 , d ) as follows. W e compute the set S of pa irwise produ cts of matrices in M 1 and M 2 and then we can compu te a maximal linear ly inde- penden t subset of S in NC 2 (see e. g. [ ABO99]). This giv es an easy NC 3 algorithm to compute a basis for the linear span of P (0 , d ) . This proves the following. Proposition 2. The pr o blem of po lynomial id entity testing for non commutative alge- braic branching pr ograms over any field (in particular , finite fi elds F ) is in deterministic NC 3 . Can we gi ve a tight complexity character ization for identity testing of noncomm utativ e ABPs ov er finite fields? W e show that the problem is in nonunifo rm Mod p L and is hard for Mod p L unde r logspac e redu ctions. Fur thermore , the problem is hard for NL. Hence, it appears difficult to improve th e upper bound to uniform Mod p L (as NL is not known to be contain ed in unifor m Mod p L). Theorem 3. The pr ob lem of polyno mial identity testing for noncommutative algebraic branching pr ograms o ve r a finite field F of characteristic p is in Mod p L / poly . Pr oof . Co nsider a n ew ABP B ′ in which we replace the variables x i , 1 ≤ i ≤ n appearin g in the linear for m associate d with an e dge from some n ode in layer l to a node in layer l + 1 of ABP B b y new variable x i,l , for layers l = 0 , 1 , . . . , d − 1 . Let g ∈ F [ X ] d enotes the po lynomial com puted by ABP B ′ in co mmuting variables x i,l , 1 ≤ i ≤ n, 1 ≤ l < d . It is easy to see that th e commu tati ve polyno mial g ∈ F [ X ] is identically zer o iff the nonco mmutative polyn omial f ∈ F h X i computed by ABP B is iden tically zero. Now , we can apply the standard Sch wartz-Zippel lemma to c heck if g is iden tically zero b y substituting random values for the variables x i,l from F (or a suitab le finite extension of F ) . After substitution of field elements, we are left with an iterated matrix prod uct o ver a field of characteristic p which can be do ne in Mod p L. This gi ves us a random ized Mod p L algorithm. B y standard amplification it follows that the problem is in Mod p L / poly. 6 Next we show that identity testing noncomm utativ e ABPs over any field is hard for NL by a reduction from directed graph reachability . Let ( G, s, t ) be a reachability instance. W ithou t loss o f generality , we assume that G is a layere d d irected acyclic grap h. The graph G defines an ABP with sourc e s and sink t as f ollows: la bel each edge e in G with a distinct variable x e and fo r eac h ab sent edge put the label 0 . Th e polyno mial computed by the ABP is nonze ro if and only if there is a directed s - t p ath in G . Theorem 4. The pr ob lem of polyno mial identity testing for noncommutative algebraic branching pr ograms o ve r any field is har d for NL . 3 Hadamard pr o duct of noncommutative cir cuits Analogou s to Theo rem 1 we show th at f ◦ g has small c ircuits if f has a sm all c ircuit and g has a small ABP . Theorem 5. Let f , h ∈ F h x 1 , x 2 , · · · , x n i be given by a de gr ee d cir cuit C and a de g r ee d ABP P respectively , where d = O ( n O (1) ) . Then we can c ompute in polynomia l time a cir cuit C ′ that comp utes f ◦ h wher e the size of C ′ is poly nomially bo unded in the sizes of C and P . Pr oof . As in the proof of T heorem 1 we can assume that both f an d h are homogeneou s polyno mials o f degree d . Let f g denote the polyn omial compu ted at gate g of circuit C . Let w boun d the number o f nod es in any layer of P . L et h i, a i d enote the a th node in the i th layer of P for 0 ≤ i ≤ d, 1 ≤ a ≤ w . Let h ( i,a ) , ( j,b ) denote the poly nomial computed by ABP P ′ , where P ′ is same as P but with source node h i, a i and sink node h j, b i . W e n ow describe the circuit C ′ computin g the po lynomia l f ◦ h . In C ′ we have gates h g , l , ( i, a ) , ( i + l , b ) i for 0 ≤ l ≤ d, 0 ≤ i ≤ d , 1 ≤ a, b ≤ w associated with each gate g of C , such th at at the gate h g, l , ( i, a ) , ( i + l , b ) i th e circuit C ′ computes r h g,l i ( i,a ) , ( i + l,b ) = f h g,l i ◦ h ( i,a ) , ( i + l,b ) (1) where f h g,l i denotes the degree l h omogen eous compo nent of the po lynomial f g . If g is a + gate of C with input gates g 1 , g 2 so that f g = f g 1 + f g 2 , we ha ve r h g,l i ( i,a ) , ( i + l,b ) = r h g 1 ,l i ( i,a ) , ( i + l,b ) + r h g 2 ,l i ( i,a ) , ( i + l,b ) , fo r 0 ≤ l ≤ d , 0 ≤ i ≤ d, 1 ≤ a, b ≤ w . In other words, h g , l , ( i , a ) , ( i + l , b ) i is a + gate in C ′ with inp ut gates h g 1 , l , ( i, a ) , ( i + l , b ) i and h g 2 , l , ( i, a ) , ( i + l , b ) i . If g is a × g ate in C we will have r h g,l i ( i,a ) , ( i + l,b ) = l X j =0 w X t =1 r h g 1 ,j i ( i,a ) , ( i + j,t ) · r h g 2 ,l − j i ( i + j,t ) , ( i + l,b ) (2) The above formula is easily comp utable by a small subcirc uit. T he outp ut gate of C ′ will be h g , d, (0 , 1) , ( d, 1) i , whe re g is the output gate of C , and (0 , 1) and ( d, 1) ar e 7 the source and the sink of the ABP P respectively . This is the description of the circu it C ′ . W e inductively argue that gate h g , l , ( i, a ) , ( i + l, b ) i of C ′ computes the poly nomial f h g,l i ◦ h ( i,a ) , ( i + l,b ) . If g is a + g ate of C the claim is obvious. Sup pose g is a × g ate of C with inputs g 1 , g 2 such that f g = f g 1 · f g 2 . Indu ctiv e ly assume tha t th e claim h olds for the gates g 1 and g 2 . Then we have f h g,l i = P l i =0 f h g 1 ,i i · f h g 2 ,l − i i . Hence, it e asily follows that f h g,l i ◦ h ( i,a ) , ( i + l,b ) = l X j =0 ( f h g 1 ,j i · f h g 2 ,l − j i ◦ h ( i,a ) , ( i + l,b ) ) = l X j =0 w X t =1 f h g 1 ,j i · f h g 2 ,l − j i ◦ h ( i,a ) , ( i + j,t ) · h ( i + j,t ) , ( i + l,b ) = l X j =0 w X t =1 ( f h g 1 ,j i ◦ h ( i,a ) , ( i + j,t ) ) · ( f h g 2 ,l − j i ◦ h ( i + j,t ) , ( i + l,b ) ) By induction hypo thesis we ha ve r h g 1 ,j i ( i,a ) , ( i + j,t ) = f h g 1 ,j i ◦ h ( i,a ) , ( i + j,t ) and r g 2 ,l − j ( i + j,t ) , ( i + l,b ) = f h g 2 ,l − j i ◦ h ( i + j,t ) , ( i + l,b ) . Now , from Eq uation 2 it is easy to obtain the desired Equation 1. Th erefore, at th e outp ut ga te h g , d, (0 , 1) , ( d, 1) i the circu it C ′ computes f ◦ h . Th e size o f C ′ is bound ed by a po lynomial in the sizes of C and P . On the o ther hand , sup pose f and g individually hav e small circuit com plexity . Does f ◦ g hav e small circuit complexity? Can we comp ute suc h a circuit for f ◦ g from circuits for f and g ? W e first consider these questions for monotone c ircuits. It is useful to und erstand the connection between monotone noncommu tati ve circu its and context- free gramma rs. W e recall the following definition. Definition 2. W e c all a context-fr ee grammar G = ( V , T , P , S ) an acyclic CFG if for any nonte rminal A ∈ V there do es not e x ist any derivation of the form A ⇒ ∗ uAw , and for each pr od uction A ⇒ β we have | β | ≤ 2 . The size siz e ( G ) of an acyclic CFG G = ( V , T , P , S ) is defined as | V | + | T | + size ( P ) , where V , T , and P are the sets of variables, ter minals, and prod uction r ules. W e note the following easy propo sition that relates acyclic CFGs to mo notone noncommu tativ e circuits over X . Proof of the Proposition 3 is in the Appendix. Proposition 3. Let C be a mon otone cir cu it o f size s computin g a polyno mial f ∈ Q h X i . Then there is an a cyclic CFG G for mon ( f ) with size ( G ) = O ( s ) . Con versely , if G is an acyclic CFG of size s comp uting some finite set L ⊂ X ∗ of mon omials over X , ther e e xists a monoton e cir c uit of size O ( s ) that computes a polynomial P m ∈ L a m m ∈ Q h X i , wher e the p ositive integer a m is the number of d erivation trees for m in th e grammar G . 8 Theorem 6. Ther e ar e mono tone cir cuits C an d C ′ computing polynomia ls f and g in Q h X i r e spectively , such th at the polyn omial f ◦ g r equ ir es monoton e circuits of size exponential in | X | , size ( C ) , and size ( C ′ ) . Pr oof . Let X = { x 1 , · · · , x n } . Define the finite lan guage L 1 = { z ww r | z , w ∈ X ∗ , | z | = | w | = n } and the corr espondin g p olynom ial f = P m α ∈ L 1 m α . Similarly let L 2 = { w w r z | z , w ∈ X ∗ , | z | = | w | = n } , and the correspon ding poly nomial g = P m α ∈ L 2 m α . It is easy to see th at there are po ly ( n ) size unam biguou s a cyclic CFGs for L 1 and L 2 . Hence, by Proposition 3 there are mo notone circuits C 1 and C 2 of size po ly ( n ) such that C 1 computes poly nomial f and C 2 computes p olyno mial g . W e first sh ow that the finite lan guage L 1 ∩ L 2 cannot b e gener ated by a ny acyclic CFG o f size 2 o ( n lg n ) . Assume to the contrary that there is an ac yclic CFG G = ( V , T , P , S ) f or L 1 ∩ L 2 of size 2 o ( n lg n ) . Notice that L 1 ∩ L 2 = { t | t = ww r w, w ∈ X ∗ , | w | = n } . Consider any deriv atio n tree T ′ for a word ww r w = w 1 w 2 . . . w n w n w n − 1 . . . w 2 w 1 w 1 . . . w n . Starting from the root of the bin ary tree T ′ , we traverse d own the tree always pick ing the child with larger yield. Clearly , there must be a nontermin al A ∈ V in this path of th e derivation tree such that A ⇒ ∗ u , u ∈ X ∗ and n ≤ | u | < 2 n . Cru cially , note that any word that A g enerates must have same len gth since every word gener ated by the gramm ar G is in L 1 ∩ L 2 and hence of leng th 3 n . Let w w r w = s 1 us 2 where | s 1 | = k . As | u | < 2 n , the string s 1 s 2 completely determin es the string w w r w . Hence, the non terminal A ca n de riv e at most one string u . Further more, this string u can oc cur in at mo st 2 n positions in a string o f length 3 n . No tice that for eac h position in wh ich u can o ccur it co mpletely determ ines a string of the form ww r w . Ther efore, A can particip ate in the deriv atio n of at most 2 n strings from L 1 ∩ L 2 . Since there are n n distinct w ords in L 1 ∩ L 2 , it follows that there must be at least n n 2 n distinct non terminals in V . Th is contradicts the size assum ption of G . Since L 1 ∩ L 2 cannot be gene rated by any acyclic CFG of size 2 o ( n log n ) , it f ollows from Lemma 3 that the polynom ial f ◦ g can not be computed by any monoton e c ircuit of 2 o ( n log n ) size. Theorem 6 shows that the Had amard pro duct of mo notone circ uits is more expressive than monotone cir cuits. It raises the question whether the permanen t polyn omial can be expressed as the Hadamard pr oduct of polyno mial-size (or even subexponen tial size) monoto ne circuits. W e note here that the per manent ca n be easily expressed as the Hadamard prod uct of O ( n 3 ) many monoton e circuits (in fact, monoton e ABPs). Theorem 7. Suppose ther e is a deterministic sube x ponentia l-time a lgorithm that takes two cir cuits as inp ut, computing polynomia ls f and g in Q h x 1 , · · · , x n i , an d outputs a cir cuit for f ◦ g . Then eith er NEXP is not in P / poly or the P erman ent d oes not have polynomia l size non commutative cir cu its. 9 Pr oof . L et C 1 be a circuit comp uting some p olynom ial h ∈ Q h x 1 , . . . , x n i . By as- sumption, we can compute a circuit C 2 for h ◦ h in sub exponential time. Therefore, h is id entically zero iff h ◦ h is iden tically zero iff C 2 ev aluates to 0 o n the all 1 ’ s inpu t. W e can easily check if C 2 ev aluates to 0 on all 1 ’ s inpu t by substitution and e valuation. This g iv es a deterministic sub exponential tim e algorithm for testing if h is identically zero. By the nonc ommutative analogue of [KI03], shown in [AMS08], it follows that ei- ther NEXP 6⊂ P / p oly or the Permanent does n ot have po lynomial size noncommutative circuits. Next, W e sho w that the iden tity testing p roblem: given f , g ∈ F h X i by circu its test if f ◦ g is identically zero is coNP hard. The pr oof of Theorem 8 is given in the Append ix. Theorem 8. Given two mo notone p olynom ial-degree circuits C an d C ′ computing polynomia l f , g ∈ Q h X i it is coNP -comp lete to c he ck if f ◦ g is identically zer o . 4 Hadamard pr o duct of monotone multilinear circu i ts In this section we study the Hadamard prod uct of commutative poly nomials (defined as in the n oncomm utative case). First we introdu ce som e no tation useful f or this section . Giv en a polyn omial f ∈ F [ X ] , an d a m onomial m over the variables X , we define f ( m ) to be the coefficient of the mono mial m in the polynom ial f . 2 Recall the Definitio n 1 of the Hadamard p roduct of two po lynomia ls in F h X i . W e define the H adamard p roduct in the commu tati ve case analog ously . Thus, for p olynom ials f , g ∈ F [ X ] we hav e F ( m ) = f ( m ) g ( m ) for any monomial m , where F = f ◦ g . In this sectio n our interest is the expressive p ower o f the Had amard pro duct. Can we express a hard e x plicit polynom ial (like the Permanent) as the Hadamard product f ◦ g where f a nd g have small arithme tic circuits? It turns out that we easily can. Proposition 4. Ther e a r e multilinear polynomia ls f , g ∈ F [ x 11 , x 12 , · · · , x nn ] su ch that both f and g h ave arithme tic formulas of size O ( n 2 ) and f ◦ g is the P ermanent polynomia l. Furthermore , for F = Q these formulas for f and g a r e mono tone . Pr oof . Define the polyno mials f and g on the v ariab les { x ij | 1 ≤ i, j ≤ n } as follows f = Q n i =1 ( P n j =1 x ij ) and g = Q n j =1 ( P n i =1 x ij ) . Clearly , their Hada mard prod uct is P erm ( x 11 , · · · , x nn ) . The formu las for f and g over ratio nals are monotone . Nev ertheless, we will define an explicit monoto ne multilin ear polyno mial that can not be written as the Hadam ard product of m ultilinear polyn omials computed by su bexpo- nential sized m onoton e arithmetic circuits. Our con struction adapts a result o f Raz a nd 2 There should be no confusion with e valuating the multiv ariate polynomial f at a point ( a 1 , · · · , a n ) as we denote that by f ( a 1 , a 2 , · · · , a n ) . 10 Y ehudayoff [R Y 08] describing an explicit mon otone polynomial that has no mono tone arithmetic circuits of size 2 ǫn , for some constant ǫ > 0 . Our proof closely follows the arguments in [R Y0 8]. Due to lack of sp ace, we pr ovide only proof sketches for several technical statements. Definition 3. F or ǫ > 0 , a multilinear p olynomial f ∈ C [ x 1 , . . . , x n ] is an ǫ -p roduct polyno mial if there a r e disjoint sets A, B ⊆ X = { x 1 , . . . , x n } such that | A | ≥ ǫ n and | B | ≥ ǫn and f = g h wher e g ∈ C [ A ] and h ∈ C [ B ] . In the sequel, we often identify a multilinear polyn omial f in C [ X ] with its coefficients vector ( indexed by m onomials in th e natural lexicographic ord er). The co mplex inne r produ ct of vectors w , w ′ ∈ C k is h w , w ′ i = P i w i w ′ i . Le t M ( X ) d enote the set of multilinear monom ials over the v ariab les i n X . Definition 4. The corr elation of multilinear polynomials f and g in C [ X ] is d efined as Corr ( f , g ) = | P m ∈M ( X ) f ( m ) g ( m ) | . Notice tha t Co rr ( f , f ) is the ℓ 2 -norm k f k of f . The explicit polynomia l from [R Y08] The explicit po lynomial F we defin e is essen- tially th e same as the one in [ R Y08] (the difference is in the constants). Let s ∈ N be a constant, to be ch osen later an d t = 40 s . Let n = tp = 4 0 sp , for a prim e p , and X = { x 1 , . . . , x n } . Partition X into t many sets of variables X (1 ) , . . . , X ( t ) with p variables each, where X ( i ) = { x ( i − 1) p + j | j ∈ [ p ] } . In poly ( n ) tim e we can construc t the field F = F 2 p which is in bijective correspond ence with { 0 , 1 } p . W e can assume that 0 ∈ F is associated with th e all 0 s vector 0 p . Fix a nontrivial additive ch aracter ψ of F . Since char ( F ) = 2 we have ψ ( x ) = ± 1 for all x ∈ F . E ach mon omial m ∈ M ( X ) defines a subset A m of X and is thus rep resented by its cha racteristic vector w ∈ { 0 , 1 } n . Split w into t blocks w 1 , . . . , w t of size p each ( w i is th e charac teristic vector o f A m ∩ X ( i ) ), and c onsider th e p field elemen ts y 1 ( m ) , y 2 ( m ) , . . . , y t ( m ) ∈ F associated with these strings. The bijection between F and { 0 , 1 } p implies fo r any m ∈ M ( X ) that y i ( m ) = 0 iff no variable x ∈ X ( i ) appears in m . Let us now define the polyn omial F . Given a mono mial m ∈ M ( X ) , we define F ( m ) to be ψ ( Q t i =1 y i ( m )) . W e d efine a polyn omial f ∈ F [ X ] to be explicit if the coef ficient f ( m ) of a ny m onomia l m can b e computed in time polyno mial in n . Note th at the polyno mial F is explicit. W e n ow state ou r ma in corre lation result using which we will obtain th e lower bou nd against the Hadamard pro duct of mo noton e multilinear polynomia ls in C [ x 1 , . . . , x n ] . A proo f sketch is gi ven in the appendix. 11 Lemma 2. Let F ∈ C [ x 1 , . . . , x n ] be the explicit multilinea r po lynomial defi ned a bove and f 1 , f 2 ∈ C [ x 1 , . . . , x n ] be any 1 / 3 -pr od uct polynomia ls. Then 1. P m ∈M ( { x 1 ,...,x n } ) F ( m ) ≥ 0 . 2. Corr ( F, f 1 ◦ f 2 ) ≤ 2 − αn k F kk f 1 ◦ f 2 k , fo r a consta nt α > 0 that is indep endent of f 1 and f 2 . Using the above lem ma bou nding the correlation between F and the Had amard produ ct of 1 / 3 -pr oduct p olynom ials, we will prove the main lower b ound . W e first recall a crucial lemma of Raz and Y ehudayoff [ R Y08]. Lemma 3. F or n ≥ 3 , let F ∈ C [ x 1 , . . . , x n ] be a monoto ne multilinea r po lynomial computed b y a mon otone cir cuit of size s (i.e. the circuit h as at most s edges). Then, ther e ar e s + 1 mo noton e 1 / 3 -p r oduct p olynomia ls f 1 , f 2 , . . . , f s +1 such th at F = P s +1 i =1 f i . Theorem 9. F or lar ge enough n ∈ N , ther e is an explicit monoto ne multilinear p olyno- mial F ′ ∈ Q [ x 1 , . . . , x n ] that cann ot be written a s a Ha damard pr o duct of two mo no- tone multilinear polyn omials f 1 , f 2 ∈ R [ x 1 , . . . , x n ] such tha t each f i is comp uted by monoton e cir cu its of size less than 2 αn , wher e α > 0 is an ab solute constant. Pr oof . B y the density of primes it suffi ces to consider n o f the f orm tp , for prime p , where t is th e c onstant in the definition of F . Let X d enote the set of variables { x 1 , . . . , x n } , and let F be th e explicit po lynomia l mentioned in Lemma 2 above. For any mono mial m ∈ M ( X ) , let F ′ ( m ) = ( F ( m ) + 1) / 2 . Clearly , the coefficients of F ′ all lie in { 0 , 1 } . C onsider the corre lation between F and F ′ : Corr ( F , F ′ ) =       X m : F ( m )=1 1       ≥ 2 n − 1 where the inequality above follows from the point 1 of Lem ma 2. Let us assume that F ′ can be written as f 1 ◦ f 2 , where f 1 and f 2 are multilinear polyno - mials co mputed b y mon otone a rithmetic circu its o f size at most s . W e assume n ≥ 3 , so that Lemma 3 is applicable . By Lemma 3, there exist mon otone 1 / 3 -prod uct polyn o- mials f 1 , 1 , . . . , f 1 ,s +1 , f 2 , 1 , . . . , f 2 ,s +1 such that f i = P s +1 j =1 f i,j , for each i ∈ { 1 , 2 } . Thus, we have, F ′ =   s +1 X j =1 f 1 ,j   ◦ s +1 X k =1 f 2 ,k ! = X 1 ≤ j,k ≤ s +1 f 1 ,j ◦ f 2 ,k T aking correlation with F on both sides, we see that, 2 n − 1 ≤ X 1 ≤ j,k ≤ s +1 Corr ( F , f 1 ,j ◦ f 2 ,k ) ≤ X 1 ≤ j,k ≤ s +1 2 − β n k F kk f 1 ,j ◦ f 2 ,k k , 12 by apply ing triang le ineq uality and then p art 2 of Lemma 2, wh ere β > 0 is some constant. Since, f 1 ,j ◦ f 2 ,k ’ s are monotone polyn omials a dding up to F ′ , it follows that for any monom ial m ∈ M ( X ) its coe fficient in f 1 ,j ◦ f 2 ,k is at most 1 . Hence, k f 1 ,j ◦ f 2 ,k k ≤ k F k and we have 2 n − 1 ≤ P 1 ≤ j,k ≤ s +1 2 − β n k F k 2 = ( s + 1) 2 2 n − β n Consequently , we ha ve s ≥ 2 β n/ 4 , for large enough n . Refer ences ABO99. E . A L L E N D E R , R . B E A L S , A N D M . O G I H A R A , T he complex ity of matrix rank and feasible systems of linear equations, Computational Complexity , 8(2):99-126, 1999. A O96. E . A L L E N D E R , M . O G I H A R A , Relationships among PL, # L and the determinant. RAIR O - Theoretical Informatics and Applications , 30:1–21, 1996. ARZ99. E . A L L E N D E R , K . R E I N H A R D T , S . Z H O U , Isolation, matching and counting uniform and nonuniform upper bounds. Journal of Computer and System Sci ences , 59(2):164–1 81, 1999. AMS08. V . A RV I N D , P. M U K H O PAD H YA Y , S . S R I N I V A S A N Ne w results on Noncommu- tativ e Polynomial Identity T esting In Proc. of Annual IEEE Conference on Computational Complexity , 268-279,20 08. Bh97. R . B H A T I A , Matrix Analysis, Springer - V erlag Publishing Com pany , 1997. Bo07. J . B O U R G A I N : “On the Construction of Affine Extractors”, Geometric & Functional Analysis GAF A, V ol. 1 7, N o. 1. (April 2007 ) , pp. 33-57. BGK06. J . B O U R G A I N , A . G L I B I C H U K , S . K O N YAG I N : “Estimate fo r the number of sums and products and for exp onential sums in fields of prime order”, J. London Math. Soc. 73 ( 2006), pp. 380-398 . GJ79. M . R . G A R E Y , D . S . J O H N S O N Computers and Intractability: A Guide to the Theory of NP-Completeness. W .H. Freeman. p. 228. ISBN 0-7167-10 45-5 , 1979. HMU. J . E . H O P C RO F T , R . M O TA WA N I , J . D . U L L M A N , Introduction t o Automata Theory Languages and Computation, Second Edition , Pearson Education Publishing Compan y . KI03. V . K A B A N E T S , R . I M PAG L I A Z Z O , Derandomization of polynomial identity test means provin g circuit lower bound s, In Proc. of 35th A CM Sym. on Theory of Computing, 355- 364,2003 . KvM02. A . K L I V A N S , D . V A N M E L K E B E E K , Graph nonisomorphism has subexp onential size proofs unless the polynomial-time hierarchy collapses. SIAM Journal on Computing , 31(5):1501– 1526, 2002. N91. N . N I S A N , Lo wer bounds for noncommutati ve computation In Proc. of 23rd A CM Sym. on Theory of Computing, 410-418, 1991. RS05. R . R A Z , A . S H P I L K A , Deterministic polynomial i dentity testing in non commutative models Computational Complexity , 14(1):1-19, 2005. R Y08. R A N R A Z , A M I R Y E H U D AYO FF , “Multilinear Formulas, Maximal-Partition Di screpanc y and Mixed-Sources Extractors. ” FOCS 2008: 273-282. T91. S . T O D A , Counting Problems Compu tationally Equi valant to the Determinant, manuscript. 13 A ppendix Proof of Pr o position 3 First we prove the fo rward direction by con structing an acyclic CFG G = ( V , T , P, S ) for mon ( f ) . Let V = { A g | g is a gate of circu it C } b e the set of nonterm inals of G . W e in clude a production in P for eac h gate of the circuit C . If g is an inp ut gate with input x i , 1 ≤ i ≤ n in clude the p roductio n A g → x i in P . If the inp ut is a nonzer o field element then add the productio n A g → ǫ . 3 f g be the polynom ial compu ted at gate g o f C . If g is a × gate with f g = f h × f k then include the productio n A g → A h A k and if it is + gate with f g = f h + f k include th e prod uctions A g → A h | A k . Le t the star t symb ol S = A g , wh ere g is the ou tput gate of C . It is easy to see fro m the above constructio n that G is acyclic moreover siz e ( G ) = O ( s ) and it gen erates the finite language mon ( f ) . The converse directio n is similar . Proof of T heorem 8 W e first show that the complem ent of the problem is in NP. The NP machine will guess a monomial m α ∈ X ∗ , X = { x 1 , . . . , x n } an d check if coef ficient of m α is nonzero in both C and C ′ . Note that we can co mpute coefficient o f m α in C and C ′ in deter ministic polyno mial time using result fro m [AMS08]. Den ote by CFGINT the pr oblem of testing emptiness of the intersectio n of two acyclic CFGs that generate poly ( n ) length strin gs. By Lemma 3 CFGINT is poly nomial tim e m any-one redu cible to testing if f ◦ g is id en- tically zero. The pro blem of testing if the intersectio n of two CFGs (with rec ursion) is empty is known to be undecidab le via a red uction from the Post Correspo ndence prob- lem [HMU, Chapter 9,Page 422] . W e can give a n analog ous reduction from bo unded Post Correspo ndence to CFGINT . The coNP-hardne ss of CFGINT follows from the coNP-hard ness of boun ded Post Correspondence [GJ79]. Pr oof of Lemma 2 Part 1 of Lemma 2 is trivial. By construction , each of the coefficients of F is ± 1 . For any z ∈ F n ote that P y ∈ F ψ ( z · y ) ≥ 0 . Hence, X m ∈M ( X ) F ( m ) = X y 1 ,...,y t ∈ F ψ ( t Y j =1 y j ) = X y 1 ,...,y t − 1 X y t ψ (( t − 1 Y j =1 y j ) y t ) ≥ 0 . where the last in equality follows as each of the terms in the o uter summation is n on- negativ e . 3 If the circuit takes as input 0 , we can first propagate it through the circuit and eliminate it. 14 The expo nential sum estimate W e now state an exponen tial sum estimate of Bourgain, Glibichuk, and Konyagin (see [BGK06],[ Bo07]) that we will need later . The result is a special case of their result, and is similar to the version used in [R Y08]. Theorem 10. Th er e exist two constan ts, an integ er s ∈ N and γ > 0 , such that for every p rime p , for e very family of sets A 1 , A 2 , . . . , A s ⊆ F 2 p of size at least 2 p/ 20 each, for every nonzer o z ∈ F 2 p , and for each non-trivial additive char acter ψ of F 2 p ,       X y 1 ∈ A 1 ,...,y s ∈ A s ψ ( z . s Y i =1 y i )       ≤ 2 − γ p | A 1 | . | A 2 | . . . | A s | W e use the abov e fixed s ∈ N as the constant s in the construction of the polynomial F of Lemma 2. A strengthening of the result of [R Y08] W e can prove, exactly along the lines of [R Y0 8, Theorem 3.1] , that the poly nomial F has low correlation with 1 / 3 -pro duct polyno mials. Howev er , we need to p rove the stro nger claim that it has low corre la- tion with Hadamard products o f such polynomials. In order to prove this claim we need to strengthen [R Y0 8, Theorem 3.1]. Let X = X ′ ∪ X ′′ be a partition of the v ariable set X . F o r a monomia l m ′′ ∈ M ( X ′′ ) , we call the tuple ( X ′ , X ′′ , m ′′ ) a suitab le r estriction if for each i ∈ [ t ] such th at | X ′′ ∩ X ( i ) | ≥ p/ 2 , som e variable in X ( i ) ap pears in m ′′ . By o ur enco ding assumptio n, this implies that for any mon omial m ′ ∈ M ( X ′ ) and any i such that | X ′′ ∩ X ( i ) | ≥ p/ 2 , y i ( m ′ m ′′ ) 6 = 0 . Giv en a suitab le ( X ′ , X ′′ , m ′′ ) , den ote by ˜ F the m ultilinear po lynomial over the variables X ′ where, for any mo nomial m ′ ∈ M ( X ′ ) , ˜ F ( m ′ ) = F ( m ′ m ′′ ) . Le t f = g h ∈ C [ X ′ ] be a multilinear p olynom ial with g ∈ C [ A ] and h ∈ C [ B ] , wher e A and B are disjoint sets. The requ ired stronger version of [R Y08, Theor em 3.1] is the following. Theorem 11. A ssume ( X ′ , X ′′ , m ′′ ) is su itable r estriction. Let ˜ F be de fined a s a bove and let f = g h as above with | A | , | B | ≥ n/ 10 . Then, Corr  ˜ F , f  ≤ k ˜ F k k f k 2 Ω ( n ) wher e th e constant in the Ω ( · ) is in depend ent of f . Pr oof Sketch . Our notatio n is from [R Y08]. For i ∈ [ t ] , let A ( i ) and B ( i ) denote A ∩ X ( i ) and B ∩ X ( i ) respectively . W e need the follo wing simple claim, the proof of which is similar to [R Y08, Proposition 9.2]. 15 Claim. There are at least s + 1 many i ∈ [ t ] such that | A ( i ) | ≥ p/ 20 a nd at least s + 1 many j ∈ [ t ] such that | B ( j ) | ≥ p/ 20 . Fix I ⊆ [ t ] of size s su ch th at | A ( i ) | ≥ p/ 20 for each i ∈ I . Let J = [ t ] \ I . By the above claim, there is a j 0 ∈ J such that | B ( j 0 ) | ≥ p/ 2 0 . Set A 1 = S i ∈ I A ( i ) , B 1 = S j ∈ J B ( j ) an d A 2 = A \ A 1 , B 2 = B \ B 1 . W e den ote by a 1 , a ′ 1 etc. monom ials from M ( A 1 ) and similarly for m onomia ls from M ( A 2 ) , M ( B 1 ) and M ( B 2 ) . Finally , we d enote by m 1 and m 2 the restriction of the m onomial m ′′ to the sets S i ∈ I X ( i ) an d S j ∈ J X ( j ) respectively . Giv en mono mials a 2 ∈ M ( A 2 ) and b 1 , b ′ 1 ∈ M ( B 1 ) , d enote by Z ( a 2 , b 1 , b ′ 1 ) th e field element Q j ∈ J y j ( a 2 b 1 m 2 ) − Q j ∈ J y j ( a 2 b ′ 1 m 2 ) . L et S ( a 2 ) denote tho se p airs ( b 1 , b ′ 1 ) su ch that Z ( a 2 , b 1 , b ′ 1 ) = 0 . Let S 1 =  a 2   | S ( a 2 ) | > 2 2 | B 1 |− p/ 40  and S 2 =  a 2   | S ( a 2 ) | ≤ 2 2 | B 1 |− p/ 40  . The quantity we wish to bound is: Corr  ˜ F , f  =       X a 1 ,a 2 ,b 1 ,b 2 ˜ F ( a 1 a 2 b 1 b 2 ) f ( a 1 a 2 b 1 b 2 )       ≤         X a 2 ∈ S 1 a 1 ,b 1 ,b 2 ˜ F ( a 1 a 2 b 1 b 2 ) f ( a 1 a 2 b 1 b 2 )         | {z } C 1 +         X a 2 ∈ S 2 a 1 ,b 1 ,b 2 ˜ F ( a 1 a 2 b 1 b 2 ) f ( a 1 a 2 b 1 b 2 )         | {z } C 2 (3) W e first bound C 1 using the following analog ue of [R Y08, Corollary 9.6]. Claim. For large enoug h p , | S 1 | ≤ 2 | A 2 |− p/ 50 . Pr oof of Claim. Le t S = { ( a 2 , b 1 , b ′ 1 ) | Z ( a 2 , b 1 , b ′ 1 ) = 0 } . W e will bound | S | . For this we first boun d the numb er of ( a 2 , b 1 , b ′ 1 ) such that Q j ∈ J y j ( a 2 b ′ 1 m 2 ) = 0 . Fix a j ∈ J . I f j is such that | X ( j ) ∩ X ′′ | ≥ p/ 2 then, as ( X ′ , X ′′ , m ′′ ) is a suitable restriction, we ha ve y j ( a 2 b ′ 1 m 2 ) 6 = 0 . Otherwise | X ( j ) ∩ X ′ | ≥ p/ 2 , and y j ( a 2 b ′ 1 m 2 ) = 0 only if non e of th e variables in X ( j ) ∩ X ′ appears in a 2 or b ′ 1 ; the numb er of such triples ( a 2 , b 1 , b ′ 2 ) is at mo st 2 | A 2 | +2 | B 1 |− p/ 2 . Thus, the num ber of ( a 2 , b 1 , b ′ 1 ) such that Q j ∈ J y j ( a 2 b ′ 1 m 2 ) = 0 is at most t 2 | A 1 | +2 | B 2 |− p/ 2 . 16 If Q j ∈ J y j ( a 2 b ′ 1 m 2 ) 6 = 0 then Z ( a 2 , b 1 , b ′ 2 ) = 0 o nly if y j 0 ( a 2 b ′ 1 m 2 ) = Q j ∈ J y j ( a 2 b 1 m 2 ) Q j ∈ J \{ j 0 } y j ( a 2 b ′ 1 m 2 ) . I.e. the scalar y j 0 ( a 2 b ′ 1 m 2 ) , and h ence the restriction of b ′ 1 to X j 0 , is com pletely deter- mined by a 2 , b 1 , and the restriction of b ′ 1 to X \ X j 0 . Since | B ( j 0 ) | ≥ p/ 2 0 , the number of ( a 2 , b 1 , b ′ 1 ) such that this is true is at most 2 | A 2 | +2 | B 1 |− p/ 20 . Hence, | S | ≤ t 2 | A 2 | +2 | B 1 |− p/ 2 + 2 | A 2 | +2 | B 1 |− p/ 20 ≤ 2 | A 2 | +2 | B 1 |− p/ 20+1 for large enough p . On the other hand, | S | is at least | S 1 | · 2 2 | B 1 |− p/ 40 . Hence, for large enoug h p we have | S 1 | ≤ 2 | A 1 |− p/ 40+1 < 2 | A 1 |− p/ 50 . It follows from the above claim, and a simple application of th e Cauchy- Schwarz in- equality , th at C 1 ≤ 2 − Ω ( n ) k ˜ F k k f k . Now we boun d C 2 . In th is case, th e p roof of [R Y0 8, Pro position 9.4 ] go es thro ugh almo st verbatim to yield the bound ; th e only change necessary is in the pr oof of [R Y08, Claim 9.7 ], where we need to use th e more general expo nential sum estimate stated in Theorem 10 ab ove. W e om it this pr oof an d simply state the obtained boun d on C 2 . C 2 ≤ 2 − Ω ( n ) k ˜ F k k f k . Putting this together with Equation ( 3 ) and the bound on C 1 yields the statement of the theorem. The correlatio n bo und W e n ow conside r part 2 of Lemma 2. Let f = f 1 ◦ f 2 be a Hadamard product of 1 / 3 -pr oduct po lynomials f 1 and f 2 . For i ∈ { 1 , 2 } , let f i = g i h i , where g i ∈ C [ A i ] , h i ∈ C [ B i ] ; A i and B i being disjoint subsets of X of size at least n/ 3 each. W e can assume that A i ∪ B i = X . Co nsider the four pairwise intersectio ns A 1 ∩ A 2 , A 1 ∩ B 2 , A 2 ∩ B 1 , and B 1 ∩ B 2 . W e can write any m onomial m ∈ M ( X ) as m 11 m 12 m 21 m 22 , where m 11 , m 12 , m 21 , an d m 22 denote the r estriction of m to A 1 ∩ A 2 , A 1 ∩ B 2 , A 2 ∩ B 1 , and B 1 ∩ B 2 respectively . If A 1 = A 2 and B 1 = B 2 or A 1 = B 2 and A 2 = B 1 , then the poly nomial f is a 1 / 3 - produ ct polynom ial. Follo win g [R Y0 8] we can show that F has very low co rrelation with all 1 / 3 -pro duct polyn omials. Hence, in this easy case we ar e do ne. In th e next claim we argue that an “approximate” v e rsion of this desirable scenario alw ays holds. Claim. Let A i , B i , fo r i ∈ { 1 , 2 } be as defined above. At least one of the following holds. Case 1: | A 1 ∩ A 2 | ≥ n/ 1 0 an d | B 1 ∩ B 2 | ≥ n/ 10 . Case 2: | A 1 ∩ B 2 | ≥ n/ 1 0 an d | A 2 ∩ B 1 | ≥ n/ 10 . 17 Pr oof of Claim. Assume Case 1 do es not hold. Let us a ssume that | A 1 ∩ A 2 | < n/ 10 (th e other c ase is symmetric to this o ne). Then, since | A 2 | ≥ n/ 3 , we kn ow that | A 2 ∩ B 1 | = | A 2 | − | A 2 ∩ A 1 | (this is because we have assume d th at A 1 ∪ B 1 = X ), wh ich is at least n/ 3 − n/ 1 0 > n/ 10 . Similarly , | A 1 ∩ B 2 | is also at least n/ 10 . Thus, Case 2 h olds. By swapping the n ames of A 2 and B 2 if n ecessary , we assume that Case 1 of claim 4 holds. Let X ′ = ( A 1 ∩ A 2 ) ∪ ( B 1 ∩ B 2 ) a nd X ′′ = X \ X ′ = ( A 1 ∩ B 2 ) ∪ ( A 2 ∩ B 1 ) . W e now note that, restricted to the set of variables X ′ , the polynom ial f h as a ‘ prod- uct p olyno mial structure ’. More precisely , fo r a monom ial m = m 11 m 12 m 21 m 22 ∈ M ( X ) , we can wr ite f ( m ) as the product of g 1 ( m 11 m 12 ) g 2 ( m 11 m 21 ) and h 1 ( m 21 m 22 ) h 2 ( m 12 m 22 ) ; for a fixed m 12 , m 21 , the former depend s only on the monom ial m 11 and the latter on ly on m 22 : th is is very much like a p roduct poly- nomial. W e u se th is fu rther below . From n ow , we deno te by g 12 ( m 11 m 12 m 21 ) and h 12 ( m 12 m 21 m 22 ) the values g 1 ( m 11 m 12 ) g 2 ( m 11 m 21 ) and h 1 ( m 21 m 22 ) h 2 ( m 12 m 22 ) . Hence, we have, Corr ( F , f ) =      X m 11 ,m 12 ,m 21 ,m 22 F ( m 11 m 12 m 21 m 22 ) f ( m 11 m 12 m 21 m 22 )      =      X m 12 ,m 21 X m 11 ,m 22 F ( m 11 m 12 m 21 m 22 ) g 12 ( m 11 m 12 m 21 ) h 12 ( m 12 m 21 m 22 )      (4) For fixed m 12 and m 21 , the inner summ ation above is the inner pr oduct of vectors that correspo nd to polyno mials over the variables in X ′ : one of the v ectors is t he restriction of F to these coordin ates, and the other is the produ ct g 12 h 12 , which is a 1 / 1 0 -pro duct polyno mial ov er the variables in X ′ . Our aim is to show that, for ‘most’ values of m 12 and m 21 , the inner summation is s mall (to prove this, we will use the proo f of [R Y08]) ; for other values of m 12 and m 21 , a brute force bound will do. W e sho w this first. Call a tuple of monomials ( m 12 , m 21 ) ∈ M ( A 1 ∩ B 2 ) × M ( A 2 ∩ B 1 ) a suitable pair of monom ials if the tuple is ( X ′ , X ′′ , m 12 m 21 ) is a suitable restriction as defined in the p revious section. Let B d enote the set { ( m 12 , m 21 ) ∈ M ( A 1 ∩ B 2 ) × M ( A 2 ∩ B 1 ) | ( m 12 , m 21 ) not suitable } of unsuitab le pairs of monomials, and let B ′ denote the 18 set of suitable pairs of monom ials. W e split the summation in Equation (4) as follows: Corr ( F , f ) ≤       X ( m 12 ,m 21 ) ∈B X m 11 ,m 22 F ( m 11 m 12 m 21 m 22 ) g 12 ( m 11 m 12 m 21 ) h 12 ( m 12 m 21 m 22 )       | {z } T 1 +       X ( m 12 ,m 21 ) ∈B ′ X m 11 ,m 22 F ( m 11 m 12 m 21 m 22 ) g 12 ( m 11 m 12 m 21 ) h 12 ( m 12 m 21 m 22 )       | {z } T 2 (5) W e bound the s ums T 1 and T 2 separately . W e tackle T 1 first. W e need a claim bo unding the number of unsuitab le pairs: Claim. |B | ≤ 2 | X ′′ | 2 Ω ( n ) , for large enough n . Pr oof of Claim. Let S ⊆ [ t ] b e the set of tho se i s.t | X ( i ) ∩ X ′′ | ≥ p/ 2 . For i ∈ S , let B i denote those pairs of mon omials ( m 12 , m 21 ) ∈ M ( A 1 ∩ B 2 ) × M ( A 2 ∩ B 1 ) such that no variable x ∈ X ( i ) ∩ X ′′ appears in them. Clearly , for i ∈ S , |B i | ≤ 2 | X ′′ \ X ( i ) | ≤ 2 | X ′′ | − p/ 2 . Also, since B = S i ∈ S B i , we have |B | ≤ | S | 2 | X ′′ | − p/ 2 ≤ t 2 | X ′′ | − p/ 2 . Since p = n/t = Ω ( n ) , we see that |B | ≤ 2 | X ′′ | / 2 Ω ( n ) , for large enough n . W e now bound T 1 . Using the Cauchy-Sch warz inequality , we ha ve, T 1 ≤ s X ( m 12 ,m 21 ) ∈B X m 11 ,m 22 | F ( m 11 m 12 m 21 m 22 ) | 2 . s X ( m 12 ,m 21 ) ∈B X m 11 ,m 22 | g 12 ( m 11 m 12 m 21 ) h 12 ( m 12 m 21 m 22 ) | 2 ≤ q |B | 2 | X ′′ | . s X m 12 ,m 21 ,m 11 ,m 22 | g 12 ( m 11 m 12 m 21 ) h 12 ( m 12 m 21 m 22 ) | 2 ≤ 2 | X ′ | + | X ′′ | 2 − Ω ( n ) . s X m | f ( m ) | 2 ≤ k F k 2 Ω ( n ) k f k = 2 − Ω ( n ) k F kk f k (6) Above, we have used the fact tha t | F ( m ) | = 1 fo r all mon omials m , an d that f or m = m 11 m 12 m 21 m 22 , f ( m ) = g 12 ( m 11 m 12 m 21 ) h 12 ( m 12 m 21 m 22 ) . 19 W e n ow bound T 2 . Fix any suitable pair of mo nomials ( m 12 , m 21 ) . For m 11 ∈ M ( A 1 ∩ A 2 ) a nd m 22 ∈ M ( B 1 ∩ B 2 ) , we denote by ˜ F ( m 11 m 22 ) , ˜ g ( m 11 ) and ˜ h ( m 22 ) th e val- ues F ( m 11 m 12 m 21 m 22 ) , g 12 ( m 11 m 12 m 21 ) , and h 12 ( m 12 m 21 m 22 ) respectively . W e think of ˜ F , ˜ g , and ˜ h as vecto rs of multilinear polyn omials over X ′ . Looked at in th is way , ˜ g ∈ C [ A 1 ∩ A 2 ] and ˜ h ∈ C [ B 1 ∩ B 2 ] . By Claim 4 , | A 1 ∩ A 2 | , | B 1 ∩ B 2 | ≥ n/ 1 0 . Hence, Theor em 11 is applicab le, and we ha ve Corr  ˜ F , ˜ g ˜ h  2 ≤ 2 | X ′ |− Ω ( n ) k ˜ g ˜ h k 2 (7) Using the above bou nd, we bou nd T 2 and finish the proof of Lemma 2. Squaring th e expression fo r T 2 in E quation 5, and using the Cauchy- Schwarz inequality , we have, T 2 2 ≤ |B ′ | X ( m 12 ,m 21 ) ∈B ′      X m 11 ,m 22 F ( m 11 m 12 m 21 m 22 ) g 12 ( m 11 m 12 m 21 ) h 12 ( m 12 m 21 m 22 )      2 ≤ |B ′ | 2 | X ′ | − Ω ( n ) X ( m 12 ,m 21 ) ∈B ′ X m 11 ,m 22 | g 12 ( m 11 m 12 m 21 ) h 12 ( m 12 m 21 m 22 ) | 2 (by Equation 7) ≤ 2 | X ′′ | + | X ′ | − Ω ( n ) X ( m 12 ,m 21 ) ∈B ′ X m 11 ,m 22 | f ( m 11 m 12 m 21 m 22 ) | 2 ( ∵ |B ′ | ≤ 2 | X ′′ | ) ≤ 2 n − Ω ( n ) X m ∈M ( X ) | f ( m ) | 2 ≤ k F k 2 2 Ω ( n ) k f k 2 . (since k F k 2 = 2 n ) ∴ T 2 ≤ 2 − Ω ( n ) k F kk f k . Using the above bou nd on T 2 , and Equations 4 and 6, we see that Corr ( F , f ) ≤ T 1 + T 2 ≤ 2 − Ω ( n ) k F kk f k . This proves part 2 of Lem ma 2. 20