On Lower Bounds for Constant Width Arithmetic Circuits

On Lo w er Bounds for Constan t Width Arithmetic Circuits V. Arvind, Pushk ar S. Jogle k ar, S rik an th Sriniv asan Institute of Mathematical Sciences C.I.T Campus,Chennai 600 113, India { arvind,p ushkar,srikanth } @ imsc.res.in Abstract. The motiv ation for this paper is to study the co mp lexit y of constan t-width arithmetic circuits. Our main results are the foll owing. 1. F or every k > 1, we provide an explicit p olynomial th at can b e computed by a linear-sized monotone circuit of width 2 k but h as no sub exp onential-sized monotone circuit of width k . It follow s, from t h e definition of t he p olynomial, that the constant-width and the constant- depth hierarchies of monotone arithmetic circuits are infinite, b oth in t he comm u t ativ e and the noncommutativ e settings. 2. W e prov e hard n ess-randomness t radeoffs for identit y testing constant-wi d th commutativ e circuits analogous to [KI03,DSY08]. 1 In tro duction Using a rank argumen t, Nisan, in a seminal pap er [N91], sho wed exp onen tial size lo wer b ounds for noncomm u tativ e formulas (and noncomm u tativ e algebraic branc h in g programs) that com- pute the noncomm u tativ e p ermanent o r determinant p olynomials in the rin g F h X i , wh ere X = { x 1 , · · · , x n } are noncomm uting v ariables. By Ben-Or a n d Cleve ’s result [BC92], we know that b oun ded-width arithmetic circuits (b oth commutat ive and n oncomm utativ e) are at least as p o w erfu l as form ulas (indeed width three is sufficient) . Can we extend Nisan’s lo we r b ound arguments to pro ve size lo w er b ounds for no nc ommutative b ound ed-width circuits? Motiv ated b y this question w e mak e some simp le motiv ating observ ations in this s ection. W e firs t recall some basic definitions. Definition 1. [N91,RS05] An Algebraic Branc hing Program (ABP) over a field F and vari- ables x 1 , x 2 , · · · , x n is a la y ered dir e cte d acyclic gr aph with one source vertex of inde gr e e zer o and one sink vertex of outde gr e e zer o. L et the layers b e numb er e d 0 , 1 , · · · , d . Edges only go fr om layer i to i + 1 for e ach i . The sour c e and sink ar e the unique layer 0 and layer d vertic e s, r esp e ctively. Each e dge in the ABP is lab ele d with a line ar form over F in the input variables. The size of the A BP is the numb er of vertic es. Each sour c e to sink p ath in the ABP c omputes the pr o duct of the line ar forms lab eling the e dges on the p ath, and the sum of these p olynomials over al l sour c e to sink p aths i s the p olynomial c ompute d by the ABP . The scalars in an ABP can come f rom an y field F . If the input v ariables X = { x 1 , x 2 , · · · , x n } are noncomm u ting th en the ABP (or circuit) computes a p olynomial in the free noncommutat ive ring F h X i . If th e v ariables are comm uting then th e p olynomial computed is in the r ing F [ X ]. Definition 2. An arithmetic cir cuit over F and variables x 1 , x 2 , · · · , x n is a dir e cte d acyclic gr aph with e ach no de of i nde gr e e zer o lab ele d by a variable or a sc alar c onstant. Each i nternal no de g of the DA G is lab ele d by + or × (i.e. it is a plus or multiply g ate) and is of inde gr e e two. A no de of the DAG is designate d as the output g ate. Each internal gate of the arithmetic cir cuit c omputes a p olynomial (by adding or multiplying its input p olynomials). The p olyno- mial c ompute d at the output gate is the p olynomial c ompute d by the cir c u it. The cir cuit i s said to b e la y ered if its vertic es ar e p artitione d into vertex sets V 1 ∪ V 2 ∪ . . . ∪ V t such that V 1 c onsists only of le aves, and given any internal no de g in V i for i > 1 , the childr en of g ar e either no des fr om V 1 (c onsisting of c onstants or variables) or no des fr om the set V i − 1 . The size of a c i r cuit is the numb er of no des in it, and the width of a layer e d cir c u it is m ax i> 1 | V i | . An arithmetic cir cui t over the field R is m onotone if al l the sc alars use d ar e nonne gative. Final ly, a layer e d arith metic cir cuit is staggered if, in e ach layer i with i > 1 , every no de exc ept p ossibly one is a pr o duct gate of the form g = u × 1 , for some gate u fr om the pr ev i ous layer. Note that the n otion of b ound ed (i.e, constant) width staggered circuits of width w is iden tical to the notion of a straigh t-line pr ogram with w registers. Th e follo wing lemma sh ows that staggered circuits of w idth w are comparable in p o wer to width w − 1 (not n ecessarily staggered) arithmetic circuits. It holds in th e comm utativ e and the noncomm utativ e settings. W e p ostp one the pr o of of the lemma to th e App endix. Lemma 1. Giv e n any layer e d arithmetic cir cui t C of width w and size s c omputing a p oly- nomial p , ther e is a stagger e d arithmetic cir cuit C ′ of width at most w + 1 and size O ( w s ) c omputing the same p olynomial. A seminal result in the area of b ound ed wid th circuits is d ue to Ben-Or and C lev e [BC92] where they sho w that size s arithmetic formulas computing a p olynomial in F [ X ] (or in F h X i in the noncomm utativ e case) can b e ev aluated by staggered arithmetic circuits of width three and size O ( s 2 n ). Bounded width circuits hav e al so b een studied und er v arious restrictions in [LMR07,MR08,JR09]. Ho we ver, th ey ha ve not consid ered the question of proving exp licit lo w er b oun ds. What is the p o w er of arithm etic circuits of width 2? It is easy to see th at the w id th-t w o circuit mo del is unive rs al. W e s tate this (folklore) obser v ation. Prop osition 1. Any p olynomial of de gr e e d with s monomials in F [ x 1 , x 2 , · · · , x n ] (or in F h x 1 , · · · , x n i ) c an b e c ompute d by a width two arithmetic cir cuit of size O ( d · s ) . F urthermor e, any monotone p olynomial (i.e, with non-ne gative r e al c o efficients) c an b e c ompute d by a width two monotone cir cuit over R of si ze O ( d · s ) . Some O bserv ations T o motiv ate the study of constan t-width circuits, we p oin t out that, for the problem of pro ving lo wer b ound s for noncommuta tive b ounded wid th circu its, Nisan’s rank argumen t is not useful. F or the noncomm utativ e “palindr omes” p olynomial P ( x 0 , x 1 ) = P w ∈{ x 0 ,x 1 } n ww R , the comm un ication matrix M n ( P ) is of rank 2 n and hence any noncommuta tive ABP for it is exp onent ially large [N91]. Ho we ver, w e can give an easy width-2 non commutativ e arithmetic circuit for P ( x 0 , x 1 ) of size O ( n ). I ndeed, we can even ensu re that eac h gate in th is circuit is homo g ene ous . Prop osition 2. The p alindr omes p olynomial P ( x 0 , x 1 ) has a width-2 nonc ommutative arith- metic cir cuit of size O ( n ) . 2 What then is a go o d candidate explicit p olynomial that is not computable by width-2 circuits of p olynomial size ? W e b eliev e that the p olynomial P ℓ k (of Sect ion 2) for suitable k is the righ t candidate. A lo wer b ou n d argu m en t stil l eludes us. Ho we ver, if w e co ns id er monotone constant-width circuits th en ev en in the co mmutativ e case w e can sho w exp onent ial size lo wer b ound s for monotone width- k circuits computin g P ℓ k . Sin ce P ℓ k is computable by depth 2 k arithmetic circuits (of u n b ounded fanin), it follo ws that the constan t-width and the constant- d epth h ierarc hies of monotone arith m etic circuits are infi nite. W e present th ese results in S ection 2 . R emark 1. Regarding the separation of the constant-depth hierarc hy of monotone circuits, w e n ote that a separation has also b een pro ved by Raz and Y ehuda y off in [R Y09 ]; their lo wer b ound s show a sup erp olynomial s eparation b et w een the p o wer of depth k multiline ar circu its and depth k + 1 monotone circuits for any k (see [R Y09] for th e definition and results regarding m ultilinear circuits). In con trast, our separation wo r ks only for monotone circuits, and only for infinitely man y k . Nonetheless, we think that our s ep aration is in teresting b ecause the separation we ac hieve is stronger. More p recisely , the results of [R Y09] sh o w a separation of the order of 2 (log s ) 1+ Ω (1 /k ) (that is, th ere is a p olynomial that can b e computed b y circuits of depth k + 1 and siz e s b ut n ot b y depth k circuits of size 2 (log s ) 1+ Ω (1 /k ) ). On the other hand, our separation is at least as large as 2 (log s ) c for an y c > 0 (see Section 2 for the precise separation). A r elated question is the comparativ e p o wer of n oncomm utativ e ABPs and noncomm u ta- tiv e formulas. Noncommutativ e form ulas ha ve p olynomial size n oncomm utativ e ABPs. Ho w- ev er, s O (log s ) is the b est kno wn formula size upp er b ound for noncommutati ve ABPs of size s . An in teresting question is wh ether we can p ro ve a separation r esult b et wee n noncommutat ive ABPs and form u las. W e note th at suc h a separation in the monotone case follo ws from an old result of S n ir [S80]. Prop osition 3. Consider two nonc ommuting variables { x 0 , x 1 } . L et L denote the se t of al l monomials of de gr e e 2 n with an e qual numb er of x 0 and x 1 , and c onsider the p olynomial E ∈ Q h x 0 , x 1 i , wher e E = P w ∈ L w . 1. Ther e is a monotone homo gene ous ABP for E of size O ( n 2 ) . 2. Any monotone formula c omputing E is of si ze n Ω (lg n ) . Pr o of . The fi rst part is directly from a standard O ( n 2 ) size DF A that accepts p r ecisely the set L = { w ∈ { x 0 , x 1 } 2 n | w has an equal num b er of x 0 ’s and x 1 ’s } . T h e second part follo ws from the fact that such a monotone formula would yield a comm u tativ e monotone formula for the sym m etric p olynomial of d egree n o ve r the v ariables y 1 , y 2 , · · · , y 2 n : this is obtained b y fi rst observing that th e form ula must compute homogeneous p olynomials at eac h gat e. F urthermore, w e can lab el eac h gate (and eac h leaf ) by a triple ( i, j, d ) where j − i + 1 = d is the degree of the homogeneous p olynomial computed at this gate such th at eac h monomial generated at this gate will o ccup y the p ositions from i to j in the output monomials co ntaining it. Hence we ha ve x 0 ’s at the leaf no des lab eled by triples ( i, i, 1) for all 2 n v alues of i . W e replace th e x 0 ’s lab eled ( i, i, 1) by y i and eac h x 1 b y 1. The resu lting form ula computes the symmetric p olynomial as claimed. Snir in [S 80 ] has sho wn a tigh t n Ω (log n ) lo w er b ound for monotone f orm ulas compu tin g the symm etric p olynomial of degree n o v er the v ariables y 1 , y 2 , · · · , y 2 n . 3 T o illustrate again the p o w er of constan t width circu its, w e note that there is, surp risingly , a width-2 circuit f or compu tin g th e p olynomial E . Prop osition 4. Ther e is a width-2 ci r cuit of size n O (1) for c omputing E i f the field F has at le ast cn 2 distinct e lements for some c onstant c . Pr o of Sketch . This is b ased on the well-kno wn Ben-Or trick [B80] for compu ting the symmetric p olynomials in depth 3. W e consid er the p olynomial g ( x 0 , x 1 , z ) = ( x 0 z 2 k +1 +1 + x 1 z + 1) 2 k +1 , where 2 k − 1 < n ≤ 2 k . ( z will eve ntually b e a scalar from F .) The co efficient of z (2 k +1 +1) n + n in g is pr ecisely the p olynomial E . F ollo wing Ben-Or’s argumen t, th e p roblem of reco v ering the p olynomial E can b e reduced to solving a system of linear equations w ith an inv ertible co efficien t matrix. Hence E can b e expressed as a sum E = P 2 n i =1 β i g ( x 0 , x 1 , z i ), where the z i s are all distinct field elemen ts. Th e terms β i g ( x 0 , x 1 , z i ) can b e ev aluated with one register using r ep eated squaring of x 0 z 2 k +1 +1 i + x 1 z i + 1. The second register is u sed as an accum u lator to compu te the su m of these terms. These observ ations are additional motiv ation for the study of constant- w id th arithmetic circuits. I n S ection 2 w e pr o v e lo wer b ound resu lts f or monotone constan t-width circuits. I n Section 3 we explore the connection b et wee n lo we r b oun ds and p olynomial iden tit y testing for constant-width commutativ e circuits analogous to the work of Dvir et al [DSY08 ]. 2 Monotone c onstan t width circuits In this s ection we study monotone constan t-width arith m etic circu its. W e p r o v e that they form an in fi nite hierarc hy . As a b y-pro du ct, the separat in g p olynomials that we construct yield the consequence that constan t-depth monotone arithmetic circuits to o form an infinite hierarc hy . All our p olynomials will b e comm utativ e, unless w e explicitly state otherwise. F or p ositive integ ers k and ℓ w e defin e a p olynomial P ℓ k on ℓ 2 k v ariables as follo ws: P ℓ 1 ( x 1 , x 2 , . . . , x ℓ 2 ) = P ℓ i =1 Q ℓ j = 1 x ( i − 1) ℓ + j P ℓ k +1 ( x 1 , x 2 , . . . , x ℓ 2 k +2 ) = P ℓ i =1 Q ℓ j = 1 P ℓ k ( x ( i − 1) ℓ 2 k +1 +( j − 1) ℓ 2 k +1 , . . . , x ( i − 1) ℓ 2 k +1 + j ℓ 2 k ) An easy in ductiv e argument from the definition giv es th e follo w in g. Lemma 2. The p olynomial P ℓ k is homo gene ous o f de gr e e ℓ k on ℓ 2 k variables and has ℓ ℓ k − 1 ℓ − 1 distinct monomials. By definition, P ℓ k can b e computed b y a depth 2 k monotone form ula of size O ( ℓ k ). F urther- more, we can argue that the p olynomials P ℓ k are the “hardest” p olynomials for constan t-depth circuits. W e make this more precise in the follo w ing observ ation. Prop osition 5. Given a depth k arithmetic ci r cuit C of size s , ther e is a pr oje ction r e duction fr om C to the p olynomial P ℓ k wher e ℓ = O ( s 2 k ) . Pr o of Sketch . W e sk etc h the easy argumen t. W e can transform C into a formula. F urthermore, w e can mak e it a la y ered formula with 2 k alternating + and × la yers suc h that the output ga te is a plus gate. T his form u la is of size at most s 2 k . Clearly , a pro jection r eduction (mapping v ariables to v ariables or constan ts) will transform P ℓ k to th is formula, for ℓ = O ( s 2 k ). It is easy to see the f ollo win g f r om the fact that a m onotone depth 2 k arithmetic circuit of size s can b e simulated by a monotone width 2 k circuit of size O ( s ). 4 Prop osition 6. F or any p ositive inte gers ℓ and k ther e is a monotone cir cuit of width 2 k and size O ( ℓ 2 k ) that c omputes P ℓ 2 k . W e no w state the main lo w er b oun d result. F or eac h k > 0 there is ℓ 0 ∈ Z + suc h that for all ℓ > ℓ 0 an y width k mon otone circuit for P ℓ k is of size Ω (2 ℓ ). W e w ill prov e this result b y indu ction on k . F or the indu ction argument it is con v enient to m ak e a stronger induction h yp othesis. F or a p olynomial f ∈ F [ X ], where X = { x 1 , x 2 , · · · , x n } let mon( f ) = { m | m is a n onzero monomial in f } . I.e. mon( f ) den otes th e set of n onzero monomials in the p olynomial f . Also, let v ar( f ) denote the set of v ariables o ccurrin g in the monomials in mon( f ). Similarly , for an arithmetic circuit C w e denote b y mon( C ) and v ar( C ) respective ly the set of nonzero monomials and v ariables o ccurrin g in the p olynomial computed b y C . W e call a la ye red circuit C minimal if there is n o smaller circuit C ′ of the same width s.t mon( C ) = mon( C ′ ). It can b e seen that f or an y monotone circuit C , th er e is a minimal circuit C ′ of the same w idth s.t mon( C ′ ) = mon( C ) and has the follo wing prop erties. – The only constan ts us ed in C ′ are 0 and 1. F urth er m ore, n o gate is ev er multiplied by a constan t. – By the minimalit y of C ′ ev ery no de g in C ′ has a path to the output no d e of C ′ . Hence, giv en an y no de g in C ′ computing a p olynomial p , there is a monomial m suc h that mon( m · p ) ⊆ mon( C ′ ). In particular, this implies that if C ′ computes a h omogeneous m ultilinear p olynomial, then p must b e a homogeneous m u ltilinear p olynomial. – If C ′ computes a h omogeneous multilinear p olynomial of degree d , and if a no de g in la y er i also computes a p olynomial p of degree d , then in la y er i + 1, ther e is a su m gate g ′ suc h that g is one of its c hildren. T h us, the gate g ′ computes a homogeneous multil in ear p olynomial p ′ of degree d such that m on( p ) ⊆ mon( p ′ ). In particular, mon( p ) ⊆ mon( C ′ ). W e call a minimal circuit satisfying th e ab ov e a go o d min imal circuit. W e no w sho w a useful prop erty of m inimal circu its C , which applies to circuits satisfying mon( C ) ⊆ P ℓ k , for all ℓ , k ≥ 1. Lemma 3. L et f = P ℓ i =1 P i b e a homo g ene ous mo notone p olynomial of de gr e e d ≥ 1 with var ( P i ) ∩ var ( P j ) = ∅ for al l i 6 = j . Given any go o d minimal cir cuit such th at mon ( C ) ⊆ mon ( f ) , we have the fol lowing: if a gate g in C c omputes a p olynomial p of de g r e e less than d , or a pr o duct of two such p olynomials, then var ( p ) ⊆ var ( P i ) for a unique i . Pr o of . F or an y p olynomial q ∈ F [ x 1 , x 2 , · · · , x n ] w e can define a bipartite graph G ( q ) as follo w s: one partition of the ve rtex set is m on( q ) and th e other partition v ar( q ). A pair { x, m } is an u ndirected edge if the v ariable x o ccurs in monomial m . It is clear that the graph G ( f ) is ju st th e d isjoin t un ion of all the G ( P i ). If the p olynomial p compu ted b y gate g is of degree d ′ < d , then , since C is go o d, there is a m onomial m of degree d ′ − d such that m on( m · p ) ⊆ mon( C ) ⊆ mon( f ). Th is imp lies that G ( m · p ) is a subgraph of G ( f ). On the other hand, G ( m · p ) is c learly seen to b e a connected graph. T his implies that, in fact, G ( m · p ) is a subgrap h of G ( P i ) for some i and hence, v ar( p ) ⊆ v ar( P i ) for a u nique i . This pr o v es the lemma in this case. Similarly , if p is a pro d uct of t wo p olynomials of degree less th an d , then G ( p ) is a conn ected graph, and by the ab o v e reasoning, it m us t b e the subgraph of some G ( P i ). Hence, the lemma follo w s. 5 W e n o w state and prov e a stronger low er b ound s tatemen t. It sho ws that P ℓ k is ev en hard to “app ro ximate” by p olynomial size width - k monotone circuits. Theorem 1. F or e ach k > 0 ther e is ℓ 0 ∈ Z + such that for al l ℓ > ℓ 0 and any width- k monotone cir cuit C such that mon ( C ) ⊆ mon ( P ℓ k ) and | mon ( C ) | ≥ | mon ( P ℓ k ) | 2 , the c i r cuit C is of size at le ast 2 ℓ 10 . Pr o of . Let us fix some notation: giv en i ∈ Z + and j ∈ [ w ], we denote by g i,j the j th no de in la y er i of C and by f i,j the p olynomial computed by g i,j . Also, give n a set of monomials M , w e say that a circuit C 1 c omputes M if mon( C 1 ) ⊇ M . Without loss of generalit y , w e assume thr oughout that C is a go o d minimal circu it. Th e pro of is by in duction on k . The case k = 1 is distinct and easy to handle. Thus, we consider as th e ind uction base case the case k = 2. C onsider a width tw o m onotone circuit C such that mon( C ) ⊆ mon( P ℓ 2 ) and | mon( C ) | ≥ | mon( P ℓ 2 ) | / 2 = ℓ ℓ +1 / 2. Let f denote the p olynomial computed by C . By Lemma 2 b oth f and P ℓ 2 are homogeneous p olynomials of degree d = ℓ 2 . W e wr ite the p olynomial P ℓ 2 as P ℓ i =1 P i , w h ere v ar( P i ) = { x ( i − 1) ℓ 3 +1 , . . . , x iℓ 3 } . Note that v ar( P i ) ∩ v ar( P j ) = ∅ for i 6 = j . L et f = P ℓ i =1 P ′ i where mon( P ′ i ) ⊆ mon( P i ) for eac h i . Since C is go o d and f is homogeneous, eac h gate of C compu tes only homogeneous p olynomials. Moreo ver, sin ce m on( C ) ⊆ mon( P ℓ 2 ) and v ar( P i ) ∩ v ar( P j ) = ∅ for i 6 = j , Lemma 3 implies that giv en any no de g in C that computes a p olynomial p of degree less than d or a pro du ct of such p olynomials satisfies v ar( p ) ⊆ v ar( P i ) for one i . Consider the lo w est la y er ( i 0 sa y) wh en the circuit C computes a degree d monotone p olynomial. W.l.o.g assu me that f i 0 , 1 is su c h a p olynomial. W e list some cru cial p r op erties satisfied by g i 0 , 1 and C . 1. By the minimalit y of i 0 , the no de g i 0 , 1 is a pro duct gate compu tin g the p ro duct of p oly- nomials of degree less than d . Hence, v ar( f i 0 , 1 ) ⊆ v ar( P i ) for exactly one i . W.l.o.g , we assume i = 1. Since deg( f i 0 , 1) = d and C is go o d, w e in f act hav e mon( f i 0 , 1 ) ⊆ mon( P 1 ). 2. Since d eg( f i 0 , 1 ) = d and C is go o d, w e kno w that th ere is a n o de g i 0 +1 ,j i 0 +1 that is a sum gate with g i 0 , 1 as c hild; g i 0 +1 ,j i 0 +1 computes a homogeneous p olynomial of degree d and mon( f i 0 +1 ,j i 0 +1 ) ⊇ mon( f i 0 , 1 ). Iterating this argumen t, w e see that th ere m u st b e a sequence of n o des g i,j i , for i > i 0 suc h that f or eac h i , g i,j i is a su m gate with g i − 1 ,j i − 1 as c hild, su c h that mon( f i 0 , 1 ) ⊆ mon( f i 0 +1 ,j i 0 +1 ) ⊆ mon( f i 0 +2 ,j i 0 +2 ) . . . , and eac h f i,j i is a homogeneous p olynomial of d egree d . W e assume, w.l.o.g, that j i = 1 for eac h i > i 0 . By the choice of i 0 , note that the no d e g i 0 , 2 either computes a p olynomial of degree less than d or computes a pro du ct of p olynomials of degree less than d . Hence, v ar( f i 0 , 2 ) ⊆ v ar( P i ) for some i . If i > 1, we assum e w.l.o.g. that v ar( p ) ⊆ v ar( P 2 ). Let us consider the circuit C with the v ariables in v ar( P 1 ) ∪ v ar( P 2 ) set to 0. The p olynomial computed b y the new circuit C ′ is no w f ′ = f − P ′ 1 − P ′ 2 = P ℓ i =3 P ′ i . Let q i,j denote the new p olynomial computed by the no de g i,j . Note that eac h q i 0 ,j is n o w a constan t p olynomial. Consider the monotone circuit C ′′ obtained f r om C ′ as follo ws : w e r emo v e all the gates b elo w lay er i 0 ; the gate g i 0 , 2 in la ye r i 0 is replaced by a pr o duct gate c × 1, wh ere c is the constan t it computes in C ′ ; from la yer i 0 on wa r ds, all no d es of the form g i, 1 are remo ved; in an y ed ge connecting n o des g i, 1 and g i +1 , 2 , the n o de g i, 1 is replaced by the constant 0. Clearly , C ′′ is a width 1 circuit. F or ease of notation, we will refer to the no des of C ′′ with th e same 6 names as th e corresp ond ing no des in C ′ . F or an y n o de g i, 2 in C ′′ ( i ≥ i 0 ), let q ′ i, 2 b e the p olynomial it n o w computes. Crucially , we observ e th e follo wing fr om th e ab o ve constru ction. Claim 2. F or e ach i ≥ i 0 , mon ( q ′ i, 2 ) ⊇ mon ( q i, 2 ) \ mon ( q i, 1 ) . W e no w fin ish the pro of of the base case. Define a sequence i 1 < i 2 < . . . < i t of la ye r s as follo ws: for eac h j ∈ [ t ], i j is the least i > i j − 1 suc h that mon( q i, 1 ) ) mon( q i j − 1 , 1 ), and mon( q i t , 1 ) = mon( f ′ ). Clearly , t is at most the size of C . Note that it must be the case that q i j , 1 = q i j − 1 , 1 + q i j − 1 , 2 . Hence, w e h a v e mon( q i j , 1 ) = mon( q i j − 1 , 1 ) ∪ mon( q i j − 1 , 2 ) = mon( q i j − 1 , 1 ) ∪ (mon( q i j − 1 , 2 ) \ mon( q i j − 1 , 1 )). By the ab ov e claim, the set m on( q i j − 1 , 2 ) \ mon( q i j − 1 , 1 ), which w e will denote by S j , can b e computed by a width-1 circuit. Th u s, mon( f ′ ) = mon( q i t , 1 ) = mon( q i 0 , 1 ) ∪ S t j = 1 S j , where eac h S j can b e compu ted b y a w id th-1 circuit. Since q i 0 , 1 is the zero p olynomial, w e hav e mon( f ′ ) = S t j = 1 S j . No w, consider any width-1 monotone circuit compu ting a set S ⊆ P ℓ 2 . It is easy to see that the set S computed must hav e a v ery restricted form. Claim 3. The set S is of the form mon ( p ) wher e p = ( P i ∈ X 1 x i ) Q j ∈ X 2 x j , and X 1 ∩ X 2 = ∅ . Clearly , as eac h set S j satisfies S j ⊆ v ar( P ′ i ) for some i , it can hav e at most ℓ 3 monomials. Therefore, if the m onotone circuit C is of ov erall size less than 2 ℓ then it can compute a p olynomial of th e form P ′ 1 + P ′ 2 + f ′ , where f ′ has at most 2 ℓ ℓ 3 monomials. Sin ce | mon( P ′ i ) | ≤ | mon( P i ) | = ℓ ℓ for eac h i , w e hav e for suitably large ℓ | mon( C ) | ≤ 2 ℓ ℓ + 2 ℓ ℓ 3 < 3 ℓ ℓ < ℓ ℓ +1 2 = | mon( P ℓ 2 ) | 2 and the b ase case follo ws. The induction step. Consider an y monoto ne circuit ˆ C of width k − 1 su c h that mon( ˆ C ) ⊆ mon( P ℓ k − 1 ) and | mon( ˆ C ) | ≥ | mon( P ℓ k − 1 ) | / 2. As induction h yp othesis w e assume that ˆ C m ust b e of size at least 2 ℓ / 10. Let P ℓ k = P ℓ i =1 P i , with v ar( P i ) = { x ( i − 1) ℓ 2 k +1 +1 , . . . , x iℓ 2 k +1 } as in the base case. By definition, th e ℓ v ariable sets v ar( P i ) are m utually d isjoin t and eac h P i has degree d = ℓ k . It is con v enient to also write P i = Q ℓ j = 1 Q ij , where eac h Q ij is of t yp e P ℓ k − 1 . W e hav e v ar( Q ij ) = { x ( i − 1) ℓ 2 k +1 +( j − 1) ℓ 2 k +1 , . . . , x ( i − 1) ℓ 2 k +1 + j ℓ 2 k } . W e start b y considering any width k − 1 circuit ˆ C of size less than 2 ℓ / 10 suc h that mon( ˆ C ) ⊆ mon( P ℓ k ). F or any i ∈ [ ℓ ], by fixin g all the v ariables outside v ar( P i ) to 0, w e obtain a wid th k − 1 circuit ˆ C i of the same size s.t mon( ˆ C i ) ⊆ mon( P i ). F urther, by setting all the v ari- ables outside v ar( Q ij ) to 1 f or some j ∈ [ ℓ ], we obtain a circuit ˆ C ij s.t mon( ˆ C ij ) ⊆ mon( Q ij ). By the induction h yp othesis, we see that | mon( ˆ C ij ) | ≤ | mon( Q ij ) | / 2. Clearly mon( ˆ C i ) ⊆ mon( ˆ C i 1 ) × mon( ˆ C i 2 ) × . . . × mon( ˆ C iℓ ). Therefore, | mon( ˆ C i ) | ≤ Q j | mon( ˆ C ij ) | ≤ | mon( P i ) | / 2 ℓ . Finally , as mon( ˆ C ) = S i mon( ˆ C i ), | mon( ˆ C ) | ≤ P i | mon( ˆ C i ) | ≤ | mon( P ℓ k ) | / 2 ℓ . W e ha ve estab- lished the follo wing claim. Claim 4. F or any width k − 1 cir cui t ˆ C of size less than 2 ℓ / 10 such that mon ( ˆ C ) ⊆ mon ( P ℓ k ) , we have | mon ( ˆ C ) | ≤ | mon ( P ℓ k ) | 2 ℓ . 7 F or the ind uction step, consider an y monot one w idth- k circuit C suc h that mon( C ) ⊆ mon( P ℓ k ) and of size at most 2 ℓ / 10. W e will show that | mon( C ) | < | mon ( P ℓ k ) | / 2. W.l.o.g, we can assume that C is a go o d minimal circuit. Let f denote the p olynomial compu ted by C ; w e write f = P ℓ i =1 P ′ i , wh ere mon( P ′ i ) ⊆ mon( P i ) for eac h i . As in the base case, let i 0 b e the fir st la y er wh ere a p olynomial of degree d is computed. W.l.o.g. w e can assume th at f i 0 , 1 is suc h a p olynomial. By the minimalit y of i 0 , the no de g i 0 , 1 m ust b e a p ro duct node with c hildr en computing p olynomials of d egree less th an d . This imp lies, as in the base case, th at v ar( f i 0 , 1 ) ⊆ v ar( P i ) for a un iqu e i . W.l.o.g. w e assume that i = 1. As b efore, we can fi x a sequence of nod es g i,j i for eac h i > i 0 suc h that g i,j i is a sum gate with g i − 1 ,j i − 1 as a c hild. It is easily seen th at mon( f i 0 , 1 ) ⊆ mon( f i 0 +1 ,j i 0 +1 ) ⊆ mon( f i 0 +2 ,j i 0 +2 ) . . . , and eac h f i,j i computes a h omogeneous p olynomial of degree d . Renaming no des if necessary , w e assu me j i = 1 for all i . No w consid er f i 0 ,j for j > 1. By the m inimalit y of i 0 , we see that eac h f i 0 ,j is either a p olynomial of degree less than d or a pro d uct of t wo su c h p olynomials. Hence, v ar( f i 0 ,j ) ⊆ v ar( P s ) for some s ∈ [ ℓ ]. Th us, there is a set S ⊆ [ ℓ ] s.t | S | = k ′ < k suc h that S j > 1 v ar( f i 0 ,j ) ⊆ S s ∈ S v ar( P s ). Without loss of generalit y , we assu me that th ose s ∈ S that are greater than 1 are among { 2 , 3 , . . . , k } . Consider the circuit C ′ obtained when eac h of the v ariables in S s ∈ [ k ] v ar( P s ) is set to 0. Let q i,j b e the p olynomial computed by g i,j in C ′ . The p olynomial compu ted by C ′ is just f ′ = f − P s ∈ [ k ] P ′ s . Note th at q i 0 ,j is no w simp ly a constant for eac h j , and that the size of C ′ is at most the size of C whic h by assumption is b oun ded by 2 ℓ / 10. Using this size b ound w e will argue that C ′ cannot compute to o man y monomials. W e no w mo dify C ′ as follo ws: w e remo v e all the gates b elo w la yer i 0 ; eac h gate g i 0 ,j with j > 1 is r eplaced by a pro d uct gate of th e form c × 1 wh ere c is the constant g i 0 , 1 computes in C ′ ; from la y er i 0 on wa r ds, all n o des of the form g i, 1 are r emo v ed; in an y edge connecting no des g i, 1 and g i +1 ,j for j > 1, the no d e g i, 1 is replaced by the constan t 0. Call this new circuit C ′′ . Clearly , C ′′ has size at most the size of C and w idth at most k − 1. F or ease of notation, w e will refer to the no d es of C ′′ with the same names as the corresp onding no des in C ′ . F or an y no d e g i,j in C ′′ ( i ≥ i 0 and j > 1), let q ′ i,j b e the p olynomial it no w compu tes. As in th e base case, we observe the follo win g from the ab o ve construction. Claim 5. F or e ach i ≥ i 0 and e ach j > 1 , mon ( q ′ i,j ) ⊇ mon ( q i,j ) \ mon ( q i, 1 ) . Using this, w e show that the circuit C ′ w as essen tially ju st u sing the gates g i, 1 to store the sum of p olynomials compu ted u sing wid th k − 1 circuits. Construct a sequence of la yers i 1 < i 2 < . . . < i t in C ′ as follo ws : for eac h j ∈ [ t ], i j is the least i > i j − 1 suc h that mon( q i, 1 ) ) mon( q i j − 1 , 1 ), and mon( q i t , 1 ) = mon( f ′ ). S urely , t is at most the size of C ′ . No w, fix any i j for j ≥ 1. Clearly , it m ust b e the case that q i j , 1 = q i j − 1 , 1 + q i j − 1 ,s for some s > 1; therefore, we hav e mon( q i j , 1 ) ⊆ mon( q i j − 1 , 1 ) ∪ (mon( q i j − 1 ,s ) \ mon( q i j − 1 , 1 )). Denote the set mon( q i j − 1 ,s ) \ mon( q i j − 1 , 1 ) by S j . S in ce the ab o ve h olds for all j , and mon( q i j − 1 , 1 ) = mon( q i j − 1 , 1 ), w e see th at m on ( f ′ ) = mon( q i t , 1 ) ⊆ mon( q i 0 , 1 ) ∪ S j S j = S j S j , sin ce q i 0 , 1 is the zero p olynomial. W e will no w analyze | S j | for eac h j . By the ab ov e claim, there is a width k − 1 circuit C ′′ of size at most th e size of C suc h that S j ⊆ mon( C ′′ ) ⊆ P ℓ k . If the size of C (and h ence that of C ′ and C ′′ ) is at most 2 ℓ / 10, it f ollo ws from Claim 4 that | S j | ≤ | mon( P ℓ k ) | / 2 ℓ . Hence, w e see that | mon( f ′ ) | ≤ t | mon ( P ℓ k ) | / 2 ℓ , wh ich is at most | mon( P ℓ k ) | / 10. But we kn o w that the p olynomial f computed by the circuit C is of the f orm f ′ + P i ∈ [ k ] P ′ i , w h ere | mon( P ′ i ) | ≤ 8 | mon( P i ) | = | mon( P ℓ k ) | /ℓ . T herefore, | mon( f ) | ≤ k ℓ | mon( P ℓ k ) | + | mon( f ′ ) | ≤ | mon( P ℓ k ) |  k ℓ + 1 10  < | mon( P ℓ k ) | 2 for large enough ℓ . Th is p ro v es th e indu ction step. F or k ∈ Z + and c > 0 let Depth k ,c and Width k ,c denote the set of families { f n } n> 0 of monotone p olynomials f n ∈ R [ x 1 , x 2 , . . . , x n ] computed by c · n c -sized monotone circuits of depth k and w id th k resp ectiv ely . F or k ∈ Z + , let Depth k = S c> 0 Depth k ,c and Width k = S c> 0 Width k ,c . T h us, Depth k and Width k denote the set of families of monotone p olynomials computed by p oly( n )-sized m onotone circuits of depth k and wid th k resp ectiv ely . Note that, for eac h k ∈ Z + w e ha v e Depth k ⊆ Width k . Mo reov er, f rom the definition of P ℓ k , we see that the family { P ⌊ n 1 / 2 k ⌋ k } n ∈ Depth 2 k . Finally , in Theorem 1 we ha ve sho wn that the f amily { P ⌊ n 1 / 2 k ⌋ k } n / ∈ Width k , for constan t k . Hence, we hav e the follo win g corollary of Theorem 1. Corollary 1. F or any fixe d k ∈ Z + , Width k ( Width 2 k and Depth k ( Depth 2 k . Theorem 1 can also b e used to giv e a separation b et wee n the p o we r of circuits of width (resp ectiv ely , depth) k an d k + 1 for infin itely man y k . W e n o w state this separation. F or an y k ∈ N and any fu nction f : N → N , let us denote by f k the k -th iter ate of f , i.e the fu n ction f ◦ f ◦ . . . ◦ f | {z } k times . Giv en non-decreasing functions f , g : N → N , call f a sub 1 /k -th iter ate of g if f k ( n ) < g ( n ), for large enough n (closely related notions h a v e b een defi ned in [Sz61] and [RR97]). It can b e verified that sub 1 /k -th iterates of exp onentia l fun ctions can gro w fairly quic kly: for example, for any ε > 0 and an y k , c ∈ N , the fu n ction 2 (log n ) c is a sub 1 /k -th iterate of 2 n ε . W e now state the pr ecise separation that can b e inferr ed from the ab o ve theorem. F or any k , n ∈ N with k ≥ 2 and an y p olynomial p ∈ R [ x 1 , x 2 , . . . , x n ], let w k ( p ) (resep ctive ly d k ( p )) denote the size of th e smallest m onotone width k (resp ectiv ely d epth k ) circuit that compu tes p . Corollary 2. Ther e is an absolute c onstant α > 0 such th at the fol lowing holds. Fix a ny k ∈ N wher e k ≥ 2 . Also, fix any non-de cr e asing function f : N → N that is a sub 1 /k -th iter ate of 2 αn 1 / 2 k . Then, for lar ge enough n , ther e is a monotone p olynomial p ∈ R [ x 1 , x 2 , . . . , x n ] such that for some k ′ , k ′′ ∈ { k , k + 1 , . . . , 2 k − 1 } , w k ′ ( p ) ≥ f ( w k ′ +1 ( p )) and d k ′′ ( p ) ≥ f ( d k ′′ +1 ( p )) . Pr o of . Let p denote th e monotone p olynomial P ⌊ n 1 / 2 k ⌋ k ∈ R [ x 1 , x 2 , . . . , x n ]. Th eorem 1 tells us that w k ( p ) = Ω (2 ⌊ n 1 / 2 k ⌋ ). T o obtain a lo wer b ound on d k ( p ), note th at an y p olynomial computed by a circuit of size s and depth k can b e computed by a wid th k circuit of size O ( s k ); th is tells u s that d k ( p ) = 2 Ω ( n 1 / 2 k ) . Hence, th er e is some constan t β > 0 s uc h that min { w k ( p ) , d k ( p ) } ≥ 2 β n 1 / 2 k , for large enough n . By definition, p = P ⌊ n 1 / 2 k ⌋ k has a depth 2 k circuit of size O ( n ), i.e d 2 k ( p ) = O ( n ). Prop o- sition 6 tells us that w 2 k ( p ) = O ( n ) also. Hence, for some constan t γ > 0 and large enough n , we h a v e max { w 2 k ( p ) , d 2 k ( p ) } ≤ γ n . The ab o v e statement s imply that w k ( p ) ≥ g ( w 2 k ( p )) and d k ( p ) ≥ g ( d 2 k ( p )), where g ( n ) = 2 αn 1 / 2 k for s ome constan t α > 0 and n is large enough. No w, fi x any non-decreasing fun ction 9 f : N → N that is a s ub 1 /k -th iterate of g . W e s ee that w k ( p ) ≥ g ( w 2 k ( p )) > f k ( w 2 k ( p )) for large enough n ; clearly , this implies that for some k ′ ∈ { k , k + 1 , . . . , 2 k − 1 } , w e m ust ha ve w k ′ ( p ) ≥ f ( w k ′ +1 ( p )). Similarly , there is also a k ′′ ∈ { k , k + 1 , . . . , 2 k − 1 } such that d k ′′ ( p ) ≥ f ( d k ′′ +1 ( p )). Similar corollaries hold for noncomm utativ e circuits to o. W e define the p olynomial P ℓ k in exactly the same w a y in the noncomm utativ e s etting. Note that an y monotone b ound ed width noncomm utativ e circuit computing P ℓ k automatica lly giv es us a monotone co mmutativ e circuit of the s ame size and wid th computing the commuta tive ve r sion of P ℓ k . Hence, the low er b ound of Th eorem 1 also holds for noncommutat ive width- k circuits. F or k ∈ Z + , let ncDepth k and ncWidth k denote the set of families of monotone p olynomials { f n ∈ R h x 1 , x 2 , . . . , x n i | n ∈ Z + } computed by p oly( n )-sized monotone (noncomm utativ e) circuits of depth k and width k resp ectiv ely . Analogous to the comm utativ e case, we obtain th e follo w ing. Corollary 3. F or any fixe d k ∈ Z + , ncWidth k ( ncWidth 2 k and ncDepth k ( ncDe pth 2 k . And finally , w e ob s erv e that the separations b et wee n width and depth k and k + 1 that hold in the comm utativ e monotone ca se also hold in the noncommutati ve m onotone case. Define, for an y k , n ∈ N w ith k ≥ 2 and an y p olynomial p ∈ R h x 1 , x 2 , . . . , x n i , let ncw k ( p ) (resep ctiv ely ncd k ( p )) d enote the size of the smallest monotone width k (resp ectiv ely depth k ) circuit that computes p . W e ha ve th e follo wing. Corollary 4. Ther e is an absolute c onstant α > 0 such th at the fol lowing holds. Fix a ny k ∈ N wher e k ≥ 2 . Also, fix any non-de cr e asing function f : N → N that is a sub 1 /k -th iter ate of 2 αn 1 / 2 k . Then, f or lar g e enough n , ther e is a monotone p olynomial p ∈ R h x 1 , x 2 , . . . , x n i such that for some k ′ , k ′′ ∈ { k , k + 1 , . . . , 2 k − 1 } , ncw k ′ ( p ) ≥ f ( ncw k ′ +1 ( p )) and ncd k ′′ ( p ) ≥ f ( ncd k ′′ +1 ( p )) . 3 Iden tity testing for constan t width circuits In this section we study p olynomial iden tity testing for constan t-width comm utativ e circuits. Impagliazzo and Kabanets [KI03] s ho we d that derandomizing p olynomial identit y testing is equiv alen t to proving arithmetic circuit lo wer b oun d s. Sp ecifically , assum ing that there are explicit p olynomials that require sup erp olynomial size arithmetic circuits, they use these p olynomials in a Nisan-Wigderson typ e “arithmetic” pseud orandom generator that can b e used to derandomized p olynomial iden tit y testing. Th is idea w as refined by Dvir et al [DSY08] to show that if there are explicit p olynomials that r equire sup erp olynomial size constan t-depth arithmetic circuits then p olynomial identit y testing for constan t-depth arith m etic circuits can b e derandomized (the precise statement in volv es the depth p arameter exp licitly [DSY08]). In this section we prov e a similar result showing th at hardness for constant -wid th arith- metic circuits yields a derandomization of p olynomial identi ty testing for constan t-width cir- cuits. W e sa y that a family of m ultilinear p olynomials { P n } n> 0 where P n ( x ) ∈ F [ x 1 , · · · , x n ] is explicit if th e co efficien t o f eac h monomial m of the p olynomial P n can b e computed in time 2 n O (1) . Recall the notion of a staggered arithm etic circuit (Definition 2). Lemma 4. L et f ∈ F [ x 1 , x 2 , · · · , x n ] of de gr e e m b e c ompute d by a stagger e d arithmetic c i r cuit of size s and width w . Then H i ( f ) (the i th homo g ene ous c omp onent of f ) c an b e c ompute d by 10 a stagger e d cir c u it of size p oly( s, m ) and width w + O (1) , pr ovide d F has at le ast d eg( f ) + 1 many elements. Pr o of . Define a new p olynomial g ( x, z ) ∈ F [ x 1 , x 2 , · · · , x n , z ] as g ( x, z ) = f ( x 1 z , x 2 z , · · · , x n z ). W e can write f ( x 1 z , x 2 z , · · · , x n z ) = P m i =0 H i ( f ) z i where m = deg ( f ) and H i ( f ) is the i th homogeneous part of f . L et { z 0 , z 1 , · · · , z m } b e m + 1 distinct fi eld elemen ts. Consider the matrix M d efined as M =     1 z 0 z 2 0 · · · z m 0 1 z 1 z 2 1 · · · z m 1 · · · · · · · · · · · · 1 z m z 2 m · · · z m m     . W e hav e the sys tem of equations M ( H 0 ( f ) , H 1 ( f ) , · · · , H m ( f )) T = ( g ( x , z 0 ) , g ( x, z 1 ) , · · · , g ( x , z m )) T . Since M is inv er tib le, it follo ws that th ere are scalars a ij ∈ F s uc h that H i ( f ) = P m j = 0 a ij g ( x, z j ). Since f ( x ) h as a width w circuit of size s , g ( x, z ) clearly has a (staggered) circuit of width w + O (1) of size O ( s ). It follo ws easily f r om the ab o v e equation for H i ( f ) that eac h H i ( f ) has a circuit of width w + O (1) and size O ( ms ). Lemma 5. L et P ( x 1 , x 2 , · · · , x n , y ) b e a p olynomial, over a sufficiently lar ge field F , c ompute d by a width w stagger e d cir cuit of size s . Supp ose the maximum de gr e e of y in P is r . Then for e ach j the j th p artial derivative ∂ j P ∂ y j c an b e c ompute d by a stagger e d cir cuit of width w + O (1) and size ( r s ) O (1) . Pr o of . Let P ( x, y ) = P r i =0 C i ( x ) y i . As in Lemma 4 eac h C i ( x ) can b e computed b y a wid th w + O (1) staggered circuit of size O ( r s ). Clearly , for eac h j the p olynomial ∂ j P ∂ y j can b e written as ∂ j P ∂ y j = r X i = j a ij C i ( x ) y i − j , for a ij ∈ F , where a ij are fi eld element s that dep end only up on j . Therefore, w e can easily giv e a staggered circuit of size O ( r 2 s ) and w idth w + O (1) for eac h p olynomial ∂ j P ∂ y j . The follo w ing lemma is pro ve d in [DSY08]. F or any p olynomial g ∈ F [ x 1 , x 2 , · · · , x n ] let H ≤ k ( g ) = P k i =0 H i ( g ). Lemma 6. [DSY08 , Lemma 3.2] L et P ∈ F [ x 1 , x 2 , · · · , x n , y ] and deg y ( P ) = r . Supp ose f ∈ F [ x 1 , x 2 , · · · , x n ] suc h that P ( x, f ( x )) = 0 and ∂ P ∂ y ( 0 , f (0)) is e qual to ξ 6 = 0 . L et P ( x, y ) = P r i =1 C i ( x ) y i . Then for e ach k ≥ 0 ther e i s a p olynomial Q k ∈ F [ y 0 , y 1 , · · · , y r ] such that H ≤ k ( f ) = H ≤ k ( Q k ( C 0 , C 1 , · · · , C r )) . Using the ab o v e lemmata w e prov e our fir st theorem. Theorem 6. L et P ∈ F [ x 1 , x 2 , · · · , x n , y ] and deg y ( P ) = r ≥ 1 such that P has a stag- ger e d cir cuit of size s and width w . Supp ose that P ( x, f ( x )) = 0 for some p olynomial f ∈ F [ x 1 , x 2 , · · · , x n ] with deg ( f ) = m . Then f has a stagger e d ci r cuit of size p oly( s, ( m + r ) r ) and width w + O (1) if char ( F ) > r and F is suffici e ntly lar ge. 11 Pr o of . First w e argue that we can assume w.l.o.g. , as in Dvir et al [DSY08], that ∂ P ∂ y ( 0 , f (0)) = ξ 6 = 0. If ∂ P ∂ y ( x, f ( x )) ≡ 0 w e can rep lace P b y ∂ P ∂ y . Since char ( F ) > r it is easy to see that there exists j : 1 ≤ j ≤ r s uc h that ∂ j P ∂ y j ( x, f ( x )) 6≡ 0. Hence, w e can assume ∂ P ∂ y ( x, f ( x )) 6≡ 0. Therefore, there is an a ∈ F n suc h that ∂ P ∂ y P ( a, f ( a )) 6 = 0. W e can assume that a = 0 by appropriately shifting P as in [DSY08]. Let P ( x, y ) = r X i =1 C i ( x ) y i . By L emma 6 there is a p olynomial Q k ∈ F [ y 0 , · · · , y r ] suc h that H ≤ k ( f ) = H ≤ k ( Q k ( C 0 , C 1 , · · · , C r )) for eac h 0 ≤ k ≤ m . Putting k = m and letting Q m = Q w e ha ve f ( x ) = H ≤ m ( Q ( C 0 , C 1 , · · · , C r )). Let y ∗ = ( C 0 (0) , · · · , C r (0)) and deg( Q ) = M . Define I M = { ( α 0 , α 1 , · · · , α r ) | α i ∈ N , P α i ≤ M } . By expand ing the p olynomial Q at the p oint y ∗ w e get Q ( y ) = P α ∈ I M Q α Q r i =0 ( y i − y ∗ i ) α i . Th u s, we can write f ( x ) = H ≤ m [ X α ∈ I M Q α r Y i =0 ( C i ( x ) − C i (0)) α i ] . As the constant term of C i ( x ) − C i (0) is zero, if we consid er Q r i =1 ( C i ( x ) − C i (0)) α i for s ome α w ith P i α i > m then we w ill get monomials of degree more than m whose n et con tribution to f ( x ) must b e zero. Hence we can write f ( x ) as f ( x ) = H ≤ m [ X α ∈ I m Q α r Y i =0 ( C i ( x ) − C i (0)) α i ] , where I m = { ( α 0 , α 1 , · · · , α r ) | α i ∈ N , P α i ≤ m } . Clearly , | I m | ≤ ( m + r ) r . No w, the p olynomial Q r i =0 ( y i − y ∗ i ) α i has a simp le O (1)-width circuit C ′ . W e can compute Q r i =0 ( C i ( x ) − C i (0)) α i b y plugging in the stagg ered width w + O (1) circuit for C i ( x ) (as obtained in Lemma 5) wh ere y i is input to C ′ . T h us, we obtain a circuit of width w + O (1) for P α ∈ I m Q α Q r i =0 ( C i ( x ) − C i (0)) α i that is of s ize p olynomial in s and ( m + r ) r . By Lemma 4 w e can compu te its h omogeneous comp onents and their partial sums with constan t increase in width. Pu tting it together, it follo w s that f ( x ) can b e computed in width w + O (1) of size p olynomial in s and ( m + r ) r . W e apply Th eorem 6 to prov e the main result of this section. Theorem 7. Ther e is a c onstant c 1 > 0 so that the fol lowing holds. Supp ose ther e is an explicit se quenc e of multiline ar p olynomials { P m } m> 0 wher e P m ( x ) ∈ F [ x 1 , · · · , x m ] and P m c annot b e c ompute d by arithmetic cir c uits of width w + c 1 and size 2 m ǫ , for c onstants w ∈ Z + and ǫ > 0 . Then, for any c onstant c 2 > 0 , ther e is a deterministic 2 (log n ) O (1) · b O (1) time algorithm that, when given as input a cir cuit C of size n O (1) and width w c omputing a p olynomial f ( x 1 , x 2 , . . . , x n ) of maximum c o efficient size b , w ith e ach variable of individual de gr e e at most (log n ) c 2 , che cks if the p olynomial c ompute d by C is identic al ly zer o, assuming that the field F is sufficiently lar ge and char ( F ) > (log n ) c 2 . 12 Pr o of Sketch . The ov erall construction is based on the Nisan-Wigderson construction as applied in Impagliazzo -Kabanets [KI03] and Dvir et al [DSY08]. Hence it s uffices to sk etc h the argumen t. 1. Let m = (log n ) c ′ ( ǫ,c 2 ) and ℓ = (log n ) c ′′ ( ǫ,c 2 ) where c ′′ is su itably larger than c ′ . 2. Construct the Nisan-Wigderson design S 1 , · · · , S n ⊂ [ ℓ ] su c h that | S i | = m f or eac h i and | S i ∩ S j | ≤ log n . 3. Consider the p olynomial F ( y 1 , y 2 , · · · , y ℓ ) = C ( P m ( y | S 1 ) , P m ( y | S 2 ) , · · · , P m ( y | S n )). F or an y in put y ∈ F ℓ w e can ev aluate F b y ev aluating P m ( y | S i ) for eac h i and then ev aluating C on the resulting v alues. Since the P m are explicit p olynomials and | S i | has p olylog( n ) size we can ev aluate P m in time 2 (log n ) O (1) . 4. W e test if F ( y ) ≡ 0 usin g a brute-force algorithm based on the Sch wartz-Zipp el lemma. Consider a fin ite set S ⊆ F , suc h that | S | is more than deg( F ). Ch ec k if F ( a ) ≡ 0 for all a ∈ S ℓ in time n O ( ℓ ) . If all the tests return ed zero then return C ≡ 0 otherwise C 6≡ 0. The pro of of correctness is exactly as in [KI03,DSY08]. Assum in g the algorithm fails, after h ybr idization and fi xing v ariables in C , we get a nonzero p olynomial F 2 of the f orm F 2 ( y | S i +1 , x i +1 ) = F 1 ( P m ( y | S 1 ∩ S i +1 ) , P m ( y | S 2 ∩ S i +1 ) , · · · , P m ( y | S i ∩ S i +1 ) , x i +1 ) . where F 1 ( x 1 , x 2 , . . . , x i +1 ) can b e computed by a wid th w circuit of size p oly( n ) and F 2 ( y | S i +1 , P m ( y | S i +1 )) ≡ 0. Note that the m u ltilinear p olynomials P m ( y | S j ∩ S i +1 ) dep end only on log n v ariables and hence, they can be computed using brute force width-2 stag- gered circuits of size O ( n log n ). Also, b y Lemma 1, we kno w that F 1 can b e computed by a staggered circuit of size p oly( n ) and w idth at most w + 1. Putting the ab ov e circuits to- gether, it is easy to see that F 2 can b e computed by a staggered circu it C ′ of size at most p oly( n ) .n log n = p oly( n ) and width w + O (1). No w, b y app lying Theorem 6 to C ′ , w e get a circuit of width w + O (1) to compute P m con tradicting the h ard ness assum ption. Finally , w e obs er ve that the follo wing analogue of [KI03, Theorem 4.1] holds for b ounded width circuits. The pro of is in th e app endix. Prop osition 7. One of the fol lowing thr e e statements is false. 1. NEXP ⊆ P / p oly . 2. The Permanent p olynomial is c omputable by p olynomial size width w arith metic cir cuit over Q , wher e w is a c onstant. 3. The identity testing pr oblem for b ounde d width arithmetic cir cu its over Q i s in NSUBEXP . Ac kno wledgements W e w ould like to thank Amir Y ehuda y off f or p ointing out the separation in [R Y09] and for man y v aluable commen ts. References B80. Michael Ben-Or. Unpub lished notes. BC92. Michael Ben-Or, Richard Cleve. Computing Algebraic F ormula s Using a Constant Number of Registers. S IAM J. Comput. 21(1): 54-58 (1992). 13 DSY08. Zeev Dvir, Am ir Shpilka, Ami r Ye huda yoff. Hardness-randomness tradeoffs for b ounded depth arithmetic circuits. Pro c. Sy mp. on Theory of Computing, 2008: 741-748. JR09. Maurice Jansen and B.V.Ragha vendra Rao. Simulation of arithmetical circuits by branching programs preserving constant width and syntactic multilinearit y . In CSR, 2009. T o A p p ear. KI03. V. Kabanets and R. Imp agliazzo. Derandomization of p olynomial identit y tests means proving circuit lo w er b ound s. In Pro c. of the thirt y-fi fth annual A CM Sym. on Theory of computing., pages 355-364, 2003. LMR07. Nut an Lima ye, Mee na Mahaja n, and B. V. Ragha ve ndra Rao. Arithmetizing classes around NC1 and L. T ec hn ical Rep ort 087, Electronic Colloquium on Computational Complexity (ECCC), 2007. Preliminary version in ST ACS 2007 , LNCS vol. 4393 pp. 477488. MR08. Meena M ahajan and B. V. Ragha vendra Rao. Arithmetic circuits, syntactic multilineari ty , and the limitations of skew formulae. In MFC S , pages 455 466, 2008. N91. N. Ni san. Low er b ounds for non-commutativ e computation. In Proc. of the 23rd annual ACM Sym. on Theory of computing., pages 410-418, 1991. RS05. R. Raz and A. Shp ilka. Deterministic p olynomial identit y testing in non comm ut ative models. Com- putational Complexit y ., 14(1):1-19, 2005. R Y09. Ran Raz and Amir Yeh uda yoff. Low er Bound s and S ep arations for Constant Dept h Multilinear Circuits. Compu t ational Complexity 18(2) : 171-207, 2009. RR97. Alexande r A. Razbor ov, Steven Rudich. N atural Pro ofs. J. Comput. Syst. Sci. 55(1): 24-35, 19 97. S80. Marc Snir. On the Size Complexit y of Monotone F ormulas. Proc. 7th Intl. Collo quium on Algorithms Languages and Programming, 1980: 621-631. Sz61. G. Szekeres. F ractional iteration of exp onentiall y gro wing fun ctions. J. Austral. Math. So c. 2 (1961/62 ) , 301-320. 14 App endix Pro of Sk etc h of Lemma 1 The c ircu it C ′ is co n structed b y sh o wing ho w to compute, for i ≥ 1, the p olynomials computed in la y er i + 1 of C fr om the p olynomials computed in the i th la ye r in C in a staggered fash ion, u s ing at most w lay ers of width at most w + 1. Equiv alen tly , it amounts to designing a s traigh t-line p r ogram w ith w + 1 registers s uc h that: initially , w of the registers con tain th e p olynomials computed in the w no d es of the i th la y er. In th e end, w of the w + 1 registers will cont ain the p olynomials computed at the i + 1 st la y er of C . Note that this is trivial for i = 2 since all no des in lay er 2 ha ve only lea v es as c hildren. F or some i > 1, let the U d en ote the n o des of C in lay er i an d V the no des of C in la y er i + 1. W e defin e an un directed multig r aph G corresp onding to la yers i and i + 1 as follo ws: its v ertex set V ( G ) is U . F or eac h gate v ∈ V in circuit C that tak es inpu ts u 1 , u 2 ∈ U we include the edge { u 1 , u 2 } in E ( G ). Notice that if u 1 = u 2 w e add a self-lo op to E ( G ). F ur thermore, if v ∈ V tak es one in put as a u ∈ U and th e other in p uts is a constan t or a v ariable, then to o we add a self-loop at vertex u . Finally , if b oth inputs to v are constan ts and/or v ariables, there is no edge in G corresp ond ing to v . W e note some prop erties of th is graph G . 1. W e ha ve | V ( G ) | ≤ w and | E ( G ) | + | V ′ | ≤ w , where V ′ is the set of those n o des in V that tak e only constants and/or v ariables as in p ut. 2. Eac h v ertex u ∈ V ( G ) corresp onds to a p olynomial p u computed at u in th e i th la y er. Eac h edge e ∈ E ( G ) is defined b y some v ∈ V and it corresp onds to the p olynomial q e computed at v . In order to compute the p olynomial corresp ond ing to e w e n eed the p olynomials corresp ond ing to its end p oints. W e ha v e w + 1 registers, w of which conta in the p olynomials p u , u ∈ U . O u r goal is to compute the p olynomials q e , e ∈ E ( G ) using these registers. Using the graph stru cture of G , w e w ill give an ordering of the edges E ( G ). If w e compute the p olynomials q e in that order then for ev ery q e computed we will ha v e a free r egister to store q e (when we do not need a p olynomial p u for f urther compu tation, w e can free the register con taining p u ). Th u s, w hat we wan t to do is compute an orderin g of th e edges E ( G ) 1 from the ve rtex set V ( G ). W e pic k edges from E ( G ) one b y one. Wh en e ∈ E ( G ) is p ic k ed, we delete e from th e graph and store q e in a f r ee register. Cr ucially , n ote that when a verte x u ∈ V ( G ) b ecomes isolated in this p ro cess the p olynomial p u is not r equired f or further computation and the register cont aining p u is freed. Thus, at any p oin t of time in this edge-deletion pro cedu re, the n umb er of r egisters required is equal to the sum of the num b er of edges remo ve d from G and the n u m b er of non-i solate d ve r tices left in G . The edge p ic king pro cedure w orks as follo ws. W e break G into its connected comp onents G 1 ∪ G 2 ∪ . . . ∪ G s ∪ G s +1 ∪ . . . ∪ G s + t , where G 1 , G 2 , . . . , G s are the acyclic comp onents and G s +1 , . . . , G s + t ha ve cycles. W e first compu te the edges of G 1 , and then those of G 2 , and so on. A t the end, w e compute the p olynomials corresp onding to the n o des in V ′ . Eac h connected comp onen t G i is pro cessed as follo ws: if there is an edge e in G i that is not a cut edge, w e p ic k the edge e and d elete it f r om th e graph ; otherwise, since ev ery edge of G i is a cut edge, G i m ust b e a tree, and in this case, we r emo v e any edge e that is incident to a d egree-1 vertex. Pro ceeding th u s , we main tain th e inv ariant th at at all p oin ts, all but one 1 W e can blur the distinction b etw een vertices and ed ges and the p olynomials they represent. 15 of the comp onents of G i are isolated v ertices. W e can use this to sh o w that the n u m b er of registers required at any p oint in the computation of q e for e ∈ E ( G i ) is at most | E ( G i ) | + 1 (in p articular, if G i is acyclic this is at most | V ( G i ) | ). Putting it all together, we can also sho w that the maxim um n umber of no des used in computing the edges of G is b ounded by max {| V ( G ) | , | E ( G ) | + 1 , | E ( G ) | + | V ′ |} ≤ w + 1. Moreo ver, since at eac h step the p olynomial of some nod e v ∈ V is computed, the to tal n umb er of steps in the straigh t-line program is at most w . This pr o v es the lemma. Pro of of Prop osition 7 The pr o of follo ws the same lines as that of [KI03, Theorem 4.1]. A similar result f or b ounde d-depth circuits is noted in [DSY08, Section 5]. W e give a br ief pro of sketc h. Assume to the contrary that all three statemen ts h old. F ollo wing the pr o of in [KI03], NEXP will collapse to NP P erm . Hence, it su ffices to sho w P P erm ⊆ NSUBEXP to deriv e a contradictio n (to the nondeterministic time hierarch y theorem). The language P erm consists of all tuples ( M , v ), where M is an inte ger matrix and v is the binary enco ding of Pe r m( M ). The NSUBEXP mac hine will guess a p olynomial size, width- w circu it C for the n × n Pe rm anen t p olynomial o v er v ariables { x ij | 1 ≤ i, j ≤ n } . Next, w e wa nt to c h ec k whether C in d eed computes the P ermanent p olynomial. W e can easily obtain a width- w p olynomial-sized circuit C k that computes the p ermanent of k × k matrix ov er v ariables { x ij | 1 ≤ i, j ≤ k } from circuit C . Next we c hec k whether B 1 = C 1 ( x ) − x ≡ 0. F or n ≥ k > 1 chec k that B k = C k ( X ( k ) ) − P k i =1 x 1 ,i C k − 1 ( X ( k ) i ) ≡ 0, where X ( k ) = ( x i,j ) i,k ∈ [ k ] is the k × k matrix and X ( k ) i is a minor obtained by deleting first ro w an d i th column of X ( k ) . It follo w s that if all the B i ’s are iden tically zero p olynomials then C compu tes the P erm anen t p olynomial. Since C k has a width- w p olynomial size ci r cuit it follo ws that B k can b e computed b y a p olynomial-size w idth w + O (1) circu it. W e can now u s e the assumed deterministic sub exp onen tial time algorithm for identi ty testing of b ounded width circuits to c hec k whether eac h B k is identica lly zero. Putting it together, w e ha ve P P erm ⊆ NSUBEXP . 16