Almost Euclidean subspaces of ell_1^N via expander codes

Almost Euclidean subspaces of ℓ N 1 via expander co des ∗ V enk atesan Guruswami † James R. Lee ‡ Alexander Razb orov § Abstract W e giv e an explicit (in particular, d et ermin istic p olynomial time) construction of subspaces X ⊆ R N of dimension (1 − o (1)) N suc h th a t for ev ery x ∈ X , (log N ) − O (log log log N ) √ N k x k 2 6 k x k 1 6 √ N k x k 2 . If we are al lo wed to use N 1 / log log N 6 N o (1) random b its and dim ( X ) > (1 − η ) N f o r an y fixed constan t η , th e lo w er b ound can b e fur ther improv ed to (log N ) − O (1 ) √ N k x k 2 . Through kn own connections b et wee n su c h Euclidean sections of ℓ 1 and compressed sensing matrices, our result also giv es explicit compr essed sen s ing matrices for lo w compression factors f o r wh ic h basis pur suit is guarante ed to reco ve r sp arse signals. Our construction mak es use of unbalanced bipartite graph s to imp ose local linear constrain ts on v ectors in the subspace, and our analysis relies on expansion pr o p erties of the graph. This is ins p ired by similar constructions of error-correcting co des. Mathematics Subje ct Classific ation (2000 ) c o des: 68R05, 68P30, 51 N20. ∗ A preliminary version of this pap er a ppeared in the Pr o c e e dings of the 19th A nnual ACM-SIAM Sym- p osium on Discr ete Algorithms , January 2008. † Univ er s it y of W ashington, Department of Computer Science and E ngineering, Box 352350 , Seattle, W A 98195 . Part of this work was done whe n the author was on leav e at the Sc ho ol o f Mathematics, Ins tit ute for Adv ance d Study , Princeton, NJ. Research supp orted in part by NSF CCF-0343 672, a P ack ar d F ellowship, and NSF gr an t CCR-0324 906 to the IAS. ve nkat@cs.wa shington.edu ‡ Univ er s it y of W ashington, Depa rtmen t of Computer Science and E ngineering, Seattle, W A 98195 . Re- search suppor ted in par t by NSF CCF-06 44037. jrl@cs. washington.edu § Institute for Adv anced Study , Scho ol of Mathematics, Princeto n, NJ a nd Steklov Mathematical Institute, Moscow, Russia. Current addres s: University of Chicago, Department o f Computer Science, Chicago, IL 60637 . razborov@ cs.uchicago.edu 1 In tro ducti o n Classical results in high- dime nsional geometry [13, 23] state that a ra ndo m (with resp ect to the Haar measure) subspace X ⊆ R N of dimension εN [13] or ev en (1 − ε ) N [23] is an almost Euclidean section in ℓ N 1 , in the sense that √ N k x k 1 and k x k 2 are within constan t factor s, uniformly for ev ery x ∈ X . Indeed, this is a particular example o f the use of the probabilistic metho d, a t echniq ue whic h is no w ubiquitous in asymptotic geometric a na ly sis. On the other hand, it is usually the case that ob jects constructe d in suc h a manner are ve ry hard to come by explicitly. Motiv ated in part by ev er gro wing connections with com binatorics a nd theoretical computer science, the problem of explicit constructions o f suc h subspaces has gained substantially in p opularit y ov er the last sev eral y ears; see, e.g. [36, Sec. 4], [30, Prob. 8], [22, Sec. 2.2]. Indeed, suc h subspace s ( view ed a s em b eddings) are imp ortan t for problems like hig h- dime nsional nearest-neigh b or searc h [19] and compressed sensing [10], and one exp ects that ex plicit constructions will lead, in particular, to a b etter understanding of the underlying geometric structure. (See also the end of the introduction for a discus sion of the relev ance to compress ed sensing.) 1.1 Previous results and our con tributions If one relaxes the requiremen t tha t dim ( X ) = Ω( N ) or allows a limited amoun t of randomness in the construction, a n um b e r o f results are kno wn. In order to review these, w e define the distortion ∆( X ) of X ⊆ R N b y ∆( X ) = √ N · max 0 6 = x ∈ X k x k 2 k x k 1 . In the first direction, it is w ell-kno wn tha t an explicit construction with distortion O (1) and dim ( X ) = Ω( √ N ) can b e extracted from Rudin [32] (see also [26] for a more accessible ex- p osition). Indyk [2 0] presen ted a deterministic p olynomial-time construction with distortion 1 + o (1) and dim ( X ) > N exp( O ( log log N ) 2 ) . Another v ery intere sting line of researc h pursued b y v arious a uthors and in quite differen t con texts is to a chiev e, in the terminology of theoretical computer science, a p artial der an- domization of the origina l (existen tial) results. The goal is to come up with a “ c onstructiv e” discr ete probabilistic measure on subspaces X of R N suc h that a random (with resp ect to this measure) subspace still has lo w distortion almost surely , whereas the en tropy of this measure ( t ha t is, the num b er of truly random bits necessary to sample fro m it) is also as lo w as p ossible. Denoting b y A k ,N a random k × N sign matrix (i.e. with i.i.d. Bernoulli ± 1 entries ), one can extract fro m the pap er [23] by Kashin that k er( A k ,N ), a subspace of co dimension at most k has, with high probabilit y , distortion p N/ k · p olylog( N /k ). Sc hec h tman [33] arriv ed at similar conclusions for subspaces generated by r ows of A N − k ,N . Artstein-Avidan and Milman [2 ] considered again the mo del ker( A k ,N ) and derandomized this further from O ( N 2 ) to O ( N log N ) bits of randomness . W e remark that the pseudorandom generator 1 approac h of Indyk [19] can b e used to efficien tly construct suc h subspaces using O ( N log 2 N ) random bits. This was f urther improv ed to O ( N ) bits b y Lo v ett and So din [27]. Subsequen t to o ur w ork, Gurusw ami, Lee, and Wigderson [16] used the construction appro ac h from this pap er to reduc e the random bits to O ( N δ ) for an y δ > 0 while ac hieving distortion 2 O (1 /δ ) . As far as deterministic constructions with dim ( X ) = Ω( N ) are concerned, we are a w are of only o ne result; implicit in v arious pap ers ( see e.g. [11]) is a subspace with dim ( X ) = N / 2 and distortion O ( N 1 / 4 ). F or dim ( X ) > 3 N/ 4, sa y , it a ppears that nothing non-trivial w as sho wn prior to our w ork. Our main result is as follows. Theorem 1.1. F or every η = η ( N ) , ther e is an explicit, de t ermin i s tic p olynomial-time c on- struction of subsp ac es X ⊆ R N with dim ( X ) > (1 − η ) N and dis tortion ( η − 1 log log N ) O (log log N ) . Lik e in [23, 2, 2 7 ], our space X has the form k er( A k ,N ) fo r a sign matrix A k ,N , but in our case this matrix is completely explicit (a nd, in particular, p olynomial time computable). Its high-lev el o v erview is giv en in Section 1.2.3 b elo w. On the other hand, if w e allow ourselv es a small num b er o f random bits, then w e can sligh tly impro v e the b ound on distortion. Theorem 1.2. F or every fix e d η > 0 ther e is a p olynom i al time algorithm using N 1 / log log N r and o m bi ts that alm ost sur ely pr o duc es a subsp ac e X ⊆ R N with dim ( X ) > (1 − η ) N and distortion (log N ) O (1 ) . 1.2 Pro of tec hniques 1.2.1 Spreading subspaces Lo w distortion of a section X ⊆ R N in tuitiv ely means that for ev ery non-zero x ∈ X , a “substan tial” p ortion of its mass is spread ov er “man y” co ordinates, and w e formalize this in tuition by in tro ducing the concept of a spr e ad subsp ac e (Definition 2.1 0). While this concept is tigh tly related to distortion, it is far more con v enien t to w ork with. In pa rticular, using a simple sp ec tral a rgumen t and Kerdo c k co des [25], [2 9 , Chap. 1 5], w e initialize our pro of b y presen ting explicit subspaces with reasonably go o d spreading prop erties. These co des app eared also in the approac h of Indyk [20], though they w ere used in a dual capacit y (i.e., as generator matrices inste ad o f c hec k matrices). In terms of distortion, ho w ev er, this construction can ac hiev e at best O ( N 1 / 4 ). 1.2.2 The main constr uct ion The k ey contribution o f our pap er consists in exploiting the natural analogy b et w een lo w- distortion subs paces o v er the reals and error- cor r ecting co des ov er a finite alphab et. Let G = ( { 1 , 2 , . . . , N } , V R , E ) b e a bipartite g raph whic h is d -regular on the righ t, and let L ⊆ R d b e any subspace. Using the notation Γ( j ) ⊆ { 1 , 2 , . . . , N } for the neigh b or set o f a v ertex j ∈ V R , we analyze the subspace X ( G, L ) = { x ∈ R N : x Γ( j ) ∈ L for ev ery j ∈ V R } , 2 where for S ⊆ [ N ], x S ∈ R | S | represen ts the v ector x restricted to the co ordinates lying in S . In other w o r ds , w e imp ose lo cal linear constrain t s (from L ) according to the structure of some bipar t ite g raph G . As Theorem 4.2 shows , one can in particular analyze the spreading prop erties of X ( G, L ) in terms of those of L and the expansion prop erties of G . 1.2.3 Putting it together: com binatorial ov erview Our final space X will b e of the form X = T r − 1 i =0 X ( G i , L i ) for suitably chosen G i , L i (see the pro of o f Theorem 1.1). Com binatorially this simply means that w e tak e k i × N sign matrices A i suc h that X ( G i , L i ) = k er( A i ) and stac k them on the top of one another to get our final matrix A k ,N . Moreo v er, ev ery A i is a stack o f | V R | copies of the sign matrix A ′ i with k er( A ′ i ) = L i in whic h ev ery copy is padded with ( N − d ) zero columns. The exact placemen t of these columns is go v erned b y the graph G that is c hosen to satisfy certain expansion pr o perties (Theorem 2.6), and it is differen t in differen t copies. And then we ha v e one more lev el of recursion: Eve ry L i has the form X ( G ′ i , L ′ i ), where G ′ i again hav e certain ( but this time different – see Prop osition 2.7) expansion prop erties and L ′ i is our initial subspace (see Section 1.2.1). 1.2.4 Connections t o discret e co des Our approac h is inspired b y L ow Density Parity Che ck Co des (LD PC ) intro duce d by Gallager [14]. They are particularly suited to o ur purp oses since, unlik e most other explicit construc- tions in co ding theory , they exploit a c ombinatorial structure of t he parity che ck matrix and rely v ery little on the a r ithme tic of the underlying finite field. Sipser and Spielman [35] sho w ed that one can ac hiev e basically t he same results (that is, simple and elegan t construc- tions o f constan t rate, constan t relativ e minimal distance co des) b y considering a djacenc y matrices of sufficien tly go o d expanders instead of a r andom sp arse matrix. Thes e co des are no w aday s called exp a n der c o des . Using an idea due to T anner [37], it w as sho wn in [35] (see also [39]) that ev en b etter constructions can b e a c hiev ed by replacing the parity c hec k b y a small (constant size) inner co de. Our results demonstrate that analogous constructions work o v er the reals: If the inner subspace L has reasonably g oo d spreading prop erties, then the spreading prop erties of X ( G, L ) are ev en b etter. Upp er b ounds on distortion follo w. 1.3 Organization In Section 2, w e pro vide necessary bac kground on bipart ite expande r graphs and define spread subspaces. In Section 3, w e initialize our construction with an explicit subspace with reasonably go o d spreading prop erties. In Section 4, w e describ e and analyze our main expander-based construction. Finally , in Section 5, we discuss wh y impro v emen ts to our b ounds ma y ha v e to come from a source other than b etter expander graphs. 3 1.4 Relationship to compressed sensing. In [9], DeV ore ask s whether probabilistically generated compressed sensing matrices can b e giv en b y deterministic constructions. The note [24] makes the connec tion b et w een distortion and compress ed sensing quite explicit. If M : R N → R n satisfies ∆(k er ( M )) 6 D , then an y ve ctor x ∈ R N with | supp( x ) | < N 4 D 2 can b e uniquely reco v ered fro m its encoding M x . Moreo v er, give n the enco ding y = M x , the reco very can b e perfo r med efficien tly b y solving the f o llo wing conv ex optimization problem: min v ∈ R N k v k 1 sub ject to M v = y . In fa c t, something more general is sho wn. D e fine, for x ∈ R N , the quan tit y σ k ( x ) 1 = min w ∈ R N : | supp( w ) | 6 k k x − w k 1 (1) as the erro r of the b es t sparse approximation to x . Then giv en M x , the ab ov e alg o rithm reco v ers a v ector v ∈ R N suc h that M x = M v and k x − v k 2 6 σ k ( x ) 1 √ k , f or k = Θ( N /D 2 ). In o ther w ords, the recov ery algorithm is stable in the sense that it can a ls o tolerate noise in the signal x , and is able t o perform appro ximate reco v ery ev en f or signals which are only appro ximately sparse. Th us our results sho w the existence of a mapping M : R N → R o ( N ) , where M is giv en b y an explicit matrix, a nd suc h that any v ector x ∈ R N with | supp ( x ) | 6 N (log N ) C log log log N can b e efficien tly reco v ered from M x (t he stable g en eralization also holds, a long the lines of (1 )). This yields the b est-kno wn explicit compressed sensing matrices for this range of parameters (e.g. where n ≈ N / po ly(log N )). Mor eov er, unlik e probabilistic constructions, our matrices are quite sparse, making compression (i.e., matrix-vec tor multiplication) and reco v ery ( via Basis Pursuit) more efficien t. F or instance, when n = N/ 2 , our matrices ha v e only N 2 − ε non-zero en tries for some ε > 0. W e refer to [21] for explicit constructions tha t ac hiev e a b etter tradeoff for n ≈ N δ , with 0 < δ < 1. W e remark that the construction o f [21 ] is not stable in the sense discusse d ab ov e (and hence only w orks for a ctual sparse signals). 2 Preliminaries 2.1 Notation F or tw o expressions A, B , we sometimes write A & B if A = Ω( B ), A . B if A = O ( B ), and w e write A ≈ B if A = Θ( B ), that is A & B and B & A . F or a p ositiv e in teger M , [ M ] denotes the set { 1 , 2 , . . . , M } . The set of nonnegativ e in tegers is denoted b y N . 2.2 Un balanced bipartite expand ers Our construction is based on un balanced bipartite gra phs with non- trivial ve rtex expansion. Definition 2.1. A bip artite gr a p h G = ( V L , V R , E ) (with no multiple e dges) is said to b e an ( N , n, D , d )-r ig h t regular graph if | V L | = N , | V R | = n , every vertex on the lef t hand siz e V L has de gr e e at most D , and every vertex on the right han d side V R has de gr e e equal to d . 4 F or a graph G = ( V , E ) a nd a v ertex v ∈ V , w e denote by Γ G ( v ) the vertex neighb orho o d { u ∈ V | ( v , u ) ∈ E } of v . W e denote b y d v = | Γ G ( v ) | the degree of a v ertex v . The neighb orho o d o f a subset S ⊆ V is defined by Γ G ( S ) = S v ∈ S Γ G ( v ). When the graph G is clear from the con text, w e ma y omit the subscript G and denote the neigh b orho ods as just Γ( v ) and Γ( S ) . Definition 2.2 (Expansion profile) . The expansion profile of a bip artite g r aph G = ( V L , V R , E ) is the function Λ G : (0 , | V L | ] → N defi ne d by Λ G ( m ) = min {| Γ G ( S ) | : S ⊆ V L , | S | > m } . Note t ha t Λ G ( m ) = min v ∈ V L d v for 0 < m 6 1. F or our w ork, w e need unb alanc e d bipartite graphs with expansion from the larger side to the smaller side. Our results are based on t w o kno wn ex plicit constructions of suc h gra phs . The first one is to tak e the edge-ve rtex incidence graph of a non-bipartite sp e ctr a l ex p ander 1 suc h as a Ra man uja n graph. These were also the graphs use d in the w ork o n expander co des [35, 39 ]. The second construction of expanders is based on a suggestion due to Avi Wigderson. It uses a result o f Barak, et. al. [4] based on sum-pro duct estimates in finite fields; see [38, § 2.8] for bac kground on suc h estimates. F or our purp oses, it is also con v enien t (but not strictly necessary) to hav e bipartite graphs t ha t are regular on the r ig h t. W e b egin by describing a simple metho d to achie ve righ t-regularity with minimal impact on the expansion and degree parameters, and then turn to stating the precise stateme nts a bout the t w o expande r constructions w e will mak e use of in Section 4 t o construct our explicit subspaces. 2.2.1 Righ t-regularization Lemma 2.3. Given a gr aph H = ( V L , V R , E ) w i th | V L | = N , | V R | = n , that is lef t -r e gular with e ach vertex in V L having de g r e e D , one c an c onstruct in O ( N D ) time an ( N , n ′ , 2 D , d ) - right r e gular gr aph G with n ′ 6 2 n and d = ⌈ N D n ⌉ such that the exp ans i on pr ofiles satisfy Λ G ( m ) > Λ H ( m ) for al l m > 0 . Pr o of. Let d a v = N D /n b e the av erage right degree of t he graph H and let d = ⌈ d a v ⌉ . Split eac h v ertex v ∈ V R of degree d v in to ⌊ d v /d ⌋ ve rtices of degree d eac h, and if d v mo d d > 0, a “r emainder” v ertex of degree r v = d v mo d d . D istribute the d v edges inciden t to v to these split v ertices in an arbitr a ry w ay . The nu mber o f newly intro duc ed vertice s is at most P v ∈ V R d v /d = nd a v /d 6 n , so the n um b er n ′ of right-side ve rtices in the new graph satisfies n ′ 6 2 n . All v ertices except the at most n “remainder” v ertices now hav e degree exactly d . F or eac h v ∈ V R , add d − r v edges to the cor r e sp onding remainder v ertex (if one exists). Since this step adds at most ( d − 1) n 6 d a v n = N D edges, it is p ossible to distribute these edges 1 That is, a regula r graph with a larg e gap b et ween the lar gest and second lar g est eigenv alue of its adjacency ma trix. 5 in suc h a wa y that no v ertex in V L is inciden t on more than D of the new edges. Therefore, the maximum left-degree of the new graph is at most 2 D . The claim a bout expansion is ob vious — just ig nore the newly added edges, and the splitting o f vertice s can only improv e the v ertex expansion. 2.2.2 Sp ectral expanders The next theorem con v erts non-bipartite expanders to un balanced bipartite expanders via the usual edge-v ertex incidence construction. Theorem 2.4. F or every d > 5 and N > d , ther e exists an explicit ( N ′ = Θ( N ) , n, 2 , Θ( d )) - right r e gular gr aph G whose exp ansion pr ofile satisfies Λ G ( m ) > min n m 2 √ d , √ 2 mN ′ d o . Pr o of. Let p, q b e any tw o primes whic h are b oth congruen t to 1 mo dulo 4. Then there exists an explicit ( p + 1 ) - regular graph Y = ( V , F ) with q ( q 2 − 1) 4 6 | V | 6 q ( q 2 − 1) 2 and suc h that λ 2 = λ 2 ( Y ) 6 2 √ p , where λ 2 ( Y ) is the second largest eigenv alue (in absolute v alue) of the adjacency matrix of Y [28]. (See [18, § 2] for a discussion of explicit constructions of expander g raphs.) Letting n = | V | , we define a ( ( p +1) n 2 , n, 2 , p + 1)-r ig h t regular bipart it e graph G = ( V L , V R , E ) where V L = F , V R = V , and ( e, v ) ∈ E if v is an endp oin t of e ∈ F . T o analyze the expansion properties of G , w e use the following lemma of Alon and Ch ung [1]. Lemma 2.5. If Y is any d -r e gular gr aph on n v e rt ic es wi th se c ond eigenva lue λ 2 , then the induc e d sub gr aph on any set of γ n vertic es in Y has at most  γ 2 + γ λ 2 d  dn 2 e dg e s. In particular, if S ⊆ V L satisfies | S | > γ 2 ( p + 1) n a nd | S | > 2 γ n √ p + 1, then | Γ G ( S ) | > γ n . Stated differen t, for an y S ⊆ V L , we hav e | Γ G ( S ) | > min n 2 p | S | n, | S | o 2 √ p + 1 Setting N ′ = ( p +1) n 2 , we see that Λ G ( m ) > min n m 2 √ d , √ 2 N ′ m d o . No w giv en parameters d > 5 and N > d , let p b e the largest prime satisfying p + 1 6 d and p ≡ 1 (mo d 4), and let q b e the smallest prime satisfying q ( q 2 − 1)( p +1) 8 > N and q ≡ 1 ( mo d 4). The theorem follows b y noting that for all integers m > 3, there exists a prime p ∈ [ m, 2 m ] whic h is congruen t to 1 modulo 4 (see [12]). The expanders of Theorem 2.4 are a lready right-regular but they ha v e one dra wbac k; w e cannot fully control the num b er of left-side v ertices N . F ortunately , this can b e easily circum v en ted with the same Lemma 2.3. 6 Theorem 2.6. F or every d > 5 and N > d , ther e exists an explicit ( N , n, 4 , Θ( d )) -right r e gular gr aph G which satisfies Λ G ( m ) > min n m 2 √ d , √ 2 N m d o . Pr o of. Apply Theorem 2.4 to get a graph with N ′ > N , N ′ ≈ N ve rtices on the left, then remo v e an arbitrary subset of N − N ′ v ertices fr om the left hand side. This do esn’t affect the expansion prop erties, but it destro ys righ t-regularity . Apply Lemma 2.3 to correct this. 2.2.3 Sum-product expanders In this section, p will denote a prime, and F p the finite field with p elemen ts. The following result is implicit in [4, § 4], and is based on a key “sum-pro duct” lemma (Lemma 3.1) from [3], whic h is it self a statistical v ersion of the sum-pro duct theorems o f Bourgain, Katz, and T ao [5], and Bourgain and Kon y agin [6] for finite fields. Prop osition 2.7. Ther e exists an absolute c onstant ξ 0 > 0 such that for al l primes p the fol lowin g holds. Consider the bi p artite gr ap h G p = ( F 3 p , [4] × F p , E ) wh er e a le ft vertex ( a, b, c ) ∈ F 3 p is adjac ent to (1 , a ) , ( 2 , b ) , (3 , c ) , and (4 , a · b + c ) on the right. The n Λ G p ( m ) > min  p 0 . 9 , m 1 / 3+ ξ 0  . Note that trivially | Γ G p ( S ) | > | S | 1 / 3 , and the ab o ve states that not-to o-large se ts S expand b y a sizeable amoun t more than the t rivial b ound. Using the ab o v e construction, w e can now prov e the fo llo wing. Theorem 2.8. F or al l inte gers N > 1 , ther e is an exp l i c it c onstruction of an ( N , n, 8 , Θ( N 2 / 3 )) - right r e gular gr aph G which satisfies Λ G ( m ) > min  1 8 n 0 . 9 , m 1 / 3+ ξ 0  . (Her e ξ 0 is the absolute c onstant fr om Pr op osition 2.7.) Pr o of. Let p b e the smallest prime such that p 3 > N ; note that N 1 / 3 6 p 6 2 N 1 / 3 . Construct the gr a ph G p , and a subgraph H o f G p b y deleting an a rbitrary p 3 − N vertice s on the left. Th us H has N v ertices o n left , 4 p v ertices o n the right, is left-regular with degree 4 and satisfies, by Prop osition 2.7, Λ H ( m ) > min  p 0 . 9 , m 1 / 3+ ξ 0  . Applying the transformatio n of Lemma 2.3 to H , we get an ( N , n, 8 , d ) - righ t regular graph with d = ⌈ 4 N 4 p ⌉ ≈ N 2 / 3 and with the same expansion prop ert y . 2.3 Distortion and spr eading F or a vec tor x ∈ R N and a subs et S ⊆ [ N ] of co ordinates, w e denote by x S ∈ R | S | the pro- jection of x onto the co ordinates in S . W e abbreviate the complemen tary set of co ordinates [ N ] \ S to ¯ S . Definition 2.9 (D istor t ion o f a subspace) . F or a subsp ac e X ⊆ R N , we define ∆( X ) = sup x ∈ X x 6 =0 √ N k x k 2 k x k 1 . 7 As w e already noted in the in tro duction, instead o f distortio n it turns out to b e mo r e con v enien t to w ork with the following notion. Definition 2.10. A subsp ac e X ⊆ R N is ( t, ε ) - spre ad if for every x ∈ X and every S ⊆ [ N ] with | S | 6 t , we have k x ¯ S k 2 > ε · k x k 2 . Let us b egin with r elat ing these t w o not io ns . Lemma 2.11. Supp os e X ⊆ R N . a) If X is ( t, ε ) -sp r e ad then ∆( X ) 6 r N t · ε − 2 ; b) c on v e rsely, X is  N 2∆( X ) 2 , 1 4∆( X )  -spr e ad. Pr o of. a). Fix x ∈ X ; w e nee d to prov e that k x k 1 > √ tε 2 k x k 2 . (2) W.l.o.g. assume that k x k 2 = 1 and that | x 1 | > | x 2 | > . . . > | x N | . Applying Definition 2.10, w e kno w that k x [ t +1 ..N ] k 2 > ε . O n the other hand, P t i =1 | x i | 2 6 1, therefore | x t | 6 1 √ t and th us k x [ t +1 ..N ] k ∞ 6 1 √ t . And no w w e get ( 2) b y the calculation k x k 1 > k x [ t +1 ..N ] k 1 > k x [ t +1 ..N ] k 2 2 k x [ t +1 ..N ] k ∞ > √ tε 2 . b). Let t = N 2∆( X ) 2 . Fix a gain x ∈ X with k x k 2 = 1 and S ⊆ [ N ] with | S | 6 t . By the b ound on distor t io n, k x k 1 > √ N ∆( X ) . On the other hand, k x S k 1 6 √ t · k x S k 2 6 √ t = p N/ 2 ∆( X ) , hence k x ¯ S k 1 = k x k 1 − k x S k 1 > √ N 4∆( X ) and k x ¯ S k 2 > k x ¯ S k 1 √ N > 1 4∆( X ) . Next, we no te spreading prop erties of random subspaces (they will b e needed only in the pro of of Theorem 1.2). The follow ing theorem is due to Ka s hin [23], with the optimal b ound essen tia lly obta ined b y Garnaev and Gluskin [15]. W e note t ha t suc h a theorem no w f ollo ws from standard to ols in asymptotic conv ex geometry , given t he en tropy b ounds of Sc h ¨ utt [34] (see, e.g. Lemma B in [27]). Theorem 2.12. If A is a uniformly r andom k × N si g n matrix, then wi th pr ob ability 1 − o (1) , ∆(k er( A )) . s N k log  N k  . 8 Com bining Theorem 2.12 with Lemma 2.11(b), w e get: Theorem 2.13. If A is a uniformly r andom k × N si g n matrix, then wi th pr ob ability 1 − o (1) , k er( A ) is a  Ω  k log( N/k )  , Ω  q k N log( N/k )  -spr e ad subsp ac e. Finally , w e in tro duce a “relativ e” ve rsion of Definition 2.10. It is somewhat less intuitiv e, but very con ve nien t to w ork with. Definition 2.14. A subsp ac e X ⊆ R N is ( t, T , ε )-spread ( t 6 T ) if for every x ∈ X , min S ⊆ [ N ] | S | 6 T k x ¯ S k 2 > ε · min S ⊆ [ N ] | S | 6 t k x ¯ S k 2 . Note tha t X is ( t, ε )-spread if and only if it is (0 , t, ε ) - spre ad, if and only if it is (1 / 2 , t, ε ) - spread. (Note t ha t t, T a re not restricted to in tegers in our definitions.) One ob vious adv antage of Definition 2 .14 is that it allows us to break the task of constructing we ll-spread subspaces into pieces. Lemma 2.15. L et X 1 , . . . , X r ⊆ R N b e line ar subsp ac es, and assume that X i is ( t i − 1 , t i , ε i ) - spr e ad, wher e t 0 6 t 1 6 · · · 6 t r . Then T r i =1 X i is ( t 0 , t r , Q r i =1 ε i ) -spr e ad. Pr o of. Ob vious. 3 An expl i cit w eakly-spread sub space No w our go a l can be stated as finding an explicit construction that gets as close as p ossible to the probabilistic b ound of Theorem 2.13. In this section w e p erform a (relativ ely simple ) “initialization” step; the bo osting argumen t (whic h is the most es sen tial con tribution of our pap er) is deferred to Section 4. Belo w, f o r a matrix A , w e denote b y k A k its op erator norm, defined a s sup x 6 =0 k Ax k 2 k x k 2 . Lemma 3.1. L et A b e any k × d ma trix whose c olumn s a 1 , . . . , a d ∈ R k have ℓ 2 -norm 1, and, mor e over, for any 1 6 i < j 6 d , |h a i , a j i| 6 τ . The n ke r ( A ) is  1 2 τ , 1 2 k A k  -spr e ad. Pr o of. Fix x ∈ k er( A ) and let S ⊆ [ d ] b e an y subset with t = | S | 6 1 2 τ . Let A S b e the k × t matrix whic h a r ises b y restricting A to the columns indexed b y S , and let Φ = A T S A S . Then Φ is the t × t matrix whose en tries are h a i , a j i for i, j ∈ S , therefore we can write Φ = I +Φ ′ where ev ery en try of Φ ′ is b ounded in ma g nitude by τ . It follows that all the eigen v alues of Φ lie in the range [1 − tτ , 1 + tτ ]. W e conclude, in particular, that k A S y k 2 2 > (1 − tτ ) k y k 2 2 > 1 2 k y k 2 2 for ev ery y ∈ R t . Let A ¯ S b e the restriction of A to the columns in the complemen t of S . Since x ∈ ker( A ), w e ha v e 0 = Ax = A S x S + A ¯ S x ¯ S 9 so tha t k A ¯ S x ¯ S k 2 = k A S x S k 2 > 1 √ 2 k x S k 2 . Since k A ¯ S x ¯ S k 2 6 k A k · k x ¯ S k 2 , it fo llo ws that k x S k 2 6 √ 2 k A k · k x ¯ S k 2 . Since k A k > 1, this implies k x ¯ S k 2 > k x k 2 2 k A k . W e now obtain matrices with small op erator norm and near-orthogonal columns from explicit constructions of Kerdo c k co des. Prop osition 3.2. F or al l p o sitive inte gers d , k wher e k is a p ower of 4 satisfying k 6 d 6 k 2 / 2 , ther e e x ists an e x p licit k × d matrix A with the fol lowin g pr op e rt ies. 1. Every en t ry of A is either ± 1 / √ k , and thus the c olumns a 1 , a 2 , . . . , a d ∈ R k of A al l have ℓ 2 -norm 1 , 2. F or al l 1 6 i < j 6 d , |h a i , a j i| 6 1 / √ k , and 3. k A k 6 q  d k  . Pr o of. The pro of is based on a construction of mutually unbiased bases ov er the reals using Kerdo c k co des [25, 7]. First, let us recall that for k a p o we r of 2, the Hadamar d co de of length k is a subspace of F k 2 of size k containing the k linear functions L a : F log 2 k 2 → F 2 , where f o r a, x ∈ F log 2 k 2 , L a ( x ) = a · x ( c omputed o v er F 2 ). A Kerdock co de is the union of a Hadamard co de H ⊆ F k 2 and a collection of its cosets { f + H | f ∈ F } , where F is a set of quadratic b en t functions with the prop ert y tha t for all f 6 = g ∈ F , the function f + g is also b en t . 2 When k is a p o w er of 4 , it is known (see [25 ] and also [29, Chap. 15, Sec. 5]) that one can construct an explicit set F of ( k 2 − 1) suc h b en t f unctions. (A simpler construction of ( √ k − 1) suc h quadratic functions app ears in [7].) The cosets of these functions together with the Hadamar d co de (the trivial coset) giv e an explicit Kerdo c k co de o f length k that has k 2 / 2 co dew ords. Interpreting binary v ectors of length k a s unit v ectors with ± 1 / √ k en tries, ev ery coset of the Hadamard co de giv es an or t ho normal basis of R k . The k / 2 cosets comprising the Kerdo c k code thus yield k / 2 orthonorma l bases B 1 , B 2 , . . . , B k / 2 of R k with the prop ert y that for ev ery pair { v , w } o f v ectors in differen t bases, one has |h v , w i| = 1 / √ k . (Suc h bases are called m utually un biased bases.) F or any d , k 6 d 6 k 2 / 2, write d = q k + r where 0 6 r < k . W e construct our k × d matrix A to consist of [ B 1 . . . B q ] follow ed by , in the case of r > 0, an y r columns of B q +1 . The first tw o prop erties of A a r e immediate from the prop ert y of the bases B i . T o b ound the op erator no r m, note tha t b eing an ortho no rmal basis, k B i k = 1 for eac h i . A simple application of Cauc h y-Sc h w artz then sh ows tha t k A k 6 p ⌈ d/k ⌉ . Plugging in the matrices guaran teed b y Prop osition 3.2 in to L emma 3.1, we can conclude the fo llo wing. 2 A function f : F a 2 → F 2 for a even is said to b e b ent if it is ma ximally fa r fro m all linear functions , or equiv alen tly if all its F ourier co efficien ts have a bsolute v a lue 1 / 2 a/ 2 . 10 Theorem 3.3. F or every inte ger k that is a p ower of 4 and every inte ger d such that k 6 d 6 k 2 / 2 , (3) ther e exists a  √ k 2 , 1 4 q k d  -spr e ad subsp ac e L ⊆ R d with co dim ( L ) 6 k , sp e cifie d as the kernel of an explicit k × d sign matrix. These subspaces will b e used a s “inner” subspaces in an expander-based construction (Theorem 4.3) to get a subspace with ev en b etter spreading properties. 4 Bo osti n g sp r e ading prop erties vi a expanders 4.1 The T anner construction Definition 4.1 (Subspaces from bipartite graphs) . Given a bip a rtite gr aph G = ( { 1 , 2 , . . . , N } , V R , E ) such that every vertex in V R has de gr e e d , and a subsp a c e L ⊆ R d , we defin e the s ubsp ac e X = X ( G , L ) ⊆ R N by X ( G, L ) = { x ∈ R N | x Γ G ( j ) ∈ L for e very j ∈ V R } . (4) The follo wing claim is straightforw ard. Claim 1. If n = | V R | , then co dim ( X ( G, L )) 6 co dim ( L ) n , that is dim ( X ( G, L )) > N − ( d − dim ( L )) n . Remark 1 (T anner’s co de construction) . Our construction is a con tin uous analog of T anner’s construction of error-correcting co des [37]. T anner constructed co des b y iden tifying the v ertices on one side of a bipartite graph with the bits of the co de and identifyin g t he other side with constrain ts. He a nalyz ed the p erformance o f suc h co des by examining the girth of the bipartite graph. Sipser and Spielman [35 ] sho w ed that graph expansion play s a k ey role in t he quality of suc h co des, and gav e a linear time deco ding algorithm to correct a constan t fraction of errors. In the co ding w orld, the sp ecial case when L is the ( d − 1)- dimensional subspace { y ∈ R d | P d ℓ =1 y ℓ = 0 } corresp onds to t he lo w-densit y parity chec k co des of Gallag er [14]. In this case, the subspace is sp ec ified as the k ernel of t he bipartite adjacency matrix of G . 4.2 The spread-b o osting theorem W e no w show how to improv e spreading prop erties using the ab o v e construc tion. Theorem 4.2. L et G b e an ( N , n, D , d ) -gr aph with exp ansi o n pr ofile Λ G ( · ) , and let L ⊆ R d b e a ( t, ε ) -spr e ad subsp a c e. Then for every T 0 , 0 < T 0 6 N , X ( G, L ) is  T 0 , t D Λ G ( T 0 ) , ε √ 2 D  - spr e ad. 11 Pr o of. Fix x ∈ X ( G, L ) with k x k 2 = 1. Fix also S ⊆ [ N ] with | S | 6 T , where T = t D Λ G ( T 0 ). W e then need to prov e that k x ¯ S k 2 > ε √ 2 D min | B | 6 T 0 k x ¯ B k 2 . (5) Let Q = { j ∈ [ n ] : | Γ( j ) ∩ S | > t } , and B = { i ∈ S : Γ( i ) ⊆ Q } . Then t | Q | < E ( S, Γ( S )) 6 D | S | 6 D T , therefore | Q | < D T t = Λ G ( T 0 ) . On the other hand, w e hav e | Q | > | Γ ( B ) | , and hence | Γ( B ) | < Λ G ( T 0 ). By the definition of the expansion profile, this implies that | B | < T 0 , and therefore (see (5)) w e are only left to sho w that k x ¯ S k 2 > ε √ 2 D · k x ¯ B k 2 (6) for our particular B . Note first that k x ¯ B k 2 2 = k x ¯ S k 2 2 + k x S \ B k 2 2 . (7) Next, sinc e ev ery v ertex in S \ B has a t least one neighbor in Γ( S ) \ Q , w e hav e X j ∈ Γ( S ) \ Q k x Γ( j ) k 2 2 > k x S \ B k 2 2 . (8) Since x ∈ X ( G, L ), L is ( t, ε )-spread, and | Γ( j ) ∩ S | 6 t for a ny j ∈ Γ( S ) \ Q , X j ∈ Γ( S ) \ Q k x Γ( j ) \ S k 2 2 > ε 2 · X j ∈ Γ( S ) \ Q k x Γ( j ) k 2 2 . (9) Finally , X j ∈ Γ( S ) \ Q k x Γ( j ) \ S k 2 2 6 X j ∈ [ n ] k x Γ( j ) \ S k 2 2 6 D · k x ¯ S k 2 2 . (10) (7)-(10) imply k x ¯ S k 2 2 > ε 2 D ( k x ¯ B k 2 2 − k x ¯ S k 2 2 ) . Since ε 6 1 and D > 1, (6) (and hence Theorem 4.2) follo ws. 12 4.3 Putting things tog ether In this section w e assem ble the pro ofs of Theorems 1.1 and 1.2 from the already av ailable blo c ks (whic h are Theorems 2.8, 2.6, 2.13 , 3.3 and 4.2). Let us first see what we can do using expanders fro m Theorem 2 .8 . 4.3.1 First step: Bo osting w it h sum-pro du ct expanders The main difference b et w een the explicit construction of Theorem 3.3 and the probabilistic result (Theorem 2.13) is the order of magnitude of t (the par a me ter from D efinition 2.10). As w e will see in the next section, this difference is v ery principal, and our first goal is to somewhat close the gap with an explicit construction. Theorem 4.3. F ix an arbitr ary c on s tant β 0 < min  0 . 08 , 3 8 ξ 0  , w her e ξ 0 is the c onstant fr om The o r em 2.8. Then for al l sufficiently lar ge N ∈ N and η > N − 2 β 0 / 3 ther e exists an explicit subsp ac e X ⊆ R N with co dim ( X ) 6 η N which is ( N 1 2 + β 0 , η O (1 ) ) -spr e ad. Pr o of. In ev erything that follows , w e assume that N is sufficien tly larg e. The desired X will b e of the fo rm X ( G, L ), where G is supplied b y Theorem 2.8, and L b y Theorem 3.3. More sp ecifically , let G b e the explicit ( N , n, 8 , d )- righ t regular graph fro m Theorem 2.8 with d ≈ N 2 / 3 (and hence n ≈ N 1 / 3 ). Using Theorem 2.8, one can c hec k that fo r m 6 N 1 2 + β 0 , w e ha v e Λ G ( m ) > md β 0 − 1 2 . (11) Indeed, since n ≈ N 1 / 3 and d ≈ N 2 / 3 , the inequalit y 1 8 n 0 . 9 > md β 0 − 1 2 follo ws (for larg e N ) from β 0 < 0 . 08, and the inequalit y m 1 3 + ξ 0 > md β 0 − 1 2 follo ws from β 0 < 3 8 ξ 0 . By o ur assumption η > N − 2 β 0 / 3 > N − 0 . 1 , along with d ≈ N 2 / 3 , w e observ e that d 6 o ( η d ) 2 . Hence (cf. the stat ement of Theorem 3.3), w e can find k 6 ηd 8 , k ≈ η d that is a p o w er o f 4 and also satisfies the restrictions (3). Let L b e an explicit  Ω  √ η d  , Ω  √ η  - spread subspace guarante ed b y The orem 3.3. The bound on co dimension of X ( G, L ) is obvious : co dim ( X ( G, L )) 6 k n 6 ηdn 8 6 η N . F or a nalyz ing spreading prop erties o f X ( G, L ), we observ e that η > N − 2 β 0 / 3 implies η d & d 1 − β 0 , hence L is (Ω( d 1 2 − β 0 2 ) , η O (1 ) )-spread. By Theorem 4.2 and (11), for ev ery T 6 N 1 2 + β 0 , w e know t hat X ( G, L ) is ( T , Ω( d β 0 2 ) T , η O (1 ) )-spread In pa r t ic ular, fo r suc h T , X ( G, L ) is ( T , N Ω(1) T , η O (1 ) )-spread. Applying Lemma 2.15 with t he same spaces X 1 := · · · := X r := X ( G, L ) and suitably large constan t r ≈ 1 /β 0 = O (1), we conclude that X ( G, L ) is  1 2 , N 1 2 + β 0 , η O (1 )  -spread, completing the pro of. 4.3.2 Second step: H a ndling large sets based on sp ectral expanders The sum-pro duct expanders of Theorem 2.8 b eha v e p o orly for very large sets (i.e., as m → N , the lo w er b ound on Λ G ( m ) b e comes constant from some p oin t). The sp ectral expande rs of Theorem 2.6 b eha ve p o orly for small sets, but their expansion still improv es as m → N . 13 In this section, w e finish the pro ofs of Theorems 1.1 and 1.2 b y exploring strong sides of b oth constructions. W e b egin with Theorem 1.2 as it is conceptually simpler (we need only sp ec tral expanders, do not rely on Theorem 4 .3 , and still use only o ne fixed space X ( G, L )). Pro of of Theorem 1.2 . By Theorem 2 .6 there exists an explicit ( N , n, 4 , d )- righ t regular graph G with N Ω ( 1 log l o g N ) 6 d 6 N 1 2 log log N (12) whic h has Λ G ( m ) > min n m 2 √ d , √ 2 N m d o . Let k = ⌊ η 4 d ⌋ ; our desired (probabilistic) space is then X ( G, k er( A )), where A is a uniformly random k × d sign matrix (due t o the upp er b ound in (12), this uses at most d 2 6 N 1 log log N random bits). Recalling that η > 0 is an absolute constan t, by Theorem 2.13 k er( A ) is an ( Ω( d ) , Ω(1))-spread subspace almost surely . The bound on co dimension is again simple: co dim ( X ( G, k er( A ))) 6 k n 6 η N . F or analyzing spreading prop erties o f X , let m 0 = 8 N /d ( which is the “critical” p oin t where m 0 2 √ d = √ 2 N m 0 d .) Then Theorem 4.2 sa ys that X ( G, L ) is a.  T , Ω( √ d ) T , Ω(1)  -spread subspace for T 6 m 0 , and b.  T , Ω( √ N T ) , Ω(1)  -spread subspace for m 0 6 T 6 N . And no w w e are once more applying Lemma 2.15 with X 1 := X 2 := . . . := X r := X ( G, L ). In O (log d m 0 ) = O (log log N ) applications of (a ) with T 6 m 0 , w e conclude t hat X ( G, L ) is ( 1 2 , m 0 , (log N ) − O (1 ) )-spread. In O (log log N ) additional applications of (b) with T > m 0 , w e conclude t ha t X ( G, L ) is ( 1 2 , Ω( N ) , (log N ) − O (1 ) )-spread. Since X ( G, L ) is a n (Ω( N ) , (log N ) − O (1 ) )-spread subspace, the statemen t of Theorem 1.2 immediately fo llo ws from Lemma 2.11(a). Pro of of Theorem 1.1. This is our most sophisticated construction: w e use a series of X ( G, L ) for di ffer ent gra phs G , and the “inner” spaces L will come fro m Theorem 4.3. In what follow s, w e assume that N is sufficien tly large (obviously for N = O (1), ev ery non- trivial su bspace has b ounded distortion). T o get star t ed, let us denote e η = η (log lo g N ) 2 , and let us first construct and analyze subspaces X ( G, L ) needed for o ur purp oses individually . F or that purp ose, fix (for the time being) an y v alue of m with 1 6 m 6 δ e η 2 β 0 / 3 N , (13) δ a sufficien tly small constan t and β 0 is the constan t from Theorem 4.3. Applying Theorem 2.6 (with d := N /m ), w e get, f or some d = Θ( N/m ), an explicit ( N , n, 4 , d )-righ t regula r g r aph G m with Λ G m ( m ) > Ω( d − 1 / 2 ) m . Note that (13) implies e η > d − 2 β 0 / 3 (pro vided the constan t δ is small enough), and thus all conditions of Theorem 4.3 14 with N := d , η := e η are met. Applying that theorem, let L m ⊆ R d b e an explicit subspace with co dim ( L m ) 6 e η d that is a ( d 1 2 + β 0 , ( η / log lo g N ) O (1 ) )-spread subs pace. Consider the space X ( G m , L m ) ⊆ R N . Since D = 4 is a constant, w e ha v e co dim ( X ( G m , L m )) . e η N = η N (log log N ) 2 . And Theorem 4.2 (applied to T := m ) implies (recalling Λ G m ( m ) & d − 1 / 2 m , t = d 1 2 + β 0 , d = Θ( N /m )) that X ( G m , L m ) is a  m, Ω   N m  β 0  m, ( η / log log N ) O (1 )  -spread subspace. W e note that it is here that w e crucially use the fact that L m has spreading prop erties for t ≫ d 1 / 2 ( t is the parameter from Definition 2.10) so that w e more than comp e nsate for the factor √ d loss in Theorem 1.2 caused b y the relativ ely p o or expansion ra te of sp ectral expanders. W e will aga in apply Lemma 2.15, but the spaces X i will no w b e distinct. In particular, for i ∈ N define X i = X ( G t i , L t i ), wh ere t i = N ·  ε N  (1 − β 0 ) i , for some sufficien tly small constan t ε , 0 < ε < 1. It is easy to see that for some r = O (log log N ), w e ha v e t r 6 δ ˜ η 2 β 0 / 3 N and t r &  δ ˜ η 2 β 0 / 3  2 N . Then for X = T r − 1 i =0 X i w e ha v e co dim ( X ) . r ηN (log log N ) 2 . ηN log log N . In particular, co dim ( X ) 6 η N for sufficie ntly large N . By the ab ov e argumen t based on Theorem 4.2 and the c hoice of the t i ’s, it is easily seen that X i is a ( t i , t i +1 , ( η / (log log N )) O (1 ) )-spread subspace. By Lemma 2.15, X is a ( ε, t r , ( η / (log log N )) O (log log N ) )-spread subs pace, or equiv alen tly a ( t r , ( η / (log log N )) O (log log N ) )- spread subspace. Since we a ls o ha v e t r > ( η / (lo g log N )) O (1 ) N , the required b ound on ∆( X ) follo ws from Lemma 2.11(a). 5 Discuss ion W e ha v e presen ted explicit subspaces X ⊆ R N of dimension (1 − η ) N with distortion ( η − 1 log log N ) O (log log N ) and, using N o (1) random bits, distortion η − O (log log N ) . It is natu- ral to wonder whether b etter explicit constructions of expanders can giv e rise to b etter b ounds. W e mak e some remarks about this p ossibilit y . 1. The GUV and CR VW expander families. The next t w o theorems es sen tially follo w f r o m [17 ] a nd [8], respectiv ely (after an appropriate application of Lemma 2.3). Theorem 5.1 ([17]) . F or e ach fixe d 0 < c, ε 6 1 , and for al l in te gers N , K with K 6 N , ther e is an explicit c onstruction of an ( N , n, D , d ) -right r e gular gr aph G with D . ((log N ) /ε ) 2+2 /c and d > N / ( D K 1+ c ) and such that Λ G ( m ) > (1 − ε ) D · min { K, m } . 15 Theorem 5.2 ([8]) . F or every fix e d 0 < ε < 1 and al l sufficiently lar ge values N and d ther e exist n 6 N , D 6 2 O (( ε − 1 log log d ) 3 ) and an explicit ( N , n, D , d ) -rig h t r e gular bip a rt ite gr aph G with Λ G ( m ) > (1 − ε ) D · min { Ω( N/d ) , m } . The main pro blem for us in b oth these constructions is that D m ust gro w with N and d , resp e ctiv ely . By plugging in the explicit subspaces of Theorem 3.3 in to Theorem 4.2 with the GUV-expanders from Theorem 5 .1, one can ac hiev e distortions ∆( X ) ≈ exp( √ log N log log N ) for X ⊆ R N with dim ( X ) > N/ 2 . Using the GUV-expanders (in place of the sum-pro duct expanders ) tog ether with sp ectral expanders in a construction similar to the pro of o f Theorem 1.1 w ould yield a distortion b ound of (log N ) O (log log N ) . 2. V ery go o d expansion for large sets. If it w ere p oss ible to construct a n ( N , n, D , d )- righ t regular bipartit e graph H with D = O (1) and suc h that for ev ery S ⊆ V L with | S | > N 1 − β , w e had | Γ( S ) | = Ω( n ), then w e would be able to achie ve O (1) distort io n using only O ( d 2 + N δ ) r andom bits for an y δ > 0 (in fact, w e could use only O ( d + N δ ) random bits with [2 7 ]). The idea w ould b e to follow the pro of of Theorem 1.2, but only for O (1) steps to show the subspace is ( N 1 − β , Ω(1))-spread. Then we w ould in tersect this with a subspace X ( H , L ), where L ⊆ R d , with the latter subs pace generated as the ke rnel of a random sign matrix (requiring d 2 bits). Unfortunately , [31, Th. 1.5] sho ws that in o r der to ac hiev e the required expansion property , o ne has to tak e D > Ω( β log N ). Ac kno wledgmen ts W e are grateful to Avi Wigderson for sev eral enligh tening discussions, and esp ecially his suggestion that the sum-pro duct expanders of [3, 4] should b e relev ant. Using the sum- pro duct expanders in place of GUV-expanders in Section 4.3.1, we w ere able to impro v e our distortion b ound from ( log N ) O (log log N ) to (log N ) O (log l og log N ) . W e are also thankful to an anon ymous referee for sev eral useful remarks. References [1] N. Alon and F. R. K. Ch ung. Explicit construction o f linear sized to leran t net w orks. In Pr o c e e dings o f the First Jap an C o nfer enc e on Gr aph Th e ory and Applic ations (Hako ne, 1986) , v olume 72, pages 15–19, 1 988. [2] S. Artstein-Avidan a nd V. D. Milman. Log arithmic reduction of the lev el of randomness in some probabilistic geometric constructions. J. F unct. A nal. , 2 3 5(1):297–329, 2006. [3] B. Barak, R. Impagliazzo, and A. Wigderson. Extracting randomness using few inde- p enden t sources. SIAM Journal on Computing , 36(4 ):1095–1118, 2006. 16 [4] B. Barak, G. Kindler, R. Shaltiel, B. Sudako v, and A. Wigderson. Sim ulating indep en- dence: New constructions of condensers, Ramsey graphs, dispersers, and extractors. In Pr o c e e dings of the 46th A CM Symp osium on The o ry of Com put ing , pages 1–10, 2005. [5] J. Bourgain, N. Katz, and T. T ao . A sum-pro duct estimate in finite fields, and appli- cations. Ge om . F unct. Anal. , 14(1):27– 57, 200 4. [6] J. Bourgain and S. V. Ko ny agin. Estimates for the num ber of sums and pro ducts and for exp onen tial sums o v er subgroups in fields of pr ime order. C. R. Math. A c ad. Sci. Paris , 337 (2):75–80, 2 0 03. [7] P . J. Cameron and J. J. Seidel. Quadratic forms ov er GF ( 2 ). Indag. Math. , 35:1 – 8, 1973. [8] M. R. Capalb o, O. Reingold, S. P . V adhan, and A. Wigderson. Randomness conductors and constan t-degree lossless expanders. In Pr o c e e dings of the 34th A CM Symp osium on The ory o f Computing , pages 659–668, 2 0 02. [9] R. A. D eV ore. Deterministic constructions of compressed sensing matr ices. Man uscript, 2007. [10] D. L. Donoho. Compressed sensing. IEEE T r ansactions on Information The ory , 52:1289–1 306, 2006. [11] D. L. Donoho and P . B. Stark. Uncertain t y principles and signal recov ery . SIAM J. Appl. Math. , 49( 3 ):906–931, 1 989. [12] P . Erd¨ os. A theorem of Sylv ester and Sc hur. J. L ondon Math. So c . , 9:282–288, 19 3 4. [13] T. Figiel, J. Lindenstrauss, a nd V. D. Milman. The dimension of almost spherical sections of con v ex b odies. A cta Math. , 139(1-2):53– 9 4, 197 7. [14] R. G. Gallager. L ow-Density Parity-Che ck Co des . MIT Pres s, 1963. [15] A. Garnaev and E. D . Gluskin. The widths of Euclidean balls. Doklad y A n. SSSR. , 277:1048– 1052, 1984 . [16] V. Gurusw ami, J. Lee, and A. Wigderson. Euclidean sections of with sublinear ran- domness a nd error-correction ov er the reals. In 12th Internationa l Wr ok shop on R an- domization and Combinatorial Optimization: Algorithms and T e chniques (RANDOM) , pages 44 4–454, 2008. [17] V. Gurusw ami, C. Umans, and S. P . V adhan. Un balanced expanders and random- ness extractors fro m Parv aresh-Vardy co des. In Pr o c e e dings of the 22nd Annual IEEE Confer enc e on Computational Complexity , pag es 96–108, 2007. [18] S. Ho ory , N. Linial, and A. Wigderson. Expande r graphs and their applications. Bul l. A mer. Math. So c. (N.S.) , 43(4) :439–561, 2006. 17 [19] P . Indyk. Stable distributions, pseudorandom generators, em b eddings, and data stream computation. Journal of the A CM , 53(3):307–323, 20 0 6. [20] P . Indyk . Uncertain t y principles, extractors, and explic it em b eddings o f L 1 in to L 2 . In Pr o c e e dings of the 39th A nnual A CM Symp osium on the The ory of Computing , pages 615–620, 2 007. [21] P . Indyk. Explicit constructions for compressed sensing of sparse signals. In Pr o c e e dings of the 19th A nnual ACM-SIAM Symp osium on Discr ete Algorithms , pag e s 30–33, 2 008. [22] W. B. Johnson and G. Sc hec h tman. Finite dimensional subspaces of L p . In Handb o ok of the ge ometry of Banach sp a c es, V ol. I , pag e s 837–8 70. North-Holland, Amsterdam, 2001. [23] B. S. Kashin. The widths of certain finite-dimens ional sets and classes of smo oth func- tions. Izv. Akad. Nauk SSSR Ser. Mat. , 41(2):334–35 1 , 478, 19 77. [24] B. S. Kashin and V. N. T emly ak o v. A remark o n compressed sensing. Av aila ble at http://www. dsp.ece.rice.edu/cs/KT2007.pdf , 2007 . [25] A. M. Kerdo c k. A class of low-rate nonlinear binary co des. Inform. Contr ol , 20:182–1 87, 1972. [26] N. Linial, E. London, and Y. Rabino vic h. The geometry of graphs and some of its algorithmic applications. Combinatoric a , 15(2):21 5–245, 199 5 . [27] S. Lov ett and S. So din. Almost Euclidean sections of the N -dimensional cross-p olytop e using O ( N ) random bits. Electronic Collo quium o n Computationa l Complexit y , Rep ort TR07-012, 20 07. [28] A. Lub otzky , R. Phillips, and P . Sarnak. Rama nujan graphs. Combinatoric a , 8(3):2 61– 277, 1988. [29] F. J. MacWilliams and N. J. A. Sloane. The The ory of Err or-C o rr e cting Co des . North- Holland, 197 7 . [30] V. Milman. T opics in asymptotic geometric analysis . Ge om . F unct. A nal. , (Sp ec ial V olume, P art II):792 –815, 2 000. GAF A 2 000 (T el Aviv, 199 9). [31] J. Radhakrishnan and A. T a-Shma. Bo unds fo r dispersers, extractors, and depth-tw o sup erc oncen trato rs. SIAM J. Disc r ete Math. , 13 (1):2–24 (electronic), 2000. [32] W. Rudin. T rigonometric series with gaps. J. Math. Me ch. , 9:203 –227, 1960. [33] G. Sc hec h tman. Random em b eddings of Euclid ean spaces in sequence spaces. Isr ael J. Math. , 40(2):1 87–192, 1981. 18 [34] C. Sc h ¨ utt. En tropy num b ers of diagonal op erators b et w een symmetric Banac h spaces. J. Appr ox. Th e ory , 40 ( 2 ):121–128, 1984. [35] M. Sipser and D. A. Spielman. Expander co des. IEEE T r ans. Inform. The ory , 42(6, part 1):1 7 10–1722, 1 996. Co des and complexit y . [36] S. Szarek. Con v exit y , complexit y , and high dimensions. In International Congr ess of Mathematicians. V ol. II , pages 1599–16 2 1. Eur. Math. So c., Z ¨ uric h, 2 006. [37] R. M. T anner. A recursiv e a pproac h to lo w complexit y co des . IEEE T r a nsactions on Information The ory , 27(5):533– 5 47, 19 8 1. [38] T. T ao and V. V u. A dditive c ombinatorics , v olume 105 of C ambridge Studies in A dvanc e d Mathematics . Cam bridge Univ ersit y Press, Cambridge, 200 6 . [39] G. Z´ emor. On expander co des. IEEE T r ansactions on Info rm ation The ory , 47 ( 2):835– 837, 2001. 19