Partitioning into Expanders

P artitioning in to Expanders Sha y a n Ov eis Gharan ∗ Luca T revisan † Decem b er 9, 2013 Abstract Let G = ( V , E ) b e an undirected graph, λ k be the k th smallest eigenv alue of the normalized laplacian matrix of G . There is a basic fact in alg ebraic graph theory that λ k > 0 if a nd only if G ha s at most k − 1 connected co mpo nents. W e prove a r obust version of this fact. If λ k > 0 , then for some 1 ≤ ℓ ≤ k − 1, V can b e p art itione d int o ℓ se ts P 1 , . . . , P ℓ such that each P i is a low-conductance set in G and induces a high conducta nce induced subgraph. In particular, φ ( P i ) . ℓ 3 √ λ ℓ and φ ( G [ P i ]) & λ k /k 2 . W e make o ur results alg orithmic by des igning a simple p olyno mia l time sp ectral a lgorithm to find such partitioning of G with a quadr atic loss in the inside conductance of P i ’s. Unlike the recent results on higher o rder Cheeg e r’s ineq uality [ LOT12 , LR TV12 ], o ur a lgorithmic results do not use higher order eigenfunctions of G . In addition, if there is a s ufficient ly large g ap b etw ee n λ k and λ k +1 , more precisely , if λ k +1 & po ly ( k ) λ 1 / 4 k then our a lgorithm finds a k partitioning of V int o sets P 1 , . . . , P k such that the induced s ubgraph G [ P i ] has a significantly larger co nducta nce than the co nductance of P i in G . Such a par titioning may repr esent the b est k clustering of G . Our algorithm is a simple lo cal search tha t only uses the Sp ectral Partitioning algor ithm as a subr o utine. W e exp ect to see further applications of this simple alg orithm in clustering applications. Let ρ ( k ) = min disjoint A 1 ,...,A k max 1 ≤ i ≤ k φ ( A i ) b e the order k conducta nc e consta nt o f G , in words, ρ ( k ) is the s ma llest v a lue of the maximum conducta nce o f any k disjoint subsets o f V . Our ma in technical lemma shows that if (1 + ǫ ) ρ ( k ) < ρ ( k + 1), then V ca n b e partitioned int o k sets P 1 , . . . , P k such that fo r ea ch 1 ≤ i ≤ k , φ ( G [ P i ]) & ǫ · ρ ( k + 1) /k a nd φ ( P i ) ≤ k · ρ ( k ). This significantly improv es a r e cent r esult of T anak a [ T an12 ] who assumed an exp onential (in k ) ga p be t ween ρ ( k ) and ρ ( k + 1). ∗ Computer Science Division, U.C. Berkeley . This material is supp orted by a Stanford graduate fello wship and a Miller fello wship. Email: oveisghara n@berkeley.edu . † Department of Computer Science, Stan ford Universit y . This ma terial is based up on w ork supp orted b y the National Science F oun dation un der grants No. CCF 1017403 and CCF 1216642. Email: trevisan@s tanford.edu . Figure 1: In this example although b oth sets in the 2-partitioning are of small conductance, in a natural clustering the red vertex (middle v ertex) will b e merged with the left cluster 1 In tro d uction Clustering is one of the f undamental primitives in mac h ine learning and data analysis with a v ariety of app lications in information retriev al, pattern recognition, recommendation systems, etc. Data clustering may b e mo d eled as a graph partitioning problem, where one mo dels eac h of the d ata p oints as a verte x of a graph and the weig ht of an edge connecting tw o v ertices represent s the similarit y of the corresp ond ing data p oin ts. W e assu me the we ight is larger if the p oints are m ore similar (see e.g. [ NJW02 ]). Let G = ( V , E ) b e an un d irected graph with n := | V | vertice s . F or all p air of v ertices u, v ∈ V let w ( u, v ) ≥ 0 b e the w eigh t of the edge b et ween u and v (we let w ( u, v ) = 0 if there is no ed ge b et w een u and v ). There are sev eral com b in atorial measures for the qualit y of a k -w a y partitioning of a graph including diameter, k -cente r , k -median, conductance, etc. Kannan, V emp ala and V etta [ KVV04 ] sho w that sev eral of these measures fail to captur e the natural clustering in simple examples. Klein b erg [ Kle02 ] sh o w that there is n o unifi ed clustering function s atisfying thr ee basic prop er ties. Kannan et al. [ KVV04 ] p r op ose condu ctance as one of the b est ob jectiv e fun ctions for measuring the qu alit y of a cluster. F or a subset S ⊆ V , let th e volume of S b e vo l ( S ) := P v ∈ S w ( v ), wh er e w ( v ) := P u ∈ V w ( v , u ) is the w eighte d degree of a vertex v ∈ V . The c onductanc e of S is defin ed as φ G ( S ) := w ( S, S ) v ol( S ) , where S = V − S , and w ( S, S ) = P u ∈ S,v ∈ S w ( u, v ) is the sum of the we ight of th e edges in the cut ( S, S ). Th e subscr ip t G in the ab o ve definition m a y b e omitted. F or example, if φ ( S ) = 0 . 1 it means th at 0 . 9 f raction of the neigh b ors of a r andom vertex of S (chosen prop ortional to degree) are in s ide S in exp ectation. The conductance of G , φ ( G ) is the s m allest cond uctance among all sets that ha ve at most half of the total v olume, φ ( G ) := min S : vo l( S ) ≤ v ol ( V ) / 2 φ ( S ) . One approac h for constructing a k -clustering of G is to fi nd k s ets of small conductance. Shi and Malik [ SM00 ] s ho w that this metho d provides high qu ality solutions in image segmen tation applications. Recen tly , Lee et al. [ LOT12 ] and Louis et al. [ LR TV12 ] designed sp ectral algorithms for findin g a k -wa y partitioning wh ere ev ery set h as a small condu ctance. It tu rns out that in many graphs ju st the fact that a set S h as a small condu ctance is not enough to argue that it is a go o d 1 Figure 2: Tw o 4-partitioning of the cycle graph . In b oth of the partitionings the num b er of edges b et ween the clusters are exactly 4, and the inside condu ctance of all comp onen ts is at least 1/2 in b oth cases. But, th e right clustering is a more natural clustering of cycle. cluster; this is b ecause although φ ( S ) is small, S can b e lo osely-connected or eve n disconnected inside (see Figure 1 ). Kannan, V empala and V etta [ KVV04 ] prop osed a bicriteria measure, where they measure the qualit y of a k -clusterin g based on the inside conductance of s ets and the num b er of edges b et ween the clusters. F or P ⊆ V let φ ( G [ P ]) b e the inside c onductanc e of P , i.e., the condu ctance of the indu ced subgraph of G on the v ertices of P . Kannan et al. [ KVV04 ] suggested that a k - partitioning int o P 1 , . . . , P k is go o d if φ ( G [ P i ]) is large, and P i 6 = j w ( P i , P j ) is small. It tu rns out that an approximat e solution for this ob jectiv e function can b e v ery different than the “correct” k -partitioning. Cons id er a 4-partitioning of a cycle as we illustrate in Figure 2 . Although the ins ide conductance of ev ery set in the left partitioning is within a factor 2 of the righ t partitioning, the left partitioning do es not p ro vide the “correct” 4-partitioning of a cycle. In this p ap er we prop ose a th ird ob jectiv e wh ic h uses b oth of the insid e/outside condu ctance of the clusters. Roughly sp eaking, S ⊆ V represen ts a go o d cluster wh en φ ( S ) is small, bu t φ ( G [ S ]) is large. In other wo r ds, although S do esn’t expand in G , the in duced sub graph G [ S ] is an expander. Definition 1.1. W e say k disjoint subsets A 1 , . . . , A k of V ar e a ( φ in , φ out ) -clustering, if for al l 1 ≤ i ≤ k , φ ( G [ A i ]) ≥ φ in and φ G ( A i ) ≤ φ out . One of the main con tribu tions of the pap er is to stud y graphs that contai n a k -partitioning suc h that φ in ≫ φ out . T o the b est of our knowle d ge, th e only th eoretical resu lt that guaran tees a ( φ in , φ out ) p artitioning of G is a recen t result of T anak a [ T an12 ]. F or an y k ≥ 2, let ρ ( k ) b e the maxim um conductance of any k d isjoin t sub sets of G , ρ ( k ) := min disjoint A 1 ,...,A k max 1 ≤ i ≤ k φ ( A i ) . F or example, observe th at ρ (2) = φ ( G ) . T anak a [ T an12 ] pr o ve d that if there is a large enough gap b et ween ρ ( k ) and ρ ( k + 1) then G has a k -partitioning that is a (exp( k ) ρ ( k ) , ρ ( k + 1) / exp( k ))- clustering. Theorem 1.2 (T anak a [ T an12 ]) . If ρ G ( k + 1) > 3 k +1 ρ G ( k ) for some k , then G has a k -p artitioning that is a ( ρ ( k + 1) / 3 k +1 , 3 k ρ ( k )) -clustering. 2 Unfortunately , T anak a requ ires a v ery large gap (exp onen tial in k ) b et w een ρ ( k ) and ρ ( k + 1). F ur th ermore, the ab o ve r esult is not algorithmic, in the sense that he needs to find the optim um sparsest cut of G or its indu ced sub graphs to construct the k -partitioning. 1.1 Related W orks Let A b e the adj acency matrix of G and L := I − D − 1 / 2 AD − 1 / 2 b e th e norm alized laplacian of G with eigen v alues 0 = λ 1 ≤ λ 2 ≤ . . . λ n ≤ 2. Cheeger’s inequalit y offers the follo wing quan titativ e connection b et ween ρ (2) and λ 2 : Theorem 1.3 (Ch eeger’s in equalit y) . F or any gr aph G , λ 2 2 ≤ φ ( G ) ≤ p 2 λ 2 . F urthermo r e, ther e is a simple ne ar-line ar time algorithm (the Sp e ctr al Partitioning algorithm) that finds a set S such that v ol ( S ) ≤ vo l ( V ) / 2 , and φ ( S ) ≤ p 4 φ ( G ) . The ab o ve inequ alit y can b e read as follo ws: a graph G is n early d isconnected if and only if λ 2 is ve r y close to zero. The imp ortance of Ch eeger’s inequalit y is that it do es not dep end on th e size of the graph G , and so it is app licable to m assiv e graphs app earing in practical applications. V ery recen tly , Lee et al. [ LOT12 ] prov ed higher ord er v ariants of Cheeger’s inequ alit y (see also [ LR TV12 ]). In particular, they show th at for an y graph G , ρ ( k ) very well c h aracterizes λ k . Theorem 1.4 (Lee et al. [ LOT12 ]) . F or any gr aph G and k ≥ 2 , λ k / 2 ≤ ρ ( k ) ≤ O ( k 2 ) p λ k . Mek a, Moitra and Sriv asta v a [ MMS13 ] studied existence of Θ( k ) expander graphs co vering most v ertices of a graph where the conductance of eac h expander is a fu nction of λ k . Kannan, V emp ala and V etta in [ KVV04 ] d esigned an appro ximation algorithm to find a par- titioning of a graph that cuts v ery few edges and eac h set in th e partitioning h as a large inside conductance. Comparing to Definition 1.1 instead of minimizing φ ( A i ) for eac h set A i they min i- mize P i φ ( A i ). V ery recent ly , Zhu, Lattanzi and Mirrokni [ ZLM13 ] designed a lo c al algorithm to find a set S such that φ ( S ) is small and φ ( G [ S ]) is large assum ing that such a s et exists. Both of these r esults do not argue ab out the existence of a partitioning with large inside cond uctance. F ur- thermore, u nlik e Cheeger type inequalities the qualit y of approximati on factor of these algorithms dep end s on the s ize of the in put graph (or th e size of the cluster S ). 1.2 Our Contributions P artitioning in t o E xpanders There is a basic fact in algebraic graph theory that f or an y graph G and any k ≥ 2, λ k > 0 if and only if G has at most k − 1 connected comp onen ts. It is a natural question to ask for a robu st version of this fact. Our main existen tial theorem provides a robust v ersion of this fact. Theorem 1.5. F or any k ≥ 2 if λ k > 0 , then for some 1 ≤ ℓ ≤ k − 1 ther e is a ℓ -p artitioning of V into sets P 1 , . . . , P ℓ that is a (Ω( ρ ( k ) /k 2 ) , O ( ℓρ ( ℓ ))) = (Ω( λ k /k 2 ) , O ( ℓ 3 ) √ λ ℓ ) clustering. The ab o v e th eorem can b e seen as a generalization of Theorem 1.4 . 3 Algorithmic Results The ab o ve r esu lt is n ot algorithmic b ut with some loss in th e parameters w e can mak e them algorithmic. Theorem 1.6 (Algorithmic Theorem) . Ther e is a simple lo c al se ar ch algorithm that for any k ≥ 1 if λ k > 0 finds a ℓ - p artitioning of V i nto sets P 1 , . . . , P ℓ that is a (Ω( λ 2 k /k 4 ) , O ( k 6 p λ k − 1 ) wher e 1 ≤ ℓ < k . If G is unweighte d the algorithm runs in a p olynomial time in the size of G . The d etails of the ab o ve algorithm are describ ed in Algorithm 3 . W e remark that the algorithm do es not use an y SDP or LP r elaxation of th e problem. I t only u ses the S p ectral Partiti onin g algorithm as a subr outine. F u rthermore, unlike the sp ectral clustering algorithms stud ied in [ NJW02 , LOT12 ], our algorithm d o es n ot use multiple eigenfunctions of the n ormalized laplacian matrix. It rather iterativ ely refines a partitioning of G by add ing non-expand in g sets that ind uce an expan d er. Supp ose that there is a large gap b et we en λ k and λ k +1 . Then, the ab o v e theorem (together with 1.13 ) imp lies that there is a k partitioning of V s u c h that inside conductance of eac h set is significan tly larger than its outside conductance in G . F urtherm ore, such a partitioning can b e found in p olynomial time. This p artitioning ma y repr esent one of the b est k -clusterings of the graph G . If instead of the Sp ectral Pa r titioning algorithm we use the O ( √ log n )-appro ximation algorithm for φ ( G ) deve lop ed in [ AR V09 ] th e same pro of imp lies that P 1 , . . . , P ℓ are a  Ω  λ k k 2 · p log( n )  , k 3 p λ k − 1  clustering. T o th e b est of our kn owledge, the ab o v e th eorem pro vides the first p olynomial time algorithm that establishes a Cheeger-t yp e inequalit y for the insid e/outside conductance of sets in a k -wa y partitioning. Main T echnic al Result The main tec hn ical result of this pap er is the follo win g theorem. W e sho w that ev en if there is a v ery small gap b et ween ρ ( k ) and ρ ( k + 1) we can guaran tee the existence of a (Ω k ( ρ ( k + 1)) , O k ( ρ ( k )))-clustering where we in Ω k ( . ) , O k ( . ) notations w e dropp ed the d ep enden cy to k . Theorem 1.7 (Existentia l Theorem) . If ρ G ( k + 1) > (1 + ǫ ) ρ G ( k ) for some 0 < ǫ < 1 , then i) Ther e e xists k disjoint su bsets of V that ar e a ( ǫ · ρ ( k + 1) / 7 , ρ ( k )) -clustering. ii) Ther e exists a k -p artitioning of V that i s a ( ǫ · ρ ( k + 1) / (14 k ) , k ρ ( k )) -clustering. The imp ortance of the ab o ve theorem is that the gap is ev en indep enden t of k and it can b e made arbitrarily close to 0. Compared to Theorem 1.2 , w e require a very s mall gap b et we en ρ ( k ) and ρ ( k + 1) and the qualit y of our k -p artitioning has a linear loss in terms of k . W e sho w tigh tness of ab o v e theorem in S ubsection 1.3 . Using the ab o ve theorem it is easy to pr o ve Theorem 1.5 . Pr o of of The or em 1.5 . Assume λ k > 0 for some k ≥ 2. By Theorem 1.4 we can assume ρ ( k ) ≥ λ k / 2 > 0. Since ρ (1) = 0 we hav e (1 + 1 /k ) ρ ( ℓ ) < ρ ( ℓ + 1) at least for one ind ex 1 ≤ ℓ < k . Let ℓ b e th e largest index su c h that (1 + 1 /k ) ρ ( ℓ ) < ρ ( ℓ + 1); it follo ws that ρ ( k ) ≤ (1 + 1 /k ) k − ℓ − 1 ρ ( ℓ + 1) ≤ e · ρ ( ℓ + 1) . (1) 4 Therefore, by p art (ii) of Theorem 1.7 th ere is a ℓ -partitioning of V into sets P 1 , . . . , P ℓ suc h that for all 1 ≤ i ≤ ℓ , φ ( G [ P i ]) ≥ ρ ( ℓ + 1) 14 k · ℓ ≥ ρ ( k ) 40 k 2 ≥ λ k 80 k 2 , and φ ( P i ) ≤ ℓρ ( ℓ ) ≤ O ( ℓ 3 ) p λ ℓ . where we used ( 1 ) and Theorem 1.4 . The follo wing corollary follo ws. Building on Theorem 1.4 we can also pro ve the existence of a go o d k -partitioning of G if there is a large en ou gh gap b et we en λ k and λ k +1 . Corollary 1.8. Ther e is a u niversal c onstant c > 0 , suc h that for any gr aph G if λ k +1 ≥ c · k 2 √ λ k , then ther e e xi sts a k -p artitioning of G that is a (Ω( λ k +1 /k ) , O ( k 3 √ λ k )) -clustering. 1.3 Tigh tness of E xisten tial Theorem In th is part we provide sev eral examples sh o wing the tight n ess of Theorem 1.7 . In the first example w e show that if ther e is no gap b etw een ρ ( k ) an d ρ ( k + 1) then G cannot b e partitioned in to expanders. Example 1.9. In the first example we c onstruct a gr aph such that ther e i s no gap b etwe en ρ ( k ) and ρ ( k + 1) and we show that in any k -p artitioning ther e is a set P such that φ ( G [ P ]) ≪ ρ ( k + 1) . Supp ose G is a star. Then, for any k ≥ 2 , ρ ( k ) = 1 . Bu t, among any k disjoint subsets of G ther e is a set P with φ ( G [ P ]) = 0 . Ther efor e, for any k ≥ 2 , ther e is a set P with φ ( G [ P ]) ≪ ρ ( k + 1) . In the next example we sho w that a line ar loss in k is necessary in the qualit y of our k -partitioning in part (ii) of Theorem 1.7 . Example 1.10. In this example we c onstruct a gr aph such that i n any k -p artitioning ther e is a set P with φ ( P ) ≥ Ω( k · ρ ( k )) . F urthermo r e, i n any k p artitioning wher e the c onductanc e of every set is O k ( ρ ( k )) , ther e is a set P such that φ ( G [ P ]) ≤ O ( ρ ( k + 1) /k ) . L et G b e a u nion of k + 1 cliques C 0 , C 1 , . . . , C k e ach with ≈ n / ( k + 1) vertic es wher e n ≫ k . Also , for any 1 ≤ i ≤ k , include an e dge b etwe en C 0 and C i . In this gr aph ρ ( k ) = Θ( k 2 /n 2 ) by cho osing the k disjoint sets C 1 , . . . , C k . F urthermor e , ρ ( k + 1) = Θ( k · ρ ( k )) . Now c onsider a k p artitioning of G . First of al l if ther e is a set P in the p artitioning that c ontains a pr op er subset of one the cliques, i.e., ∅ ⊂ ( P ∩ C i ) ⊂ C i for some i , then φ ( P ) ≥ Ω k (1 /n ) = Ω k ( n · ρ ( k )) . Otherwise, every clique i s mapp e d to one of the sets in the p artitioning. Now, let P b e the set c ontaining C 0 ( P may c ontain at most one other clique). It fol lows that φ ( P ) = Ω( k · ρ ( k )) . Now, su pp ose we have a p artitioning of G into k sets such that the c onductanc e of e ach set is O k ( ρ ( k )) . By the ar g u ments in ab ove p ar agr aph none of the sets in the p artitioning c an have a pr op er subset of one cliques. Sinc e we have k + 1 cliques ther e is a set P that c ontains exactly two cliques C i , C j , for i 6 = j . It fol lows that φ ( G [ P ]) ≤ O ( ρ ( k ) /k ) . 1.4 Notations F or a function f : V → R let R ( f ) := P ( u,v ) ∈ E w ( u, v ) · | f ( u ) − f ( v ) | 2 P v ∈ V w ( v ) f ( v ) 2 5 The su p p ort of f is the set of ve r tices with non-zero v alue in f , supp( f ) := { v ∈ V : f ( v ) 6 = 0 } . W e say t wo fu nctions f , g : V → R are d isjoin tly sup p orted if supp( f ) ∩ supp( g ) = ∅ . F or S ⊆ P ⊆ V we use φ G [ P ] ( S ) to denote the condu ctance of S in the in duced sub graph G [ P ]. F or S, T ⊆ V w e use w ( S → T ) := X u ∈ S,v ∈ T − S w ( u, v ) . W e r emark that in the ab o ve definition S and T are not necessarily d isjoin t, so w ( S → T ) is not necessarily the same as w ( T → S ) . F or S ⊆ B i ⊆ V w e define ϕ ( S, B i ) := w ( S → B i ) v ol( B i − S ) v ol( B i ) · w ( S → V − B i ) Let us motiv ate the ab o v e defin ition. Supp ose B i ⊆ V such that φ G ( B i ) is v ery small bu t φ ( G [ B i ]) is ve r y large. Then, an y S ⊆ B i suc h that v ol( S ) ≤ v ol ( B i ) / 2 satisfy the f ollo w ing prop erties. • Sin ce φ G [ B i ] ( S ) is large, a large f raction of edges adjacen t to v ertices of S m u st lea ve this s et. • Sin ce φ G ( B i ) is small, a small fraction of edges adjacen t to S may lea v e B i . Putting ab o ve prop erties together we obtain that w ( S → B i ) & w ( S → V − B i ), thus ϕ ( S, B i ) is at least a constan t. As we describ e in the next section the conv erse of this argument is a cru cial part of our pro of. In particular, if for any S ⊆ B i , ϕ ( S, B i ) is large, then B i has large inside conductance, and it can b e used as the “bac kb one” of our k -partitioning. 1.5 Ov erview of the Pro of W e p ro ve Theorem 1.7 in tw o steps. Let A 1 , . . . , A k b e an y k disjoint sets suc h that φ ( A i ) ≤ (1 + ǫ ) ρ ( k + 1). In th e first step we find B 1 , . . . , B k suc h th at for 1 ≤ i ≤ k , φ ( B i ) ≤ φ ( A i ) with the crucial prop er ty that any sub set of B i has at least a constant f r action of its outgoing ed ges inside B i . W e then use B 1 , . . . , B k as the “bac kb one” of our k -partitioning. W e merge the remaining v ertices with B 1 , . . . , B k to obtain P 1 , . . . , P k making sure that f or eac h S ⊆ P i − B i at least 1 /k fraction of the outgoing edges of S go to P i (i.e., w ( S → P i ) ≥ w ( S → V ) /k ). W e sh o w th at if 2 max 1 ≤ i ≤ k φ ( A i ) < ρ ( k + 1) then we can construct B 1 , . . . , B k suc h that ev ery S ⊆ B i satisfies ϕ ( S, B i ) ≥ Ω (1) (see Lemma 2.1 ). F or example, if v ol( S ) ≤ vo l ( B i ) / 2, w e obtain that w ( S → B i − S ) & w ( S → V ) . This prop erty sho w s that eac h B i has an inside conductance of Ω( ρ ( k + 1)) (see Lemma 2.3 ). In addition, it implies th at any sup ers et of B i , P i ⊇ B i , has an inside conductance φ ( G [ P i ]) & α · ρ ( k + 1) as long as for any S ⊆ P i − B i , w ( S → B i ) ≥ α · w ( S → V ) (see Lemma 2.6 ). By latter observ ation w e just need to merge the vertice s in V − B 1 − . . . − B k with B 1 , . . . , B k and obtain a k -partitioning P 1 , . . . , P k suc h that for any S ⊆ P i − B i , w ( S → P i ) ≥ w ( S → V ) /k . 6 1.6 Bac kground on H igher Order Cheeger’s Inequalit y In this sh ort s ection we u se the m achinery develo p ed in [ LOT12 ] to s h o w that for any partitioning of V int o ℓ < k sets P 1 , . . . , P ℓ the min im um inside condu ctance of P i ’s is p oly ( k ) √ λ k . Theorem 1.11 (Lee et al.[ LOT12 ]) . Ther e is a universal c onstant c 0 > 0 such that for any gr aph G = ( V , E ) and 1 ≤ k ≤ n ther e exists k disjointly supp orte d functions f 1 , . . . , f k : V → R such that for e ach 1 ≤ i ≤ k , R ( f i ) ≤ c 0 k 6 λ k . Prop osition 1.12 (Kwo k et al. [ KLL + 13 ]) . F or any gr aph G = ( V , E ) , any k ≥ 1 and any k disjointly supp orte d functions f 1 , . . . , f k : V → R we have λ k ≤ 2 max 1 ≤ i ≤ k R ( f i ) . Lemma 1.13. Ther e is a universal c onstant c 0 > such that for any k ≥ 2 and any p artitioning of V into ℓ sets P 1 , . . . , P ℓ of V wher e ℓ ≤ k − 1 , we have min 1 ≤ i ≤ ℓ λ 2 ( G [ P i ]) ≤ 2 c 0 k 6 λ k . wher e λ 2 ( G [ P i ]) is the se c ond eigenvalue of the normalize d laplacian matrix of the induc e d gr aph G [ P i ] . Pr o of. Let f 1 , . . . , f k b e the fi rst k eigenfunctions of L corresp onding to λ 1 , . . . , λ k . By definition R ( f i ) = λ i . By Theorem 1.11 th ere are k d isjoin tly supp orted functions g 1 , . . . , g k suc h that R ( g i ) ≤ c 0 k 6 λ k . F or an y 1 ≤ j ≤ ℓ , let g i,j b e th e restriction of g i to the indu ced su bgraph G [ P j ]. It follo ws that R ( g i ) ≥ P ℓ j =1 P ( u,v ) ∈ E ( P j ) | g i ( v ) − g i ( u ) | 2 P ℓ j =1 P v ∈ P j g i ( v ) 2 ≥ min 1 ≤ j ≤ ℓ P ( u,v ) ∈ E ( P j ) | g i ( u ) − g i ( v ) | 2 P v ∈ P j g i ( v ) 2 = min 1 ≤ j ≤ ℓ R ( g i,j ) . (2) F or eac h 1 ≤ i ≤ k let j ( i ) := argmin 1 ≤ j ≤ ℓ R ( g i,j ). Since ℓ < k , by the p igeon hole principle, there are tw o ind ices 1 ≤ i 1 < i 2 ≤ k such th at j ( i 1 ) = j ( i 2 ) = j ∗ for some 1 ≤ j ∗ ≤ ℓ . Since g 1 , . . . , g k are disj oin tly supp orted, b y Prop osition 1.12 λ 2 ( G [ P j ∗ ]) ≤ 2 max {R ( g i 1 ,j ∗ ) , R ( g i 2 ,j ∗ ) } ≤ 2 max {R ( g i 1 ) , R ( g i 2 ) } ≤ 2 c 0 k 6 λ k . where the second in equalit y follo ws by ( 2 ). The ab o v e lemma is u sed in the pr o of of T heorem 1.6 . 2 Pro of of Existen tial Theorem In this s ection we prov e Theorem 1.7 . Let A 1 , . . . , A k are k disjoin t sets suc h th at φ ( A i ) ≤ ρ ( k ) for all 1 ≤ i ≤ k . In the first lemma we construct k disj oin t sets B 1 , . . . , B k suc h that their conductance in G is only b etter th an A 1 , . . . , A k with the additional prop ert y that ϕ ( S, B i ) ≥ ǫ/ 3 for any S ⊆ B i . 7 Lemma 2.1. L et A 1 , . . . , A k b e k disjoint sets s.t. (1 + ǫ ) φ ( A i ) ≤ ρ ( k + 1) for 0 < ǫ < 1 . F or any 1 ≤ i ≤ k , ther e exist a set B i ⊆ A i such that the fol lowing holds: 1. φ ( B i ) ≤ φ ( A i ) . 2. F or any S ⊆ B i , ϕ ( S, B i ) ≥ ǫ/ 3 . Pr o of. F or eac h 1 ≤ i ≤ k we r un Algorithm 1 to construct B i from A i . Note that although the algorithm is constructiv e, it m a y not r u n in p olynomial time. T he reason is that we don’t know an y (constan t f actor approximat ion) algorithm for m in S ⊆ B i ϕ ( S, B i ). Algorithm 1 Construction of B 1 , . . . , B k from A 1 , . . . , A k B i = A i . lo op if ∃ S ⊂ B i suc h that ϕ ( S, B i ) ≤ ǫ/ 3 then , Up date B i to either of S or B i − S w ith the smallest condu ctance in G . else return B i . end if end lo op First, obs erv e that the algorithm alw a ys terminates after at most | A i | iterations of the lo op s in ce | B i | d ecreases in eac h iteration. The output of the algorithm alw a ys satisfies conclusion 2 of the lemma. So, w e only n eed to b oun d the conductance of the output s et. W e show that throu gh ou t the algorithm we alwa ys ha v e φ ( B i ) ≤ φ ( A i ) . (3) In fact, w e p ro ve something stronger. That is, the conductance of B i nev er increases in the entire run of th e algorithm. W e pro v e th is by ind u ction. A t the b eginnin g B i = A i , so ( 3 ) ob viously holds. It remains to prov e the inductive step. Let S ⊆ B i suc h that ϕ ( S, B i ) ≤ ǫ/ 3. Among the k + 1 d isj oin t sets { A 1 , . . . , A i − 1 , S, B i − S, A i +1 , A k } there is one of conductance ρ G ( k + 1). So, max { φ ( S ) , φ ( B i − S ) } ≥ ρ G ( k + 1) ≥ (1 + ǫ ) φ ( A i ) ≥ (1 + ǫ ) φ ( B i ) . The ind uctiv e step follo ws from the follo wing lemma. Lemma 2.2. F or any set B i ⊆ V and S ⊂ B i , if ϕ ( S, B i ) ≤ ǫ/ 3 and max { φ ( S ) , φ ( B i − S ) } ≥ (1 + ǫ ) φ ( B i ) , (4) then min { φ ( S ) , φ ( B i − S ) } ≤ φ ( B i ) . Pr o of. Let T = B i − S . Since ϕ ( S, B i ) ≤ ǫ/ 3, w ( S → T ) ≤ ǫ 3 · v ol( T ) v ol( B i ) · w ( S → V − B i ) ≤ ǫ 3 · w ( S → V − B i ) . (5) W e consider t wo cases d ep end in g on whether φ ( S ) ≥ (1 + ǫ ) φ ( B i ). 8 Case 1: φ ( S ) ≥ (1 + ǫ ) φ ( B i ) . First, by ( 5 ). (1 + ǫ ) φ ( B i ) ≤ φ ( S ) = w ( S → T ) + w ( S → V − B i ) v ol( S ) ≤ (1 + ǫ/ 3) w ( S → V − B i ) v ol( S ) (6) Therefore, φ ( T ) = w ( B i → V ) − w ( S → V − B i ) + w ( S → T ) v ol( T ) ≤ w ( B i → V ) − (1 − ǫ/ 3) w ( S → V − B i ) v ol( T ) ≤ φ ( B i )(v ol( B i ) − v ol ( S )(1 + ǫ/ 2)(1 − ǫ/ 3)) v ol( T ) ≤ φ ( B i ) v ol( T ) v ol( T ) = φ ( B i ) . where the fir s t inequalit y f ollo ws b y ( 5 ) and the second in equalit y follo ws by ( 6 ) and that ǫ ≤ 1. Case 2: φ ( T ) ≥ (1 + ǫ ) φ ( B i ) . First, (1 + ǫ ) φ ( B i ) ≤ φ ( T ) = w ( S → T ) + w ( T → V − B i ) v ol( T ) (7) Therefore, φ ( S ) = w ( B i → V ) − w ( T → V − B i ) + w ( S → T ) v ol ( S ) ≤ w ( B i → V ) − (1 + ǫ ) φ ( B i ) v ol( T ) + 2 w ( S → T ) v ol( S ) ≤ φ ( B i )(v ol( B i ) − (1 + ǫ ) v ol ( T )) + 2 ǫ 3 · vo l ( T ) · φ ( B i ) v ol( S ) ≤ φ ( B i ) v ol( S ) v ol( S ) = φ ( B i ) . where the first inequalit y follo ws b y ( 7 ), the second inequalit y follo ws b y ( 5 ) and that w ( S → V − B i ) ≤ w ( B i → V − B i ). So we get φ ( S ) ≤ φ ( B i ). This completes the pro of of Lemma 2.2 . This completes the pro of of Lemma 2.1 . Note that sets that we construct in th e ab o v e lemma do n ot necessarily d efi ne a partitioning of G . In the n ext lemma w e sho w that the sets B 1 , . . . , B k that are constructed ab ov e hav e large inside conductance. Lemma 2.3. L et B i ⊆ V , and S ⊆ B i such that v ol ( S ) ≤ vol( B i ) / 2 . If ϕ ( S, B i ) , ϕ ( B i − S, B i ) ≥ ǫ/ 3 for ǫ ≤ 1 , then φ G [ B i ] ( S ) ≥ w ( S → B i ) v ol( S ) ≥ ǫ 7 · max { φ ( S ) , φ ( B i − S ) } , 9 Pr o of. Let T = B i − S . First, we low er b ound φ G [ B i ] ( S ) by ǫ · φ ( S ) / 7. S in ce ϕ ( S, B i ) ≥ ǫ/ 3, w ( S → B i ) v ol( S ) = ϕ ( S, B i ) · v ol( T ) v ol( B i ) · w ( S → V − B i ) v ol( S ) ≥ ǫ · w ( S → V − B i ) 6 v ol ( S ) where the first inequalit y follo ws by the assumption v ol( S ) ≤ vol( B i ) / 2. Summing up b oth sides of the ab o ve inequality w ith ǫw ( S → B i ) 6 vol( S ) and divid ing by 1 + ǫ/ 6 we obtain w ( S → B i ) v ol( S ) ≥ ǫ/ 6 (1 + ǫ/ 6 · · w ( S → V ) v ol( S ) ≥ ǫ · φ ( S ) 7 . where we used ǫ ≤ 1. It remains to φ G [ B i ] ( S ) by ǫ · φ ( B i − S ) / 7. Since ϕ ( T , B i ) ≥ ǫ/ 3, w ( S → B i ) v ol ( S ) = w ( T → B i ) v ol( S ) = ϕ ( T , B i ) · w ( T → V − B i ) v ol( B i ) ≥ ǫ 3 · w ( T → V − B i ) v ol( B i ) ≥ ǫ 6 · w ( T → V − B i ) v ol( T ) where the last inequ alit y follo ws by the assu mption v ol ( S ) ≤ vol( B i ) / 2. Su mming up b oth sides of the ab o v e inequalit y with ǫ · w ( S → B i ) 6 vol( S ) w e obtain, (1 + ǫ/ 6) w ( S → B i ) v ol ( S ) ≥ ǫ 6 · w ( T → V ) v ol ( T ) ≥ ǫ · φ ( T ) 6 . where we used the assump tion v ol( S ) ≤ v ol ( B i ) / 2. Th e lemma follo ws u sing the fact that ǫ ≤ 1. Let B 1 , . . . , B k b e the sets constructed in Lemma 2.1 . T hen, for eac h B i and S ⊆ B i since φ ( B j ) < ρ ( k + 1) for all 1 ≤ j ≤ k , we get max( φ ( S ) , φ ( T )) ≥ ρ ( k + 1) . Therefore, by the ab o v e lemma, for all 1 ≤ i ≤ k , φ ( G [ B i ]) ≥ ǫ · ρ ( k + 1) / 7, and φ ( B i ) ≤ max 1 ≤ i ≤ k φ ( A i ) ≤ ρ ( k ) . This completes the pro of of part (i) of Theorem 1.7 . It remains to p ro ve part (ii). T o p ro ve part (ii) we hav e to tur n B 1 , . . . , B k in to a k -p artitioning. W e run the follo win g algorithm to mer ge th e v ertices that are not included in B 1 , . . . , B k . Again, although this algorithm is constructiv e, it m a y not run in p olynomial time. The m ain d ifficult y is in fin ding a set S ⊂ P i − B i suc h that w ( S → P i ) < w ( S → P j ), if suc h a set exists. First, observe that ab ov e algorithm alw a ys terminates in a fin ite n umb er of steps. This is b ecause in eac h iteration of the lo op the w eight of edges b etw een the sets decreases. That is, X 1 ≤ i 1 suc h that λ k > 0. Output: A ( φ 2 in / 4 , φ out ) ℓ -partitioning of G for some 1 ≤ ℓ < k . 1: Let ℓ = 1, P 1 = B 1 = V . 2: while ∃ 1 ≤ i ≤ ℓ suc h that w ( P i − B i → B i ) < w ( P i − B i → P j ) for j 6 = i , or Sp ectral P artitioning fin d s S ⊆ P i s.t. φ G [ P i ] ( S ) , φ G [ P i ] ( P i − S ) < φ in do 3: Assume (after renaming) v ol( S ∩ B i ) ≤ vo l ( B i ) / 2. 4: Let S B = S ∩ B i , S B = B i ∩ S , S P = ( P i − B i ) ∩ S and S P = ( P i − B i ) ∩ S (see Figure 3 ). 5: if max { φ ( S B ) , φ ( S B ) } ≤ (1 + 1 /k ) ℓ +1 ρ ∗ then 6: Let B i = S B , P ℓ +1 = B ℓ +1 = S B and P i = P i − S B . Set ℓ ← ℓ + 1 and goto step 2 . 7: end if 8: if max { ϕ ( S B , B i ) , ϕ ( S B , B i ) } ≤ 1 / 3 k , then 9: Up date B i to either of S B or S B with the smallest conductance, and goto step 2 . 10: end if 11: if φ ( S P ) ≤ (1 + 1 /k ) ℓ +1 ρ ∗ then 12: Let P ℓ +1 = B ℓ +1 = S P , P i = P i − S P . Set ℓ ← ℓ + 1 and goto step 2 . 13: end if 14: if w ( P i − B i → P i ) < w ( P i − B i → B j ) for j 6 = i , then 15: Up date P j = P j ∪ ( P i − B i ), and let P i = B i and goto s tep 2 . 16: end if 17: if w ( S P → P i ) < w ( S P → P j ) for j 6 = i , then 18: Up date P i = P i − S P and merge S P with argmax P j w ( S P → P j ). 19: end if 20: e nd while return P 1 , . . . , P ℓ . Observe th at in the entire ru n of the algorithm B 1 , . . . , B ℓ are alw a ys disjoint , B i ⊆ P i and P 1 , . . . , P ℓ form an ℓ -partitioning of V . W e pro ve Algorithm 3 by a sequence of steps. Claim 3.1. Thr oughout the algorithm we always have max 1 ≤ i ≤ ℓ φ ( B i ) ≤ ρ ∗ (1 + 1 /k ) ℓ . Pr o of. W e pr o ve the claim in ductiv ely . By d efinition, at the b eginning φ ( B 1 ) = 0. In eac h iteration of th e algorithm, B 1 , . . . , B ℓ only c hange in steps 6 , 9 and 12 . It is straigh tforward that by executing either of steps 6 and 12 w e satisfy in d uction claim, i.e., we obtain ℓ + 1 sets B 1 , . . . , B ℓ +1 suc h th at for all 1 ≤ i ≤ ℓ + 1, φ ( B i ) ≤ ρ ∗ (1 + 1 /k ) ℓ +1 . 13 On the other hand, if step 9 is executed, then the condition of 5 is not satisfied, i.e., max { φ ( S B ) , φ ( S B ) } > (1 + 1 /k ) ℓ +1 ρ ∗ ≥ (1 + 1 /k ) φ ( B i ) . where the last inequalit y follo ws b y the indu ction h yp othesis. Since min { ϕ ( S B , B i ) , ϕ ( S B , B i ) } ≤ 1 / 3 k f or ǫ = 1 /k by Lemma 2.2 w e get min { φ ( S B ) , φ ( S B ) } ≤ φ ( B i ) ≤ (1 + 1 /k ) ℓ ρ ∗ , whic h completes the pr o of. Claim 3.2. In the entir e run of the algorithm we have ℓ < k . Pr o of. The follo ws from the p revious claim. If ℓ = k , then b y previous claim we h a ve disjoint sets B 1 , . . . , B k suc h that max 1 ≤ i ≤ k φ ( B i ) ≤ ρ ∗ (1 + 1 /k ) k ≤ e · ρ ∗ ≤ eλ k / 10 < λ k / 2 . where we used ( 8 ). But, th e ab ov e inequ alit y implies ρ ( k ) < λ k / 2 which contradicts Theorem 1.4 . Claim 3.3. If the algorithm terminates, then it r eturns a ℓ - p artitioning of V tha t is a ( φ 2 in / 4 , φ out ) - clustering. Pr o of. Supp ose the algorithm terminates w ith sets B 1 , . . . , B ℓ and P 1 , . . . , P ℓ . Since by the lo op condition, for eac h 1 ≤ i ≤ ℓ , w ( P i − B i → B i ) ≥ w ( P i − B i → V ) /ℓ, b y Lemma 2.5 , φ ( P i ) ≤ ℓφ ( B i ) ≤ ℓ · e · ρ ∗ ≤ 90 c 0 · k 6 p λ k − 1 . where the second inequalit y follo ws by Claim 3.1 , and the last inequ ality f ollo ws b y C laim 3.2 and ( 8 ). On the other hand, by the condition of the lo op and the p erformance of Sp ectral Partitio n ing algorithm as describ ed in Theorem 1.3 , for eac h 1 ≤ i ≤ k , φ ( G [ P i ]) ≥ φ 2 in / 4 = Ω( λ 2 k /k 4 ) . It remains to sho w that the algorithm indeed terminates. First, we show th at in eac h iteration of the lo op at least one of the conditions are satisfied. Claim 3.4. In e ach iter ation of the lo op at le ast one of the c onditions hold. Pr o of. W e use Lemma 2.6 to s h o w that if none of the conditions in the lo op are satisfied then φ ( S ) ≥ φ in whic h is a contradictio n . So, for the sake of con tradiction assume in an iteration of the lo op none of the conditions hold. 14 First, since conditions of 8 and 17 do not hold, for ǫ = 1 /k assump tions (1) and (2) of Lemma 2.6 are satisfied. F u rthermore, sin ce condition of steps 5 and 11 d o not hold max { φ ( S B , S B ) } = max { φ ( B 1 ) , . . . , φ ( B i − 1 ) , φ ( S B ) , φ ( S B ) , φ ( B i +1 , . . . , φ ( B ℓ ) } ≥ max { ρ ∗ , ρ ( ℓ + 1) } . φ ( S P ) = max { φ ( B 1 ) , . . . , , . . . , φ ( B ℓ ) , φ ( S P ) } ≥ max { ρ ∗ , ρ ( ℓ + 1) } . where we used Claim 3.1 . So, for ρ = ρ ∗ and ǫ = 1 /k b y Lemma 2.6 we get φ ( S ) ≥ ǫ · ρ 14 k = max { ρ ∗ , ρ ( ℓ + 1) } 14 k 2 . (9) No w, if ℓ = k − 1, then by Theorem 1.4 we get φ ( S ) ≥ ρ ( k ) 14 k 2 ≥ λ k 28 k 2 ≥ φ in , whic h is a con tr ad iction and we are d one. Otherwise, w e m u st ha ve ℓ < k − 1. T hen by Lemma 1.13 , φ ( S ) ≤ min 1 ≤ i ≤ ℓ p 2 λ 2 ( G [ P i ]) ≤ p 4 c 0 k 6 λ k − 1 , ( 10) where the fi rst inequalit y f ollo w s by the Cheeger’s inequ alit y ( Theorem 1.3 ), Putting ( 9 ) and ( 10 ) together we ha ve ρ ∗ ≤ 14 k 2 p 4 c 0 k 6 λ k − 1 . But, by definition of ρ ∗ in equation ( 8 )), we m u st hav e ρ ∗ = λ k / 10. Therefore, by ( 9 ), φ ( S ) ≥ λ k 140 k 2 = φ in , whic h is a con tradiction, and we are done. It r emains to sho w that the algorithm actually termin ates and if G is u n weigh ted it termin ates in p olynomial time. Claim 3.5. F or any gr aph G the algorithm terminates in finite numb er of iter ations of the lo op. F urthermo r e, if G is unweighte d, the algorithm terminates after at most O ( k n · | E | ) i ter ations of the lo op. Pr o of. In eac h iteration of the lo op at least one of conditions in lines 5 , 8 , 11 , 14 and 17 are satisfied. By Claim 3.2 , Lines 5 and 11 can b e satisfied at most k − 1 times. Line 8 can b e satisfied at most n times (this is b ecause eac h time the s ize of one B i decreases by at least one v ertex). F ur thermore, for a fi xed B 1 , . . . , B k , 14 , 17 ma y h old only fi nite num b er of iterations, b ecause eac h time the total w eight of the edges b et wee n P 1 , . . . , P k decreases. In particular, if G is unw eigh ted, the latter can happ en at most O ( | E | ) times. So, f or undirected graphs the algorithm terminates after at most O ( k n · | E | ) iterations of the lo op. This completes the pro of of Theorem 1.6 . 15 4 Concluding Remarks W e prop ose a new mo del for measuring the qu alit y of k -partitionings of graphs whic h in volv es b oth the insid e and the outside conductance of the sets in the p artitioning. W e b eliev e th at this is often an accurate mo del of the qu alit y of solutions in practical applications. F urthermore, the simple lo cal searc h Algorithm 3 can b e used as a prunin g step at the end of an y graph clustering algorithm. F rom a theoretical p oin t of view, there has b een a long line of w orks on the sparsest cut problem and p artitioning of a graph int o sets of small outside conductance [ LR99 , LLR95 , AR V09 , ALN08 ] but n one of these wo rk s s tu dy the insid e conductance of the s ets in the partitioning. W e think it is a fascinating op en pr oblem to stud y efficient algorithms based on linear programming or semidefinite programming relaxations that pr o vide a bicriteria appro ximation to the ( φ in , φ out )- clustering p roblem. Sev eral of our results can b e generalized or improv ed. In Theorem 1.7 w e signifi can tly imp r o ve Theorem 1.2 of T anak a [ T an12 ] and we sh o w that ev en if there is a small gap b et ween ρ ( k ) and ρ ( k + 1), f or some k ≥ 1, th en the graph admits a k -partitioning th at is a (p oly( k ) ρ ( k + 1) , p oly ( k ) ρ ( k ))- clustering. Unfortunately , to carry-out this result to the domain of eigen v alues w e need to lo ok for a significan tly larger gap b et wee n λ k , λ k +1 (see Corollary 1.8 ). It r emains an op en p roblem if suc h a partitionings of G exists under only a constant gap b etw een λ k , λ k +1 . Ac knowledge ments W e w ould lik e to thank anon ymous review ers f or helpful comments. Also, w e w ould lik e to thank P a vel Kolev f or a carefu l reading of the p ap er and exclusiv e comments. References [ALN08] Sanjeev Arora, James R. Lee, and Assaf Naor. Euclidean distortion and the Sp arsest Cut. J. Amer. Math. So c. , 21(1):1 –21, 2008. 16 [AR V09] Sanjeev Arora, Satish R ao, and Umesh V azirani. Expander flows, geometric em b eddings and graph p artitioning. J. ACM , 56:5:1–5:3 7, April 2009. 4 , 16 [Kle02] Jon Klein b erg. An imp ossibilit y theorem for clustering. p ages 446–453. MIT Press, 2002. 1 [KLL + 13] Tsz C hiu K w ok, Lap C hi Lau, Yin T at Lee, Shay an Oveis Gharan, and Lu ca T revisan. Improv ed c heeger’s inequalit y: analysis of sp ectral p artitioning algorithms through higher order sp ectral gap. In STOC , pages 11–20, 2013. 7 [KVV04] Ra vi Kannan, San tosh V empala, and Adrian V etta. On clusterings: Go o d, bad and sp ectral. J. ACM , 51(3):497 –515, May 2004. 1 , 2 , 3 [LLR95] Nathan Linial, Eran London, and Y ur i Rabinovic h. The geometry of graphs and some of its algorithmic app lications. Combinat oric a , 15:577–59 1, 1995. 16 [LOT12] James R. Lee, S ha yan Ove is Gharan, and Luca T revisan. Multi-w a y sp ectral partitioning and higher-ord er c h eeger in equalities. In STOC , pages 1117–11 30, 2012. 1 , 3 , 4 , 7 16 [LR99] T om Leigh ton and Satish Rao. Multicommo dit y max-flow min-cut theorems and their use in designing approximat ion algorithms. Journal of The ACM , 46:78 7–832, No vem b er 1999. 16 [LR TV12] Anand Louis, Prasad Ragha v end ra, Prasad T etali, and San tosh V empala. Man y sp arse cuts via higher eigen v alues. In STOC , pages 1131–114 0, 2012. 1 , 3 [MMS13] Ragh u Mek a, Ankur Moitra, and Nikhil Sriv asta v a, 2013. P ersonal Communicat ion. 3 [NJW02] Andrew Ng, Mic hael Jordan, and Y air W eiss. On sp ectral clustering: Analysis and an algorithm. In N IPS , 2002. 1 , 4 [SM00] Jian b o Shi and Jitendr a Malik. Normalized Cuts and Image Segmen tation. IEEE T r ansactions on Pattern Analysis and Machine Intel ligenc e , 22(8): 888–905, 2000. 1 [T an12] Mamoru T anak a. Higher eigen v alues and partitions of a graph. arXiv:1112.3 434, 2012. 1 , 2 , 16 [ZLM13] Zeyuan A. Zh u, Silvio Lattanzi, and V ahab Mirrokni. A local algorithm for finding w ell-connected clusters. In ICML , 2013. 3 17