A Note on Preconditioning by Low-Stretch Spanning Trees

A Note on Preconditioning b y Lo w -Stretc h Spanning T rees ∗ Daniel A. Spielman Departmen t o f Computer Science Program in Applied Mathematics Y a le Univ ersit y Jae Oh W o o Program in Applied Mathematics Y a le Univ ersit y Octob er 23, 2018 Abstract Boman and Hendrickson [BH01] obser ved that one can solve linear systems in La placian matrices in time O  m 3 / 2+ o (1) ln(1 /ǫ )  by preconditioning with the Laplacian of a low-stretch spanning tree. By examining the distribution of eigenv alues of the preconditioned linear system, we pro ve that the pr econditioned co njugate gr a dient will a c tually solve the linear system in time e O  m 4 / 3 ln(1 /ǫ )  . 1 In tro d uction F or bac kground on the sup p ort-theory approac h to solving symmetric, diagonally domin an t systems of linear equations, we refer the reader to one of [BGH + 06, BH03, ST08]. Giv en a w eighte d , u ndirected graph G = ( V , E , w ), w e recall that the Laplacian of G ma y b e defin ed by L G = X ( u,v ) ∈ E w ( u, v ) L ( u,v ) , where L ( u,v ) is the Laplacian of the weig ht-1 edge from u to v . Th is is, L ( u,v ) is th e matrix that is zero ev eryw here, except for the s u bmatrix in ro ws and columns { u, v } wh ic h has form:  1 − 1 − 1 1  . Note that t h is last matrix ma y be writte n as the outer pro duct of the v ector ψ u − ψ v with itself, where w e let ψ u denote the elemen tary unit v ector with a 1 in its u -th comp onen t. F or a connected graph G , w e recall that a spann ing tree of G is a connected graph T = ( V , F, w ) wher e F is a subset of E ha ving exactly n − 1 edges. As we in tend for the edges th at app ear in T to ha ve the same we ight as they do in G , w e u se th e same w eight f unction w . As T is a tree, ev ery pair of v ertices of V is connected by a unique path in T . ∗ This materi al is based upon w ork supp orted b y the National Science F oundation u nder Gran t CCF - 0634957. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Scie n ce F oundation. 1 F or an y edge e ∈ E , w e n o w d efine the str etch of e with resp ec t to T . Let e 1 , . . . , e k ∈ F b e the edges on the u nique path in T connecting the endp oin ts of e . T he str etch of e with resp ect to T is giv en by st T ( e ) = w ( e ) k X i =1 1 /w ( e i ) ! . The stretc h of the graph G with resp e ct to T w as defined by Alon, Karp, Pe leg, and W est [AKPW95] to b e st T ( G ) def = X e ∈ E st T ( e ) . A low-str etch sp anning tr e e of G is a graph for whic h the ab o v e quan tit y is reasonably small. The b est kno w n b oun d on attainable str etc h was obtained b y Abr aham, Bartal and Neiman [ABN08], who pr esen t an alg orithm that, on input a graph with n vertice s and m edges, r uns in time e O ( m ) and pro d uces a spanning tree T of stretc h O ( m log n log l og n (log log log n ) 3 ). The adv an tage o f using a spanning tree as a preco n ditioner is that (afte r a p erm utation) one can compute an LU-factorizat ion of the Laplacian of a tree in time O ( n ), and that one can use this LU-factorizat ion to solv e linear systems in the Laplacian of the tree in linear time as wel l. 2 Preconditioning W e prov e the follo wing th ree results. Theorem 2.1. L et G = ( V , E , w ) b e a c onne c te d gr aph and let T = ( V , F , w ) b e a sp anning tr e e of G . L et L G and L T b e the L apla cian matric es of G and T , r esp e ctively. Then, T r  L G L T †  = st T ( G ) , wher e L T † denotes the pseudo-inverse of L T . As T is a subgraph of G , all the nonzero eigen v alues of T r  L G L T †  are at least 1. The analysis of Boman and Hendric ks on [ BH01] follo w ed from the fact that the largest eigen v alue of L G L T † is at most st T ( G ). W e u se th e b oun d on the trace to sho w that n ot to o man y of these eigen v alues are large. Corollary 2.2. F or every t > 0 , the numb er of eigenvalues of L G L T † gr e ater than t is at most st T ( G ) /t . Theorem 2.3. If one uses the pr e c onditione d c onjugate gr adient (PCG) to solve a line ar e qua- tion in L G while using L T as a pr e c onditioner, it wil l find a solution of ac cu r acy ǫ in at most O  st T ( G ) 1 / 3 ln(1 /ǫ )  iter ations. As the dominant co s t of eac h iteration of PCG is th e time requ ired to multiply a vec tor by L G , which is O ( m ), and the time requ ir ed to solv e a sy s tem of equations in L T , which is O ( n ), the lo w-stretc h sp anning trees of Abraham, Bartal and Neiman enable PCG to run in time O  m 4 / 3 (log n ) 1 / 3 (log log n ) 2 / 3 (log 1 /ǫ )  . The follo wing lemma is the key to the pro of of Theorem 2.1. 2 Lemma 2.4. L et T = ( V , F , w ) b e a tr e e, let u, v ∈ V , and let x = ψ u − ψ v . Then, x T L T † x = k X i =1 1 /w ( e i ) , wher e e 1 , ..., e k ar e the e dges on the uniqu e simple p ath in T fr om u to v . Pr o of. T he quan tit y x T L T † x is kn o wn to equal the effect ive resistance in the e lectrical n et wo rk corresp ondin g to T in which the resistance of eve r y edge is the recipro cal of its we ight (see, for example, [S S08]). As only edges on the path from u to v can con tribute to the effectiv e resistance in T fr om u to v , the effectiv e resistance is the same as the effectiv e resistance of the path in T from u to v . As the effectiv e resistance of resistors in serial is j ust the sum of their resistances, the lemma follo ws. Pr o of of The or em 2.1. W e compute T r  L G L T †  = X ( u,v ) ∈ E w ( u, v )T r  L ( u,v ) L T †  = X ( u,v ) ∈ E w ( u, v )T r  ( ψ u − ψ v )( ψ u − ψ v ) T L T †  = X ( u,v ) ∈ E w ( u, v )T r  ( ψ u − ψ v ) T L T † ( ψ u − ψ v )  = X ( u,v ) ∈ E w ( u, v ) k X i =1 1 /w ( e i ) (where e 1 , . . . , e k are the edges on the simple path in T from u to v ) = X ( u,v ) ∈ E st T ( u, v ) = st T ( G ) . Pr o of of Cor ol lary 2.2. As b oth L G and L T are p ositiv e semi-definite, all the eigen v alues of L G L T † are real and non-negativ e. The corollary follo ws imm ediately . T o sho w that the PCG will qu ic kly solve linear systems in L G with L T as a preconditioner, w e use the analysis of Axelsson and Lind sk og [AL86, (2.4)], whic h we summarize as Theorem 2.5. Theorem 2.5. L et A and C b e p ositive semi-definite matric es with the same nul lsp ac e such that al l but q of the eigenvalues of AC † lie in the interval [ l , u ] , and the r emaining q ar e lar ger than u . If b is in the sp an of A and one u se s the Pr e c onditio ne d Conjugate Gr adient with C as a pr e c onditioner to solve the line ar system Ax = b , then after k = q +  ln(2 /ǫ ) 2 r u l  3 iter ations, the algorithm wil l pr o duc e a solution x satisfying    x − A † b    A ≤ ǫ    A † b    A . W e r ecall that k x k A def = √ x T Ax . While Axelsson and Lindsko g do not explicitly deal with the case in whic h A and C are p ositiv e-semidefinite with the s ame n u llsp ace, the extension of their analysis to this case is immediate if one applies the ps eudo-in verse of C wh enev er they refer to the inv erse. Pr o of of The or em 2.3. As G and T are connected, b oth L G and L T ha ve the same nullspace: the span of the all-1s vec tor. Set u = (st T ( G )) 2 / 3 and l = 1. Corollary 2.2 tells us that L G L T † has at most q = (st T ( G )) 1 / 3 eigen v alues greater than u . T he theorem no w follo ws from T h eorem 2.5. References [ABN08] I. Abraham, Y. Bartal, and O. Neiman. Nearly tight low stretc h spanning trees. In Pr o c e e dings of the 49th Annual IEE E Symp osium on F oundations of Computer Scienc e , pages 781– 790, O ct. 2008. [AKPW95] Noga Alon, Richard M. Karp, Da vid Pe leg, and Douglas W est. A graph-theoretic game and its application to the k -server problem. SIAM J ournal on Computing , 24(1): 78–100, F ebr u ary 1995. [AL86] Ow e Axelsson and Gun hild Lindskog. On th e rate of con verge n ce of the precondi- tioned conjugate gradien t metho d. Numerische Mathematik , 48(5):499– 523, 1986. [BGH + 06] M. Bern, J. Gilb ert, B. Hend ric kson, N. Nguy en, and S . T oledo. S upp ort-graph preconditioners. SIAM J. Matrix Anal. & Appl , 27(4): 930–951, 200 6. [BH01] Erik Boman and B. Hendr ic kson. On spannin g tree preconditioners. Ma nuscript, Sandia National Lab., 2001. [BH03] Erik G. Boman a n d Bru ce Hendric kson. S upp ort theory for preconditioning. SIA M Journal on Matrix Analysis and Applic ations , 25(3):694 –717, 2003. [SS08] Daniel A. Spielman and Nikhil Sriv asta v a. Graph sp arsification b y effectiv e resis- tances. In Pr o c e e dings of the 40th annual A CM Symp osium on The ory of Computing , pages 563–56 8, 2008 . [ST08] Daniel A. Spielman and Sh ang-Hua T eng. Nearly-linear time algo r ithms for pr e- conditioning and solving symmetric, diagonally dominant linear sys tems. CoRR , abs/cs/060 7105, 2008 . Av ailable at http:// www.arxi v.org/ab s/cs.NA/0607105 . 4