An extension of the Moser-Tardos algorithmic local lemma

An extension of the Moser-T ardos algorithmic lo cal lemma W esley P egden ∗ Marc h 12, 2011 Abstract A recent theorem o f Bissacot, et al. p roved using results ab o ut the clus- ter expansion in statistical mec hanics extend s the Lov´ asz Local Lemma by w eak ening the conditions under which its conclusions holds. In this note, w e prov e an a lgorithmic analog of this result, extending Moser and T ardos’s recent algorithmic Lo cal Lemma, and pro viding an alternativ e proof of the theorem of Bissacot, et al. ap p licable in the Moser-T ardos algorithmic framework. 1 In tro duction If even ts A 1 , A 2 , . . . , A n are independent, then we have P( T ¯ A i )) > 0 so long as P ( A i ) < 1 for each i . A ce ntral to ol in pro babilistic co m binator ics is the Lov´ asz Lo cal Lemma prov ed b y Erd˝ os and Lov´ asz [5], which can be seen as generalizing this simple fac t to situations where some depe ndencie s among the A i are allowed, in exchange for better b ounds on the probabilities P( A i ). The Loca l Lemma is commonly presented through the framework of a de- p en dency gr aph on the ev ents A i , where if C is any fa mily of non-neighbors of some A i , then we ha ve that A i is indep endent of the family C of even ts. The Lov´ asz Lo ca l Lemma is th en as follows: Theorem 1. 1 (Lov´ asz Loc al Lemma) . L et G b e any dep endency gr aph for a finite family A of event s, and supp ose that t her e ar e r e al numb ers 0 < x A < 1 ( A ∈ A ) such that for al l A ∈ A we have P( A ) ≤ x A Y B ∼ A (1 − x B ) . (1) Then P \ A ∈A ¯ A ! > Y A ∈A (1 − x A ) , ∗ Couran t Institute of Mathematical Sciences, New Y ork Univ ersity , 251 Mercer St, Rm 921, New Y ork, NY 10012 Email: pegden@math.n yu.edu. P artially supported b y NSF MSPRF gran t 1004696. 1 and so in p articular, we have P \ A ∈A ¯ A ! > 0 . (2) The first breakthro ugh in finding an algor ithmic version o f the Loca l Lemma was made by B eck, who demonstrated his method on the class ic al Lo cal Lemma application to 2-co lorable hyperg raphs. Beck’s metho d was s ubsequently refined and given a more genera l framework [1, 4, 9, 1 4], but requir e d stronger bounds on the proba bilities of the ev ents than w ere req uired by the nonalg orithmic version. In Moser and T ar dos’ recent br eakthroug h pap er [10], they give an algor ith- mic p ro o f o f the Lov´ as z Lo cal Lemma in a setting whic h is gener al eno ugh f or nearly all applicatio ns of the Lemma in com binator ics, with b ounds identical to those required b y the nonalgorithmic version. In the framework Mo ser and T ardos cons ider, the ev ents in A dep end on some underlying set V of indep en- dent random v ariables, and they denote by vbl( A ) ( A ∈ A ) the minimal set of random v ariables fro m V on which each A dep ends; A is said to b e ‘v iolated’ with resp ect to a pa rticular ev alation of the v ariables in vbl( A ) if the even t o ccurs for that ev alua tion. A Moser -T ardos dependency graph is one whic h implies that if even ts A and B are nonadjacent, then vbl( A ) is disjoint from vbl( B ). (Note that this notion o f a dep endency graph is more restr ictive than the L ov´ a sz version based on pr obabilistic ind ep endence, as is demo ns trated by an exa mple of Kolipak a and Szegedy [8].) Moser and T ardos ’s theorem is then the following: Theorem 1. 2 (Moser and T ardos) . L et V b e a finite set of mutual ly indep endent variables in a pr ob ability sp ac e, and let A b e a finite family of events determine d by these variables. If ther e ar e r e al numb ers 0 < x A < 1 ( A ∈ A ) such that P( A ) ≤ x A Y B ∼ A (1 − x B ) (3) then ther e exists an assignment to the variables V which c orr esp onds t o no o c- curr enc e of any event fr om A . Mor e over, t he r andomize d algorithm describ e d b elow r esamples an event A at most an exp e cte d x A 1 − x A times b efor e finding t he evaluation, thus the total numb er of r esampling st eps is P A ∈A x A 1 − x A in ex p e c- tation. The Moser-T ar dos a lgorithm consists just of b eginning with a rando m e v alu- ation of all the v ariables in V , and then resa mpling vbl( A ) for an y violated even ts A un til no violated even ts remain. Of co urse, the e fficiency of the algo- rithm depends on the a bilit y to resample v ariables efficien tly and c heck whether individual even ts a re viola ted; this is generally an ea sy implemen tation problem, how ever, making the analysis of the n um b er of r e s ampling s teps the im p ortant issue. Recently , Bissa cot, F er n´ andez, Pro ca cci and Sco ppo la prov ed the following improv ement of the L ov´ a sz Lo cal Lemma : 2 Theorem 1.3 (Bissa cot, et a l. [3]) . Consider a finite fa mily A of events in some pr ob ability sp ac e Ω , with some dep endency gr aph G . If t her e ar e r e al numb ers 0 < µ A < ∞ such that P ( A ) ≤ µ A X I ⊂ ¯ Γ( A ) I in dep. Y B ∈ I µ B (4) then P  T A ∈A ¯ A  > 0 . It is not difficult to check that co ndition (4) is w eaker than condition (1) b y considering the substitution µ A = x A 1 − x A . (Condition (1) would be equiv alen t to (4) w itho ut the condition in the sum that the sets I be indep endent.) In [3], they also give exa mples where this theo rem improv es some c lassical theorems prov ed with the Loca l Lemma. Theorem 1 .3 has also since b een applied to improv e s ome theorems o n graph co lo rings in [11]. Their pro o f of Theorem 1.3 is based o n Shearer ’s characterization of lab eled depe ndency gr aphs to which the co nclusion of the Lo cal Lemma applies [13] and tw o of thos e authors’ recent results o n the radius of conv erg ence of log s of partition functions [7]. (The connection betw een the Lo cal Lemma and the partition functions of statistical mechanics was first made by Scott and Sok al [12].) In this short note, we prov e an algo rithmic a nalog to the r esult of Bissac o t, et. al. That is , we will show that in the s e tting of Mose r and T ardos’s algo - rithmic Lo ca l Lemma, Moser and T ardos’s b ounds on the running time of their algorithm hold even with their condition (3) re pla ced with condition (4): Theorem 1.4 . L et V b e a finite set of mutual ly indep endent variables in a pr ob ability sp ac e, and let A b e a finite family of events determine d by these variables. If ther e ar e r e al n u mb ers 0 < µ A < ∞ ( A ∈ A ) su ch that P ( A ) ≤ µ A X I ⊂ ¯ Γ( A ) I inde p. Y B ∈ I µ B , (5) then ther e exists an assignment to the variables V whic h c orr esp onds to no o c- curr enc e of any event fr om A . Mor e over, the Moser-T ar dos algorithm r esamples an event A at most an ex p e cte d µ A times b efor e fi nding the evaluation, thus the total numb er of r esampling st eps is P A ∈A µ A in exp e ctation. (The r unning time b ound here is equiv alent to the Moser -T ardos b ound under the substituation µ A = x A 1 − x A .) The proof of Theor em 1.4 consists simply of re- doing one part o f the proo f of Moser and T ardos’s theorem, taking a dv antage of some constr aints whic h were not necessar y for Moser a nd T ardos’s orig inal r esult. Theorem 1.4 can b e seen a s doing tw o things: fir st, it extends the result of Mose r and T ar dos by giving a weaker condition under which the iden tical 3 result holds—note that this has also been done in a more genera l sense b y Kolipak a a nd Sz e gedy [8], who directly connect Shea rer’s condition with the Moser/T ar dos a lgorithmic framework. Seco ndly , it gives an a lter native pr o of of the result of Bissacot, et al. (in the slig h tly more restrictive a lgorithmic setting) which is indep endent o f Shea rer’s theorem and the cluster expans io n metho ds used in [7]. Bissacot, et al. note that their Theorem 1.3 ca n be extended to L opside d dep endency gr aphs , first considered b y Erd˝ os and Sp encer in [6]. In their pa- per on their algorithmic Lo ca l Le mma , Mo ser and T ar dos define an ana log of lopsidep endency in the a lgorithm/v ariable setting, and a reader fa miliar with Moser and T ar do s’s paper can eas ily verify that our improvemen t to Moser and T ardos’s theor em applies to their theorem on algor ithmic lopsided dep endency graphs a s well, as we only r e-do their branching arg umen t, which is applied to the lopsided ca se in the same way as in their main r esult. 2 Pro of The Moser-T ar dos alg orithm is as follows: 1: pro cedure Moser-T ardos ( P ) 2: for all P ∈ P do 3: v P ← (random ev aluation of P ) 4: end for 5: while ∃ A s.t. A is v iolated when P = v P ( ∀ P ) do 6: for all P ∈ vbl( A ) do 7: v P ← (new ra ndo m ev a luation of P ) 8: end for 9: end whi le 10: end pro cedure (Note that when m ultiple even ts exist satisfying line 5, one o f the satisfying even ts is chosen a rbitrarily .) Moser and T ardos’ pr o of that this a lg orithm terminates in p oly nomial time (under condition (3 )) is ba sed on the notion of a ‘witness tree’. As the algorithm runs and bad even ts are found and resampled, a witness tree is assigned to each step of the algorithm (where a step consists of a resampling of an e vent). A witness tree is a ro o ted tree with lab els from A . The witness tree W t for s tep t of the alg orithm is constr ucted a s follows: cho ose as its ro ot a vertex lab eled with whatever ev ent A 0 was resa mpled at step t . If the even t A 1 which was resampled at step t − 1 ov erlaps the label of the ro o t, a v ertex is added as a child of the ro ot lab eled with A 1 . (W e may ha ve A 1 = A 0 .) In genera l, for each step i = t − 1 , t − 2 , . . . , 1 of the algorithm, if the even t A i which was added at step i ov erla ps any of the even ts curr ent ly lab els o f vertices of our par tially constructed W t , w e a dd a vertex lab eled with A i as the child of a vertex of maximum depth whose lab el ov erlaps A i . In the result, W t , children always ov erlap their parents, a nd c hildren of a co mmon parent alw ays ge t distinct 4 lab els (other wise, whichev er was added after the other would have been added as a child of the other). An y tr e e T with labels fro m A with these tw o pro pe rties is called a pr op er witness t r e e . Moser a nd T ar dos’s pro of of their algorithm’s efficiency cons is ts of tw o parts: first, they s how that a n y prop er witnes s tree T has probability at most Y v ∈ T P( A v ) (6) of o ccurring as a witness tree a t any p oint in the r unning of the a lg orithm, where here A v denotes the even t lab eling the vertex v . Now, if an even t A is resampled a t step t , the num b er of o ccurr ences of A as a lab el in the witness tree W t is equa l to the num b er of times A ha s been resampled on s teps 1 , . . . , t —in particular, all witness trees which will o ccur in a run of the algorithm will be dis tinct. Thu s if we let T A denote the set of prop er witness trees with r o ot lab el A , the exp ected v alue of the num b er N A of resamplings of A which o ccur in a run of the alg o rithm is equal to E( N A ) = X T ∈T A P( T occur s in the lo g) ≤ X T ∈T A Y v ∈ T P( A v ) . (7) The second part of Mos e r a nd T ardos’s pro of consists of b ounding the s um of pro ducts in line (7). They do this by cons idering a ra ndo m proc ess for constructing trees: Suppo s e x A ( A ∈ A ) are r eal n umbers betw een 0 and 1 . Fix now any e vent A 0 . In the firs t round of the pr o cess, a vertex lab eled A 0 is created. In each subsequent ro und, for e a ch even t vertex v with lab el A v created in the previo us round, a nd for each event A u ∈ ¯ Γ( A v ) (in the dep endency graph), a vertex u with lab el A u is added a s a child of v with pro babilit y x A u . (All o f these choices a re are made indep endently .) Moser and T ardos prov e: Lemma 2. 1 (Moser T ardos Branching Lemma) . F or any pr op er witness tr e e T with r o ot lab ele d A 0 , the pr ob abili ty p T that t he pr o c ess ab ove pr o duc es exactly the tr e e T is p T = 1 − x A 0 x A 0 Y v ∈ T x A v Y B ∼ A v (1 − x B ) ! . (8) Thu s, the Lemma gives us that 1 ≥ X T ∈T A p T ≥ 1 − x A x A X T ∈T A Y v ∈ T x A v Y B ∼ A v (1 − x B ) ! (9) Thu s the bound P ( A ) ≤ ( x A Q B ∼ A (1 − x B )) for all A implies, toge ther with line (7), that E( N A ) ≤ x A 1 − x A . (10) 5 Our impr ov ement comes just from a slightly more careful br a nching ar gu- men t. Note that any witness tree whic h o c c urs in the log of the a lgorithm has the prop erty that any children of a common vertex hav e lab els whic h are nonadjacent in the dep endency graph. This condition—let’s call it st r ongly pr op er —is stro ng er than requiring s imply that c hildren be distinct. Thus, we can strengthen line 7, a s we hav e the b ound E( N A ) = X T ∈T S A P( T occur s in the lo g) ≤ X T ∈T S A Y v ∈ T P( A v ) . (11) where T S A ⊂ T A is the set of strongly prop er witness trees. T o b ound the sum in (11), we conside r a mo dified branching pr o cess which pro ceeds as follows. Given re a l n umbers 0 < µ A < ∞ , w e define x A = µ A µ A +1 (note that 0 < x A < 1) and fix a ny even t A 0 . In the first round of the pro cess, a vertex lab eled A 0 is created. In each subsequent r ound, for e a ch even t vertex v with lab el A v in the previo us round, we car ry out a ‘subpr o cess’, where for each A u ∈ ¯ Γ( v ) (in the dep endency g raph), a vertex u with la be l A u is added a s a child o f v with probability x A u (the choices are independent). At the end of the subpr o cess, w e chec k if the la be l- set of the resulting s et of children for v is an indep endent set in the dep endency gra ph. If it is not, we delete the children created and r estart the subpro cess . Note that x A < 1 (for all A ) implies tha t the subpro cess will even tually end (with probability 1) having pro duced a n indep endent set. Note that the pro cess describ ed ab ove is equiv alen t to o ne in which, in each round and for ea ch vertex v fro m the previous round, w e create a s et of children u with lab els from a set chosen fr om all indep endent sets I v ⊂ ¯ Γ( v ), where the likelihoo d of the c hoice of each indep endent set I v is w eighted according the the pro duct w ( I v ) = Y u ∈ I v x A u !   Y u ∈ ¯ Γ( v ) \ I v (1 − x A u )   . Lemma 2.2 (Impr oved Br anching Lemma) . F or any str ongly pr op er witness tr e e T with r o ot la b ele d A 0 , the pr ob ability p ′ T that the mo difie d br anching pr o c ess describ e d ab ove pr o duc es exactly the t r e e T is p ′ T = µ − 1 A 0 Y v ∈ T µ A u X I ⊂ ¯ Γ( A v ) I inde p. Y A ∈ I µ A . (12) Pr o of. Letting W v = ¯ Γ G ( A v ) \ ℓ (Γ + T ( v )), where ℓ ( v ) is the lab el o f vertex v , we hav e p ′ T = Y v ∈ T Y u ∈ Γ + T ( v ) x A u Y B ∈ W v (1 − x B ) X I ⊂ ¯ Γ( A v ) I i ndep. Y A ∈ I x A Y B ∈ ¯ Γ G ( A v ) \ I (1 − x B ) . 6 This can be rewr itten as p ′ T = Y v ∈ T Y u ∈ Γ + T ( v ) x A u 1 − x A u X I ⊂ ¯ Γ( A v ) I i ndep. Y A ∈ I x A 1 − x A by dividing the top and bo ttom b y Q B ∈ ¯ Γ G ( A v ) (1 − x B ). Since taking the double pro duct Q v ∈ T Q u ∈ Γ + T ( v ) is a eq uiv alent to taking a pro duct Q v ∈ T \{ v 0 } , where v 0 denotes the ro ot vertex o f T , this g ives line (12 ), r ecalling that x A = µ A µ A +1 and so x A 1 − x A = µ A . This is applied now in the same wa y as the br a nching lemma used b y Mo ser and T ar dos, but with r egards to the family T S A of str ongly pro per witness trees ro oted with A , instead of the family T A of prop er witness trees ro o ted with A . W e hav e 1 ≥ X T ∈T S A p ′ T ≥ µ − 1 A 0 X T ∈T S A Y v ∈ T µ A u X I ⊂ ¯ Γ( A v ) I i ndep. Y A ∈ I µ A . (13) Putting this together with line (11), w e see then that the c o ndition (5) of the theorem implies that E( N A ) ≤ µ A , (14) completing the pro o f the the Moser- T ar dos algorithm still terminates in ex- pec ted time X A ∈A µ A . Ac kno wledgeme n t I’d like to thank J o el Spe ncer for some helpful discussions o n this note. References [1] N. Alo n. A parallel algo rithmic version of the lo ca l lemma, R andom St r u c- tur es and Algorithms 2 (199 1) 3673 78. [2] J´ oszef Beck. An Algor ithmic Appro ach to the Lov´ asz Lo ca l Lemma, Ra n- dom St ructur es and Algo rithms 2 (19 9 1) 3 43–36 5. [3] R. Bissa cot, R. F ern´ andez, A. Pr o cacci, B. Sco pp ola . Title: An Improv ement of the Lo vsz Lo ca l Lemma via Cluster Expansion. http:/ /arxi v.org/abs/0910.1824v2 (10 pa ges). 7 [4] A. Czuma j and C. Scheideler. Coloring no n-uniform h yp ergr aphs: a new algorithmic appro ach to the gener al Lov´ as z loca l lemma, S ymp osium on Discr ete Algori thms (20 0 0) 3 039. [5] P . Erd˝ os and L. Lov´ a s z. Proble ms and r esults o n 3- chromatic h yp e r graphs and so me related questions, in Infinite and Finite sets I I , North-Holla nd, pp. 609 627, A. Ha jnal, R. Rado, a nd V. T. S´ os (E ds.) (197 5). [6] P . Erd˝ os, J. Sp encer. Lopsided Lov´ asz Lo ca l Lemma and L a tin tra nsversals, Discr ete Applie d Math. 3 0 (19 9 1) 1 51–15 4. [7] R. F ernandez, A. Pro ca cci. Cluster expansio n for abstract p olymer mod- els: New b ounds from an old approa ch. Communic ations in Mathematic al Physics 2 74 (2007 ) 123 –140 (20 07). [8] K. K olipak a, M. Szegedy . Moser and T ardos meet Lov´ as z . Man uscript (2010). [9] M. Molloy and B. Reed. F urther Algorithmic Asp ects of the Lo cal Lemma, Pr o c e e dings of the 30th Annual ACM Symp osium on the The ory of Com- puting (19 98) 5 24529 . [10] R. Moser and G. T ar do s. A constructive pro o f of the ge ne r al L ov´ asz Lo cal Lemma, J. ACM 57 (2010) Article 11, 15 pag es. [11] S. Ndre ca, A. Pro cacc i, B. Scopp ola . Improv ed b ounds o n color ing of graphs, h ttp:// arxiv. org/abs/1005.1875 . [12] A. Scott a nd A. Sok al. The repulsive lattice g a s, the indep endent-set po ly- nomial, a nd the Lov´ asz lo cal le mma , J. Stat. Phys. 118 (2005) 115 11261 . [13] J. Shea rer. On a problem of Sp encer , Combinatoric a 5 (1985 ), 241-2 45. [14] A. Sriniv asan. Improved algo rithmic versions of the Lov´ a sz Lo cal Lemma, Pr o c e e dings of the ninete enth annu al ACM-SIAM symp osium on D iscr ete algorithms (S ODA) (2008 ) 611 620. 8