Dependent Randomized Rounding for Matroid Polytopes and Applications

Depende nt Randomized Rounding for Matroid Polytopes and Applicatio ns Chandra Chekuri ∗ Jan V ondr ´ ak † Rico Zenklusen ‡ May 30, 2018 Abstract Motiv ated by se veral applications, we co nsider the pro blem of rando mly round ing a fractional solu tion in a m atroid (base) polyto pe to an integral one. W e con sider the pipage r ounding technique [5, 6, 3 6] and also present a ne w technique, rando mized swap r ounding . Our main technical results are concen tration bound s fo r functio ns o f ran dom variables arising from these ro unding techniqu es. W e p rove Chernoff- type con centration boun ds for linear fu nctions of ran dom variables arising from b oth techniqu es, and also a lower -tail exponential bound for monotone submodular functions of variables arising from rando mized swap roundin g. The following are examples of our applications. • W e gi ve a (1 − 1 /e − ε ) -ap proxim ation algorithm fo r the problem o f maximizing a m onoto ne submod - ular function subject to 1 matro id and k linear constraints, for any constant k ≥ 1 and ε > 0 . W e also giv e the same result fo r a super-constant number k of ”loose” linear con straints, where the right-h and side domin ates the matrix entries by an Ω( ε − 2 log k ) factor . • W e presen t a re sult o n min imax pack ing pro blems that inv olve a matroid b ase c onstraint. W e g iv e an O (log m/ log log m ) -approx imation for the g eneral p roblem min { λ : ∃ x ∈ { 0 , 1 } N , x ∈ B ( M ) , Ax ≤ λb } where m is the number of packing constraints. Exam ples include the low-cong estion multi-path routing problem [34] and spanning -tree problems with capacity constraints on cuts [4, 16]. • W e g eneralize the continuou s greedy alg orithm [35, 6] to pr oblems involving m ultiple submo dular function s, and use it to ﬁnd a (1 − 1 / e − ε ) -appr oximate p areto set for the prob lem o f m aximizing a con stant num ber of mo notone submod ular functions subject to a ma troid constraint. An examp le is the Submo dular W elfare Problem wher e we are lo oking for an ap proxim ate pareto set with respect to individual players’ utilities. ∗ Dept. of Computer Science, Univ . of Ill inois, Urbana, IL 6180 1. Partially supported by NSF grant CCF-0728782. E- mail: chekuri@cs.il linois.edu † IBM Almaden Research Center , San Jose, CA 9512 0. E -mail: jvondrak @us.ibm.com ‡ Institute for Operations Research, ETH Zurich. E-mail: rico.z enklusen@ifor .math.ethz.c h 1 Introd uction Randomize d roundin g is a fundamental te chnique intr oduced by R agha van and Thomp son [29] in ord er to round a fractio nal solution of an LP into an integr al solution. Numerous application s and varian ts hav e since been explore d and it is a standard techniqu e in the design of approxima tion algorithms and related areas. The origin al techni que from [29] (and sev eral subseque nt papers) relies on independe nt roundin g of the va riables which allo ws one to use Chernof f-Hoeff ding concentratio n bounds for linear functio ns of the varia bles; these bound s are critical for se veral applicatio ns in packing and cov ering problems. Ho wev er , there are many situa- tions in which independen t rounding is not feasib le due to the pr esence of constrain ts that can not be violated by the rounded solution. V arious techniques are used to handle such scenarios. T o name just a few: alteration of soluti ons obt ained b y independen t rounding , careful derandomiza tion or constructi ve metho ds wh en probability of a feasible solution is non-zero b ut small (for e xample when using the L ov ´ asz Local Lemma), and va rious forms of correlate d or depende nt randomized rounding schemes. These methods are typicall y succes sful when one i s in terested in preserving the expected v alue of the sum of se veral ra ndom v ariables; the r ounding sch emes approx imately preserve the ex pected va lue of each random v ariable and then one reli es on linearity of ex pecta- tion for the su m. There are, ho wev er , applic ations where one ca nnot use independ ent rounding and ne verth eless one needs concentra tion bounds and/or the ability to handle non-line ar objecti ve functions such as con vex or submodu lar functions of the v ariable s; the wor k of Srini v asan [34] and othe rs [14, 19] highli ghts some of thes e applic ations. Our focus in this paper is on such schemes. In particular we consid er the problem of roundi ng a point in a matr oid polytope to a ve rtex. W e compare the existin g approach es and propose a new roundin g scheme which is simple and has multiple applica tions. Backgr ound: Matroid polyto pes, whos e study was initiate d by Edmonds in the 70’ s, fo rm one of the most importan t classes of polytopes as sociated with combinator ial optimization problems. (For a de ﬁnition, see Section 2 .) Even tho ugh the full des cription of a matroi d polyt ope is exponen tially larg e, matroid po lytopes can be optimized ove r , separated over , and they hav e strong integralit y propertie s such as total dual inte gra lity . As a consequ ence, the basic solution of a linear optimizatio n problem ov er a matroid polyto pe is always integra l and no roundin g is neces sary . More re cently , vari ous applic ations emerged whe re a mat roid constrai nt appears with ad ditional constra ints and/or the object iv e function is non-line ar . In such cases, the issue of rounding a fractional solutio n in the matroid polytope re-appears as a non-t riv ial question . One such applicatio n is the submod ular welfare prob- lem [12, 22], w hich can be formulated as a submodular maximization problem subject to a partition matroid constr aint. The roundin g techni que that turned out to be useful in this contex t is pipag e r ounding [5]. Pipage roun ding was introduce d by Agee v and Sviriden ko [3], who used it for round ing fractional solu- tions in the bipartite matching polytope. T hey used a linear program to obtain a fractional solution to a certain proble m, but the rounding procedure was based on an auxiliary (non-lin ear) objecti ve. T he auxiliary objecti ve F ( x ) was deﬁned in such a way that F ( x ) would alw ays increase or stay cons tant througho ut the rounding proced ure. A comparis on between F ( x ) and the original objecti ve yields an appro ximation guarantee. Cali- nescu et al. [5] adapted the pipage rounding techniqu e to problems in volvi ng a matroid constr aint rather than bipart ite matching s. Moreov er , they showed that the necessary con vexity pr operties are sa tisﬁed w hene ver the auxiliary function F ( x ) is a multilinear extens ion of a submodula r set function f . This turned out to be crucia l for furthe r dev elopmen ts on submodu lar maximiza tion proble ms - in particu lar an optimal (1 − 1 /e ) - approx imation for maximizing a monotone submodular function subject to a matroid constr aint [35, 6], and a (1 − 1 /e − ε ) -appro ximation for maximizing a mono tone submodul ar function subject to a constant number of linear constrai nts [18]. A s one of our application s, we consider a common generalizat ion of these two problems. Srini v asan [34], and bu ilding on his work Gandhi et al. [14], consider ed depende nt rando mized roun ding for poin ts in the b ipartite matching poly tope (and more ge nerally the as signment polytope ); their te chnique can be viewed as a randomized (and obli vious) versio n of pipage rounding. The m oti v ation for this randomize d scheme ca me from a diff erent set of applicat ions (see [34]). The res ults in [ 34, 14] showed ne gative corr elation proper ties for their round ing scheme which implied concentr ation bounds (via [28]) that were then useful in 1 dealin g with addition al constra ints. W e make some observ ations regard ing the resul ts and applicati ons in [3, 34, 14]. Although the schemes round a point in the assignment polytope, each constraint and objecti ve function is restric ted to depend on a subset of the edges incident to some verte x in the underlying bipartite graph. Furthe r , se vera l of the application s in [3, 34, 14] can be naturally modeled via a matroid constraint instead of using a bipart ite graph with the abov e mentioned restricti on; in fact the simple part ition matroid sufﬁces . The pipage round ing technique for matroids, as presented in [5], is a determin istic procedu re. Ho wev er , it can be randomized similarly to Sriniv asan’ s work [34], and this is the varian t presented in [6]. This varian t starts with a fractional solution in the mat roid base poly tope, y ∈ B ( M ) , and pro duces a rando m base B ∈ M such that E [ f ( B )] ≥ F ( y ) ; here F is th e multiline ar extens ion of the su bmodular functio n f . A further round ing stage is nee ded in case th e start ing point is ins ide the matroi d polytop e P ( M ) rather than the m atroid base polytope B ( M ) ; pipage rou nding has been e xtended to this case in [36 ]. In the analysis of [6, 36], the approximatio n guarante es are only in ex pectatio n. Stronger guara ntees could be obtained and additional applic ations would arise if we could prove concentr ation boun ds on the v alue of linear/ submodula r functi ons under such a round ing procedure. This is th e focus of this paper . V ery recen tly , anot her applica tion has emer ged where rounding in a matroid polyt ope plays an essen tial role. Asadpour et al. [2] present a new approach to the Asymmetric Tra veling S alesman problem achiev ing an O (log n / log log n ) -appro ximation, improving upo n the long-standi ng O (log n ) -appr oximation. A crucial step in the algorithm is a rounding procedure , which giv en a fractiona l solution in the spannin g tree polytop e pro- duces a sp anning tree sati sfying certai n ad ditional cons traints. The authors of [2] use t he te chnique o f maximu m entr opy sampling w hich giv es negati ve correlatio n propertie s and Chernof f-type concen tration bounds for any linear function on the edges of the graph. S ince spanni ng trees are bases in the graphic matr oid for any graph, this rou nding procedur e also falls in the framew ork of rand omized rounding in the mat roid polytop e. Howe ver , it is not clear w hether the technique of [2] can be generalize d to any matroid or whether it could be used in applic ations with a submodular objecti ve function. 1.1 Our work In this paper we stu dy th e problem of randomly rounding a point in a matr oid po lytope to a v ertex of the polyto pe. 1 W e consider the techn ique of ran domized pipag e r ounding and also introdu ce a ne w rounding proced ure called randomized swap r ounding . Give n a starting point x ∈ P ( M ) , the proced ure produces a random indepe ndent set S ∈ I such that Pr [ i ∈ S ] = x i for each element i . Our main technica l results are concen trati on bounds for linear and submodular functions f ( S ) unde r this new roundi ng. W e demonstrate the useful ness of these concentrat ion bounds via sev eral applicati ons. The randomize d swap rounding procedu re bears some similarity to pipage rounding and can be used as a replac ement for pipage rounding in [6, 36]. It can be also u sed as a replacement f or max imum ent ropy sa mpling in [2]. Ho wev er , it has se veral adv antages over previ ous rounding proced ures. It is easy to descri be and implement, and it is very efﬁcient. Moreo ver , th anks to the simplicity of randomize d sw ap roundin g, we are able to deri ve results that are not kno w n for pre vious techniqu es. O ne examp le is the tail estimate for submodula r functi ons, T heorem 1.4. O n the other hand, our concentr ation bound for linear functions (Corollar y 1.2) holds for a more gener al class of round ing techniques including pipage rounding (see also L emma 4.1). Randomize d swap rounding sta rts fro m an arb itrary rep resentat ion of a starting point x ∈ P ( M ) as a con ve x combinatio n of inc idence vectors of ind ependen t sets. (T his repr esentatio n can be obtai ned by standard techni ques and in some a pplicatio ns it is exp licitly a vaila ble.) Once a con ve x repre sentation of the starting p oint is ob tained, the ru nning time of randomize d swap rou nding is bounded by O ( nd 2 ) call s to t he membersh ip ora- cle of the matro id, where d is the rank of the matroi d and n is the size of the gro und set. In comparis on, pipag e round ing perfor ms O ( n 2 ) iterations ea ch of which requires an expen siv e call to submodu lar functi on minimiza- tion (se e [6]). Maximum entrop y sampling for sp anning trees in a g raph G = ( V , E ) is e ven more complicate d; 1 Our results ex tend easily to the case of roundin g a point in the polytope of an integer v alued polymatr oid . Additional applications may follo w from this. 2 [2] does not pro vide an explic it running time, bu t it states that the procedur e in vo lves O ( | E | 2 | V | log | V | ) itera- tions, where in each iteration one needs to compute a determinant (from Kirchhof f ’ s matrix theore m) for each edge. A lso, maximum entrop y sampling preserv es the margina l probabilitie s Pr[ i ∈ S ] = x i only approxi- mately , and the running time depends on the desired accurac y . First, we show that randomiz ed swap round ing as well as pipage rounding hav e the property that the indi- cator v ariable s X i = [ i ∈ S ] hav e expectati ons exactly x i , and are ne gatively corr elated . Theor em 1.1. Let ( x 1 , . . . , x n ) ∈ P ( M ) be a f ractio nal solution in the matr oid polytope and ( X 1 , . . . , X n ) ∈ { 0 , 1 } n an inte gra l solution obtained using either randomize d swap r oundin g or randomize d pipag e r oundin g. Then E [ X i ] = x i , and for any T ⊆ [ n ] , (i) E [ Q i ∈ T X i ] ≤ Q i ∈ T x i , (ii) E [ Q i ∈ T (1 − X i )] ≤ Q i ∈ T (1 − x i ) . This yields Chernof f-type concentr ation bounds for any linear function of X 1 , . . . , X n , as prov ed by Pan- conesi and Srini v asan [28] (see also Theorem 3.1 in [14]). T ogethe r w ith Theorem 1.1 we obtain: Cor ollary 1.2. Let a i ∈ [0 , 1] and X = P a i X i , wher e ( X 1 , . . . , X n ) ar e obta ined by eit her ran domized swap r ounding or randomize d pipag e r ounding fr om a starting point ( x 1 , . . . , x n ) ∈ P ( M ) . • If δ ≥ 0 and µ ≥ E [ X ] = P a i x i , then Pr[ X ≥ (1 + δ ) µ ] ≤  e δ (1+ δ ) 1+ δ  µ ; for δ ∈ [0 , 1] , the bound can be simpliﬁed to Pr[ X ≥ (1 + δ ) µ ] ≤ e − µδ 2 / 3 . • If δ ∈ [0 , 1] , and µ ≤ E [ X ] = P a i x i , then Pr[ X ≤ (1 − δ ) µ ] ≤ e − µδ 2 / 2 . In particu lar , these bounds hold for X = P i ∈ S X i where S is an arbitrary subset of the variab les. W e remark that in contrast, whe n randomized pipage roundi ng is performed on bipartite graphs, negati ve correlat ion holds only for subset s of edges inciden t to a ﬁxed v ertex [14]. More general ly , we consider concentra tion properties for a monotone submodular function f ( R ) , where R is the outcome of randomized rounding. Equiv alently , we can also write f ( R ) = f ( X 1 , X 2 , . . . , X n ) where X i ∈ { 0 , 1 } is a rando m v ariable in dicating whethe r i ∈ S . F irst, we conside r a sc enario where X 1 , . . . , X n are indepe ndent random variab les. W e prov e that in this case, Chernof f-type bounds hold for f ( X 1 , X 2 , . . . , X n ) just lik e they would for a linear functi on. Theor em 1.3. Let f : { 0 , 1 } n → R + be a monotone submodular functio n with mar ginal values in [0 , 1] . Let X 1 , . . . , X n be inde pendent random variable s in { 0 , 1 } . L et µ = E [ f ( X 1 , X 2 , . . . , X n )] . T hen for any δ > 0 , • Pr[ f ( X 1 , . . . , X n ) ≥ (1 + δ ) µ ] ≤  e δ (1+ δ ) 1+ δ  µ . • Pr[ f ( X 1 , . . . , X n ) ≤ (1 − δ ) µ ] ≤ e − µδ 2 / 2 . W e remark that Theorem 1.3 can be used to simplify prev ious results for submodular maximization under linear const raints, where variab les are rounded independe ntly [18]. Furthermore, we prov e a lo wer-t ail bound in the depen dent round ing case, w here X 1 , . . . , X n are produc ed by randomized swap rounding . Theor em 1.4. Let f ( S ) be a monoto ne submodular function with mar ginal values in [0 , 1] , and F ( x ) = E [ f ( ˆ x )] its multilinear exten sion. Let ( x 1 , . . . , x n ) ∈ P ( M ) be a point in a matr oid polyto pe and R a random indepe ndent set obtained fr om it by randomized swap r ounding . Let µ 0 = F ( x 1 , . . . , x n ) and δ > 0 . Then E [ f ( R )] ≥ µ 0 and Pr[ f ( R ) ≤ (1 − δ ) µ 0 ] ≤ e − µ 0 δ 2 / 8 . W e do not kno w how to deri ve this result using only the property of negati ve correla tions; in particul ar , we do not hav e a proof for pipage rounding, although w e suspect that a similar tail estima te holds. (W eaker tail estimates in v olving a dependenc e on n follo w directly from martingal e concentra tion bou nds; the main dif ﬁculty here is to obtain a bound which does not depend on n .) W e remark that the tail estimate is with respec t to the v alue of the starting point, µ 0 = F ( x 1 , . . . , x n ) , rath er than the actual expectat ion of f ( R ) , 3 which could be larg er (it would be equal for a linear function f , or under independen t roundin g). For this reason , we do not ha ve an upper tail bound. Howe ver , µ 0 is the val ue that we want to achie ve in applicat ions and hence this is the bound that we need. Applica tions: W e next discuss sev eral applicatio ns of our rounding scheme. W hile some of the application s are concrete , others are couche d in a genera l framewor k; speciﬁc instantiatio ns lea d to var ious applicati ons ne w and old, and we defer some of these to a later versio n of the paper . Our rounding procedure can be used to improv e the running time of some pre vious applicatio ns of pipage rounding [6, 36] and maximum entropy sampling [2]. In parti cular , our techniqu e signiﬁcantly simpliﬁes the al gorithm and analy sis in the recent O (log n / log log n ) -appro ximation for the Asymmetric Tr av eling Salesman problem [2]. In other applicat ions, we obtain ap proximatio ns with h igh probability in stead o f in e xpecta tion [6, 36]. D etails of these impr ov ements are deferre d. Our ne w applica tions are as follo w s. Submodu lar maximization subject to 1 m atr oid and k linear constrain ts. Giv en a monotone submodu lar func- tion f : 2 N → R + , a matroid M on the same ground set N , and a system of k linear packin g constrain ts Ax ≤ b , we consider the follo wing problem: max { f ( x ) : x ∈ P ( M ) , Ax ≤ b, x ∈ { 0 , 1 } n } . This problem is a common generaliza tion of two previ ously studied problems, monoton e submodular maximization subject to a matroid constraint [6] and subject to a constant number of linear constraints [18]. For an y ﬁxed ε > 0 and k ≥ 0 , we obtain a (1 − 1 /e − ε ) -app roximation for this proble m, which is optimal up to the arbitrarily small ε (e ven for 1 matroid or 1 linear constrai nt [25, 11 ]), and genera lizes the prev iously kno wn results in the two specia l cases. W e also obtain a (1 − 1 /e − ε ) -ap proximati on w hen the c onstraint s are suf ﬁciently ”loose” ; that is b i ≥ Ω( ε − 2 log k ) · A ij for all i, j . Minimax Inte ger Pr ogr ams sub ject to a matr oid const raint . Let M be a matroid on a grou nd set N (let n = | N | ). Let B ( M ) be the base polytope of M . W e consi der the problem min { λ : Ax ≤ λb, x ∈ B ( M ) , x ∈ { 0 , 1 } n } where A ∈ R m × n + and b ∈ R n + . W e gi ve an O (log m/ log log m ) -approx imation for this problem, and a si milar result for the min-cost vers ion (with giv en pac king constraints and element costs). This genera lizes earl ier results on minima x inte ger pr ograms whic h were conside red in th e con text of routi ng and partitionin g problems [29, 2 3, 33, 34 , 14 ]; the underlyin g matroid in these setti ngs is th e partit ion matroid. Another ap plicatio n ﬁtting in this frame work is the minimum cr ossing spanni ng tre e pr oblem and its geometric v ariant, the minimum stabbi ng spanning tre e pr oblem . W e elabo rate on these in S ection 6. Multiobj ective optimiz ation with submodula r functions . Suppose we are giv en a matroid M = ( N , I ) and a consta nt number of monotone submod ular functions f 1 , . . . , f k : 2 N → R + . Give n a set of ”tar get value s” V 1 , . . . , V k , we either ﬁ nd a certiﬁcate that there is no solution S ∈ I such that f i ( S ) ≥ V i for all i , or we ﬁnd a solution S such that f i ( S ) ≥ (1 − 1 /e − ε ) V i for all i . Using the frame work of multiobjec tiv e optimization [27], this implies that we can ﬁnd ef ﬁciently a (1 − 1 /e − ε ) -approximat e pare to curve for the problem of maximizing k m onoton e submodular func tions subject to a matroid constrain t. A natural spec ial case of this is the Submodular W elfare problem, w here each objecti ve fun ction f i ( S ) repres ents the utility of player i . I.e., we can ﬁnd a (1 − 1 /e − ε ) -approx imate pareto curve with respect to the utilit ies of the k players (for k cons tant). This result in v olves a new varia nt of the continuou s gr eedy algorit hm from [35], which in some sense optimizes m ultiple submodular functions at the same time. W ith linear objecti ve functions f i , we obtain the same guara ntees w ith 1 − ε instead of 1 − 1 /e − ε . W e giv e more details in Section 7. Organiz ation: In Section 2, w e pr esent the neces sary deﬁnitions. In Section 3 the randomized swap round ing procedur e is intro duced. In Sectio n 4, we prov e a negati ve corre lation proper ty for a class of rounding proced ures including randomized swap rounding and pipage rounding . In Section 5, we present our algorithm for m aximizin g a monoto ne submod ular function subject to 1 matroid and k linea r cons traints. In Section 6, we present our resul ts on minimax integer programs. In Section 7, w e prese nt our results on multiobje ctiv e optimiza tion. In A ppendi x A, we giv e a complete description of randomized pipage rounding . In Appendix B, we present a generaliz ation of swap rounding for rounding points in the matroid polytope rather than the base polyto pe. In Appendix C, we present our concen tration bounds for submodula r functions under inde pendent round ing, and in Appendix D our lower -tail bound under randomized swap roun ding. 4 2 Pr eliminaries Matr oid polytopes. Giv en a matroid M = ( N , I ) with rank funct ion r : 2 N → Z + , two polytopes associat ed with M are the matroid polyto pe P ( M ) and the matroid base polyto pe B ( M ) [9] (see also [30]). P ( M ) is the con ve x hull of characteris tic vectors of the indepe ndent sets of M . P ( M ) = con v { 1 I : I ∈ I } = { x ≥ 0 : ∀ S ; X i ∈ S x i ≤ r ( S ) } B ( M ) is the con vex hull of the characteris tic vecto rs of the bases B of M , i.e. independen t sets of maximum cardin ality . B ( M ) = con v { 1 B : B ∈ B } = P ( M ) ∩ { x : X i ∈ N x i = r ( N ) } . Matr oid exchange properties . T o simplify notation, we use + and − for the addition and deletion of single elements from a set, for exampl e S − i + j deno tes the set ( S \ { i } ) ∪ { j } . The follo w ing base exc hange proper ty of matroids is crucial in the design of our roundin g algori thm. Theor em 2.1. Let M = ( N , I ) be a matr oid and l et B 1 , B 2 ∈ B . F or a ny i ∈ B 1 \ B 2 ther e e xists j ∈ B 2 \ B 1 suc h that B 1 − i + j ∈ B and B 2 − j + i ∈ B . T o ﬁnd an ele ment j that correspo nds to a giv en el ement i as desc ribed i n th e abo ve theorem, o ne can simp ly check all ele ments in B 2 \ B 1 . Thus a co rrespon ding element j can be foun d by O ( d ) calls to an indep endence oracle , w here d is the rank of the matroid. For many matroids, a correspon ding element j can be found fast er . In particul ar , for the graphic matroid, j can be chosen to be any element 6 = i that lies simultaneous ly in the cut deﬁned by the connec ted components of B 1 − i and in the uniq ue cycle in B 2 + i . Submodu lar fun ctions. A functio n f : 2 N → R is submodular if for any A, B ⊆ N , f ( A ) + f ( B ) ≥ f ( A ∪ B ) + f ( A ∩ B ) . In addit ion, f is monotone if f ( S ) ≤ f ( T ) whene ver S ⊆ T . W e denot e by f A ( i ) = f ( A + i ) − f ( A ) the marg inal value of i with respect to A . An important concept in recent work on submodu lar functions [5, 35, 6, 18, 20, 36] is the multilinea r e xtension of a submodular function: F ( x ) = E [ f ( x )] = X S ⊆ N f ( S ) Y i ∈ S x i Y i ∈ N \ S (1 − x i ) . Rounding in the matr oid polytope. A roundin g proced ure takes a point in the matroi d polytope x ∈ P ( M ) and rounds it to an indepe ndent set R ∈ I . In its randomiz ed version , it is obli vious to any objecti ve function and pro duces a random indep endent set, with a distr ibu tion depending only on the s tarting point x ∈ P ( M ) . If the startin g point is in the matroid base polyto pe B ( M ) , th e rounded solution is a (random) base of M . One candidate for such a round ing proc edure is pipag e r ounding [6, 36]. W e giv e a complete description of the pipage rounding technique in the appendix . In particul ar , this rounding satisﬁes that Pr[ i ∈ R ] = x i for each element i , and E [ f ( R )] ≥ F ( x ) for any submodular function f and its multilinear extensio n F . Our new round ing, w hich is descr ibed in S ection 3, satis ﬁes the same properties and has additiona l adv antage s. 3 Randomized swap r ounding Let M = ( N , I ) be a matro id of rank d = r ( N ) and let n = | N | . Randomized swap rounding is a rando mized proced ure that rou nds a point x ∈ P ( M ) to a n inde pendent set. W e pre sent the pro cedure for po ints in th e ba se polyto pe. It can easily be general ized to round any poi nt in the matroid polytope (see Appendix B.2). Assume that x ∈ B ( M ) is the point we want to round. The procedure needs a repre sentatio n of x as a con ve x combination of bases, i.e., x = P m ℓ =1 β ℓ 1 B ℓ with P m ℓ =1 β ℓ = 1 , β ℓ ≥ 0 . Notice that by C arath ´ eodory ’ s 5 theore m there exi sts such a con vex repr esentatio n using at most n bases. In some applic ations, the vector x comes along with a con vex representat ion. Otherwise, it is well-kno w n that one can ﬁ nd such a con vex repre- sentat ion in polynomial time using the fact that one can separa te (or equiv alently optimize ) over the polytop e in polynomial time (see for example [31]). For m atroid polytopes , Cunnin gham [8] proposed a combinato rial algori thm that allo ws to ﬁnd a con vex re presenta tion of x ∈ B ( M ) usi ng at most n bases an d whose ru ntime is bound ed by O ( n 6 ) calls to an independ ence oracle. In special cases, faster algorithms are known; for example any point in the spanning tree polytope of a graph G = ( V , E ) can be decomposed into a con vex combinatio n of span ning trees in ˜ O ( | V | 3 | E | ) time [13]. In genera l this would be the domin ating term in the runn ing time of randomiz ed swap rounding. Giv en a con vex combination of bases x = P n ℓ =1 β ℓ 1 B ℓ , the procedure takes O ( nd 2 ) calls to a matroid indepe ndence oracle. The ro unding p roceeds in n − 1 stages, where in th e ﬁrst stage w e merge the bas es B 1 , B 2 (rando mly) into a new base C 2 , and replace β 1 1 B 1 + β 2 1 B 2 in the linear combina tion by ( β 1 + β 2 ) 1 C 2 . In the k -th stage, C k and B k +1 are mer ged into a new base C k +1 , and ( P k ℓ =1 β ℓ ) 1 C k + β k +1 1 B k +1 is replaced in the linear combination by ( P k +1 ℓ =1 β ℓ ) 1 C k +1 . A fter n − 1 stages, we obtain a linear combination ( P n ℓ =1 β ℓ ) 1 C n = 1 C n , and the base C n is returned . Algorith m MergeBases ( β 1 , B 1 , β 2 , B 2 ) : While ( B 1 6 = B 2 ) do Pick i ∈ B 1 \ B 2 and ﬁnd j ∈ B 2 \ B 1 such that B 1 − i + j ∈ I and B 2 − j + i ∈ I ; W ith probability β 1 / ( β 1 + β 2 ) , { B 2 ← B 2 − j + i } ; Else { B 1 ← B 1 − i + j } ; EndWhile Output B 1 . The procedure we use to merg e two bases, called Mer geBases , takes as i nput two base s B 1 and B 2 and two positi ve scalars β 1 and β 2 . It is descri bed in the adjacen t ﬁgure. N otice that the proced ure relie s heavil y on the basis exchang e proper ty giv en by Theorem 2.1 to guarantee the exi stence of the elements j in th e while loop. As discus sed in Section 2, j can be found by chec k- ing all elements in B 2 \ B 1 . Furthermore, since the cardinality of B 1 \ B 2 decrea ses at each iteration by one, the total number of iterations is bounded by | B 1 | = d . Algorith m Sw apRound ( x = P n ℓ =1 β ℓ 1 B ℓ ) : C 1 = B 1 ; For ( k = 1 to n − 1 ) do C k +1 = MergeBa ses ( P k ℓ =1 β ℓ , C k , β k +1 , B k +1 ) ; EndFor Output C n . The m ain algorithm Sw apRound is described in the ﬁgure. It uses MergeBases to repeat edly mer ge bases in the con vex decompositi on of x . For further analys is we present a diff erent viewpoi nt on the al- gorith m, namely as a random process in the matroid base polyto pe. T his also allo ws us to present the al- gorith m in a common framew ork w ith pipage round- ing and to dra w parallels between the approaches more easily . W e denote by an elemen tary oper ation of the swap r ounding a lgorithm on e iterat ion of the while loop in the Merge Bases proced ure, which is repeatedly called in Sw apRound . Hence, an elementa ry operation change s two components in one of the bases used in the con vex repres entation of the curr ent point. For example, if the ﬁrst elementary operation transfor ms the base B 1 into B ′ 1 , then this can be interpreted on the matroid base polyto pe as transforming the point x = P n ℓ =1 β ℓ 1 B ℓ into β 1 1 B ′ 1 + P n ℓ =2 β ℓ 1 B ℓ . H ence, the SwapRound algori thm can be seen as a sequence of dn elementary operation s leading to a random sequen ce X 0 , . . . , X τ where X t denote s the con vex combinati on after t elementary operation s. 4 Negative corr elation for dependent rounding pr o cedur es In this section, w e pro ve a result which shows that the statement of Theorem 1.1 is true for a lar ge class of random vec tor -va lued processes that only change at m ost two componen ts at a time. Theorem 1.1 then easily follo ws by observin g that randomized swap rounding as well as pipage rounding fall in this class of random proces ses. The proof follo w s the same lines as [14] in the case of bipa rtite graphs. The intui tiv e reason for 6 neg ati ve correlatio n is that whene ver a pair of varia bles is being modiﬁed, their sum remains consta nt. Hence , kno w ing that one vari able is high can only make the expect ation of another vari able lower . Lemma 4.1. L et τ ∈ N and let X t = ( X 1 ,t , . . . , X n,t ) for t ∈ { 0 , . . . , τ } be a non-ne gative vector -valued ran dom pr ocess with initial distrib ution given by X i, 0 = x i with pr obabili ty 1 ∀ i ∈ [ n ] , and satisfy ing the followin g pr operties: 1. E [ X t +1 | X t ] = X t for t ∈ { 0 , . . . , τ } and i ∈ [ n ] . 2. X t and X t +1 dif fer in at most two components for t ∈ { 0 , . . . , τ − 1 } . 3. F or t ∈ { 0 , . . . , τ } , if two co mponents i, j ∈ [ n ] chang e between X t and X t +1 , then their su m is pr eserved: X i,t +1 + X j,t +1 = X i,t + X j,t . Then for any t ∈ { 0 , . . . , τ } , the compone nts of X t satisfy E [ Q i ∈ S X i,t ] ≤ Q i ∈ S x i ∀ S ⊆ [ n ] . Pr oof. W e are interest ed in the quan tity Y t = Q i ∈ S X i,t . A t the beginn ing of the process, we hav e E [ Y 0 ] = Q i ∈ S x i . The main claim is that for each t , we ha ve E [ Y t +1 | X t ] ≤ Y t . Let us condition on a particular conﬁguration of va riables at time t , X t = ( X 1 ,t , . . . , X n,t ) . W e consider three cases : • If no v ariable X i , i ∈ S , is modiﬁed in step t , we ha ve Y t +1 = Q i ∈ S X i,t +1 = Q i ∈ S X i,t = Y t . • If exactl y one varia ble X i , i ∈ S , is modiﬁed in step t , then by prope rty 1 of the lemma: E [ Y t +1 | X t ] = E [ X i,t +1 | X t ] · Y j ∈ S \{ i } X j,t = Y j ∈ S X j,t = Y t . • If two variab les X i , X j , i, j ∈ S , are m odiﬁed in step t , we use the property that their sum is preserv ed: X i,t +1 + X j,t +1 = X i,t + X j,t . This also implies that E [( X i,t +1 + X j,t +1 ) 2 | X t ] = ( X i,t + X j,t ) 2 . (1) On the other hand, the valu e of each v ariable is preserve d in expectat ion. Applyin g this to their differe nce, we get E [ X i,t +1 − X j,t +1 | X t ] = X i,t − X j,t . Since E [ Z 2 ] ≥ ( E [ Z ]) 2 holds for any random var iable, we get E [( X i,t +1 − X j,t +1 ) 2 | X t ] ≥ ( X i,t − X j,t ) 2 . (2) Combining (1) and (2), and using the formula X Y = 1 4 (( X + Y ) 2 − ( X − Y ) 2 ) , we get E [ X i,t +1 X j,t +1 | X t ] ≤ X i,t X j,t . Therefore , E [ Y t +1 | X t ] = E [ X i,t +1 X j,t +1 | X t ] · Y k ∈ S \{ i,j } X k ,t ≤ Y k ∈ S X k ,t = Y t , as claimed. By tak ing expe ctation ov er all conﬁgura tions X t we obtain E [ Y t +1 ] ≤ E [ Y t ] . Consequentl y , E [ Q i ∈ S X i,t ] = E [ Y t ] ≤ E [ Y t − 1 ] ≤ . . . ≤ E [ Y 0 ] = Q i ∈ S x i , as claimed by the lemma. Any process that satis ﬁes the condit ions of Lemma 4.1 thus also sati sﬁes the ﬁrst stat ement of T heo- rem 1.1. Furthermore, the second statement of Theorem 1.1 also follo ws by observing that for any proce ss ( X 1 ,t , . . . , X n,t ) that satisﬁes the conditi ons of Lemma 4.1, also the proc ess (1 − X 1 ,t , . . . , 1 − X n,t ) satis- ﬁes the condition s. As we mentioned in S ection 1, these results imply strong concentr ation bound s for linear functi ons of the v ariables X 1 , . . . , X n (Corollar y 1.2). Both randomiz ed swap rounding and pipage roundin g satisfy the condi tions of Lemma 4.1 (proof s can be found in the Appendix). This implies T heorem 1.1. Note that the sequences X t created by randomiz ed swap round ing or pi page roun ding – besides satis fying the c ondition s of Lemma 4.1 – are Mark ovian, and hence the y are vect or- val ued martingal es. 7 5 Submodular maximization subject to 1 matroid and k li near constraints In this section, w e present an algorithm for t he problem of m aximizin g a m onoton e submod ular fu nction subject to 1 matroid and k linear (”knapsack” ) constrain ts. Pro blem deﬁn ition. Given a monotone submodula r funct ion f : 2 N → R + (by a value oracle), and a matr oid M = ( N , I ) (by an i ndepend ence orac le). F or each i ∈ N , we have k paramet ers c ij , 1 ≤ j ≤ k . A se t S ⊆ N is feasi ble if S ∈ I and P i ∈ S c ij ≤ 1 for each 1 ≤ j ≤ k . The goal is to maximize f over all feasible sets. Kulik et al. ga ve a (1 − 1 /e − ε ) -approxi mation for the same problem with a cons tant number of linear constr aints, b ut without the matroid constraint [18]. Gupta, Nagarajan and R a vi [15] sho w that a knapsack constr aint can in a techni cal sense be simulate d in a black-bo x fash ion by a collec tion of partition matroid con- straint s. Using their reductio n and kno wn results on submodula r set function maximizatio n subject to matroid constr aints [12, 21], the y obtain a 1 / ( p + q + 1) -appro ximation with p knapsac ks and q matroids for an y q ≥ 1 and ﬁxed p ≥ 1 (or 1 / ( p + q + ε ) for any ﬁxed p ≥ 1 , q ≥ 2 and ε > 0 ). 5.1 Constant number of knapsack constraints W e consider ﬁrst 1 matroid and a consta nt number k of linear constraints, in which case each linear constraint is thought of as a ”knapsack ” constrain t. W e sho w a (1 − 1 /e − ε ) -approximat ion in this case, bui lding upon the algorith m of Kulik, Shachnai and T amir [18], which works for k knapsack constraint s (without a matroid constr aint). The basic idea is that we can add the knap sack constraint s to the multilinear optimization problem max { F ( x ) : x ∈ P ( M ) } which is used to achie ve a (1 − 1 /e ) -appro ximation for 1 matroid constraint [6]. Using standard technique s (partia l en umeration), we get rid of all items of lar ge v alue or size, and then scale do wn the constrai nts a littl e bit, so that w e hav e some room for ov erﬂo w in the round ing stage. W e can still solve the multilinear optimization proble m within a fa ctor of 1 − 1 /e and then round the frac tional solution using randomized swap round ing (or pipage rounding ). Usin g the fact that randomized swap roun ding makes the size in each knapsack strongly concen trated, w e conclu de that our solution is feasible with constant probabilit y . Algorithm. • Assume 0 < ε < 1 / (4 k 2 ) . E numerate all sets A of at most 1 /ε 4 items which form a feasible solution. (W e are tryin g to guess the most valuab le items in the optimal soluti on under a greed y ordering.) For each candidat e set A , repeat the follo wing. • Let M ′ = M / A be the matro id where A has been contracted . For each 1 ≤ j ≤ k , l et C j = 1 − P i ∈ A c ij be the remain ing cap acity in knapsack j . L et B be the s et o f items i / ∈ A such that e ither f A ( i ) > ε 4 f ( A ) or c ij > kε 3 C j for some j (the item is relati vely big compared to the size of some knapsack) . Thro w awa y all the items in B . • W e consider a reduced problem on the item set N \ ( A ∪ B ) , with the matroid constraint M ′ , knapsack capaci ties C j , and objec tiv e function g ( S ) = f A ( S ) . Deﬁne a polyto pe P ′ = n x ∈ P ( M ′ ) : ∀ j ; X c ij x i ≤ C j o (3) where P ( M ′ ) is the matroi d pol ytope of M ′ . W e solve (approximat ely) the follo wing op timization proble m: max  G ( x ) : x ∈ (1 − ε ) P ′  (4) 8 where G ( x ) = E [ g ( ˆ x )] is t he multili near exten sion of g ( S ) . S ince li near funct ions can be optimized o ver P ′ in polynomial time, w e can use the continuou s greedy algorit hm [35 ] to ﬁnd a fractional soluti on x ∗ within a fac tor of 1 − 1 /e of optimal. • Giv en a fractio nal solution x ∗ , we apply randomized pipage roundin g to x ∗ with respect to the matroid polyto pe P ( M ′ ) . Call the resulting set R A . A mong all candidat e sets A such that A ∪ R A is feasible, return the one maximizing f ( A ∪ R A ) . W e remark that th e v alue of this algorit hm (unlike the (1 − 1 /e ) -a pproximati on for 1 matroid cons traint) is purely theore tical, as it relies on enumerati on of a huge (constan t) number of elements. Theor em 5.1. W ith constan t positiv e pr obabil ity , the algorith m above r eturns a solut ion of valu e at least (1 − 1 /e − 3 ε ) O P T . Pr oof. Consider an optimum solution O , i.e. O P T = f ( O ) . Order the elements of O greedily by decreasin g mar ginal v alues, and let A ⊆ O be the elements whose mar ginal val ue is at least ε 4 O P T . There can b e at most 1 /ε 4 such elements , and so the algorithm will consider them as one of the candidate sets. W e assume in the follo wing that this is the set A chosen by the algorithm. W e co nsider the red uced instance, where M ′ = M / A and the knap sack capacit ies are C j = 1 − P i ∈ A c ij . O \ A is a feasible solutio n for this instance and we ha ve g ( O \ A ) = f A ( O \ A ) = O P T − f ( A ) . W e kno w that in O \ A , there are no items of marg inal valu e m ore than the last item in A . In particu lar , f A ( i ) ≤ ε 4 f ( A ) ≤ ε 4 O P T for all i ∈ O \ A . W e throw away all items where f A ( i ) > ε 4 f ( A ) b ut this does not af fect any item in O \ A . W e also thro w away the set B ⊆ N \ A of items whose size in some knapsack is more then k ε 3 C j . In O \ A , there can be at most 1 / ( kε 3 ) such items for each knapsack , i.e. 1 /ε 3 items in total. Since their margin al value s with respect to A are bounded by ε 4 O P T , these items together ha ve value g ( O ∩ B ) = f A ( O ∩ B ) ≤ εO P T . O ′ = O \ ( A ∪ B ) is still a feasible set for the red uced problem, and using submodu larity , its va lue is g ( O ′ ) = g (( O \ A ) \ ( O ∩ B )) ≥ g ( O \ A ) − g ( O ∩ B ) ≥ O P T − f ( A ) − εOP T . No w consid er the multilinear proble m (4). Note that the indicato r v ector 1 O ′ is feasibl e in P ′ , and hence (1 − ε ) 1 O ′ is feasib le in (1 − ε ) P ′ . Using the conc av ity of G ( x ) along the line fr om the origin to 1 O ′ , we ha ve G ((1 − ε ) 1 O ′ ) ≥ (1 − ε ) g ( O ′ ) ≥ (1 − 2 ε ) OP T − f ( A ) . Using the continuous greedy algori thm [35], we ﬁnd a fractio nal solution x ∗ of v alue G ( x ∗ ) ≥ (1 − 1 /e ) G ((1 − ε ) 1 O ′ ) ≥ (1 − 1 /e − 2 ε ) O P T − f ( A ) . Finally , w e apply randomized swap rounding (or pipage roundi ng) to x ∗ and call the resultin g set R . By the construct ion of randomized swap rounding, R is indepe ndent in M ′ with probabi lity 1 . Howe ver , R m ight violat e some of the knapsack constraints . Consider a ﬁxed knapsack constraint, P i ∈ S c ij ≤ C j . Our frac tional so lution x ∗ satisﬁes P c ij x ∗ i ≤ (1 − ε ) C j . Also, we kno w that all sizes in the reduced insta nce are bounded by c ij ≤ k ε 3 C j . By scali ng, c ′ ij = c ij / ( kε 3 C j ) , we can apply Corollar y 1.2 with µ = (1 − ε ) / ( kε 3 ) : Pr[ X i ∈ R c ij > C j ] ≤ Pr[ X i ∈ R c ′ ij > (1 + ε ) µ ] ≤ e − µε 2 / 3 < e − 1 / 4 kε . On the other hand, consid er the objecti ve function g ( R ) . In the reduced instance, all items hav e v alue g ( i ) ≤ ε 4 O P T . Let µ = G ( x ∗ ) / ( ε 4 O P T ) . Then, Theorem 1.4 implies Pr[ g ( R ) ≤ (1 − δ ) G ( x ∗ )] = Pr[ f ( R ) / ( ε 4 O P T ) ≤ (1 − δ ) µ ] ≤ e − δ 2 µ/ 8 = e − δ 2 G ( x ∗ ) / 8 ε 4 O P T . 9 W e set δ = O P T G ( x ∗ ) ε and obtain Pr[ g ( R ) ≤ G ( x ∗ ) − εO P T ] ≤ e − O P T / 8 ε 2 G ( x ∗ ) ≤ e − 1 / 8 ε 2 . By the union boun d, Pr[ g ( R ) ≤ G ( x ∗ ) − εO P T or ∃ j ; X i ∈ R c ij > C j ] ≤ e − 1 / 8 ε 2 + ke − 1 / 4 kε . For ε < 1 / (4 k 2 ) , this prob ability is at most e − 2 k 4 + ke − k < 1 . If this ev ent does not occur , we hav e a feasible soluti on of value f ( R ) = f ( A ) + g ( R ) ≥ f ( A ) + G ( x ∗ ) − εO P T ≥ (1 − 1 /e − 3 ε ) O P T . 5.2 Loose packing constraints In this section w e consider the case when the number of linear packing constraints is not a ﬁ xed constan t. The notati on w e use in this case is that of a packi ng integer progr am: max { f ( x ) : x ∈ P ( M ) , Ax ≤ b, x ∈ { 0 , 1 } n } . Here f : 2 N → R is a monoton e submodular functi on with n = | N | , M = ( N , I ) is a matroid, A ∈ R k × n + is a non-ne gati ve matrix and b ∈ R k + is a non-ne gati ve vector . This proble m has been studied exten siv ely when f ( x ) is a linear function, in other words f ( x ) = w T x for some non-ne gati ve weight vector w ∈ R n . E ven this case w ith A, b ha ving only 0 , 1 entries captures the maximum indepen dent set problem in graphs and hence is NP-hard to approximate to within an n 1 − ε -fac tor for any ﬁxed ε > 0 . For this reason a var iety of restriction s on A, b hav e been studied. W e consider the case when the constrain ts are suf ﬁciently loose, i.e. the right-hand side b is signiﬁcantly lar ger th an ent ries in A : in par ticular , we assume b i ≥ c log k · max j A ij for 1 ≤ i ≤ k . In this case, we propose a straig htforwar d algorithm w hich works as fo llo ws. Algorithm. • Let ε = p 6 /c . S olve ( approxima tely) the follo wing optimization problem: max { F ( x ) : x ∈ (1 − ε ) P } where F ( x ) = E [ f ( ˆ x )] is the multilinear extension of f ( S ) , and P = { x ∈ P ( M ) | ∀ i ; X j ∈ N A ij x j ≤ b i } . Since linear functions can be optimized over P in poly nomial time, we can use the conti nuous greedy algori thm [35] to ﬁnd a fract ional solution x ∗ within a factor of 1 − 1 /e of optimal. • Apply randomize d pipage rounding to x ∗ with respec t to the m atroid polytope P ( M ) . If the resultin g soluti on R satisﬁes the packing constrain ts, return R ; othe rwise, fail. Theor em 5.2. Assume that A ∈ R k × n and b ∈ R k suc h that b i ≥ A ij c log k for all i, j and some const ant c = 6 /ε 2 . Then the algorith m above gives a (1 − 1 /e − O ( ε )) -appr oximation with constant pr obability . W e remark that it is NP-hard to achie ve a better than (1 − 1 /e ) -ap proximatio n eve n when k = 1 and the constr aint is very loose ( A ij = 1 and b i → ∞ ) [11]. 10 Pr oof. The proof is similar to that of Theorem 5.1, but simpler . W e only highlight the main differe nces. In the ﬁrst stage we obtain a fractional solution such that F ( x ∗ ) ≥ (1 − ε )(1 − 1 /e ) O P T . Randomized swap round ing yields a rand om solutio n R which satisﬁes the matroid const raint. It remains to chec k the packing constr aints. For e ach i , we hav e E [ X j ∈ R A ij ] = X j ∈ N A ij x ∗ j ≤ (1 − ε ) b i . The v ariables X j are nega tiv ely correlated and by C orollar y 1.2 with δ = ε = p 6 /c and µ = c log k , Pr[ X j ∈ R A ij > b i ] < e − δ 2 µ/ 3 = 1 k 2 . By the u nion bound, all pack ing constrai nts are satisﬁed with pro bability at least 1 − 1 /k . W e assu me here that k = ω (1) . By using T heorem 1.4, w e can also conclu de that the val ue of the solution is at least (1 − 1 /e − O ( ε )) O P T with con stant probabili ty . 6 Minimax integer programs with a matroid con straint Minimax integ er p rograms are moti vate d by ap plicatio ns to routing and partitioning . The setup is as follows; we follo w [33]. W e hav e boolean va riables x i,j for i ∈ [ p ] and j ∈ [ ℓ i ] for integers ℓ 1 , . . . , ℓ p . Let n = P i ∈ [ p ] ℓ i . The goal is to minimize λ subject to: • equali ty constr aints: ∀ i ∈ [ p ] , P j ∈ [ ℓ i ] x i,j = 1 • a system of linear inequal ities Ax ≤ λ 1 where A ∈ [0 , 1] m × n • inte grality constraints : x i,j ∈ { 0 , 1 } for all i, j . The variab les x i,j , j ∈ [ ℓ i ] for each i ∈ [ p ] capture the fact that exactly one option amongst the ℓ i option s in gr oup i shoul d be chosen . A c anonical exa mple is the congestion minimizatio n problem for in tegral rout ings in graphs where for each i , the x i,j v ariables represent the dif ferent paths for routin g the ﬂow of a pair ( s i , t i ) and the matrix A encodes the capacity constrai nts of the edges. A natura l appr oach is to solve the natural LP relaxation for the above problem and then apply randomized rounding by choosing independe ntly for each i exact ly one j ∈ [ ℓ i ] where the probability of choosin g j ∈ [ ℓ i ] is exactly equal to x i,j . This follo ws the randomiz ed rounding met hod of Raghav an and Thomp son for congestion minimiz ation [29] and one obtains an O (log m / log log m ) -approx imation with respec t to the fractiona l solutio n. Using L ov ´ asz Local Lemma (and complica ted derandomizat ion) it is possible to obtain an improv ed bound of O (log q / log log q ) [23, 33] where q is the m aximum n umber of n on-zero entri es in an y col umn of A . This re ﬁned bound h as v arious app lications . Interes tingly , the abo ve pr oblem becomes non-tr iv ial if we mak e a sligh t change to the e quality cons traints. Suppose for each i ∈ [ p ] we no w ha ve an equality constraint of the form P j ∈ [ ℓ i ] x i,j = k i where k i is an inte ger . For routi ng, this corres ponds to a requirement of k i paths for pair ( s i , t i ) . Now the standar d rando mized round ing doesn’ t quite work for this low conge stion multi-pa th r outing pr oblem . Srini v asan [34], motiv ated by this genera lized routing problem, dev eloped dependent randomized rounding and used the negati ve correlation proper ties of this rounding to obtain an O (log m/ log log m ) -approximatio n. T his was further general ized in [14] as randomize d ver sions of pipage roundin g in the contex t of other applicatio ns. 6.1 Congestion minimization under a matroid b ase constraint Here we show that our dependen t rounding in matroids allo ws a clean generaliz ation of the type of constrain ts consid ered in se vera l applica tions in [34, 14]. L et M be a matroid on a ground set N . Let B ( M ) be the base polyto pe of M . W e consider the problem min  λ : ∃ x ∈ { 0 , 1 } N , x ∈ B ( M ) , Ax ≤ λ 1  11 where A ∈ [0 , 1] m × N . W e observe that the pre vious problem with the v ariables partitione d into groups and equali ty constr aints can be cast na turally as a special cas e of this matroid co nstraint pro blem; the equality constr aints simply corre spond to a partition matroid on the ground set of all varia bles x i,j . Ho wev er , our frame work is much more ﬂexible. For example, consid er the spanning tree problem with packin g constrain ts: each edge has a weight w e and we want to m inimize the maximum load on an y vertex , max v ∈ V P e ∈ δ ( v ) w e . This problem also falls within our frame work. Theor em 6.1. Ther e is an O (log m/ log log m ) -appr oximation for the pr oblem min  λ : ∃ x ∈ { 0 , 1 } N , x ∈ B ( M ) , Ax ≤ λ 1  , wher e m is the number of pack ing constrai nts, i.e. A ∈ [0 , 1] m × N . Pr oof. Fix a va lue of λ . Let Z ( λ ) = { j | ∃ i ; A ij > λ } . W e can force x j = 0 for all j ∈ Z ( λ ) , becaus e no element j ∈ Z ( λ ) can be in a feasible solution for λ . In polynomial time, we can check the feasibil ity of the follo wing L P: P λ =  x ∈ B ( M ) : Ax ≤ λ 1 , x | Z ( λ ) = 0  (becau se w e can separate over B ( M ) and the additional packing constrain ts efﬁcie ntly). By binary search, we can ﬁnd (withi n 1 + ε ) the minimu m valu e of λ such that P λ 6 = ∅ . This is a l ower bound on the actua l optimum λ O P T . W e also obtain the corresp onding fractional solution x ∗ . W e apply rando mized swap rounding (or randomized pipage rounding) to x ∗ , obtai ning a random set R . R satisﬁes the matroid base const raint by deﬁnition. Consider a ﬁxed packing constraint (the i -th row of A ). W e ha ve X j ∈ N A ij x ∗ j ≤ λ and all entries A ij such that x ∗ j > 0 are bounded by λ . W e set ˜ A ij = A ij /λ , so that we can use Corollary 1.2. W e get Pr[ X j ∈ R A ij > (1 + δ ) λ ] = Pr[ X j ∈ R ˜ A ij > 1 + δ ] <  e δ (1 + δ ) 1+ δ  µ . For µ = 1 and 1 + δ = 4 log m log log m , this prob ability is bounded by Pr[ X j ∈ R A ij > (1 + δ ) λ ] ≤  e log log m 4 log m  4 log m log log m <  1 √ log m  4 log m log log m = 1 m 2 for suf ﬁciently large m . Therefore, all m constrain ts are satisﬁed within a factor of 1 + δ = 4 log m log log m with high probab ility . W e remark that the approximati on guarantee can be made an ”almost additi ve” O (log m ) , in the follo wing sense: Assuming that the optimum v alue is λ ∗ , for any ﬁxed ε > 0 w e can ﬁnd a solution of value λ ≤ (1 + ε ) λ ∗ + O ( 1 ε log m ) . Scaling is importa nt here: recall that we ass umed A ∈ [0 , 1] N × m . W e omit the p roof, which follo ws by a similar applicati on of the Chernof f bound as abo ve, wit h µ = λ ∗ and δ = ε + O ( 1 ελ ∗ log m ) . Minimum S tabbing and Cr ossing T r ee Pr oblems: Another int eresting appl ication of Theorem 6.1, is to the minimum sta bbing and cro ssing tree problems. Bilo et al. [4], motiv ated by se veral app lication s, conside red the crossi ng spanning tree problem. T he input is a graph G = ( V , E ) and an explit set C of m cuts in G . The goal is to ﬁnd a spanning tree tha t minimizes the number of edges crossing any cut in C . T he algorithm in [4] returns a tree that crosses any cut in C at most O ((log m + log n )( γ ∗ + log n )) times where γ ∗ is the optimal solution v alue; the authors claim an improv ed bound of O ( γ ∗ log n + log m ) in a subsequ ent versi on of the paper . 12 The minimum stabbing tree proble m arises in compu tational geometry: the input i s a set V = { v 1 , . . . , v n } of po ints in R d ; it is assumed that d is a constant and the case of 2 -dimens ions is of partic ular interest. The task is to constru ct a spanning tree on V by connecting vertices with straight lines such that the crossing number , which is the maximum number of edges that are inters ected by any hyperp lane, is minimized. This problem was sho wn to be NP-hard by Fekete et al. [10 ]. It is relati vely easy to see tha t the stabbing tree pro blem is a special case of the crossing spannin g tree problem; th e number of combinatori ally distinct cuts induced by the hyperplan es is O ( n d ) , one for each set of d poin ts that deﬁne a hyperplane through them. Thus, the result in [4] implies that there is an algori thm for the stabbin g tree problem that retur ns a tree with crossing number O ( λ ∗ log n ) where λ ∗ is the tree with the smallest crossing number (note that this is via the impro ved bound claimed by the authors of [4 ] in a longer vers ion). Unawa re of the work in [4], HarPeled very recentl y [16] ga ve a polynomial time algorithm for the stabbing tree proble m that outputs a tree with crossing number O ( λ ∗ log n + log 2 n/ log log n ) . Both of the abov e problems can be cast as speci al cases of the minimizatio n problem presente d in The- orem 6.1 , where M is the graphic matroid and each row of A corres ponds to the incidence vector of a cut. Theorem 6.1 implies that using dependen t randomized rounding, an O (log n/ log log n ) -appro ximation can be obtain ed for the stabbing tree problem and an O (log m/ log log m ) -approx imation for the crossi ng spanning tree problem. The approx imation guarantee can be trans formed into an almost additi ve one as well, leading to a solution of val ue λ ≤ (1 + ε ) λ ∗ + O ( 1 ε log n ) for the stabbing tre e prob lem and a solution of v alue γ ≤ (1 + ε ) γ ∗ + O ( 1 ε log m ) for the crossing spanning tree problem. Note that these additi ve results imply a consta nt factor approximatio n if the optimal v alue is Ω(log n ) and Ω(log m ) respecti vely . W e remark th at the results we ob tain for the abo ve problems can also be ob tained by the maximum ent ropy sampling approach for spanning trees from [2]; our algorithms hav e the adv antage of being simpler and more ef ﬁcient. 6.2 Min-cost matr oid bases with packing constraints W e can similarl y ha ndle the case where in additio n we want to minimize a linear objec tiv e functi on. An exampl e of such a problem would be a multi-path routin g problem minimizing the total cost in addition to congestion . Another example is the minimum-cost spanning tree with packing constr aints for the edges incident with each ver tex. W e remark that in case the packing constra ints are simply degre e bound s, strong results are kno w n - namely , there is an algorithm that ﬁ nds a spanning tree of optimal cost and violatin g the degr ee bound s by at most one [32]. In the general case of ﬁnding a matroid base satisfy ing certain ”de gree const raints”, there is an algorithm [17] that ﬁnds a base of optimal cost and violating the degre e constrain ts by an additi ve error of at most ∆ − 1 , where each element participate s in at most ∆ constr aints (e.g. ∆ = 2 for degree-b ounded spann ing trees). The algorithm of [17] also works for upper and lo wer bounds , violatin g each constraint by at most 2∆ − 1 . See [17] for more details. W e consider a varian t of this pro blem where the packing co nstraints can in vo lve ar bitrary w eights and capaci ties. W e show that we can ﬁnd a mat roid ba se of near -optimal cost which violates the packing c onstrain ts by a multiplic ativ e factor of O (log m / log log m ) , where m is the total number of packin g constr aints. Theor em 6.2. Ther e is a (1 + ε, O (log m/ log log m )) -bicri teria appr oximation for the pr oblem min  c T x : x ∈ { 0 , 1 } N , x ∈ B ( M ) , Ax ≤ b  , wher e A ∈ [0 , 1] m × N and b ∈ R N ; the ﬁrst gu aran tee is w .r .t. the cost o f the so lution and the second gua rantee w .r .t. the ove rﬂow on the packi ng constrain ts. Pr oof. W e gi ve a sketch of t he proo f. First, we thro w away all elements that o n their o wn violate some packing constr aint. Then, we solv e the followin g LP : min  c T x : x ∈ B ( M ) , Ax ≤ b  . 13 Let the optimum solution be x ∗ . W e apply randomize d swap roundin g (or randomized pipage rounding ) to x ∗ , yieldi ng a random solution R . Since each of the m constrai nts is satis ﬁed in exp ectation , and each element alone satisﬁes each packing constra int, w e get by the same analysis as abov e th at with high probabi lity , R violat es e very cons traint by a factor of O (log m / log log m ) . Finally , the ex pected cost of our solution is c T x ∗ ≤ O P T . B y Marko v’ s inequalit y , the probabi lity that c ( R ) > (1 + ε ) O P T is at most 1 / (1 + ε ) ≤ 1 − ε/ 2 . W ith proba bility at lea st ε/ 2 − o (1) , c ( R ) ≤ (1 + ε ) O P T and all packing const raints are satisﬁed within O (log m/ log log m ) . Let us rephras e this result in the more familiar setting of spannin g tree s. Giv en packing constra ints on the edges incide nt with each verte x, using arbitrary weights and capacities , we can ﬁnd a spanning tree of near -optimal cost, violating each packing constraint by a multiplicati ve fac tor of O (log m/ log log m ) . As in the previ ous secti on, if we assume that the weights are in [0 , 1] , this can be replaced by an additi ve fa ctor of O ( 1 ε log m ) w hile making the multipli cati ve factor 1 + ε (see the end of Section 6.1). In the general case of matroid bases, our result is incomparabl e to that of [17], which provides an additi ve guaran tee of ∆ − 1 . (The assumptio n here is that each element participates in at most ∆ degree constraints ; in our frame work, this correspond s to A ∈ { 0 , 1 } m × N with ∆ -sparse columns.) When elements participate in many degree const raints ( ∆ ≫ log m ) and the degree boun ds are b i = O (log m ) , our r esult is act ually stronge r in terms of the pack ing constraint guarantee. Asymmetric T rav eling Salesman and Maximum Entr opy S ampling: In a recent breakth rough, [2] ob- tained an O (log n / log log n ) -approximatio n for the A TSP problem. A crucial ingredient in the approach is to round a point x in the spanning tree polytope to a tree T such that no cut of G contains too m any edges of T , and the cost of the tree is within a constant factor of the cost of x . For this purpo se, [2] uses the maximum entrop y sampling approach which also enjo ys nega tiv e correlation properties and hence one can get Chernof f- type bounds for linear sums of the vari ables; moreov er T cont ains each edge e with probabi lity x e . W e note that the n umber of cuts i s ex ponentia l in n . T o add ress this issue, [2] us es Karg er’ s result on the nu mber of cuts in a graph within a certain weight range: assuming that the minimum cut is at least 1 , there are only O ( n 2 α ) cuts of weight in ( α/ 2 , α ] for any α ≥ 1 . Maximum entropy sampling is technically quite in volv ed and also computa tionally exp ensi ve. Our roundin g procedur es can be used in place of m aximum entropy sampling to simplify the algorith m and the analysis in [2]. 7 Multiobjectiv e optimization with submodular functions In this se ction, we consider the foll owing pro blem: Give n a matroi d M = ( N , I ) and k monotone submodular functi ons f 1 , . . . , f k : 2 N → R + , in what sense can we maximize f 1 ( S ) , . . . , f k ( S ) simulta neously ov er S ∈ I ? This questio n has been studied in the framew ork of m ultiobj ective optimization , popularize d in the CS community by the work of Papa dimitriou and Y annakakis [27]. T he set of all soluti ons which are optimal with respec t to f 1 ( S ) , . . . , f k ( S ) is capture d by the notio n of a par eto set : the set of all solution s S such that for any other feasible solution S ′ , there exists i for which f i ( S ′ ) < f i ( S ) . Since the pareto set in general can be exp onential ly lar ge, we settle for the notion of a ε -approximat e pareto set, where the condition is replaced by f i ( S ′ ) < (1 + ε ) f i ( S ) . Papadimitrio u and Y annaka kis show the foll owing equi valen ce [27, Theorem 2]: Pro position 7.1. An ε -appr oximate par eto set can be foun d in polynomial time, if and only if the follo wing pr oblem can be solved: Given ( V 1 , . . . , V k ) , either r eturn a solution with f i ( S ) ≥ V i for all i , or answer that ther e is no solutio n such tha t f i ( S ) ≥ (1 + ε ) V i for all i . The latter problem is exac tly w hat we addre ss in this section. W e show the foll owin g result. Theor em 7.2. F or any ﬁxed ε > 0 and k ≥ 2 , given a m atr oid M = ( N , I ) , monotone submodular functio ns f 1 , . . . , f k : 2 N → R + , and valu es V 1 , . . . , V k ∈ R + , in polyno mial time w e can either 14 • ﬁnd a solution S ∈ I such that f i ( S ) ≥ (1 − 1 /e − ε ) V i for all i , or • r eturn a certiﬁcat e that ther e is no solution with f i ( S ) ≥ V i for all i . If f i ( S ) ar e linear functions, the guaran tee in the ﬁrst case becomes f i ( S ) ≥ (1 − ε ) V i . This together with Proposi tion 7.1 implies t hat f or an y cons tant nu mber of linear objecti ve f unctions s ubject to a matroid constraint , an ε -appr oximate pareto set can be found in polynomial time. (This was known in the case of multio bjecti ve spanning trees [27].) Furthermore, a straigh tforward m odiﬁcatio n of Prop. 7.1 (see [27], Theorem 2) implies that for m onoton e submodular function s f i ( S ) , we can ﬁnd a (1 − 1 /e − ε ) -approx imate pareto set. Our al gorithm requir es a modiﬁcation of the cont inuous greedy algori thm from [35, 6]. W e sho w the follo wing, which might be useful in other applicati ons as w ell. In the followin g lemma, we do not require k to be constan t. Lemma 7.3. Conside r monotone submodular functions f 1 , . . . , f k : 2 N → R + , their multiline ar ex tensions F i ( x ) = E [ f i ( ˆ x )] and a down-monotone polyto pe P ⊂ R N + suc h that we can optimize linear functions over P in polyn omial time. T hen given V 1 , . . . , V k ∈ R + we can either • ﬁnd a point x ∈ P suc h that F i ( x ) ≥ (1 − 1 /e ) V i for all i , or • r eturn a certiﬁcat e that ther e is no point x ∈ P suc h that F i ( x ) ≥ V i for all i . Pr oof. W e refer to Sectio n 2.3 of [6] for intuition and notation. Assumin g that there is a so lution S ∈ I achie ving f i ( S ) ≥ V i , Section 2.3 in [6] implies that for any fractiona l solution y ∈ P ( M ) there is a direction v ∗ ( y ) ∈ P ( M ) such that v ∗ ( y ) · ∇ F i ( y ) ≥ V i − F i ( y ) . Moreov er , the way this direction is construc ted is by going tow ards the actual optimum - i.e., this direction is the same for all i . Assuming that such a direction exi sts, we can ﬁ nd it by linear programming. If th e LP is infeasible, we ha ve a cer tiﬁcate that t here is no solution satisfy ing f i ( S ) ≥ V i for all i . Otherwise , we follo w the continuous greedy algorithm and the analysis implies that dF i dt ≥ v ∗ ( y ( t )) · ∇ F ( y ( t )) ≥ V i − F i ( y ( t )) which implies F i ( y (1)) ≥ (1 − 1 /e ) V i . Giv en L emma 7.3, w e sketch the proof of Theorem 7.2 as follo ws. First, we guess a consta nt number of elements so that for each remaining element j , the mar ginal v alue for eac h i is at most ε 3 V i . In the followin g, we just assume that f i ( j ) ≤ ε 3 V i for all i, j . For each objecti ve function f i , we consider the multilinear relaxati on of the probl em: max { F i ( x ) : x ∈ P ( M ) } where F i ( x ) = E [ f i ( ˆ x )] . W e app ly Lemma 7.3 to ﬁnd a fractional solution y ∗ satisfy ing F i ( y ∗ ) ≥ (1 − 1 /e ) V i for a ll i (or a certiﬁcate that there is no soluti on y ∈ P ( M ) such that F i ( y ) ≥ V i for a ll i ; this implies that there is no feasible solution S such that f i ( S ) ≥ V i for all i ). For linear object iv e funct ions, the problem is much simpler: then F i ( x ) are l inear func tions and we can ﬁnd a fra ctional solut ion satisfyin g F i ( y ∗ ) ≥ V i directl y by linear programmin g. W e apply randomize d swap rounding to y ∗ , to obtain a random soluti on R ∈ I satisf ying the lo w er -tail concen tration bound of Theorem 1.4. The marg inal valu es of f i are bounded by ε 3 V i , so by standard scaling we obtain Pr[ f i ( R ) < (1 − δ ) F i ( y ∗ )] < e − δ 2 F i ( y ∗ ) / 8 ε 3 V i ≤ e − δ 2 / 16 ε 3 . Hence, we can set δ = ε and obtain error probability at most e − 1 / 16 ε . B y the union bound, the probability that f i ( R ) < (1 − ε ) F i ( y ∗ ) for any i is at most k e − 1 / 16 ε . For s uf ﬁciently small ε > 0 , this is a consta nt probabili ty smaller than 1 . T hen, f i ( R ) ≥ (1 − 1 /e − ε ) V i for all i . This prov es T heorem 7.2. 15 T o concl ude, w e are able to ﬁnd a (1 − 1 /e − ε ) -ap proximate pareto set for any con stant number of mono- tone submodular funct ions and an y matroid constrain t. This has a natural interpr etation in the setting of the Submodula r W elfare Problem (which is a special case, see [12, 22]). Then each objecti ve function f i ( S ) is the utilit y functi on of a player , and we want to ﬁnd a pareto set with respe ct to all possible allocation s. T o summarize, we can ﬁnd a set of al l allocatio ns that are not dominate d by any other allo cation w ithin a fac tor of 1 − 1 /e − ε per player . Acknowledgments: CC and JV thank Anupam Gupta for asking about the approximab ility of maximizing a monotone submodular set function subject to a m atroid constraint and a constan t numbe r of knapsac k con- straint s; this moti v ated their work . JV thank s Ilias Diakon ikola s for fru itful discuss ions concernin g multiobje c- ti ve optimization that inspired the applica tion in Section 7. RZ is grateful to Michel Goemans for introduc ing him to a v ersion of Shannon ’ s switchi ng game that ins pired t he randomized sw ap rounding alg orithm. W e thank Mohit Singh for pointin g out [4, 16]. Refer ences [1] N. Alon and J. Spencer . The Proba bilistic Method. 2nd Edition, John W iley & Sons. [2] A. Asadpour , M. Goemans, A. Madry , S.O. Gharan, an d A . Saberi. An O (log n / log log n ) -appr oximation algori thm for the assymetric trav eling salesman problem. T o appear in Pr oc. of ACM-SIAM S ODA , 2 010. [3] A. A gee v and M. Sviridenk o. Pipage roun ding: a ne w method of const ructing algorithms with prov en perfor mance guarantee. J. of Combina torial O ptimizati on , 8:307–32 8, 2004. [4] V . Bilo, V . Goyal, R. Rav i and M. Singh. O n the Crossing Spanning Tree Problem. Pr oc. of APP R OX , 51–60 , 2004. [5] G. Calinesc u, C. C hekuri , M. P ´ al and J. V ond r ´ ak. Maximizing a submodul ar set function subject to a matroid cons traint. P r oc. of IPCO , 182–196, 2007. [6] G. Calinesc u, C. C hekuri , M. P ´ al and J. V ond r ´ ak. Maximizing a submodul ar set function subject to a matroid cons traint. T o appear in SIAM J ournal on Computing , special issue for STOC 2008 . [7] P .M. C amerini, G. Galbiati, and F . Maf ﬁoli. Random pseud o-polyno mial alg orithms for exac t matroid proble ms. J . of algorithms 13, 258–273, 1992. [8] W . H. Cunningh am. T esting membership in m atroid polyhe dra. J ournal of C ombinato rial Theory , Series B , 36(2): 161–188 , April 1984. [9] J. E dmonds. Matroids, submodular functio ns and certain polyhedra. Combinatorial Structures and T heir Applicat ions, 69–87, 1970. [10] S. P . F eke te, M. E. L ¨ ubbeck e and H. Meijer . Minimizing the stabbi ng number of matchin gs, trees, and triang ulations. P r oc. of ACM-SIAM S ODA , 43 7–446, 2004. [11] U. Feige. A threshold of ln n for approx imating set cover . Jour nal of the ACM , 45(4 ):634–6 52, 1998. [12] M. L. Fisher , G. L. Nemhauser and L. A . W olsey . An analysis of approx imations for maximizing sub- modular set funct ions - II. Math. Pr og . Study , 8:73–8 7, 1978. [13] H.N. Gabo w and K.S. Manu. Packing algorith ms for arbores cences (and spanni ng trees) in capacit ated graphs . Math ematical Pr ogr amm ing 82(1–2):8 3–109, 1998. 16 [14] R. Gandhi, S. Khulle r , S. Partha sarathy and A. Sriniv asan. Depen dent round ing and its applicat ions to approx imation algorithms. Jou rnal of the ACM 5 3:324-3 60, 2006. [15] A. Gupta, V . Nagarajan and R. Ravi . Personal communicatio n, 2009. [16] S. H ar -Peled. Approxima ting spanning trees with low crossing n umber . T echnical re port, http:// va lis.cs.uiuc.ed u/ sariel/pa pers/09/ crossing/. [17] T . K ir ´ aly , L. C. Lau, and M. Singh. D egre e bound ed matroids and submodular ﬂows. P r oc. of IPCO , 2008. [18] A. Kulik, H. Shachnai and T . T amir . Maximizing submodular set functio ns subje ct to multiple linear constr aints. P r oc. of ACM-SIAM SOD A , 545–554 , 2009. [19] V . S. A. Kumar , M. V . Marathe, S. Parthasar athy and A. S rini vasan. A Uniﬁed Approach to Scheduling on Unrelated Paralle l Machines. J ACM , V ol. 56, 2009. [20] J. L ee, V . Mirrokni, V . N agaraja n and M. Sviriden ko. Maximizin g non-monotone submodular functio ns under matroid and knaps ack constraints . Pr oc. of ACM ST O C 2009 , 323–33 2. [21] J. Lee, M. Sviridenk o, and J. V ondr ´ ak. S ubmodul ar maximization over multip le matroids via generalized exc hange properties. Pr oc. of APPR O X , Sprin ger LNC S, 244 –257, 2009. [22] B. Lehmann, D. J. Lehmann, and N. Nisan. Combinatorial aucti ons with decreasing mar ginal utilities. Games and Economic Behavio r 55:270–2 96, 2006. [23] T . Leighton, C.-J.Lu, S . R ao, and A. Srini v asan. New algor ithmic aspects of the local lemma with appli- cation s to routing and partitionin g. SIAM J . on Computing , V ol. 31, 626–641, 2001. [24] G. L. Nemhauser , L . A. W olsey and M. L. Fisher . An analysis of approx imations for maximizing sub- modular set funct ions - I. Math. Pr og . , 14:265 –294, 1978. [25] G. L. Nemhauser and L. A. W olsey . B est al gorithms for a pproximati ng the maximum of a submodula r set functi on. M ath. Oper . Resear ch , 3(3):17 7–188, 1978. [26] J. Pach and P . K. Agarwal. Combinatorial Geometry W iley-Inte rscience , 1995. [27] C. H. Papadimitrio u and M. Y annaka kis. The comple xity of tradeof fs, and optimal ac cess of web s ources. In Pr oc. of FOC S 200 0 , 86–92. [28] A. Pancones i and A. Sriniv asan. Randomize d distrib uted edge coloring via an extensi on of the Chernof f- Hoef fding bounds. SIAM Jou rnal on C omputing 26:35 0-368, 1997. [29] P . Ragha va n and C. D . Thompson. Randomize d roundi ng: a technique for prov ably good algorit hms and algori thmic proofs. Combinatorica 7(4):365– 374, 1987. [30] A. Schrijv er . Combina torial optimizatio n - polyhedra and efﬁcie ncy . Springer , 2003. [31] A. Schrijv er . Theory of Linear and Inte ger Pr ogr amming . John W iley & Sons, 19 98. [32] M. Singh and L.C. Lau. Approximating minimum bounded degree spanning tress to within one of optimal, Pr oc. of ACM ST O C (2007). [33] A. Srini va san. An extens ion of the L ov ´ asz Local Lemma, and its application s to inte ger programming. SIAM J . on Comput ing , V ol 36 , 609–634, 2006. Preliminary v ersion in Pr oc. of A CM-SIAM SODA , 1996. 17 [34] A. Sriniv asan. Distrib utions on lev el-sets with applic ations to appro ximation algorithms, Pr oc. IEEE Symposiu m on F oundation s of Computer Science (FOCS) , 588–597, 2001. [35] J. V ondr ´ ak. Optimal approximatio n for the submodular welfare problem in the valu e oracle model. Pr oc. of A CM ST OC , 67–74, 2008. [36] J. V ondr ´ ak. Symmetry and approxi mability of submodu lar maximization proble ms. In Pr oc. of IEE E FOCS , 251–27 0, 2009. A Randomized pipage roun ding Let us s ummarize the pip age rou nding tec hnique in the conte xt of matroid polytope s [5, 6 ]. The basic v ersion of the techniqu e assu mes tha t w e start w ith a point in the mat roid base po lytope, a nd we wa nt to round i t to a v erte x of B ( M ) . In each step, we ha ve a fractiona l solution y ∈ B ( M ) and a tight set T (satisf ying y ( T ) = r ( T ) ) contai ning at least tw o fracti onal va riables. W e modify the two frac tional var iables in such a way that their sum remains constant, until some varia ble becomes integra l or a new constraint becomes tight. If a ne w constraint becomes tigh t, we continu e with a ne w tight set, which c an be sho wn to be a proper subset of the pre vious tight set [5, 6]. Hence, after n steps we produce a new inte gral varia ble, and the process terminates after n 2 steps. In the randomized versi on of the techni que, each step is randomized in such a way that the expecta - tion of each varia ble is preserv ed. H ere is the randomize d v ersion of pipage rounding [6]. The subrou tine HitConstraint ( y, i, j ) starts from y and tries to increase y i and decre ase y j at the same rate, as long as the the solution is inside B ( M ) . It returns a new point y and a tight set A , which would be violated if we go any furthe r . This is used in the main algorithm PipageRound ( M , y ) , w hich repeats the process until an integr al soluti on in B ( M ) is found. Subr outine HitConstraint ( y , i , j ): Denote A = { A ⊆ X : i ∈ A, j / ∈ A } ; Find δ = min A ∈A ( r M ( A ) − y ( A )) and a set A ∈ A attaining the abov e minimum; If y j < δ then { δ ← y j , A ← { j }} ; y i ← y i + δ , y j ← y j − δ ; Return ( y , A ) . Algorith m PipageRound ( ( M , y ) ): While ( y is not inte gral) do T ← X ; While ( T contains fractional varia bles) do Pick i, j ∈ T frac tional; ( y + , A + ) ← HitConstraint ( y, i, j ) ; ( y − , A − ) ← HitConstraint ( y, j, i ) ; p ← || y + − y || / || y + − y − || ; W ith probability p , { y ← y − , T ← T ∩ A − } ; Else { y ← y + , T ← T ∩ A + } ; EndWhile EndWhile Output y . Subsequ ently [36], pipag e roun ding wa s extended to the case when the starting po int is in the m atroid polyto pe P ( M ) , rather than B ( M ) . T his is not an issue in [6], but it is necessary for applicati ons with non- monoton e submodular functions [36 ] or with addi tional constraints , such as in this paper . 18 The follo wing procedu re takes care of the case when we start with a frac tional solutio n x ∈ P ( M ) . It adjust s the solution in a randomized way so that the expec tation of each varia ble is preserv ed, and the ne w fractio nal solution is in the base polytope of a (possib ly reduce d) m atroid. Algorith m Ad just ( ( M , x ) ): While ( x is not in B ( M ) ) do If (there is i and δ > 0 such that x + δ e i ∈ P ( M ) ) do Let x max = x i + max { δ : x + δ e i ∈ P ( M ) } ; Let p = x i /x max ; W ith probability p , { x i ← x max } ; Else { x i ← 0 } ; EndIf If (there is i such that x i = 0 ) do Delete i from M and remove the i -coo rdinate from x . EndIf EndWhile Output ( M , x ) . T o summariz e, the complete proce dure works as follo ws. For a gi ven x ∈ P ( M ) , we run ( M ′ , y ) := Adjust ( M , x ) , follo wed b y PipageRound ( ( M ′ , y ) ). The outcome is a base in the restricted matroid w here some elemen ts ha ve been delet ed, i.e. an independe nt set in the original matroid. B Pr oofs and generalizations for randomized swap roundin g In this section we proof that randomized swap rounding satisﬁes the conditions of Lemma 4.1 and generali ze the proced ure to points in the matroid polytop e. B.1 Proof of cond itions for negativ e corr elation Lemma B.1. Randomiz ed swap r oundin g satisﬁ es the condition s of Lemma 4.1. Pr oof. Let X i,t denote the i -th compone nt of X t . T o prov e the ﬁrst condition of Lemma 4.1 we conditio n on a particula r vect or X t at time t of the process and on its con ve x represen tation X t = P k ℓ =1 β ℓ 1 B ℓ . The vector X t +1 is obtain ed fr om X t by an element ary o peration . W ithout loss of genera lity we assume that the elementary operat ion does a swap between the bases B 1 and B 2 in vo lving the elements i ∈ B 1 \ B 2 and j ∈ B 2 \ B 1 . Let B ′ 1 and B ′ 2 be the ba ses after the swap. Hence, with probab ility β 1 / ( β 1 + β 2 ) , B ′ 1 = B 1 and B ′ 2 = B 2 − j + i , and with proba bility β 2 / ( β 1 + β 2 ) , B ′ 1 = B 1 − i + j and B ′ 2 = B 2 . Thus, E [ β 1 1 B ′ 1 + β 2 1 B ′ 2 ] = β 1 β 1 + β 2 ( β 1 1 B 1 + β 2 ( 1 B 2 − e j + e i )) + β 2 β 1 + β 2 ( β 1 ( 1 B 1 − e i + e j ) + β 2 1 B 2 ) = β 1 1 B 1 + β 2 1 B 2 , where e i = 1 { i } and e j = 1 { j } denote the canoni cal basis vect ors corresp onding to element i and j , resp ec- ti vely . Since the vector X t +1 is gi ven by X t +1 = β 1 1 B ′ 1 + β 2 1 B ′ 2 + P k ℓ =3 β ℓ 1 B ℓ , we obtain E [ X t +1 | X t ] = X t . The second condition of L emma 4.1 is satisﬁed since an elementary operation only changes two elements in one base of the con vex representat ion as discussed abo ve. T o check the third condition of the lemma, assume without loss of generality that X t +1 is ob tained fro m X t = P k ℓ =1 β ℓ 1 B ℓ by replacing B 1 by B 1 − i + j . Hence, X i,t +1 = X i,t + β 1 and X j,t +1 = X j,t − β 1 , implyin g that the third conditio n of the lemma is satisﬁed. 19 B.2 Adapting randomized swap r ounding to points in the matroid polytope In this secti on we show ho w randomized swap roundi ng can be genera lized to roun d a point in the matroid polyto pe to an indepen dent set, such that the conditio ns of Lemma 4.1 are still satisﬁed. W e ﬁrst presen t a genera lization where the round ing is done by applying randomized swap rounding for base polyto pes to an ext ension of the underly ing matroid. In a second step we show that this proced ure can easily be interpreted as a procedure on the initial matroid, leading to a simpler descriptio n of the method. An adv antage of presenting the method as a special case of base rounding, is that results presen ted for randomized swap rounding on base polyto pes easily carry ove r to the general rounding procedure . Let x ∈ P ( M ) be the point to round. Similar as for the base polyto pe case, w e need a repres entation of x as a con vex combin ation of independ ent sets. Again, the algor ithm of Cunningha m [8] can be used to obtain a con vex combination of x using at m ost n + 1 independe nt sets with a running time which is bounde d by O ( n 6 ) oracle calls. Thus, we assume that such a con ve x combinatio n of x using n + 1 independe nt sets I 1 , . . . , I n +1 ∈ I is gi ven, i.e., x = P n +1 ℓ =1 β ℓ 1 I ℓ . Let M ′ = ( N ′ , I ′ ) be the follo wing extensi on of the matroid M = ( N , I ) . The set N ′ is obtain ed from N by adding d addit ional dummy elements { s 1 , . . . , s d } , N ′ = N ∪ { s 1 , . . . , s d } . The independ ent sets are deﬁned by I ′ = { I ⊆ N ′ | I ∩ N ∈ I , | I | ≤ d } . Thus, a base of M is also a base of M ′ . The tas k of roun ding x in M can b e trans formed into roundin g a point in the base polytope of M ′ as fo llo ws. Every independe nt set I ℓ that is used in the con ve x represent ation of x , is extended to a base B ′ ℓ of M ′ by adding an arbitrary subset of { s 1 , . . . , s d } of cardinality d − | I ℓ | . Hence, y = P n +1 ℓ =1 β ℓ 1 B ′ ℓ is a point in the base polyto pe of M ′ . Then the randomized swap roundi ng proce dure as presented in Section 3 for points in the base polytope is used to get a point 1 B ′ in B ( M ′ ) . The point 1 B ′ is ﬁnally transfor med into a point x that is a verte x of P ( M ) by projec ting 1 B ′ onto the component s corresp onding to elements in N . The point x is returned by the algorithm. By Lemma B. 1, the rand om point 1 B ′ satisﬁes the conditions of Lemma 4.1. Since the proje ction does not chang e the distrib ution of the compone nts of 1 B ′ , also x satisﬁes the same properties . The dummy elements can be interprete d as elements that do not ha ve any inﬂuence in the ﬁnal outcome, since they will be remov ed by the projectio n. Consider for example an elementary operation on two bases B ′ 1 , B ′ 2 ∈ B w hich are extensio ns of two independen t set I 1 , I 2 ∈ I to the matr oid M ′ , an d let i ∈ B ′ 1 \ B ′ 2 and j ∈ B ′ 2 \ B ′ 1 be the two elements in v olved in the swap. If i is a dummy element, i.e., i ∈ { s 1 , . . . , s d } , then replac ing B ′ 2 by B ′ 2 − j + i corres ponds to removing ele ment j from I 2 . Consider the abo ve algorithm using dummy elements w ith the follo wing modiﬁcation: A t each elementa ry operat ion, if pos sible, two non-d ummy element s are chosen. O ne can e asily ob serve that d escribing thi s v ersion of the a lgorithm without du mmy e lements correspond s to re placing th e M erg eBases pr ocedure with the follo w- ing procedur e to merge two indep endent sets. The procedure, called MergeI ndepS ets , takes two indepe ndent sets I 1 , I 2 ∈ I and two positi ve scalars β 1 , β 2 as input. T o simp lify the descript ion of the procedur e, w e assume | I 1 | ≥ | I 2 | , othe rwise the roles of I 1 and I 2 ha ve to be ex changed in the algorithm. Algorith m MergeIndepSets ( β 1 , I 1 , β 2 , I 2 ) : Find a set S ⊆ I 1 \ I 2 of cardi nality | I 1 | − | I 2 | such that I 2 ∪ S ∈ I ; I ′ 2 = I 2 ∪ S ; While ( I 1 6 = I ′ 2 ) do Pick i ∈ I 1 \ I ′ 2 and ﬁnd j ∈ I ′ 2 \ I 1 such that I 1 − i + j ∈ I and I ′ 2 − j + i ∈ I ; W ith probability β 1 / ( β 1 + β 2 ) , { I ′ 2 ← I ′ 2 − j + i } ; Else { I 1 ← I 1 − i + j } ; EndWhile For ( i ∈ S ) do W ith probability β 2 / ( β 1 + β 2 ) , { I 1 ← I 1 − i } ; EndFor Output I 1 . The existence of a set S as used in the algorithm easily follo ws from the matroid axioms [30]. It can be found 20 by successi vely choosing elements in I 1 \ I 2 that can be added to I 2 still m aintain ing indepe ndence. Once the element i ∈ I 1 \ I ′ 2 is chosen in the while loop of the algo rithm, the ex istence of an element j ∈ I ′ 2 \ I 1 satisfy ing I 1 − i + j ∈ I and I ′ 2 − j + i ∈ I is guarantee d by applyin g Theorem 2.1 to the matroid M ′ = ( N , I ′ ) giv en by I ′ = { I ∈ I | | I | ≤ | I 1 |} . C Cher noff bounds for sub modular functions Here we prov e Theorem 1.3, a Chernof f-type bound f or a monotone submodular function f ( X 1 , . . . , X n ) where X 1 , . . . , X n ∈ { 0 , 1 } are independen t random v ariables . S imilarly to the proof of Chernof f bounds for linear functi ons, the main trick is to prov e a bound on the expone ntial moments E [ e λf ( X 1 ,...,X n ) ] . For that purpose, we write the v alue of f ( X 1 , . . . , X n ) as follo w s: f ( X 1 , . . . , X n ) = P n i =1 Y i , where Y i = f ( X 1 , . . . , X i , 0 , . . . , 0) − f ( X 1 , . . . , X i − 1 , 0 , . . . , 0) . The ne w complicatio n is that the v ariables Y i are not independen t. There could be negati ve and ev en positi ve correla tions b etween Y i , Y j . What is important for us, ho wev er , is that we can show negati ve correla tion between e λ P k − 1 i =1 Y i and e λY k , and by induc tion the followin g bound. Lemma C.1. F or any λ ∈ R , a monotone submodular function and Y 1 , . . . , Y n deﬁne d as above , E [ e λ P n i =1 Y i ] ≤ n Y i =1 E [ e λY i ] . Pr oof. Denote p i = Pr[ X i = 1] . For any k , w e ha ve E [ e λ P k i =1 Y i ] = E [ e λf ( X 1 ,...,X k , 0 ,..., 0) ] = p k E [ e λf ( X 1 ,...,X k − 1 , 1 ,..., 0) ] + (1 − p k ) E [ e λf ( X 1 ,...,X k − 1 , 0 ,..., 0) ] = p k E [ e λf ( X 1 ,...,X k − 1 , 0 ,..., 0) e λF k ( X 1 ,...,X k − 1 , 0 ,..., 0) ] + (1 − p k ) E [ e λf ( X 1 ,...,X k − 1 , 0 ,..., 0) ] where F k ( X 1 , . . . , X k − 1 , 0 , . . . , 0) = f ( X 1 , . . . , X k − 1 , 1 , . . . , 0) − f ( X 1 , . . . , X k − 1 , 0 , . . . , 0) denote s the marg inal valu e of X k being set to 1 , gi ven the precedi ng v ariables. Observe that E [ F k ( X 1 , . . . , X k − 1 , 0 , . . . , 0)] = E [ Y k | X k = 1] . By submodularity , F k is a decreasing function of ( X 1 , . . . , X k − 1 ) . On the other hand, P k − 1 i =1 Y i = f ( X 1 , . . . , X k − 1 , 0 , . . . , 0) is an incr easing functi on of ( X 1 , . . . , X k − 1 ) . W e get th e same mo notonici ty properties for the expo nential fun ctions e λf ( ... ) and e λF k ( ... ) (with a switch in monotonicity for λ < 0 ). By the FKG inequality , e λf ( X 1 ,...,X k − 1 , 0 ,..., 0) and e λF k ( X 1 ,...,X k − 1 , 0 ,..., 0) are nega tiv ely correlated, and w e get E [ e λf ( X 1 ,...,X k − 1 , 0 ,..., 0) e λF k ( X 1 ,...,X k − 1 , 0 ,..., 0) ] ≤ E [ e λf ( X 1 ,...,X k − 1 , 0 ,..., 0) ] E [ e λF k ( X 1 ,...,X k − 1 , 0 ,..., 0) ] = E [ e λ P k − 1 i =1 Y i ] E [ e λY k | X k = 1] . Hence, we ha ve E [ e λ P k i =1 Y i ] ≤ p k E [ e λ P k − 1 i =1 Y i ] E [ e λY k | X k = 1] + (1 − p k ) E [ e λ P k − 1 i =1 Y i ] = E [ e λ P k − 1 i =1 Y i ] · ( p k E [ e λY k | X k = 1] + (1 − p k ) · 1) = E [ e λ P k − 1 i =1 Y i ] · ( p k E [ e λY k | X k = 1] + (1 − p k ) E [ e λY k | X k = 0]) = E [ e λ P k − 1 i =1 Y i ] · E [ e λY k ] . By induct ion, w e obta in the lemma. 21 Giv en this lemma, we can ﬁnish the proof of T heorem 1.3 follo wing the same outline as of proof of the Chernof f bound . Pr oof. Let Y i = f ( X 1 , . . . , X k , 0 , . . . , 0) − f ( X 1 , . . . , X k − 1 , 0 , . . . , 0) as ab ov e. Let us denote E [ Y i ] = ω i and µ = P n i =1 ω i = E [ f ( X 1 , . . . , X n )] . By the con vexity of the expon ential and the fact that Y i ∈ [0 , 1] , E [ e λY i ] ≤ ω i e λ + (1 − ω i ) = 1 + ( e λ − 1) ω i ≤ e ( e λ − 1) ω i . Lemma C.1 then implies E [ e λf ( X 1 ,...,X n ) ] = E [ e λ P n i =1 Y i ] ≤ n Y i =1 E [ e λY i ] ≤ e ( e λ − 1) µ . For the u pper -tail bound, w e use Mark ov’ s inequality as follo ws: Pr[ f ( X 1 , . . . , X n ) ≥ (1 + δ ) µ ] = Pr[ e λf ( X 1 ,...,X n ) ≥ e λ (1+ δ ) µ ] ≤ E [ e λf ( X 1 ,...,X n ) ] e λ (1+ δ ) µ ≤ e ( e λ − 1) µ e λ (1+ δ ) µ . W e choose e λ = 1 + δ w hich yields Pr[ f ( X 1 , . . . , X n ) ≥ (1 + δ ) µ ] ≤ e δµ (1 + δ ) (1+ δ ) µ . For the l ower -tail bound, we use Markov ’ s inequality with λ < 0 as follo ws: Pr[ f ( X 1 , . . . , X n ) ≤ (1 − δ ) µ ] = Pr[ e λf ( X 1 ,...,X n ) ≥ e λ (1 − δ ) µ ] ≤ E [ e λf ( X 1 ,...,X n ) ] e λ (1 − δ ) µ ≤ e ( e λ − 1) µ e λ (1 − δ ) µ . W e choose e λ = 1 − δ w hich yields Pr[ f ( X 1 , . . . , X n ) ≤ (1 − δ ) µ ] ≤ e − δµ (1 − δ ) (1 − δ ) µ ≤ e − µδ 2 / 2 using (1 − δ ) 1 − δ ≥ e − δ + δ 2 / 2 for δ ∈ (0 , 1] . D Lower -tail estimate f or submodular functions under depend ent round ing In this secti on, w e prov e Theorem 1.4, i.e. an e xponenti al estimate for the lower tail of the distrib ution of a monoton e submodular function under randomize d swa p round ing. W e note that the bound on the exp ected v alue of the rounded solution , E [ f ( R )] ≥ µ 0 , follo ws by the con vexit y of F ( x ) along direction s e i − e j just like in [6 ]; we omit the detai ls. T he e xponen tial tail bound is much more in volv ed. W e sta rt by setting up some notati on. Notation. The round ing procedure starts from a con vex linear combinati on of bases, x 0 = n X i =1 β i 1 B i . The rounding proc eeds in stages, where in t he ﬁrst stage we mer ge the bases B 1 , B 2 (rando mly) into a ne w base C 2 , and replace β 1 1 B 1 + β 2 1 B 2 in the linear combinati on by γ 2 1 C 2 , with γ 2 = β 1 + β 2 . More generall y , in the k -th stage, we merg e C k and B k +1 into a ne w base C k +1 (we set C 1 = B 1 in the ﬁrst stage), and replace γ k 1 C k + β k +1 1 B k +1 in the linear comb ination by γ k +1 1 C k +1 . Inducti vely , γ k +1 = γ k + β k +1 = P k +1 i =1 β i . After n − 1 stages, we obtain a linear combination γ n 1 C n and γ n = P n i =1 β i = 1 ; i.e., this is an integer solutio n. W e use the follo wing notatio n to describe the vect ors produced in the process: 22 • b i = β i 1 B i • c i = γ i 1 C i • y k = P n i = k b i = P n i = k β i 1 B i • x k = c k +1 + y k +2 = γ k +1 1 C k +1 + P n i = k +2 β i 1 B i In other words, b i are the initial vec tors in the linear combination, which get gradually replaced by c i , and x k is the fractio nal solution after k stages. W e emphasiz e that x k denote s the entire fractional solution at a certain stage and not the value of its k -th coordi nate. The coordin ates of the fractio nal solution are the variab les X i . If we want to refer to the val ue of X i after k stages, w e use the nota tion X i,k . W e work with the multilinear extensi on of a submodular function , F ( x ) = E [ f ( ˆ x )] . In the followin g, we use the follo w ing shorthand notation and basic properti es: • F i ( x ) denotes the partial deri vati ve ∂ F ∂ X i e va luated at x . The interpre tation of F i ( x ) is the mar ginal va lue of i with respect to the fractional solution x . • W e use e i = 1 { i } to deno te the canonical basis vector corres ponding to element i . • If only one variab le is changi ng while others are ﬁ xed , F ( x ) is a linear function. There fore, we can use the follo wing formula: F ( x + t e i ) = F ( x ) + tF i ( x ) . • Due to submodu larity , ∂ 2 F ∂ X i ∂ X j ≤ 0 for any i, j . This implies that F i ( x ) = ∂ F ∂ X i is non-in creasing as a functi on of each coordinat e of x . If y dominates x in all coord inates ( x ≤ y ), w e ha ve F i ( x ) ≥ F i ( y ) . Pro of over view . The random process in terms of the ev olution of F ( x ) is a submartin gale, i.e. the valu e in each step can only increa se in expec tation. This is a good sign; ho w e ver , a straightfo rward applicat ion of concen tration bounds for martingales yie lds a dependenc y of the number of variab les n whic h woul d rende r the bound meaningle ss. More reﬁned bounds for martingales rely on bounds on the va riance in success iv e steps. Unfortun ately , these are also dif ﬁcult to use since we do n ot hav e a good a prio ri bound on the v ariance in eac h step. The va riance can depend on preceding steps and taking worst-cas e bounds leads to the same dependenc y on n as mentioned abov e. In or der to pro ve a bo und which dep ends only on t he parameter s δ and µ 0 , we sta rt from scra tch and follo w the standa rd recipe : estimat e the exponen tial moment E [ e λ ( µ 0 − f ( R )) ] , where µ 0 is the initia l value and R is the round ed soluti on. W e deco mpose the expressi on e λ ( µ 0 − f ( R )) into a telesc oping product: e λ ( µ 0 − f ( R )) = e λ ( F ( x 0 ) − F ( x n − 1 )) = e λ ( F ( x 0 ) − F ( x 1 )) · e λ ( F ( x 1 ) − F ( x 2 )) · . . . · e λ ( F ( x n − 2 ) − F ( x n − 1 )) . The f actors in this product are not independen t, b ut we can pro ve bo unds on the co nditiona l expect ations E [ e λ ( F ( x k − 1 ) − F ( x k )) | x 0 , . . . , x k − 1 ] , in other words condition ed on a giv en history of the roundin g process. These bou nds depend on the h istory , but we are a ble to char ge the arising fac tors to the v alue of µ 0 = F ( x 0 ) in such a way that the ﬁnal bou nd depends only on µ 0 . W e start from the bot tom, by a nalyzing the basic rou nding step for tw o v ariables. The fo llo wing eleme ntary inequa lity will be helpful. Lemma D.1. F or any p ∈ [0 , 1] and ξ ∈ [ − 1 , 1] , pe ξ (1 − p ) + (1 − p ) e − ξ p ≤ e ξ 2 p (1 − p ) . 23 Pr oof. If ξ < 0 , w e can replace ξ by − ξ and p by 1 − p ; the statement of the lemma remains the same. So w e can assume ξ ∈ [0 , 1] . Fix any p ∈ [0 , 1] and deﬁne φ p ( ξ ) = e ξ 2 p (1 − p ) − pe ξ (1 − p ) − (1 − p ) e − ξ p . It is easy to see that φ p (0) = 0 . Our goal is to pro ve that φ p ( ξ ) ≥ 0 for ξ ∈ [0 , 1] . Let us compute the deriv ati ve of φ p ( ξ ) with respect to ξ : φ ′ p ( ξ ) = 2 ξ p (1 − p ) e ξ 2 p (1 − p ) − p (1 − p ) e ξ (1 − p ) + p (1 − p ) e − ξ p = p (1 − p ) e − ξ p  2 ξ e ξ 2 p (1 − p )+ ξ p − e ξ + 1  ≥ p (1 − p ) e − ξ p  2 ξ − e ξ + 1  . For ξ ∈ [0 , 1] , we ha ve e ξ ≤ 1 + 2 ξ and henc e φ ′ p ( ξ ) ≥ 0 . This m eans that φ p ( ξ ) is non-decre asing and φ p ( ξ ) ≥ 0 for ξ ∈ [0 , 1] . Note that the lemma does not hold for arbitrarily larg e ξ , e.g. when p = 1 /ξ 2 and ξ → ∞ . Next, we apply this lemma to the basic step of the roundin g proced ure. Lemma D.2. Let F ( x ) be the m ultiline ar ex tension of a m onoton e submodular function w ith marg inal values in [0 , 1] , and let λ ∈ [0 , 1] . Consider one elementary opera tion of randomiz ed swap r oundin g, wher e two variab les X i , X j ar e modiﬁed . Let x denote the fraction al solution befor e, x ′ after this step, and let H denot e the complete his tory pr ior to this r ounding step . Assume th at the va lues of the two variables be for e the r oundi ng step ar e X i = γ , X j = β . T hen E [ e λ ( F ( x ) − F ( x ′ )) | H ] ≤ e λ 2 β γ ( F j ( x ) − F i ( x )) 2 wher e F i ( x ) = ∂ F ∂ X i ( x ) and F j ( x ) = ∂ F ∂ X j ( x ) . Pr oof. Fix the history H ; this includes the point x before the rounding step. W ith probabil ity p = γ β + γ , the round ing step is X ′ i = X i + β and X ′ j = X j − β . I.e., x ′ = x + β e i − β e j . S ince F ( x ) is linear when only one coordi nate is m odiﬁed, we get F ( x ′ ) = F ( x ) + β F i ( x ) − β F j ( x + β e i ) . By submodu larity , F j ( x + β e i ) ≤ F j ( x ) and hence F ( x ′ ) = F ( x ) + β F i ( x ) − β F j ( x + β e i ) ≥ F ( x ) + β ( F i ( x ) − F j ( x )) . W ith probability 1 − p , we set X ′ i = X i − γ and X ′ j = X j + γ . By similar reasoning , in this case we get F ( x ′ ) = F ( x ) − γ F i ( x ) + γ F j ( x − γ e i ) ≥ F ( x ) − γ ( F i ( x ) − F j ( x )) . T aking exp ectation over the two c ases, we get E [ e λ ( F ( x ) − F ( x ′ )) | H ] ≤ pe λβ ( F j ( x ) − F i ( x )) + (1 − p ) e − λγ ( F j ( x ) − F i ( x )) = pe λ (1 − p )( β + γ )( F j ( x ) − F i ( x )) + (1 − p ) e − λp ( β + γ )( F j ( x ) − F i ( x )) . W e in vok e Lemma D .1 with ξ = λ ( β + γ )( F j ( x ) − F i ( x )) (we ha ve | ξ | ≤ 1 due to λ, β + γ , F i ( x ) , F j ( x ) all being in [0 , 1] ). W e get E [ e λ ( F ( x ) − F ( x ′ )) | H ] ≤ e ξ 2 p (1 − p ) = e λ 2 β γ ( F j ( x ) − F i ( x )) 2 . 24 Note that the expone nt on the right-hand side of Lemma D.2 correspo nds to the varian ce in one step of the round ing procedu re. The next lemma estimates these contrib utions , aggregat ed ov er one stage of the rounding proces s, i.e., the mergi ng of the bases C k and B k +1 . The expon ent on the right-ha nd side of Lemm a D. 3 corres ponds to the va riance of t he random process a ccumulated over the k -th st age. It is crucial that we compare this quant ity to certain valu es which can be ev entual ly char ged to µ 0 . Lemma D.3. Let F ( x ) be the m ultiline ar ex tension of a m onoton e submodular function w ith marg inal values in [0 , 1] , and let λ ∈ [0 , 1] . Consider the k -th stage of the r ounding pr ocess, when bases C k and B k +1 (with coef ﬁcients γ k and β k +1 ) ar e mer ged int o C k +1 . The fr actiona l solution befor e this stag e is x k − 1 and aft er this sta ge x k . Conditio ned on any history H of the r ounding pr ocess thr oughout the ﬁrst k − 1 sta ges, E [ e λ ( F ( x k − 1 ) − F ( x k )) | H ] ≤ e λ 2 ( β k +1 F ( c k )+ γ k ( F ( y k +1 ) − F ( y k +2 ))) . Pr oof. The k -th stage merges bases C k and B k +1 into C k +1 by taking elements in pairs and performing round ing steps as in Lemma D.2. Let us denot e the pairs of elements consid ered by the rounding proced ure ( c 1 , b 1 ) , . . . , ( c d , b d ) , w here C k = { c 1 , . . . , c d } and B k +1 = { b 1 , . . . , b d } . T he matchi ng is not determined before hand: ( c 2 , b 2 ) might depend on the random choic e between c 1 , b 1 , etc. In the follo wing, w e drop the inde x k and denote by x i the fractiona l solutio n obtained after processing ( c 1 , b 1 ) , . . . , ( c i , b i ) . W e start w ith x 0 = x k − 1 and after process ing all d pairs, we get x d = x k . W e also replace β k +1 , γ k simply by β , γ . W e denote by H i the complete history prior to the rounding step in volv ing ( c i +1 , b i +1 ) ; in partic ular , this includes the fractio nal solution x i . Using Lemma D.2 for the roundin g step in v olving ( c i +1 , b i +1 ) , we get E [ e λ ( F ( x i ) − F ( x i +1 )) | H i ] ≤ e λ 2 γ β ( F c i +1 ( x i ) − F b i +1 ( x i )) 2 ≤ e λ 2 γ β ( F c i +1 ( x i )+ F b i +1 ( x i )) , using the fact that the parti al deriv ati ves F j ( x i ) are in [0 , 1] . Further , we modify the expo nent of the right-ha nd side as follo ws. The vecto r x i is obtain ed after pro- cessin g i pairs and still contai ns the coordinat es c i +1 , . . . , c d of c k = γ 1 C k untou ched: in othe r words, x i ≥ γ 1 { c i +1 ,...,c d } . Let us deﬁne • c i = γ 1 { c i +1 ,...,c d } . I.e., x i ≥ c i ≥ c i +1 . By submodu larity , we ha ve F c i +1 ( x i ) ≤ F c i +1 ( c i +1 ) . Similarly , the vecto r x i also contain s the coordinates b i +1 , . . . , b d of b k +1 and all of y k +2 = P n j = k +2 b j uncha nged: x i ≥ β 1 { b i +1 ,...,b d } + y k +2 . Let us deﬁne • y i = β 1 { b i +1 ,...,b d } + y k +2 . I.e., x i ≥ y i ≥ y i +1 . By submodu larity , we get F b i +1 ( x i ) ≤ F b i +1 ( y i +1 ) . T herefor e, we can w rite E [ e λ ( F ( x i ) − F ( x i +1 )) | H i ] ≤ e λ 2 γ β ( F c i +1 ( c i +1 )+ F b i +1 ( y i +1 )) . (5) W e claim that by induct ion on d − i , this implies E [ e λ ( F ( x i ) − F ( x d )) | H i ] ≤ e λ 2 ( β F ( c i )+ γ ( F ( y i ) − F ( y d ))) (6) for all i = 0 , . . . , d . For i = d , the claim is triv ial. For i < d , we can write E [ e λ ( F ( x i ) − F ( x d )) | H i ] = E h e λ ( F ( x i ) − F ( x i +1 )) E [ e λ ( F ( x i +1 ) − F ( x d )) | H i +1 ]   H i i and using the inducti ve hypothesis (6) for i + 1 , E [ e λ ( F ( x i ) − F ( x d )) | H i ] ≤ E h e λ ( F ( x i ) − F ( x i +1 )) · e λ 2 ( β F ( c i +1 )+ γ ( F ( y i +1 ) − F ( y d ))) | H i i = e λ 2 ( β F ( c i +1 )+ γ ( F ( y i +1 ) − F ( y d ))) · E h e λ ( F ( x i ) − F ( x i +1 )) | H i i 25 where we used the f act that t he inducti ve bo und is determin ed b y H i , and s o we can tak e it out of the expect ation (it depends only on the sets { c i +2 , . . . , c d } and { b i +2 , . . . , b d } which are determine d ev en before performing the roundi ng step on ( c i +1 , b i +1 ) ). T aking logs and using (5) to estimate the last expec tation, we obtain log E [ e λ ( F ( x i ) − F ( x d )) | H i ] ≤ λ 2  β F ( c i +1 ) + γ  F ( y i +1 ) − F ( y d )   + λ 2 γ β  F c i +1 ( c i +1 ) + F b i +1 ( y i +1 )  = λ 2  β  F ( c i +1 ) + γ F c i +1 ( c i +1 )  + γ  F ( y i +1 ) + β F b i +1 ( y i +1 ) − F ( y d )   = λ 2  β F ( c i ) + γ  F ( y i ) − F ( y d )   where we used F ( c i +1 ) + γ F c i +1 ( c i +1 )) = F ( c i ) and F ( y i +1 ) + β F b i +1 ( y i +1 ) = F ( y i ) (see the deﬁnitions of c i , y i abo ve). This prov es our inducti ve claim (6). For i = 0 , since x 0 = x k − 1 , x d = x k , c 0 = c k , y 0 = y k +1 and y d = y k +2 , this gi ves the statemen t of the lemma. No w we can proceed ﬁnally to the proof of Theorem 1.4. Pr oof. W e prov e inducti vely the follo wing statement: For an y k and any λ ∈ [0 , 1] , E [ e λ ( µ 0 − F ( x k )) ] ≤ e λ 2 ( µ 0 (1+ P k i =1 β i +1 ) − F ( y k +2 )) . (7) W e remind the reader that µ 0 = F ( x 0 ) , x k is the fractiona l solutio n after k stages, and y k +2 = P n i = k +2 b i . W e proceed by inducti on on k . For k = 0 , the claim is triv ial, since F ( y 2 ) ≤ F ( x 0 ) = µ 0 by monotonic ity . For k ≥ 1 , we unroll the exp ectation as follows: E [ e λ ( µ 0 − F ( x k )) ] = E h e λ ( µ 0 − F ( x k − 1 )) E [ e λ ( F ( x k − 1 ) − F ( x k )) | H ] i where H is the complete histo ry prior to stag e k (up to x k − 1 ). W e estimate th e inside e xpectati on using Lemma D.3: E [ e λ ( F ( x k − 1 ) − F ( x k )) | H ] ≤ e λ 2 ( β k +1 F ( c k )+ γ k ( F ( y k +1 ) − F ( y k +2 ))) ≤ e λ 2 ( β k +1 F ( x k − 1 )+ F ( y k +1 ) − F ( y k +2 )) using monotoni city , c k ≤ x k − 1 , y k +2 ≤ y k +1 and γ k ≤ 1 . Therefore, E [ e λ ( µ 0 − F ( x k )) ] ≤ E h e λ ( µ 0 − F ( x k − 1 )) e λ 2 ( β k +1 F ( x k − 1 )+ F ( y k +1 ) − F ( y k +2 )) i = e λ 2 ( β k +1 µ 0 + F ( y k +1 ) − F ( y k +2 )) E h e ( λ − λ 2 β k +1 )( µ 0 − F ( x k − 1 )) i . By the inducti ve hypothesi s (7) with λ ′ = λ − λ 2 β k +1 ∈ [0 , 1] , E [ e ( λ − λ 2 β k +1 )( µ 0 − F ( x k − 1 )) ] ≤ e ( λ − λ 2 β k +1 ) 2 ( µ 0 (1+ P k − 1 i =1 β i +1 ) − F ( y k +1 )) ≤ e λ 2 ( µ 0 (1+ P k − 1 i =1 β i +1 ) − F ( y k +1 )) . In the last inequal ity we used F ( y k +1 ) ≤ µ 0 , which holds by monotonicity . Plugging this into the preced ing equati on, E [ e λ ( µ 0 − F ( x k )) ] ≤ e λ 2 ( β k +1 µ 0 + F ( y k +1 ) − F ( y k +2 )) e λ 2 ( µ 0 (1+ P k − 1 i =1 β i +1 ) − F ( y k +1 )) = e λ 2 ( µ 0 (1+ P k i =1 β i +1 ) − F ( y k +2 )) which prov es (7). Finally , for k = n − 1 we obtain F ( x n − 1 ) = f ( R ) wher e R is the rounded solution, y n +1 = 0 , and E [ e λ ( µ 0 − f ( R )) ] ≤ e λ 2 µ 0 (1+ P n − 1 i =1 β i +1 ) ≤ e 2 λ 2 µ 0 (8) 26 becaus e P n − 1 i =1 β i +1 ≤ 1 . The ﬁnal step is to apply Markov ’ s inequalit y to the expo nential m oment. From Marko v’ s inequality and E quatio n (8), w e get Pr[ f ( R ) ≤ (1 − δ ) µ 0 ] = Pr h e λ ( µ 0 − f ( R )) ≥ e λδµ 0 i ≤ E [ e λ ( µ 0 − f ( R )) ] e λδµ 0 ≤ e 2 λ 2 µ 0 − λδµ 0 . A choic e of λ = δ / 4 giv es the statement of the theorem. 27

Dependent Randomized Rounding for Matroid Polytopes and Applications

Original Paper

Comments & Academic Discussion

Leave a Comment

Original Paper

Related Papers

Comments & Academic Discussion

Leave a Comment