Early Experiences in Traffic Engineering Exploiting Path Diversity: A Practical Approach

apport   de recherche ISSN 0249-6399 ISRN INRIA/RR--6474--FR+ENG Thème COM INSTITUT N A TION AL DE RECHERCHE EN INFORMA TIQUE ET EN A UTOMA TIQUE Early Experiences in T raf fic Engineering Exploiting P ath Dive rsity: A Practical Approac h Luca Muscariello — Diego P erino N° 6474 February 2008 Unité de recherche INRIA Rocquenco urt Domaine de V oluceau, Rocquen court, BP 105 , 78153 Le Chesnay Cedex (France) Téléphone : +33 1 39 63 55 11 — Télécopie : +33 1 39 63 53 30 Early Exp erienes in T ra Engineering Exploiting P ath Div ersit y: A Pratial Approa h Lua Musariello ∗ , Diego P erino † Thème COM  Systèmes omm unian ts Pro jets Gang Rapp ort de re her he n ° 6474  F ebruary 2008  21 pages Abstrat: Reen t literature has pro v ed that stable dynami routing algorithms ha v e solid the- oretial foundation that mak es them suitable to b e implemen ted in a real proto ol, and used in pratie in man y dieren t op erational net w ork on texts. Su h algorithms inherit m u h of the prop erties of ongestion on trollers implemen ting one of the p ossible om bination of A QM/ECN s hemes at no des and o w on trol at soures. In this pap er w e prop ose a linear program form ulation of the m ulti-ommo dit y o w problem with ongestion on trol, under max-min fairness, omprising demands with or without exogenous p eak rates. Our ev aluations of the gain, using path div ersit y , in senarios as in tra-domain tra engineering and wireless mesh net w orks enourages real implemen tations, esp eially in presene of hot sp ots demands and non uniform tra matries. W e prop ose a o w a w are p ersp etiv e of the sub jet b y using a natural m ulti-path extension to urren t ongestion on trollers and sho w its p erformane with resp et to urren t prop osals. Sine o w a w are ar hitetures exploiting path div ersit y are feasible, salable, robust and nearly optimal in presene of o ws with exogenous p eak rates, w e laim that our solution rethink ed in the on text of realisti tra assumptions p erforms as b etter as an optimal approa h with all the additional b enets of the o w a w are paradigm. Key-w ords: Multi-path Routing, Congestion on trol, T ra Engineering, Flo w-A w are Ar hi- tetures ∗ lua.m usarielloorange-ftgroup.om † diego.p erinoorange-ftgroup.om Ingènierie du T ra par Flot Une Appro  he Multi- hemins Pratique Résumé : Les algorithmes de routage dynamique p ossèden t de solides fondemen ts théoriques qui les renden t aptes á une implémen tation rèelle dans dièren ts rèseaux op èrationnels. Ces algorithmes p ossèden t de nom breuses propriètès propres aux on trles de ongestion grâe á l'utilisation de méanismes de signalisation expliite et de on trle des ots á la soure. Dans et artile, nous prop osons une form ulation linèaire du problème de m ultiot a v e on trle de ongestion et ritère d'èquitè de t yp e "max-min". Les p erformanes obten ues par l'exploitation de  hemins m ultiples son t enouragean ts, que ela soit en routage in tra-domaine ou dans les rèseaux mesh sans l, et initen t á une implèmen tation rèelle, en partiulier dans le as de matries de tra non-uniformes et de "p oin ts  hauds" de demandes. Nous prop osons une appro  he par ots qui in tègre naturellemen t les m ulti- hemins aux mèa- nismes atuels de on trle de ongestion, et nous l'èv aluons par rapp ort aux solutions atuelles. Les ar hitetures par ot son t rèalisables, robustes, apables de passer á l'è helle, et quasi-optimales lorsque les ots on t un dèbit-rête expliite. Dans un on texte rèaliste, notre solution p ossède don les propriètès d'une solution optimale ainsi que les a v an tages de l'appro  he par ots. Mots-lés : Multi- hemins, Con trle de ongestion, Ingènierie du tra, Ar hiteture par ots Flow-A war e T r a Engine ering 3 1 In tro dution T ra engineering is usually p ereiv ed as an o-line funtionalit y in order to impro v e p erformane b y b etter mat hing net w ork resoures to tra demands. Optimisation is p erformed o-line as demands are a v erages o v er the long term that do not onsider tra utuation o v er smaller time sales. In tra-domain routing optimisation, b y means of OSPF ost parametrisation [8 , 9 ℄, is t ypial example of the aforemen tioned problem. The max degree of resp onsiv eness is guaran teed at the long term, in a daily or w eekly basis for instane. The ob jetiv e of ISPs is to get rid of a giv en tra matrix at the minim um ost, whi h is estimated as global exp enditures for link upgrades. Therefore minimising the maxim um link load is a natural ob jetiv e. Su h an engineered net w ork is not robust to an y ner grained tra utuation, as large n um b er of failures, tra ash ro wds, BGP re-routes, appliation re-routing. In urren t ba kb ones these eets are mitigated b y o v er- pro visioned links. In other on texts, of inreasing imp ortane no w ada ys as wireless mesh net w orks, sareness of resoures push net w ork engineers to aept a higher degree of resp onsiv eness in order to exploit an y single piee of un used apait y . W e only onsider wireless mesh resulting from a radio engineered net w ork, where links guaran tee a minim um a v ailabilit y . Literature on optimal routing in net w orks starts from seminal w orks as [ 3, 4℄ or [6 , 10 ℄. This omprises en tralised and deen tralised strategies prop osed in the v ery b eginning of the ARP Anet pro jet. Historially the main outome of optimal routing has b een shortest-path or, at most, minim um-ost routing. Early exp erienes on dynami routing [37 ℄ ha v e k ept it ba k in spite of its p oten tial, as sensitivit y to ongestion has long b een refused for b eing omplex, unstable, not prone to b e easily deplo y ed. There is a quite large reen t literature on m ulti-path routing, raised from the need to dev elop dynami y et stable algorithms, sensitiv e to ongestion for man y appliations that ould eetiv ely exploit un used resoures within the net w ork. In the In ternet, w e b eliev e that v ery sp ei appliations an tolerate su h dynami en viron- men t. A daptiv e video streaming is, probably , the main appliation that w ould denitely b e able to tolerate v ariabilit y and, at the same time, tak e adv an tage of un used apait y . It is w orth re- alling that video appliations are going to b e the main part of In ternet tra v ery so on. Video streams usually last v ery long, and will probably last longer as b etter appliations and on ten ts will b e a v ailable. F urthermore it is lik ely that su h on ten ts will b e transp orted b y a n um b er of non-sp eied proto ols, sub jet to non b etter sp eied fairness riteria. Con v ersational appliations should b e k ept a w a y from b eing routed o v er m ultiple routes but also other data servies that are made of o ws that last p oten tially v ery short. Also, m u h of the presen t data tra if it inludes mail, w eb, instan t messaging. Ho w ev er, P2P le sharing appliations or on ten t deliv ery net w orks (CDN) are prone to w ell exploit path div ersit y as they in trinsially are robust to rate utuations. Other data appliations are do wnloads of soft w are up dates ev en though they an b e inluded in the lass of P2P le sharing systems. In the last few y ears, in tense resear h on m ulti-path routing has progressed. Theory of optimi- sation has b een applied to dev elop distributed algorithms solving a global optimisation problem, see [11 , 12 , 13 , 14 , 21 , 31 , 35 , 38 ℄. Optimisation expliits the problem of resoure allo ation under a  hosen fairness riteria. Con trol theory has b een used to obtain dela y stabilit y of dis- tributed optimal s hemes, see [11 , 16 , 21 , 35 ℄. Other resear h onsiders dynami o w lev el mo dels [22 , 23 , 24 , 30 ℄ in order to tak e in to aoun t arriv als and departures of user's sessions. In this pap er w e supp ort the deplo ymen t of a o w a w are ar hiteture exploiting path div ersit y for a sp ei lass of appliations, rate adaptiv e video streaming for instane or CDN and P2P le sharing. In su h set up w e sho w that o w a w are paradigm is nearly optimal for an y tipial tra demand requiring the use of m ultiple paths without the need to assume an y kind of om- mon transp ort proto ol among users, an y ommon fairness seman ti and an y kind of o op eration b et w een users, and net w ork no des as w ell. In Setion 2 w e explain our denition of net w ork o ws while in Setion 3 the mo delling frame- w ork for optimal routing and ongestion on trol is in tro dued. The setion inludes a set of examples on to y net w orks. RR n ° 6474 4 Lu a Mus ariel lo and Die go Perino Finally w e in tro due our main outome in term of p erformane ev aluation of the optimal solution of large problems, with an original linear program form ulation under max-min fairness that is used to ev aluate large problems in Setion 4 . Setion 5 in tro dues our main original outome as a new m ulti-path ongestion on troller alled MIR TO. The algorithm is b orn inferring an optimal strategy from previous setions. Moreo v er this algorithm is ev aluated within the framew ork of a o w a w are ar hiteture, bringing new argumen ts in fa v our of su h net w ork paradigm. 2 T ra Charateristis IP tra on a net w ork link an b e onsidered as a sup erp osition of indep enden t sessions, ea h session relating to some piee of user ativit y and b eing manifested b y the transmission of a olletion of o ws. Sessions and o ws are dened lo ally at a onsidered net w ork in terfae. Flo ws an generally b e iden tied b y ommon v alues in pa k et header elds (e.g., the 5-tuple of IP addresses, p ort n um b ers and transp ort proto ol) and the fat that the in terv al b et w een su h pa k ets is less than some time out v alue (20s, sa y). It is not usually p ossible to iden tify sessions just from data in pa k ets and this notion annot therefore b e used for resoure allo ation. Nev ertheless w e are more inlined to think ab out user sessions than proto ol dened o ws. A more signian t o w  harateristi is the exogenous p eak rate at whi h a o w an b e emitted. This is the highest rate the o w w ould attain if the link w ere of unlimited apait y . This limit ma y b e due to the user aess apait y , the maxim um TCP reeiv e windo w, or the urren t a v ailable bandwidth on other links of the path, or the stream rate in ase of video appliations for instane. In the rest of the pap er w e will use in ter hangeably the terms demand and o w. 3 Mo delling framew ork The net w ork top ology is mo delled b y a onneted graph G = ( N , L ) giv en as a set of no des and links. Let A = [ a ij ] the adjaeny matrix, a ij = 1 if there exists a diretional link b et w een i and j and a ij = 0 otherwise. The net w ork arries tra generated b y a set of demands Γ , ea h demand d is giv en with a triple ( s d , e d , p d ) , with s ∈ S , e ∈ E , soure and destination no des with S , E ⊆ N , and p ∈ R + exogenous p eak rate. In our mo del a net w ork o w d gets a share x d ij of the apait y c ij at ea h link 0 ≤ x d ij ≤ min( c ij , p ) . x d ij ( t ) is a uid appro ximation of the rate at whi h the soure d is sending at time t through link ij . The net w ork o w an b e slit among dieren t paths that are made a v ailable b y a net w ork proto ol at an ingress no de. W e mak e no mo delling assumption whether paths are disjoin t, ho w ev er the abilit y to reate more path div ersit y helps design highly robust net w ork routing proto ols. In paragraph 3.3 w e mo del route seletion and bandwidth sharing as an optimisation prob- lem that maximises user satisfation and minimise net w ork ongestion under a sp eied fairness riteria. 3.1 Minim um ost routing The abilit y to reate the set of optimal paths at the ingress of the net w ork and mak e them a v ailable to the routing proto ol requires a ertain kno wledge of the net w ork status, as link load, path dela y and length. Ho w ev er, as this an b e done in pratie b y disseminating lo al measures, the proto ol m ust b e also robust to state inauray . Assuming p erfet kno wledge of net w ork state, optimal routing an b e form ulated through the follo wing non linear optimisation problem INRIA Flow-A war e T r a Engine ering 5 Sym b ol Meaning N no de set L link set Γ demand set d demand n um b er S soure set E destination set s d demand d soure no de e d demand d destination no de p d demand d exogenous p eak rate P d path set of demand d k path n um b er L d k link set of demand d o v er its k th path C ij apait y of link ( i, j ) x d ij rate of demand d o v er link ( i, j ) x d k rate of demand d o v er its k th path x d rate of demand d ( P k x d k ) ρ ij load on link ( i, j ) ( P d ∈ Γ x d ij C ij ) T able 1: Summary of notation used with linear onstrained. minimise X i,j ∈ N C P d ∈ Γ x d ij c ij ! sub jet to X k ∈ N a ik x d ki − X j ∈ N a j i x d ij =    p d i if i ∈ S − p d i if i ∈ E 0 otherwise ∀ d ∈ Γ (1) onstrain t (4) mo dels zero net o w for rela y no des, p ositiv e for soure no des and negativ e for destination no des. This allo ws to obtain optimal routes diretly from the optimisation problem. C an b e though t mo delling the link dela y often used in tra engineering form ulations of the m ulti-ommo dit y o w problem, C ( x ij ) = x ij c ij − x ij (2) with this form ula the ost funtion b eomes the a v erage dela y in a M/M/1 queue as a on- sequene of the Kleinro  k indep endene appro ximation and Ja kson's Theorem. This problem form ulation dates ba k to [6, 10 ℄ in the on text of minim um dela y routing. Using standard te h- niques in on v ex onstrained optimisation (on v ex optimisation o v er a simplex ) in [3, 4℄, it is sho wn that the optimal solution alw a ys selet paths with minim um (and equal) rs ost deriv ativ es for an y stritly on v ex ost funtion. Therefore the problem an b e re-form ulated as a shortest path problem where path lengths are the rst deriv ativ es of the ost funtion along the path, that an b e written with abuse of notation, C ′ ( x d p ) = C ′ P d ∈ Γ x d ij c ij ! (3) where x d p is the p ortion of o w of demand d o wing through path p . This is what, in [ 4℄, Bertsek as and Gallager all rst deriv ativ e path lengths. Therefore, at optim um all paths ha v e equal lengths. This fat will b e use in the follo wing setion rep eatedly . RR n ° 6474 6 Lu a Mus ariel lo and Die go Perino Another plausible ob jetiv e is to minimise the most loaded link, frequen t in tra engineering net w ork op erator's ba kb one optimization in onjuntion with link apait y o v er-pro visioning. In the on text of m ulti-path routing this has b een used to design TEX CP [16 ℄. 3.2 Bandwidth sharing and fairness Bandwidth is shared b et w een o ws aording to a ertain ob jetiv e realised b y one transp ort proto ol as TCP for data transfers making use of one of its ongestion on trol proto ols (Reno, V egas, Cubi, high sp eed et.) or TCP friendly rate on trol (TFR C) for adaptiv e streaming appliations. In general these proto ols realise dieren t fairness riteria, whilst in this on text w e assume that all o ws are sub jet to a ommon fairness ob jetiv e. The problem form ulation dates ba k to Kelly [18 ℄ where this problem is form ulated as a non linear optimization problem with linear onstrained with ob jetiv e giv en b y a utilit y funtion U ( x ) of the o w rate x . maximise X i ∈S ,d ∈ Γ U d ( φ d i ) sub jet to X k ∈ N a ik x d ki − X j ∈ N a j i x d ij =    φ d i if i ∈ S − φ d i if i ∈ E 0 otherwise ∀ d ∈ Γ (4) X d ∈ Γ x d ij ≤ c ij ∀ i, j ∈ L (5) demands are assumed elasti, meaning that they ould get as m u h bandwidth as net w ork status allo ws, and ha v e no exogenous p eak rates. A general lass of utilit y funtions has b een in tro dues in [27 ℄, U d ( x ) =  w d log x α = 1 w d (1 − α ) − 1 x 1 − α α 6 = 1 (6) if α → ∞ fairness riteria is max-min. This form ulation is widely and suesfully used in net w ork mo deling. [11 , 12 , 19 , 20 , 21 , 33 ℄ ha v e onsidered the problem of single and m ultiple path routing and ongestion on trol under this framew ork as onstrain t (4) ma y oun t either a single or a m ultiple set of routes. 3.3 User utilit y and net w ork ost User utilit y and net w ork ost are t w o oniting ob jetiv e in a mathematial form ulation. [ 12 , 13 , 16 ℄ ha v e used the ost funtion as TE ob jetiv e lik ely mo deled b y the ISPs in ordered to k eep lo w link loads, i.e. minimise osts for upgrades. [11 , 14 , 21 , 31 , 35 , 38 ℄ ha v e just used the net w ork ost as p enalt y funtion in plae of hard onstrain ts in the optimisation framew ork. Congestion sensitiv e m ultiple routes seletion an b e form ulated as a mathematial program with non linear ob jetiv e and linear onstrain ts. maximise X i ∈S ,d ∈ Γ U d ( φ d i ) − X i,j ∈ N C P d ∈ Γ x d ij c ij ! sub jet to X k ∈ N a ik x d ki − X j ∈ N a j i x d ij =    φ d i if i ∈ S − φ d i if i ∈ E 0 otherwise ∀ d ∈ Γ (7) INRIA Flow-A war e T r a Engine ering 7 X d ∈ Γ x d ij ≤ c ij ∀ i, j ∈ L (8) φ d i ≤ p d ∀ d ∈ Γ ∀ i ∈ S (9) As a new additional onstrain ts w e add exgenous rates as this has signian t impat in the pro ess of seletion of optimal routes. In this form ulation the user utilit y is a funtion of the total o w rate tra v ersing the net w ork through the a v ailable paths. In a path-demand form ulation the o w rate φ of a giv en user an b e re-written as the sum of the rates o v er the set of a v ailable paths P , i.e. φ = P p ∈P φ p . A user is free to o ordinate sending rates o v er the paths join tly , aiming at maximise its o wn utilit y . Consider no w the folllo wing relaxation of the ob jetiv e U ( X p ∈P φ p ) ≥ X p ∈P U ( φ p ) (10) ea h user's path w ould b e seen as indep enden t, in other w ords as if it w ere a separate user and the fairness ob jetiv e w ould b e at a path, and not user base. A proto ol designed observing su h rule w ould break path o ordination, while a net w ork imp osing p er link fair bandwith sharing, w ould realise this ob jetiv e for an y m ulti-path on troller regardless its original design. 3.4 T o y Examples In this setion w e onsider t w o simple net w ork top ologies, a triangle and a square as depited in g.3.4 with all a v ailable paths. All links are bi-dertional with the same apait y C . Capait y C 12 = C 31 is inreased from C to 15 × C . F or b oth senarios no des 1 and 3 send data to a single destination no de n um b er 2. W e nd the global optim um of problem ( 7), using utilit y funtion (6) with α = 2 and ost funtion (2). W e assume demands fully elasti. 1 2 3 1 2 3 4 (a) (b) Figure 1: F ull mesh triangle and square top ologies. Hot-sp ot destination in no de 2 and soures in no de 1 and 3. 3.4.1 T riangle Fig.2 sho ws the split ratios o v er the t w o routes (one-hop and t w o-hops) and the ratio of the total users' rate (global go o dput) o v er the total onsumed net w ork bandwidth (net w ork utilisation). Let us all this ratio GCR (go o dput to ost ratio). T op plot relates to o ordinated m ulti-path (CM), whilst b ottom plot to uno ordinated (UM). When C 12 = C 31 = C the t w o problems ha v e ompletely dieren t solutions as CM splits all tra to the shortest-path and UM splits rates equally . A t this stage GCR for CM is 30% larger than UM's. As C 12 = C 31 > C inreases CM lo oks for more resoures for the seond demand along the t w o-hop path, resulting in more net w ork ost and GCR dereases. Ho w ev er using UM, GCR is insensitiv e to the split ratios. After a ertain p oin t CM and UM ha v e the same p erformane. RR n ° 6474 8 Lu a Mus ariel lo and Die go Perino à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ ò ò ò ò ò ò ò ò ò ò ò ò ò ò ò 0 2 4 6 8 10 12 14 0.0 0.2 0.4 0.6 0.8 1.0 C 12  C = C 31  C x i d  x d ò H x 1 + x 2 L  Cost æ x 2 à x 1 à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ ò ò ò ò ò ò ò ò ò ò ò ò ò ò ò 0 2 4 6 8 10 12 14 0.0 0.2 0.4 0.6 0.8 1.0 C 12  C = C 31  C x i d  x d ò H x 1 + x 2 L  Cost æ x 2 à x 1 Figure 2: Rate distribution among the a v ailable paths and user rate to net w ork ost ratio. 3.4.2 Square In g.3 w e ha v e similar p erformane to the triangle, despite CM needs to use more paths to attain the optim um, ev en to gather a small amoun t of bandwidth. F urthermore GCR for CM is not that m u h larger than in ase UM is used. In a real proto ol, seondary paths w ould not b e used if the attained gain do es not meet the ost for the o v erhead that is not onsidered here in the mo del, ho w ev er signiativ e in pratie. 3.4.3 Disussion MC has a larger stabilit y region, as it onsumes less bandwith to pro vide the same global go o dput as UM. This migh t turn out not b e true in pratie as w eakly used seondary paths ould ost to m u h in terms of o v er-head due to signalling to set up the onnetion, or prob es to monitor paths that are b eing o ordinated. à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ ò ò ò ò ò ò ò ò ò ò ò ò ò ò ò 0 2 4 6 8 10 12 14 0.0 0.2 0.4 0.6 0.8 1.0 C 12  C = C 31  C x i d  x d ò H x 1 + x 2 L  Cost æ x 2 à x 1 à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à à æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ æ ò ò ò ò ò ò ò ò ò ò ò ò ò ò ò 0 2 4 6 8 10 12 14 0.0 0.2 0.4 0.6 0.8 1.0 C 12  C = C 31  C x i d  x d ò H x 1 + x 2 L  Cost æ x 2 à x 1 Figure 3: Rate distribution among the a v ailable paths and user rate to net w ork ost ratio. 3.5 A Linear Program form ulation In this setion w e onsider problem ( 7) in se.3.3 and selet one partiular fairness riteria: max- min. W e write an original form ulation of problem (4 ) as an iterativ e linear program assuming linear osts C ( x ) . A t ea h iteration a linear sub-problem is solv ed whom, at optim um, giv es the same net w ork o w share to ev ery demand. This share is the maxim um bandwidth that an b e allo ated to the most onstrained demand. The net w ork graph G is redued to ˜ G through the follo wing transformation: ˜ c ij = c ij − P d ∈ Γ x d ij , i.e. apaities are replaed b y residual apaities after the allo ation of this share of bandwidth. if ˜ c ij = 0 the link is remo v ed from the graph. The sub-problems are formalised as follo ws. INRIA Flow-A war e T r a Engine ering 9 maximise z − X i,j ∈ N C P d ∈ Γ x d ij c ij ! sub jet to X k ∈ N a ik x d ki − X j ∈ N a j i x d ij =    z if i ∈ S − z if i ∈ E 0 otherwise ∀ d ∈ Γ (11) X d ∈ Γ x d ij ≤ ˜ c ij ∀ i, j ∈ L (12) z ≤ p d ∀ d ∈ Γ ∀ i ∈ S (13) In the follo wing setion w e use this iterativ e LP in order to obtain the gain that an b e obtained exploiting path div ersit y for large net w orks with a large n um b er of demands. 4 Analysis of large problems 4.1 Sim ulation set-up W e study t w o top ologies as sho wn in g. 4. The rst is the Abilene ba kb one net w ork [1 ℄, while the seond is a p ossible wireless mesh net w ork ba khaul. The link apait y distribution is a Normal distribution with a v erage ¯ C and standard deviation ¯ C / 10 . The Abilene top ology oun ts N = 11 no des and w e p erform sim ulations with mean link apait y set to t w o senarios: ¯ C = 100 M b/ s and ¯ C = 50 M b/s . The wireless mesh top ology oun ts N = 1 6 no des and, similarly , t w o senarios are onsidered: ¯ C = 50 M b/s ¯ C = 25 M b/s . In this latter ase apaities are redued to represen t radio  hannels with lo w er a v ailable bitrate. 1 3 2 4 5 6 7 8 9 10 11 1 3 2 4 5 7 6 8 9 11 10 12 13 15 14 16 (a) (b) Figure 4: (a) Abilene ba kb one top ology; (b) a planned wireless mesh net w ork. The distribution of p eak rate is tak en Log-Normal with parameters µ = 16 . 6 and σ = 1 . 04 . These v alues are tak en from a set of ts p erformed on measuremen ts gathered from Sprin t ba kb one [29 ℄ . T w o tra matries are onsidered:  Uniform . Ev ery no de sends tra to all other no des. There are th us N ( N − 1) demands where N is the n um b er of no des. T o mat h o w rates to soure-destination pairs w e use a te hnique desrib e in [29 ℄, assuming minim um ost-path the routing used. This te hnique RR n ° 6474 10 Lu a Mus ariel lo and Die go Perino a hiev es the b est mat h b et w een a set of demands and a set of SD pairs onneted b y a giv en routing. This tra pattern migh t b e represen tativ e of an In tra-domain optimised ba kb one. Sine the Abilene net w ork has 11 no des, in our sim ulations w e generate a total n um b er of 110 demands. W e do not onsider this tra matrix o v er a wireless mesh top ology as unlik ely all no des send tra to all no des in su h net w orks.  Hot-Sp ot | S | no des ha v e r o ws direted to a ommon sink no de. W e x the sink no de and randomly selet | S | soure no des. | S | r demands are randomly assigned to these | S | Soure- Sink pairs. In our exp erienes w e set | S | = 4 , r = 2 5 , for a total n um b er of 100 demands. No de 6 is seleted as sink in b oth top ologies. This tra pattern an b e rapresen tativ e of a data en ter lo ated in a ba kb one top ology or a gatew a y no de in a wireless mesh net w ork. A ording to the aforemen tioned set-up w e sim ulate a tra matrix whi h is used as input to the LP desrib ed in se.3.5 and solv ed using the MA TLAB optimisation to olb o x. F or ev ery senario w e ev aluate the satisfation of ea h demand as the ratio b et w een its attained rate and its exogenous p eak rate. A demand is fully elasti if its exogenous rate is larger that the maxim um attainable bandwidth in an empt y net w ork. Hene, satisfation is alw a ys dened as w e need not assume innite p eak rate to elasti o ws. W e use this p erformane parameter as it is able to expliit ho w bandwidth is allo ated with resp et to the distribution of the exogenous rates. Output data are a v eraged o v er m ultiple runs. 4.2 Numerial results Results are rep orted in g. 5 and 6 and, as exp eted, sho w m ulti-path routing outp erforms mini- m um ost routing o v er b oth net w ork top ologies and for b oth tra patterns. Ho w ev er, the p oin t is to measure the en tit y of the impro v emen t. In partiular, the gain is larger for wireless mesh top ology and for senarios adopting larger link apaities. The gain of m ulti-path is, in great part, limited to o ws with larger p eak rate, whilst o ws with lo w er p eak rate are ompletely satised b y b oth routing s hemes. This b eause MinCost routing, under max-min fairness, p enalises larger o ws b y fairly sharing the apait y of the link that ats as b ottlene k among all o ws there in progress. Multi-path routing, under max-min fairness, ats similarly exept that o ws an retriev e band- width, not only on their minim um ost path, but also on their seondary routes. Indeed, sim ula- tions sho w that lo w rate o ws do not tak e an y adv an tage of path div ersit y , while high rate o ws retriev e additional bandwidth along other paths. Max-min m ulti-path routing a v oids to use more than one path to route lo w rate o ws. This has b eneial eet in pratie as it a v oids w astage of resoures due to the o v erhead, that migh t b e justied only ab o v e a ertain minim um rate. 20 40 60 80 100 0 0.2 0.4 0.6 0.8 1 Demand ID Satisfation Abilene Uniform Scenario m in - co st ¯ C = 10 0 mu l ti- pa t h ¯ C = 10 0 m in - co st ¯ C = 50 mu l ti- pa t h ¯ C = 50 20 40 60 80 100 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Abilene Hot−Spot Scenario Demand ID Satisfation m in- cost ¯ C =1 00 mu lti- pa th ¯ C =1 00 m in- cost ¯ C =5 0 mu lti- pa tht ¯ C =5 0 Figure 5: Abilene top ology . Satisfation distribution for uniform (top) and hot-sp ot (b ottom) tra matrix. INRIA Flow-A war e T r a Engine ering 11 Fig. 5 rep orts results for the Abilene net w ork. The gain of max-min m ulti-path routing is v ery large for hot-sp ot tra matries, and ev en more signian t when net w ork apaities are larger. The routing s heme is able to transfer 65% additional tra when ¯ C = 10 0 and 41% when ¯ C = 50 , with resp et to minim um ost routing. Ho w ev er, under uniform tra, the gain is m u h smaller; i.e. only 4% when ¯ C = 10 0 and 3% when ¯ C = 50 . Under uniform tra, demands are spread all around no des and ev en high rate o ws do not exploit path div ersit y as they are almost ompletely satised o v er their minim um ost path. If resoures b eome sare high rate o ws annot exploit path div ersit y b eause all links are already saturated b y o ws routed along their minim um ost paths. Therefore as the apait y is inreased, the gain is distributed to lo w rate o ws rst, and to those with higher rate at last. Of ourse, as the apait y is fairly large, resulting in a ligh tly loaded net w ork, b oth s hemes ha v e similar p erformane as all o ws are satised along the minost route. The v ariane of the satisfation is quite small, and b et w een 0.05 and 2.3 for all senarios. Ho w ev er it is not uniformly distributed among all o ws. In fat, large o ws are aeted b y larger v ariane than small o ws. This is due to the max-min fairness riteria that allo ates a minim um amoun t of bandwidth to all o ws. This alw a ys assures omplete satisfation for small o ws while satisfation of large o ws dep ends on the net w ork apaities that v aries from sim ulation to sim ulation. 0 20 40 60 80 100 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Demand ID Satisfation Hot Spot Wireless Mesh Scenario m in -co st ¯ C =5 0 mu l ti-p a th ¯ C =5 0 m in -co st ¯ C =2 5 mu l ti-p a th C =2 5 Figure 6: Wireless mesh top ology . Satisfation distribution hot-sp ot tra matrix. 5 Ar hitetures and proto ols 5.1 Ar hitetures An y of the prop osals for ongestion on trol, and ongestion on trol exploiting path div ersit y , an b e restated in the on text of an optimisation problem of the kind of ( 7) or the uno ordinated oun terpart (10 ). Su h problems lead to dieren t ar hitetures based on dieren t theoretial foundations. W e divide su h ar hitetures in three groups: fully deen tralised, quasi deen tralised, o w a w are. This lassiation is for the ease of exp osition and for sak e of larit y , ho w ev er it la ys itself op en to ritis.  F ully deen tralised (FD) . In su h ar hiteture net w ork no des store, swit h and forw ard pa k ets from an input to an output in terfae. Neither s heduling nor ativ e queue managa- men t is implemen ted in to the no des. A form of o op eration is assumed b et w een soures, that implemen t a ommon rate on trol algorithm and, if needed, a ommon m ultiple path splitter. In general soures need to b e onforman t to a ommon fairness riteria. This is the ase for the urren t In ternet and m ulti-path TCP [11 ℄ is one example of su h on troller. RR n ° 6474 12 Lu a Mus ariel lo and Die go Perino  Quasi deen tralised (QED) . No des implemen t an y form of ativ e queue managemen t (A QM), to prev en t ongestion, and expliit ongestion notiation (ECN) that is triggered aording to some temp orisation. ECN migh t also b e result of a lo al no de alulation. Soures exploit su h notiation to on trol sending rate and split deisions. [12 , 13 , 16 ℄ are t w o examples in the on text of in tra-domain tra engineering.  Flo w a w are (F A) . No des implemen t pa k et s hedulers that realise fair bandwidth sharing b et w een o ws. Soures balane tra among the a v ailable routes sub jet to the restrition imp osed b y pa k et s hedulers and autonomously deide ho w to exploit as b etter as they an net w ork resoures. Congestion on trol and bandwidth allo ation are solv ed separately . [30 ℄ prop ose a routing s heme inspired b y the the te hnique of "trunk reserv ation" used in PSTN to allo ate iruits to seondary paths in ase the diret paths w ere exp eriening o v erload. [30 ℄ Ea h of ab o v e-men tioned ar hitetures ha v e its oun terpart that mak e no use of path div er- sit y . It app ears diult to unequiv o ally selet one approa h. F or instane, the question of the deplo ymen t of a ertain ongestion on trol algorithm is still hotly debated (see the newsletter [40 ℄). The use of exp erimen tal proto ols (e.g. ubi in Lin ux) sares, as FD ar hitetures assume a ommon form of o op eration b et w een users and anar h y migh t b e ostly , whether degenerates in ongestion ollapse, or b eing less negativ e, in unfairness in sharing resoures. Manifest truth is that FD are v ery simple and do not require an y sort of parametrisation, esp eially at no des, and extensions that mak e use of path div ersit y are easy to deplo y as soure routing is made a v ailable. Ho w ev er, the main onern that obstruts the deplo ymen t of soure routing is seurit y . QD enhanes FD and mak e it more eien t and stable. X CP [17 ℄ is an example of proto ols of this kind while, in the on text of in tra-domain TE, TEX CP [ 16 ℄ and TR UMP [12 ℄ b elong to this lass. 5.2 Flo w a w are ar hiteture 5.2.1 P er-o w fair queueing The b enet of p er-o w fair queueing has long b een reognised [ 7 , 28 ℄. Besides, it is robust again ts unfair use of resoures b eause of non standard onforman t use of net w ork transp ort proto ols resulting from bad implemen tations (rare), absene of ommon agreemen t (the ase of ubi in Lin ux [40 ℄) as w ell as maliious use that exploits others' w eakness (more aggressiv e ongestion on trollers for instane). P er-o w fair queueing reliev es the net w ork to assume standard onfor- mane of end to end proto ols. Assured fairness allo ws new transp ort proto ols to b e in tro dued without relying on detailed fairness prop erties of preexisting algorithms. 5.2.2 Ov erload on trol P er-o w fair queueing is feasible and salable in presene of o v erload on trol [25 , 26 ℄. When demand exeeds apait y the s heduler assures equal p erformane degradation to all o ws. Ho w ev er, arising ongestion at o w lev el is a transien t phenomena, as users w ould quit the servie in ro wds bringing ba k utilisation to normal loads. This is p ereiv ed b y the user as servie break-do wn. A t presen t, no o v erload on trol is implemen ted, in an y form, within the net w ork. A t a ertain exten t, this is assured within the ISPs ba kb one, and in part within the aess, b y o v er-pro visioning link apaities aording to an estimated tra matrix (using neto w or similar to ols). Su h metho dology do es not solv e lo al ongestion in small p erio ds of time and fore users to quit proting from a servie. In wireless mesh ba khaul o v erpro visioning w ould not b e an ymore a feasible solution and m ulti- path routing migh t b e neessary . Ov erload on trol has to b e dynami and fast reating to ongestion. This an b e realised lo oking for resoures from other a v ailable w a ys, for instane the a v ailabilit y m ultiple routes to join a servie. INRIA Flow-A war e T r a Engine ering 13 5.2.3 P er-o w path seletion and admission on trol A sub optimal approa h is to selet one single path among man y others. In presene of a n um b er of paths to rea h a destination, a o w an b e deeted to a b etter route, e.g. with larger fair rate or with minim um ost. Su h a greedy s heme is not stable and w ould led to osillations if the driving metri is not stable enough. This is the ase of fair rate. 5.2.4 P er-o w m ultiple path routing The presene of p er-o w fair queueing in ev ery link imp ose p er-path fair bandwidth sharing. This means that the utilit y of a single user is not a funtion of the total attained rate as he is not allo w ed to get more bandwith of the fair share along its minim um ost path. Therefore the utilit y is giv en b y the sum of the utilit y of ea h singular path as (10 ). W e kno w that in general this problem is sub optimal with resp et to a o ordinated splitter. In this ase the optim um attained rate is giv en b y x d i = ∂ x d i U ( q d i ) − 1 b eing q d i the ost of path i for user d . The split ratio is the in v ersely prop ortional to the ost. In a reen t pap er Key et al. [ 22 ℄ pro v e that in absene of o ordination, the stabilit y region of the n um b er of o w in progress in the net w ork, is redued. This is sho wn for a triangle top ology and uniform tra matrix. The o w mo del assumes that o ws arriv e aording to a sto  hasti pro ess and lea v e the system after b eing serv ed an amoun t of data whi h is distributed. A redued rate region at o w lev el is a onsequene of ( 10 ) as, the same rate is obtained at a larger ost. As w e sho w in the example 3.4.1 . Notie also that in presene of uniform tra matries, the o v er-head requested b y m ulti-path is not really justied with resp et to minim um ost path seletion as there is almost no gain, as w e ha v e sho wn in the previous setion. 5.3 Multi-path Iterativ e Routing T ra Optimizer In this setion w e prop ose one of the main outome of the pap er: Multi-path Iterativ e Routing T ra Optimiser (MIR TO), a fully distributed algorithm aimed at obtaining, in a deen tralised w a y , a m ulti-path optimal strategy prop osed in se.3.5 . It is designed around TCP (AIMD) and it allo ws o ordinated tra split o v er m ultiple paths. MIR TO migh t lik ely run on end-hosts and an w ork in an y of the ar hitetures desrib ed in se. 5.1 . In order to run MIR TO, hosts should diso v er a v ailable routes and link apaities along them to rea h a destination. The w a y su h information is olleted is out of the sop e of our algorithm but they an for example b e diso v ered through link-state routing proto ols. The algorithm w orks ev en whether information is inomplete but in a less eetiv e w a y , as in ase paths are partially diso v ered and path div ersit y limited. Flo w splitting o v er m ultiple paths an b e p erformed in sev eral w a ys aording to the appliation on text. By means of path omputation apabilities of MPLS for in tra-domain TE, or through middlew ares running on pro xy no des ativ e as middle-la y er. Sym b ol Meaning ∆ + p ositiv e step ∈ R + ∆ − negativ e step ∈ R − Q d k ( t ) ost of k th path of o w d at time t RT T d k Round T rip Time of the k th path of o w d T able 2: Algorithm's notation Algorithm 1 sho ws MIR TO pseudo-o de for rate on trol o v er a giv en path k for a giv en o w d . Notation is rep orted in table 2. Op erations are p erformed ev ery RT T d k or in general when new information are a v ailable on the state of path k . P ath osts are omputed aording to ( 14 ). They only dep end on apait y of the link along the route. Indeed, in ase of linear ost ( 3) is equal to P ij ∈ L d k 1 C ij . Link with larger apait y are b etter rank ed regardeless of utilisation. Innite ost in RR n ° 6474 14 Lu a Mus ariel lo and Die go Perino (14 ) just indiates ongestion notiation à la TCP and the route is mark ed ongested. Q d k ( t ) = ( P ij ∈ L d k ∆ + C ij if ∀ ( i, j ) ∈ L d k ρ ij ( t ) < C ij ∞ if ∃ ( i, j ) ∈ L d k ρ ij ( t ) ≥ C ij (14) Algorithm 1 MIR TO algorithm for a giv en demand d for k ∈ P d do ompute Q d k ( t + RT T d k ) end for if Q d k ( t + RT T d k ) = ∞ ∀ k ∈ P d then for k ∈ P d do x d k ( t + RT T d k ) ← x d k ( t ) − x d ( t )∆ − end for else if p d ( t ) ≥ x d ( t ) and Q d k ( t + RT T d k ) = min k Q d k ( t + RT T d k ) then x d k ( t + RT T d k ) ← x d k ( t ) + ∆ + else if p d ( t ) < x d ( t ) and Q d k ( t + RT T d k ) = max k Q d k ( t + RT T d k ) then x d k ( t + RT T d k ) ← x d k ( t ) − ∆ − end if An inrease of ∆ + is done o v er the minim um ost path when there is at least one non ongested path, and the total o w rate an still get inreased if lo w er than its o wn p eak rate. The rate inrease is  hosen onstan t and indep enden t to o w rate not to fa v our higher rate o ws. This inrease an b e onsidered as a prob e in order to diso v er the global optim um split. In priniple all path migh t b e prob ed as, when those b etter rank ed are ongested, w orse rank ed routes an b e exploited. The imp ortane of probing in su h kind of on troller to a v oid to b e trapp ed in non optima equilibria has b een highligh ted in [21 ℄. Flo w rate is dereased in t w o w a ys aording to net w ork onditions. Firstly MIR TO redues sending rate o v er all paths when they are all ongested. The rate derease is prop ortional to the total o w rate. This guaran tees fairness. Otherwise, small o ws or new oming o ws starting with lo w er rates w ould b e disadv an taged with resp et to higher o ws. Moreo v er deremen t is p erformed o v er all paths at the same time in order to allo w o w split re-arranging. In fat, after this derease, there is newly a v ailable bandwidth and o ws will gro w aording to the aforemen tioned rules. So, if the urren t split is only lo ally optim um the on troller w ould mo v e to w ards a global optim um aording to this sear h strategy . On the other hand, a rate derease is p erformed an y time the total o w rate is larger than o w p eak rate and at least one non-ongested path is a v ailable. This means that, as a b etter rank ed route is willing to inrease its rate, this m ust b e done to the detrimen t of a w orse rank ed route, without altering the global rate whi h is b ounded b y the exogenous rate. In fat ev en if the total o w rate is equal to p eak rate, the path splitting ould b e only lo ally optim um i.e. more exp ensiv e. It w orth realling that a more exp ensiv e path migh t also b e  haraterized b y larger end to end dela ys. If more than one maxim um/minim um ost paths exist, the derease/inrease is shared among them. This me hanism allo ws MIR TO to rea h a global optim um in presene of demands with or without exogenous rate as it follo ws the lassial w ater lling pro edure that la ys at the base of the max-min fairness riteria. The seletion of the minim um-maxim um ost path when p erforming inrease/derease op er- ations guaran tees o ordination b et w een dieren t paths of the same o w as long as the net w ork do es not imp ose an y additional fairness seman ti. 5.4 Stabilit y and optimalit y MIR TO an b e desrib ed through a uid equation that appro ximates its b eha viour and an b e used to pro v e on v ergene and stabilit y . W e supp ose rst no net w ork dela y and then w e generealise to the realisti ase. INRIA Flow-A war e T r a Engine ering 15 5.4.1 Absene of net w ork dela ys First onsider the ase the o w has no exogenous rate. dx i ( t ) dt = ∆ + [1 − q ( t )] κ i ( t ) − ∆ − q ( t ) N X k =1 x k ( t ) κ i ( t ) is the probabilit y that i is the minim um ost path at time t . q ( t ) is the probabilit y that all path are ongested. Therefore at steady state X k x k ( ∞ ) = κ max ( ∞ ) 1 − q ( ∞ ) q ( ∞ ) ∆ + ∆ − where κ max ( ∞ ) = max k κ j ( ∞ ) . This means tra is split among minim um ost paths, p ossibly a single path. All the others are sub jet to on tin uous probing su h that x i ( t ) ∼ κ i ( t ) . This feature allo ws the on troller not to b e trapp ed in equilibria p oin ts that are not optima. A path is prob ed as frequen t as it is rank ed the b est among the others. This an b e seen from sim ulations in Setion 5.5 in g. 8 . In ase the o w is p eak rate limited the previous equation an b e rewritten as follo ws. dx d i ( t ) dt = { ∆ + κ i ( t ) s d ( t ) − ∆ − r i ( t )[1 − s d ( t )] } [1 − q d ( t )] + − ∆ − q d ( t ) X k x k ( t ) where s d ( t ) = P r [ x d ( t ) > p d ] and r i ( t ) is the probabilit y that i is the more exp ensiv e path. Con v ergene an b e disussed as in the previous ase when there w ere no p eak rate. 5.4.2 Presene of net w ork dela ys In presene of net w ork dela ys κ ( t ) , q ( t ) , r ( t ) and s ( t ) are dela y ed information at the soure. A problem of stabilit y in the sense of theory of on trol arises. W e do not pro vide here rules on ho w to set ∆ + and ∆ − in order to k eep the system asin totially stable in presene of dela ys. Using standard te hniques as desrib ed in [33 ℄ this an b e easily obtained for simple top ologies. Cum b ersome alulations, and the use of the generalised Nyquist riterion an b e used to pro v e stabilit y for a general top ology . 5.5 A ase study In this setion w e ev aluate the p erformane of the ab o v e men tioned ar hitetures b y means of uid sim ulations in order to dipla y the on v ergene b eha viour of three seleted senarios. Ea h senario represen ts one of the three ar hitetures onsidered in Setion 5.1 1 3 2 4 5 100 Mb/s 2.5 ms 100 Mb/s 2.5 ms 100 Mb/s 2.5 ms 100 Mb/s 2.5 ms 1 Mb/s 250 ms 10 Mb/s 25 ms 100 Mb/s 2.5 ms 1 3 2 4 5 X1 X2 X3 X4 1 3 2 4 5 X1 X2 X3 X4 1 3 2 4 5 X1 X2 X3 X4 Figure 7: Simple net w ork top ology with the set of p ossible path for ea h of the three demands. RR n ° 6474 16 Lu a Mus ariel lo and Die go Perino 5.5.1 Sim ulation setup The FD and the F A ar hitetures are analyzed b y supp osing no des run the MIR TO algorithm. Reall MIR TO has b een sp eially designed for FD solutions and requires to set rate inrease and derease v alues. They should allo w the algorithm to o v erome lo al optimal split to attain the global optim um and, at the same time, limit tra utuation. The problem of setting inrease- derease v alues is a w ell kno wn trade-o of TCP and TCP-lik e proto ols and MIR TO inherits it as w ell. In our sim ulations w e set them to ∆ + = 0 . 5 M b / s and ∆ − = 0 . 01 3 . As onern QD ar hitetures, w e implemen t a mo died v ersion of the TR UMP [ 13 ℄ algorithm. It diers from the original one simply b eause it an w ork ev en in presene of o ws with a giv en p eak rate. This is a hiev ed b y dereasing the rate of a o w o v er all paths when its total rate o v eromes its p eak rate. TR UMP requires to set three parameters. A rst parameter, alled w is a w eigh t to adjust balane b et w een utilit y and ost funtion. It is strongly related to top ology and link apaities and it tunes the maxim um net w ork load. A seond one, alled β , w eighs the impat of a ongested link, while a third one, alled γ , is the inrease/derease step. F or our omparison purp oses w e set them to w = 1 0 − 2 to allo w o ws to fully utilize links, β = 10 − 3 to resp et link apaities and γ = 1 0 − 3 to limit rate osillations. Time [se℄ 0-5 5-10 10-18 18-25 25-40 40-60 60-80 X 1 0 0 70 ∞ ∞ ∞ ∞ X 2 0 30 30 ∞ ∞ ∞ ∞ X 3 50 50 50 50 50 ∞ 55 T able 3: P er user p eak rate ev olution o v er time. Rates v alues are expressed in Mb/s. 5.5.2 Results Figure 7 rep orts the top ology used in the sim ulations w e sho w in this setion to sho w the b eha viour of MIR TO. Link apaities and latenies ha v e b een seleted in order to ha v e path div ersit y and dieren t resp onse time. There are three o ws in the net w ork and w e selet as soure-destination pairs 1-2, 3-2 and 4-5. Flo w p eak rates  hange o v er time as sp eied in table 3 . Note that, ∞ is used to indiate elasti o ws. This just means o w p eak rate is larger than the a v ailable net w ork resoures. Ev ery o w an b e split o v er four dieren t paths as sho wn in Figure. Figure 8 sho ws the total o w rates and the o ws splitting obtained b y running MIR TO o v er a FD ar hitetures. MIR TO rates are ompared to those obtained b y running the LP optimizer desrib ed in setion 4 . As exp eted, total o w rates and rate splits o v er paths a hiev ed b y MIR TO follo w those obtained b y the LP . Flutuations o ur and are more pronouned when t w o or more o ws are elasti. This is due to the probing nature of the algorithm that allo ws the global optim um attainmen t. Figure 9 sho ws o w rates obtained b y running MIR TO o v er a F Q ar hiteture and a omparison with the LP . Ov er this ar hiteture only total o w rates follo w the trend of the LP while the o w spitting is quite dieren t from optim um. This onrms what stated in setion 5 as F A breaks o ordination among o ws. In that w a y , o ws annot a hiev e rates larger than their fair rate o v er paths. This leads to a sub-optimal solution ev en if total o w rates are the same as b efore. In fat, they ha v e b een a hiev ed with a larger net w ork ost. As exp eted, with a F A ar hiteture rate utuations are redued and the algorithm on v erges in a shorter time. Finally , Figure 10 rep orts p erformane of the TR UMP algorithm. TR UMP a hiev es optimal rates and splitting but it tak es long time to on v erge and some long term utuations o ur. This is a onsequene of the set of parameters w e used. With dieren t v alues w e w ould ha v e seen a dieren t b eha viours with m u h faster on v ergene and no utuations. Ho w ev er the optimisation INRIA Flow-A war e T r a Engine ering 17 0 10 20 30 40 50 60 70 0 20 40 60 80 100 Rate [Mb/s] User rates X 3 X 1 X 2 0 10 20 30 40 50 60 70 0 10 20 30 40 50 60 70 80 90 Rate [Mb/s] Rate split User 1 X 1 X 3 0 10 20 30 40 50 60 70 0 10 20 30 40 50 60 70 80 90 Rate [Mb/s] Rate split User 2 X 1 X 4 X 3 X 2 0 10 20 30 40 50 60 70 0 10 20 30 40 50 60 70 80 90 Time [sec] Rate [Mb/s] Rate split User 3 X 1 X 3 X 4 Figure 8: Time ev olution of the rate allo ation for MIR TO in a FD ar hiteture. P oin ts indiate the optimal allo ation. 0 10 20 30 40 50 60 70 0 10 20 30 40 50 60 70 80 90 100 110 Rate [Mb/s] User rates X 2 X 3 X 1 0 10 20 30 40 50 60 70 0 10 20 30 40 50 60 70 80 90 Rate [Mb/s] Rate split User 1 X 1 X 3 X 4 X 2 0 10 20 30 40 50 60 70 0 10 20 30 40 50 60 70 80 90 Rate [Mb/s] Rate split User 2 X 1 X 3 X 3 X 4 0 10 20 30 40 50 60 70 0 10 20 30 40 50 60 70 80 90 Time [sec] Rate [Mb/s] Rate split User 3 X 1 X 3 X 4 X 2 Figure 9: Time ev olution of the rate allo ation for MIR TO in a F A ar hiteture. P oin ts indiate the optimal allo ation. RR n ° 6474 18 Lu a Mus ariel lo and Die go Perino solution w ould ha v e b een signian tly dieren t from that w e w an t to a hiev e. A tually this is not a dra wba k of TR UMP as it has b een designed in the on text of in tra-domain TE to limit the link loads within the net w ork, ev en though authors assume p ossible to implemen t su h on troller at end hosts. Unfortunately TR UMP do es not allo w to predit at whi h lev el of utilisation link an b e set. Hene the  hoie of parameters migh t b e v ery omplex in net w orks with heterogeneous apaities. W e ha v e tested TR UMP for dieren t setup and dieren t parameters and measured its go o d prop erties in term of fast on v ergene as the authors in [13 ℄. This sho w the  hoie of parameters in su h algorithms is a k ey p oin t and ould b e quite ompliated. Moreo v er it is strongly related to top ology and ould driv e to v ery dieren t solutions. 0 10 20 30 40 50 60 70 0 20 40 60 80 100 Rate [Mb/s] User rates X 2 X 3 X 1 0 10 20 30 40 50 60 70 0 20 40 60 80 Rate [Mb/s] Rate split User 1 X 1 X 3 X 2 0 10 20 30 40 50 60 70 0 20 40 60 80 Rate [Mb/s] Rate split User 2 X 1 X 3 X 4 0 10 20 30 40 50 60 70 0 20 40 60 80 Rate [Mb/s] Time [sec] Rate split User 3 X 1 X 3 X 4 Figure 10: Time ev olution of the rate allo ation for TR UMP in a QD ar hiteture. P oin ts indiate the optimal allo ation. 6 Disussion and Conlusions The outomes of this pap ers are m ultiple:  w e ha v e studied ongestion on trol and m ulti-path routing in presene of demands with exogenous p eak rates and w e ha v e found that m ultiple routes an b e lik ely exploited b y higher rate o ws.  in net w ork senario that are ommon in pratie, o ordinated and uno ordinated m ulti-path routing p erform the same.  in t ypial senarios where o ordination outp erforms un-o ordination, m ultiple path do es not gain m u h with resp et to single path routing.  furthermore w e prop ose a new m ulti-path optimal on troller alled MIR TO. This on troller is able to allo ate max-min bandwidth among demands exploiting path div ersit y . The on- INRIA Flow-A war e T r a Engine ering 19 troller has b een ompared with other solutions and has pro v ed to b e easy to b e deplo y ed in dieren t ar hitetures. The ab o v e men tioned outomes allo w to p ositiv ely rethink a o w a w are ar hiteture, in trinsially uno ordinated, in presene of m ulti-path routing. Indeed m ulti-path on trollers as MIR TO an b e eetiv ely used in order to exploit path div ersit y ev en when a o w a w are net w ork imp oses fairness at a link base. T ask of the on troller remains to dynamially dene split ratios o v er the paths as net w ork onditions  hange. W e suggest the use of m ultiple paths for a ertain lass of appliations that are naturally robust to v ariabilit y as adaptiv e video streaming, P2P le sharing and CDN as they an ee- tiv ely exploit un used apait y within the net w ork in b oth In ternet ba kb ones and wireless mesh ba khauling systems. On the other hand, taking in to aoun t the o v erhead required b y m ultiple routes transmissions, this should not b e used for more on v ersational tra as v oie, w eb, or short transations as mail or instan t messaging. Referenes [1℄ Abilene Ba kb one. http://abilene.i nt ern et 2. edu / . [2℄ Andersen D.G., Sno eren A.C., Balakrishan H., Best-P ath vs. Multi-P ath Ov erla y Routing A CM IMC 2003. [3℄ Bertsek as, D. Gafni, E. Gallager, R. Seond Deriv ativ e Algorithms for Minim um Dela y Dis- tributed Routing in Net w orks, IEEE T ransations on Comm uniations 1984 [4℄ Bertsek as, D. and Gallager, R., Data Networks . Pren tie-Hall.1987. [5℄ Bertsek as, D. P . (1999). Nonline ar pr o gr amming . Belmon t, MA 02178-9998: A thena Sien ti. Seond edition. [6℄ Can tor D. G. and Gerla M., Optimal routing in a pa k et swit hed omputer net w ork, IEEE T rans. Comput., v ol. C-23, pp. 1062-1069, Ot. 1974. [7℄ Demers A. , Kesha v S. , Shenk er S., Analysis and sim ulation of a fair queueing algorithm, In ternet w orking: Resear h and exp eriene, V ol 1, 3-26, 1990. (Also in pro eedings of A CM Sigomm 89). [8℄ F ortz, B. Thorup, M. In ternet tra engineering b y optimizing OSPF w eigh ts, IEEE Info om 2000 [9℄ F ortz, B. Thorup, M. Optimizing OSPF/IS-IS w eigh ts in a  hanging w orld, IEEE JSA C 2002 [10℄ Gallager, R. A Minimum Delay R outing A lgorithm Using Distribute d Computation . IEEE T ransation on Comm uniations, V ol. 25, No. 1, Jan. 1977. [11℄ Han H. , Shakk ottai S. , Hollot C.V. , Srik an t R. and T o wsley D.; Multi-path TCP: a join t ongestion on trol and routing s heme to exploit path div ersit y in the in ternet IEEE/A CM T r ans. on Networking V ol. 14 , Issue 6 (De. 2006). [12℄ He, J.; Bresler, M.; Chiang, M. and Rexford, J. T o w ards Robust Multi-La y er T ra Engi- neering: Optimization of Congestion Con trol and Routing IEEE Journal on Seleted Areas in Comm uniations, June 2007 V ol. 25, No. 5O [13℄ He, J.; Su hara M.; Bresler, M.; Chiang, M. and Rexford, J. Rethinking In ternet T ra Man- agemen t: F rom Multiple Deomp osition to a Pratial Approa h, Pro eeding of CONEXT 2007. RR n ° 6474 20 Lu a Mus ariel lo and Die go Perino [14℄ Lo w, S.H. Optimization o w on trol with on-line measuremen t or m ultiple paths, in Pro eed- ings of 16th In ternational T eletra Congress,1999 [15℄ Lo w, S.H. A dualit y mo del of TCP and queue managemen t algorithms IEEE/A CM T ransa- tions on Net w orking, Aug. 2003 V ol. 11, No. 4 [16℄ Kandula, S.; Katabi, D.; Da vie B.; Charnie A. W alking the tigh trop e: resp onsiv e y et stable tra engineering A CM SIGCOMM 2005 [17℄ Katabi, D; Handley , M and Rohrs, C.. Congestion on trol for high bandwidth-dela y pro dut net w orks A CM SIGCOMM 2002. [18℄ Kelly , F. Char ging and r ate  ontr ol for elasti tr a. Europ ean T ransations on T eleomm u- niations, v olume 8 (1997) pages 33-37. [19℄ Kelly , F.; Maullo o, A. and T an,D. R ate  ontr ol in  ommuni ation networks: shadow pri es, pr op ortional fairness and stability. Journal of the Op erational Resear h So iet y 49 (1998) 237-252. [20℄ Kelly , F., F airness and stability of end-to-end  ongestion  ontr ol . Europ ean Journal of Con trol, 9:159176, 2003. [21℄ Kelly , F. and V oie, T. Stability of end-to-end algorithms for joint r outing and r ate  ontr ol. A CM SIGCOMM Computer Comm uniation Review 35:2 (2005) 5-12. [22℄ Key P . , Massoulié L. Fluid mo dels of in tegrated tra and m ultipath routing. Queueing System 53: 65-98, 2006 [23℄ Key P . , Massoulié L., T o wsley D. Com bining m ultipath routing and ongestion on trol for robustness, Pro eedings of CISS 2006. [24℄ Key P . , Massoulié L., T o wsley D., P ath seletion and m ultipath ongestion on trol; IEEE Info om2007. [25℄ K ortebi A., Musariello L., Oueslati S., Rob erts J., On the Salabilit y of F air Queueing, Pro . of A CM SIGCOMM HotNets I I I, 2004. [26℄ K ortebi A., Musariello L., Oueslati S., Rob erts J, Ev aluating the n um b er of ativ e o ws in a s heduler realizing fair statistial bandwidth sharing, Pro . of A CM SIGMETRICS 2005. [27℄ Mo, J. and W alrand, J., 2000. F air end-to-end windo w-based ongestion on trol. IEEE/A CM T r ans. on Networking , 8(5):556567. [28℄ Nagle J. On P a k et Swit hes with Innite Storage, RF C 970, IETF, 1985. [29℄ Nui A., Sridharan A., T aft N. , The problem of syn thetially generating IP tra matries: initial reommendations, A CM SIGCOMM Computer Comm uniation Review, 2005. [30℄ Oueslati S. and Rob erts J. Comparing Flo w-A w are and Flo w-Oblivious A daptiv e Routing Pro eedings of CISS 2006. [31℄ P aganini F. Congestion on trol with adaptiv e m ultipath routing based on optimization, Pro- eedings of CISS 2006. [32℄ Raduno vi, B.; Gk an tsidis, C.; Key , P; Ro driguez,P; Hu, W; An Optimization F ramew ork for Pratial Multipath Routing in Wireless Mesh Net w orks Mirosoft T e hnial Rep ort ,MSR- TR-2007-81, June 2007. [33℄ Srik an t, R. The Mathematis of Internet Congestion Contr ol . Birkhauser, 2004. INRIA Flow-A war e T r a Engine ering 21 [34℄ Sriniv asan V., Chiasserini C. ,Nuggehalli P . and R. Rao, Optimal rate allo ation for energy- eien t m ultipath routing in wireless ad ho  net w orks, IEEE T ransations on Wireless Com- m uniations, v ol. 3, no. 3, pp. 891899, 2005. [35℄ V oie, T. Stability of Multi-Path Dual Congestion Contr ol A lgorithms to app ear in IEEE/A CM T ransations on Net w orking. [36℄ V oie, T. A global stabilit y result for primal-dual ongestion on trol algorithms with routing, Computer Comm uniation Review, v ol. 34, no. 3, pp. 3541, 2004. [37℄ W ang Z. and Cro w roft J. Analysis of shortest-path routing algorithms in a dynami net w ork en vironmen t A CM SIGCOMM CCR 1992. [38℄ Xiao jun Lin and Shro, N.B. ; Utilit y maximization for omm uniation net w orks with m ul- tipath routing IEEE T ransations on Automati Con trol, Ma y 2006. V ol. 51 No. 5 [39℄ Zhang-Shen R and MKeo wn N., Designing a Preditable In ternet Ba kb one Net w ork, Hot- Nets I I I, San Diego, No v em b er 2004. [40℄ [e2e℄ Are w e doing sliding windo w in the In ternet?. from the ar hiv e of End-to-End Resear h Group Charter. Jan 3 2008. RR n ° 6474 Unité de recherche INRIA Rocquenco urt Domaine de V oluce au - R ocq uencour t - BP 105 - 78153 Le Chesnay Cedex (France) Unité de reche rche INRIA Futurs : Parc Club Orsay Uni versité - ZAC d es Vi gnes 4, rue Jacques Monod - 91893 ORSA Y Cedex (France ) Unité de reche rche INRIA Lorraine : LORIA, T echnopôle de Nancy-B rabois - Campus scientifique 615, rue du Jardin Botani que - BP 101 - 54602 V illers-lès-Nanc y Cede x (France) Unité de reche rche INRIA Rennes : IRISA, Campus univ ersitai re de Beaulie u - 35042 Rennes Cedex (France ) Unité de reche rche INRIA Rhône-Alpes : 655, aven ue de l’Europe - 38334 Montbonnot Saint-Ismier (France) Unité de reche rche INRIA Sophia Antipolis : 2004, route des Lucioles - BP 93 - 06902 Sophia Antipolis Cedex (France) Éditeur INRIA - Domaine de V oluceau - Rocquencourt , BP 105 - 78153 Le Chesnay Cedex (Franc e) http://www.inria.fr ISSN 0249 -6399