Throughput metrics and packet delay in TCP/IP networks

Thr oughput metrics and pac ket dela y in TCP/IP netw orks [W ork in prog ress] Andrei M. Sukhov ∗ Samara State Aerospace University Mosko vskoe sh., 34 Samara, 443086, Russia amskh@yande x.ru Timur Sultano v Samara State Aerospace University , T ogliatti branch V oskresenska ya st., 1 T ogliatti, 445000, Russia tursul@ramb ler .r u Mikhail V . Strizhov Samara State Aerospace University Mosko vskoe sh., 34 Samara, 443086, Russia strizhov@ip4tv .ru Ale x ey P . Platonov Russian Institute for Public Networks K urchatov a sq. 1 Moscow , 123182, Russia plat@ripn.net ABSTRA CT In the paper the metho d for estimation of throughput met- rics lik e av ailable bandwidth and end-t-end capacity is sup- posed. This method is based on measuremen t of net work dela y D i for pac k ets o f di fferent s izes W i . The simple expres- sion for av ailable bandwidth B av = ( W 2 − W 1 ) / ( D 2 − D 1 ) is substan tiated. The num ber of exp erimen ts on matc hing of the results received new and traditional methods is spent. The received results testify to p ossibilit y of application of new model. Categories and Subject Descriptors C.2.3 [ Computer-communication net works ]: Net work Operations— network monitoring ; C.4 [ P erformance of sys- tems ]: Measuremen t techniques K eywords new model for a v ailable bandwidth, end-to-end capacit y , de- la y for pack ets of differen t sizes, RIPE T est Box 1. INTR ODUCTION Measuremen t of throughput metrics lik e av ailable bandwidth and capacit y gives a great chance to predict the end-to-end performance of applications, for dynamic path selection and traffic engineering, and select among num bers of differenti- ated classes of service. The throughput metric is an imp or- tan t metric for several applications, such as grid, video and ∗ corresponding author v oice streaming, ov erla y routing, p2p file transfers, serv er selection, and in terdomain path monitoring. V arious real-time applications in Intern et, first of all, trans- mission audio a nd video information become more and more popular, how ever for their qualitative transmission high- speed netw orks are required. The ma jor factors defining qualit y of the service are: quality of the equipmen t (the codec and a video server) and an av ailable bandwidth of the c hannel. Pro viders and their customers should provide a demanded a v ailable bandwidth for voic e and video applica- tions to guaran tee presence of demanded services in a global net work. In this paper we define a net work path as the sequence of links that forward pack ets from the path sender to the re- ceiv er. There are v arious definitions for the throughput met- rics, but we will adhere to the approaches accepted in a series of papers by Do vrolis et al [5, 10, 18]. Tw o bandwidth metrics that are commonly associated with a path are the capacit y C and the av ailable bandwidth B av . The capacit y C is the maximum IP-la y er throughput that the path can provide to a flow, when there is no comp et- ing traffic load (cross traffic). The av ailable bandwidth B av , on the other hand, is the maximum IP-la y er throughput that the path can provide to a flo w, given the path’s cur- ren t cross traffic load. The link with the minimum trans- mission rate determines the capacit y of the path, while the link with the minimum unused capacity limits the av ailable bandwidth. Measuring av ailable bandwidth is not only for kno wing the netw ork status, but also to pro vide information to net work applications on ho w to con trol their outgoing traffic and fairly share the net work bandwidth. Another related throughput metric is the Bulk-T ransfer- Capacit y (BTC). The BTC of a path in a certain time perio d is the throughput of a bulk TCP transfer, when the transfer is only limited by the net work resources and not by limita- tions at the end-systems. The intuitiv e definition of BTC is the exp ected long-term av erage data rate (bits p er sec- ond) of a single ideal TCP implement ation o ver the path in question. In order to measure different capacit y metrics, the installa- tion of sp ecial utilities [12] is required at b oth ends of path. This is uncomfortable pro cess especially for usual In ternet users who try to install modern netw ork applications lik e videoconference service. F or to day also there are the v arious systems, allowing defin- ing an a v ailable bandwidth, but they hav e the disadv antages, therefore search of new solutions is claimed. Among them, suc h as ipe rf, netp erf, p athr ate, p athlo ad and abget , and also a n umber of little-kno wn programs ncs, netest, pip e char . W e will consider the cores from the abov e describ ed products. Eac h of the pro ducts set forth ab ov e has disadv an tages. Utilities Ip erf, netp erf, p athr ate hav e one feature which is their essential disadv antage. T o estimate capacit y of a net- w ork it is required to instal client and server parts of the program. The utilit y ab get demands HTTP a serv er on the remote server and the privilege of the sup eruser, and as to programs ncs, netest, pip e char so they are not adapted for operation with netw ork screens that in mo dern conditions does their a little used. A t the same time these programs use algorithms of an es- timation of av ailable bandwidth, grounded on transmission the considerable quantit y of pack ages on a data link that reduces capacity of a netw ork suffices and demands consid- erable time. In order to construct a perfect picture of a global net work (monitoring and b ottlenecks troubleshooting) and develop the standards describing new app endices, the mo dern mea- suring infrastructure should be installed. In Russia different measuremen t pro jects are rea lized in the area of net w orking, for example, PingER [16] in Institute of Theoretical an d Ex- perimental Physics (ITEP), but full access to the collected data is limited for researchers. Unfortunately , current mea- suring area do not reflect structure of the Russian segment of a global net work. A t present time p o w erful measuremen t system lik e RIPE T est B ox is expanded [7]. Unfortunately , this system doesn’t measure the a v ailable bandwidth, but it collects the n umer- ical v alues characterized the netw ork heals like delay , jitter, routing path, etc. This data allo ws us to inv estigate the basic in terdep endencies of av ailable bandwidth from basic net work parameters. Our aim is to estimate the av ailable bandwidth from the delay v alue, received from one p oint of path. In our work we try to present the uniform model, allow- ing measuring all known throughput metrics. Our metho d is based on testing of a netw ork by pack ages of the differ- en t size. Earlier such techni que called V ariable Pac k et Size (VPS) was applied in work [6]. The VPS technique can es- timate the capacity of a hop i based on the relation betw een the Round-T rip Time (R TT) up to hop i and the probing pac ket size W . Figure 1: Pac k et Size vs Delay 2. MODEL The w ell-kno wn expression for throughput metric describing a ratio betw een a netw ork delay and the pack et size is a v ersion of the Little’s Law [13]. B = W/D (1) Here W is the size of transmitted pack et and D is the net- w orking pack et delay . This form ula is ideally for calculation of av ailable bandwidth b et ween t wo net w ork p oints that are connected immediately (in other words for distantion in one hop). In general case the delay v alue is caused by such constan t netw ork factors as propagation delay , transmission dela y , p er-pac ket router pro cessing time, etc [18]. In 1999 Do wney [6] for the first time has detected linear de- pendence of the minimum p ossible round trip time on the size of transferred pac k ets. In 2004 precise exp eriments by Choi et al [2] prov ed that the minimum fixed dela y comp o- nen t for a pac k et of size W is a line ar (or precisely , an affine ) function of its size, D f ixed ( W ) = W h X i =1 1 /C i + h X i =1 δ i (2) where C i is each link of capacit y of h hops and δ i is prop- agation delay . T o v alidate this assumption, they chec k the minim um dela y of pack ets of the same size for three path, and plot the minim um delay against the pack et size. Let D ( W ) represen ts the p oin t-to-point dela y of a pack et. Here we refer to it as the minimum path transit time for the giv en pac k et size W , denoted b y D f ixed ( W ) = min D ( W ). With the fixed delay component D f ixed ( W ) identified, w e can no w substract it from the point-to-point delay of each pac ket to study the v ariable dela y comp onent d var . The v ariable delay component of the pac k et, d var , is given by D ( W ) = D f ixed ( W ) + d var (3) On the Fig. 1 the graphic shows the linear dep endence b e- t ween av erage netw ork delay D av ( W ) = E [ D ( W )] and pac ket size W like it is constructed in pap er [2]. Slop e angle con- cerning Y axe could b e considered as av ailable bandwidth B av in contrast to b ottleneck capacit y C (maximum through- put) for computed minimal dela y D f ixed ( W ): D f ixed ( W ) = D min + W /C, (4) where D min = lim W → 0 D f ixed ( W ) (5) Prolongation of line D ( W ) from Fig. 1 to Y axe giv es the in tercept v alue a = P h i =1 δ i . Then the Equation (1) for the throughput metric which path consists of tw o or more hops should be mo dernized to the following view: B av = W / ( D av − a ) (6) The v alue a is related to the distance b etw een the sites (i.e. propagation delay) and p er-pac k et router pro cessing time at each hop along the path b etw een the sites [3, 4]. This v alue represen ts as the minim um dela y D min for which the v ery small pack age can b e transmitted on a netw ork from one p oin t in another. In the general case a ( n, l ) could b e considered as the linear function depended on n and l , a = f ( n, l ) ≈ αn + β l (7) where n is the num ber of hops (routers) that is measured b y the traceroute utilit y and l = P n l n is the sum of single length of routing path. The Equation (6) gives us the simple wa y for estimation of throughput metrics including active bandwidth B av and capacit y C . Our metho d supp oses the v ariation of pack et size on the same path for measurement of the throughput metrics. If the testing pro cess betw een t w o fixed points is organized b y pack ets with differen t sizes W 1 and W 2 then the delay times D i get tw o different v alues. Experiments should show the identical v alue for av ailable bandwidth B av independently from pack et size W i . The system from tw o equations with different v alues of v ariables D i = E [ D ( W i )] and W i is easy solv ed to find B av and a : B av = W 2 − W 1 D 2 − D 1 (8) It should b e noted that similar result was first time received for b andwidth-dominate d path in classical pap er of Jacob- son [9] dedicated congestion and a v oidance control. Fig. 2 illustrates a schematic representation of transfer of pac k ages of the differen t sizes on the slo w est link in the path (the b ottleneck). The v ertical dimension is bandwidth, the horizon tal dimension is time. Another result for capacit y C will turn out, if instead of the a verage v alue D av ( W ) in an analogue of the equation (6) C = W D f ixed ( W ) − D min (9) the minim um fixed delay comp onent D f ixed ( W ) is used C = W 2 − W 1 D f ixed ( W 2 ) − D f ixed ( W 1 ) (10) It is necessary to notice that exp erimen tal definition of an y throughput metrics demands carrying out of several mea- suremen ts for a netw ork delay . After these measurements Figure 2: Av ailable Bandwidth Illustration are sp ent for pack ages of the differen t sizes, it is necessary to choose from them the minim um and av erage v alues. The minim um v alue will b e used for calculation of av ailable band- width B av , and a verage v alue for capacit y C . Even in work of Downey [6] it w as noticed that are many data p oin ts near the minim um and we can find the minimum delay D min with a small n umber of prob es at each pack et size. It should b e noted that the method presented in given work allows mea- suring the av ailable bandwidth and capacity of the outgoing c hannel. The minimal delay of datagram transmission D min ma y b e calculated as D min = W 2 D 1 − W 1 D 2 W 2 − W 1 (11) This v alue as well as the metho ds of its measuremen t has a important significance in applied tasks of con trol theory [19]. The second significan t question of n etw orking con trol theory is the distribution type for v ariable dela y comp onent d var whic h should b e studied. T o know the expression for this parameter we may easy calculate the duration of buffer for streaming aplication on receiving side. 3. PRECISE EXPERIMENTS A n um b er of measurements in a global netw ork hav e been spent for ackno wledgemen t of our metho d. In this work the v ery first results which are already pro cessed are presented only . F or practical realization of our metho d the sizes W 1 and W 2 should different in sev eral times, it is reasonable to choose 64 and 1064 bytes for Lin ux based systems, 32 and 1032 b ytes for Windows correspondingly . The basic problem of experimental testing is the precise of delay measuremen ts that is necessary for accurate result. The exact metering demands micro second precision for delay measuremen ts; w e are reaching such accuracy with help of RIPE T est Box mec hanism [17]. In order to prepare the exp erimen ts three T est Bo xes ha ve been installed in Mosco w, Samara and Ros- to v on Don during 2006-2008 years in framework of RFBR gran t 06-07-89074. Each RIPE T est Box represents a serv er under management of an F reeBSD operating system with the GPS receiv er connected to it. Characteristic times of inv estigated pro cesses (a pack et de- la y , jitter) hav e the order from 10 ms to 1 sec , therefore is quite enough accuracy of system hours of a RIPE T est Box for th eir reliable measuremen t. At the first stage experiment betw een tt01.rip e.net (RIPE NCC at AMS-IX, Amsterdam) and tt143.rip e.net (Samara, SSA U) hav e b een made whic h included • Precision measuremen t of pac ket delay in the size 100 and 1100 b ytes with accuracy 2-12 µs • Measuremen t of av ailable bandwidth by means of util- it y ip erf [12] • Measuremen t of bandwidth by a metho d of do wnload- ing of a file on FTP Th us, at us it will be generated al ternatively measured three sizes of throughput metrics for the subsequent comparative analysis. It is necessary to notice that the utility ip erf is started with an option -u and measures sp eed of a stream b etw een t wo p oin ts that precisely enough corresponds to av ailable bandwidth. Sp eed of downloading on ftp measures a Bulk- T ransfer-Capacit y (BTC) and gives strongly underestimated v alue. Unfortunately , at the given stage w e could not spend more exact measuremen ts, but further we assume to find partners for installation of exact utilities. The design of the RI PE TTM system meets all requiremen ts sho wn by our method, namely it allows to ch ange the size of a testing pack age and to find netw ork dela y with a split-hair accuracy . By default, testing is conducted by pack ages in the size of 100 byte, but there is a page corresp onding to p oin t of the men u ´ nConfiguration ˙ z of local T est Box. On whic h it is possible to add testing pac k ages to RIPE Box up to 1500 b yte in size with demanded frequency . In our case it is reasonable to add testing 1100 (1024) byte pac k ages with frequency of 60 times in a minute. It is nec- essary to notice that the results of tests will b e av ailable on next da y . T esting results are av ailable in telnet to RIPE T est Box on port 9142. It is important to come and write down simulta- neously the data on both ends of the in vestigated c hannel, in the case present ed here it is tt01.rip e.net and tt143.ripe.net. Obtained data will contain required dela y of pack ages of the differen t sizes. Also, we need to distinguish pack ages. Therefore at first it is reversible to sending Bo x and we will find lines, see T able 1. Last v alue in string is sequence num ber of the pack et. It is necessary to us to find this num ber on the receiving side at the channel . The string example on the receiving side is lo wer resulted, see T able 2. F or set n um b er of a pack age it is easy to find netw ork delay , in our case it ma kes 0.044084 seconds. The follo wing pac k age 1039148557 has the size of 100 bytes and its delay makes 0.043591 seconds. Th us, the difference will make 0.000493 second. Our mo del assumes operations with minimal and a v erage v alues; therefore w e should note a verage v alues, not less than fiv e pairs for the delay , going consistently . In our case, a v- erage difference E [ D (1024) − E [ D (100) is 0.000571 seconds. (tt143 - > tt01). Then the required bandwidth of the link (tt143 - > tt01) can be calculated as B av ( tt 143 → tt 01) = 924 × 8 0 . 000571 = 12 . 9[ M bps ] (12) The minimal and av erage v alues of the return link (tt01 - > tt143) are E [ D (1024)] − E [ D (100)] = 0 . 000511 second and D f ixed (1024) − D f ixed (100) = 0 . 000492 second/ Then a v ailable bandwidth and capacity can be calculated as C ( tt 01 → tt 143) = 924 × 8 0 . 000492 = 15 . 0[ M bps ] (13) B av ( tt 01 → tt 143) = 924 × 8 0 . 000511 = 14 . 7[ M bps ] (14) The main problem of the offered method consists in under- standing, what v alue is measured. Actually , it can b e bulk transport capacit y or a v ailable bandwidth . Alternative mea- suremen ts of the giv en v alues are necessary fo r sp ecification. It is ideal to compare the width receiv ed by our method to the v alues measured b y alternativ e metho ds, first of all by means of the utility iperf. Unfortunately , such tests are not spent y et, w e allo cate only in the speed of FTP do wnloading. It makes 3.04 - 3.20 Mbps in a direction from tt143.ripe.net to tt01.rip e.net and 3.2-3.3 Mbps in the opp osite direction. That is additional researc hes for whic h carrying out partners are required are necessary . It should be noted that T able II from pap er [2] giv es us these v alues; calculated slop e is inv erse v alue to end-to-end capacit y . The corresp onding capacities for data set 1, 2, 3 (path 1 and 2) are 285 Mbps, 128 Mbps, 222 Mbps and 205 Mbps . 4. A VB AND UTILITY Routinely the sp ecial utilities could b e used for delay mea- suremen ts; we tried to test traditional ping , the new UDP- ping and other utility . In result of test the simplest utilit y ping w as found to be a b est choice for delay measuremen ts. Utilit y AvB and (Av ailable Bandwidth) has been developed, realizing the abov e describ ed method, using in the basis al- gorithm ping. This algorithm has been dev eloped b y Mike Muus in 1983 in the USA for op erating system BSD [14]. Its adv antage consists that it is possible to wo rk with an y router or the host whic h resp onds to pac k ages of inquiries ICMP Ec ho. The giv en v ersion of t he utilit y is dev eloped for platform Windo ws and uses library ICMP Windo ws API. In the near future we plan working out of the utility for Unix systems, first of all for family Lin ux. The giv en utilit y defines av ailable bandwidth of outgoing c hannel betw een host from whic h measuremen t and a remote SNDP 9 1240234684 -h tt01.ripe.net -p 6000 -n 1024 -s 1039148464 SNDP 9 1240234685 -h tt164.ripe.net -p 6000 -n 100 -s 1039148548 SNDP 9 1240234685 -h tt01.ripe.net -p 6000 -n 100 -s 1039148557 T able 1: The data of sending b o x RCDP 12 2 89.186.245.200 60322 193.0.0.228 6000 1240234684.785799 0.044084 0X2107 0X2107 1039148464 0.000002 0.000008 RCDP 12 2 89.186.245.200 53571 193.0.0.228 6000 1240234685.788367 0.043591 0X2107 0X2107 1039148557 0.000002 0.000008 T able 2: The data of receivig b o x Figure 3: The AvBand Screenshot serv er in teresting us is sp ent. F or this purpose the program measures R TT (Round T rip Time) that is the time b etw een sending of inquiry and answer reception. Thus at first pack- ages in 32 bytes (standard Windows size) are generated and their R TT is defined, and the following step forms pac k ages of the size in 1032 b ytes and is measured their R TT. On Fig. 3 the screenshot of the program is presen ted. In the field “Host” it is en tered a host name, av ailable band- width to which we are going to measure. In the field “Re- tries” the quan tity of the echo-inquiries which will b e sent on a remote host is underlined. After that enough to press button “Start” and the utility will send the set quan tity of pac k ages of the size of 32 b ytes, further the same quan tit y of pac k ages in the size of 1032 b ytes. The collected v alues of the receiv ed delays on each of groups of pack ages are av eraged, and then by means of our mo del the av ailable bandwidth of the c hannel pa ys off and is displa y ed. It is necessary to notice that the av ailable bandwidth of the outgoing c hannel is measured. F or chec k of utility AvBand a series of exp eriences with use of follo wing measuring mechanism s also has b een spent: • Utilit y AvBand • Standard ping • Ip erf • FTP Measuremen ts with Samara State Aerospace Univ ersity (SSA U), Institute of Organic Chemistry of the Russian Academy of Sciences (IOC RAS), con trol cent re RIPE in Amsterdam (RIPE), Ohio State Unive rsity (OSU), and also a num ber of local experiments with use of netw orks of v arious Inter- net Service Providers of the Samara region (Infolada, AIST, V olgaT elecom, etc.) ha ve b een curren tly sp en t. All data on experiments is resulted in the table more lo w. As a case in p oin t ADSL connection in Samara region could be c hosen for illustration of our approach. The delay mea- suremen ts give D 1 = 18 ms , D 2 = 42 ms , that corresponds to 350 Kbps of av ailable bandwidth. During FTP session the dela y gro ws to 300 ms and 425 ms that corresponds appro ximately to 60 Kbps of av ailable bandwidth. This is v ery rough computation, but it could b e made quickly and independently . 5. CONCLUSION No w measurements are not completed yet, is planned to t yp e the data from not less than 50 p oin ts scattered on ter- ritory of a planet. F rom these measuremen ts not less than 10 should b e fulfilled with application of RIPE T est Boxes. Th us, summing up to the done op eration, it is p ossible to dra w the main output: the theoretical mo del of calculation of an a v ailable bandwidth pro v es to b e true. F urther it is planned to contin ue researches to establish t yp e of distribution for a net w ork delay . At definition of t yp e of distribution it is supp osed to use analogy to molec- ular physics, namely ab out distribution of molecules in the speeds Maxswell. Probably , in our case required distribu- tion should be presen ted in the form of pro duct of normally (Gaussian) distribution and the inv erse function defined by the Equation 6. The kno wledge of density of distribution in TCP/IP netw orks will help to find a new class the decision in the net work ed con trol systems. In summary we w ould like to express sp ecial gratitude of Prasad Calyam and Gregg T rueb from Ohio State Univer- sit y for the in v aluable help at carrying out of measuremen ts. Also it w ould b e desirable to thank all collectiv e of tec hnical service RIPE ncc and esp ecially Rub en v an Stav eren and Roman Kaly akin for constant assistance in comprehension of subtleties of a measuring infrastructure. 6. REFERENCES [1] Ben F redj, S., Bonald, T., Proutiere, A., Regnie, G., Roberts, J.: Statistical Bandwidth Sharing: A Study of Congestion at Flo w Level. In: ACM SIGCOMM (2001) [2] Choi, B.-Y., Moon, S., Zhang, Z.-L., Papagiannaki, K. and Diot, C.: Analysis of Poi nt-T o-P oin t Pac k et Dela y Hosts Av ailable bandwidth testing remote ping or FTP Ip erf serv er host AvB and SSA U IOC RAS 20-20.6 17.6-27.4 Mbps Mbps SSA U serv er2.hosting.reg.ru 1150 1140 Kbps Kbps OSU SSA U 2500 2450 Kbps Kbps AIST SSAU 536 600 659 Kbps Kbps Kbps Infolada SSA U 346 374 Kbps Kbps V olgaT elecom SSAU 274 283 Kbps Kbps T able 3: Exp erimen tal results In an Operational Netw ork. In: Info com 2004, Hong Kong, pp. 1797-1807 (2004) [3] Cottrell, L., Matthews, W. and Logg C.: T utorial on In ternet Monitoring & PingER at SLAC. h ttp://www.slac.stanford.edu/comp/net/w an- mon/tutorial.h tml [4] Cro vella, M.E. and Carter, R.L.: Dynamic Server Selection in the In ternet. In: Pro c. of the Third IEEE W orkshop on the Arc hitecture and Implementat ion of High P erformance Communicati on Subsystems (1995) [5] Do vrolis C., Ramanathan P ., and Mo ore D., P ack et-Dispersion T echniques and a Capacit y-Estimation Methodology , IEEE/ACM TRANSA CTIONS ON NETWORKING, VOL. 12, NO. 6, DECEMBER 2004, p. 963-977 [6] Do wney A.B., Using Pathc har to estimate in ternet link c haracteristics, in Pro c. A CM SICCOMM, Sept. 1999, pp. 222 ˝ U223. [7] Georgatos, F., Gruber, F., Karrenberg, D., Santcroos, M., Susanj, A., Uijterw aal, H. and Wilhelm R., Pro viding active measurements as a regular service for ISP’s. In: P AM2001 [8] Guo jun, J.: Av ailable Bandwidth Measuremen t and Sampling, h ttp://www.caida.org/w orkshops/isma/0312/abstracts/guo jun.pdf [9] Jacobson, V. Congestion a voida nce and control. In Proceedings of SIGCOMM 88 (Stanford, CA, Aug. 1988), A CM [10] Jain, M., Do vrolis, K.: End-to-end Estimation of the Av ailable Bandwidth V ariation Range. In: SIGMETRICS’05, Banff, Alberta, Canada (2005) [11] H.323 Beacon T ool, h ttp://www.osc.edu/net working/itecoh io.net/b eacon/ [12] Iperf, dast.nlanr.net/Pro jects/Iperf/ [13] Kleinrock, L. Queueing Systems, v ol. II. John Wiley & Sons, 1976. [14] Mik e Muus, Ping do cumen tation, h ttp://ftp.arl.mil/ ∼ mik e/ping.html [15] P adhy e, J., Firoiu, V., T o wsley , D., Kurose, J.: Modeling TCP Throughput: A Simple Mo del and its Empirical V alidation. In: Proc. SIGCOMM Symp. Comm unications Archit ectures and Proto cols, pp. 304-314 (1998) [16] PingER, h ttp://www-iepm.slac.stanford.edu/pinger/ [17] Ripe T est Bo x, http://ripe.net/pro jects/ttm/ [18] Prasad R.S., Do vrolis C., and B. A. Mah B.A., The effect of la yer-2 storeand-forward devices on per-hop capacit y estimation, in Pro c. IEEE INF OCOM, Mar. 2003, pp. 2090 ˝ U2100. [19] Zhang, W., Branic ky , M.S., Phillips S.M.: Stability of Net work ed Con trol Systems. In: IEEE Control System Magazine, 21(1), pp. 84-99 (2001)