Non-Cooperative Game Theory Based Rate Adaptation for Dynamic Video Streaming over HTTP

1 Non-Cooperativ e Game Theor y Based Rate Adaptation f or Dynamic Video Streaming ov er HTTP Hui Y uan, Senior Member , IEEE, Hua yong Fu, Ju Liu, Senior Member , IEEE, Junhui Hou, Member , IEEE, and Sam Kwong, F ellow , IEEE Abstract —Dynamic Adaptiv e Streaming o ver HTTP (D ASH) has demonstrated to be an emerging and promising multimedia streaming technique, owing to its capability of dealing with the v ariability of networ ks. Rate adaptation mechanism, a challenging and open issue, plays an important role in DASH based systems since it aff ects Quality of Experience ( QoE ) of users, network utilization, etc. In this paper , based on non-cooperative game theory , we propose a nov el algorithm to optimally allocate the limited export bandwidth of the server to multi-users to maximize their QoE with f air ness guaranteed. The proposed algorithm is proxy-free . Specifically , a nov el user QoE model is derived b y taking a variety of factors into account, like the receiv ed video quality , the reference b uffer length, and user accumulated buff er lengths, etc. Then, the bandwidth competing problem is f or mulated as a non-cooperation game with the e xistence of Nash Equilibrium that is theoretically proven. Finally , a distr ibuted iterativ e algor ithm with stability analysis is proposed to find the Nash Equilibrium. Compared with state-of-the-ar t methods, extensiv e experimental results in terms of both simulated and realistic networking scenarios demonstrate that the proposed algorithm can produce higher QoE , and the actual buff er lengths of all users keep nearly optimal states, i.e., moving around the ref erence buff er all the time. Besides, the proposed algorithm produces no playbac k interruption. Index T erms —Non-cooperative Game, Nash Equilibrium, D ASH, Bitrate Adaptation, QoE. F 1 I N T R O D U C T I O N N O WA D A Y S , with the incr ease of Internet bandwidth and the tremendous gr owth of web platforms, Hy- pertext T ransfer Protocol (HTTP) str eaming has become a cost-effective method for multimedia delivery [1][2]. Dy- namic Adaptive Streaming over HTTP (DASH) is a typical HTTP based multimedia delivery standar d that can transmit multimedia content adaptively between multimedia servers and users with a limited and varied network bandwidth [3]. Fig. 1 illustrates a typical DASH-based video delivery system. In this system, the media content (e.g., videos or audios) is first divided into multiple segments (or chunks) [4] with the same display time. Each segment is then encoded/transcoded with differ ent bitrates (corresponding to different quality levels). At the same time, the server generates a Media Presentation Description (MPD) file that recor ds the information of the available video content, e.g., URL addresses, segment lengths, quality levels, resolutions, etc. The users first download the MPD file from the server using HTTP protocol, and then request segments with differ ent quality levels to adapt to the bandwidth varia- This work was supported in part by the National Natural Science Foundation of China under Grants 61571274,61672443; in part by the Shandong Natural Science Funds for Distinguished Y oung Scholar under Grant JQ201614; in part by Hong Kong RGC General Research Fund(GRF) under Grant 9042322 (CityU 11200116); and in part by the Y oung Scholars Program of Shandong University (YSPSDU) under Grant 2015WLJH39. • H. Y uan, H. Fu, and J. Liu are with the School of Information Science and Engineering, Shandong University , Ji’nan 250100, China. Email: huiyuan@sdu.edu.cn, johy .fu@gmail.com, juliu@sdu.edu.cn • J. Hou and S. Kwong are with the Department of Computer Science, City University of Hong Kong, Kowloon, Hong Kong. Email: jh.hou@cityu.edu.hk, cssamk@cityu.edu.hk tion. The main advantage of DASH is that it can achieve bandwidth adaptation and reduce the number of playback interruptions under fluctuating network conditions [5]. An effective rate adaptation algorithm is necessary in a DASH system, with which the DASH user can adaptively request video segments with dif ferent bitrates based on the network condition and its buffer length. However , this challenging issue is not specified in the DASH standard. W ithout an effective rate adaptation algorithm, the DASH user might suffer from frequent interruptions. Moreover , recent studies show that the DASH user ’s selfish behavior (i.e., making requests without considering other users sharing the net- work resources) will result in network underutilization (or congestion), fluctuating and unfair throughout allocation [6]. This paper aims to develop an effective rate adaptation algorithm to address the above issues. Many rate adaptation methods have been proposed (see Section II). However , most of them optimize the HTTP streaming of multiple DASH users sharing the same net- work resour ces separately , regar dless of the influence be- tween each other; thus, user fairness cannot be well guaran- teed. In contrast, the proposed rate adaptation algorithm optimizes the HTTP str eaming of multiple DASH users simultaneously who compete for higher quality video seg- ments fr om a single server with a limited export bandwidth . As a branch of game theory , the non-cooperative game theory [7] can resolve the conflicts among interacting play- ers involved in a certain game, in which each player behaves selfishly to optimize its own profit usually quantified as an objective function. The non-cooperative game can provide meaningful solutions for many applications wher e the inter - 2 Se gment Se g m e n t Se g m e n t Se g m e n t S e gm e n t S e gm e n t S e gm e n t S e gm e n t S e gm e n t S e gm e n t S e gm e n t S e gm e n t S e gm e n t Se gm e n t Se gm e n t S e gm e n t Se gment Se gment Se gment Se gment H TT P S e rver Media Pres entation Description (MPD) DA S H User MPD Parser Media Pla ye r Segment Parser Connection Manager DASH S treaming Control Low Bitrate High Bit r ate Fig. 1. D ASH system architecture. action among several players is negligible and centralized approaches are not suitable [8][9]. The typical application of a non-cooperative game is the oligopoly market problem in economics, in which all corporations compete for market share of the same commodity in or der to maximize their own profits [10], and the market share of each corporation tends to be stable and reaches Nash Equilibrium. In this paper , we consider formulating the rate adapta- tion problem for improving user QoE , as well as preserving user fairness, as a non-cooperative game in which DASH users try to consume the limited export bandwidth of the server as much as possible to maximize their profits (i.e., QoE ). The optimal bitrate that pr oduces the optimal QoE can be obtained when the Nash Equilibrium of the contradiction problem is achieved. More specifically , the proposed non- cooperative game theory based rate adaptation algorithm determines the requested bitrate for a DASH user based on both the local information (e.g., the requested bitrate of the last segment, reference buffer length, and current buffer length) and the global payoff variation information obtained from the server . Each user can gradually adapt the requested bitrate to convergence. The bitrate’s convergence speed is controlled by the learning rate. Note that there is a limitation that additional HTTP sessions between users and servers are needed in the proposed method, which may reduce the transmission efficiency of the streaming system. Even so, extensive experimental results demonstrate that the proposed algorithm can produce higher QoE , while the actual buffer lengths of all users move around the reference buffer all the time when compared with state-of-the-art algo- rithms. Moreover , the proposed algorithm is proxy-fr ee and playback interruption-free. T o the best of our knowledge, this is the first time to address the DASH rate adaptation problem using the non-cooperative game theory . The major contributions of this paper are summarized as follows. • W e formulated the rate adaptation problem as a non- cooperative game with the existence of the Nash Equilibrium that is theoretically proven. The pro- posed rate adaptation algorithm optimizes streaming for multiple DASH users simultaneously to guaran- tee their fairness and improve their QoE . • W e proposed a novel QoE model for the DASH users by taking the current buffer length, reference buffer length, and video quality into account. • W e designed an efficient distributed iterative algo- rithm to obtain the Nash Equilibrium of the game by additional HTTP sessions between the server and users, the stability of which is theoretically analyzed. The rest of this paper is organized as follows. In Section II, the related work on rate adaptation methods for DASH is presented. In Section III, a user QoE model is proposed, and the corr esponding non-cooperative game is formulated. Besides that, the existence of Nash Equilibrium and the stability of the non-cooperative game ar e also demonstrated. Simulation and realistic experimental results are given in Section IV and V , respectively . Finally , Section VI concludes the paper and discusses the limitations of the proposed method. 2 R E L A T E D W O R K In order to adapt to the varying bandwidths, a straight- forward method is to estimate the bandwidth or through- put of the transmission link. Thang et al . [11] proposed a channel throughput estimation-based adaptive request method to deal with short-term bandwidth fluctuations and stabilize the bitrates of segments. Romero [12] developed a Java client for HTTP streaming on the Android platform and proposed a smoothed throughput estimation method to cope with short-term fluctuations. But the user ’s buffer length (evaluated by remaining playback time) has not been considered. In [13], a r ound-trip time and pr evious values of the instant throughput based throughput estimation method is pr oposed for adaptive str eaming so as to stabilize both the bitrates of segments and the buffer length of users. Liu et al . [14] developed a throughput estimation based rate adap- tation method by using the ratio of the expected segment fetch time (ESFT) and the measured segment fetch time. However , in practical applications, since bandwidth and throughput are affected by a lot of factors, it is a non-trivial task to estimate them accurately . Huang et al . [15] showed that inaccurate throughput estimation at the user side can cause the degeneration of the video quality . Recently , Mao et al . [16] proposed a deep reinforcement learning based rate adaptation algorithm by accurately estimating the channel throughput accurately . Besides, in order to improve the QoE [17][18] of users directly , some resear chers proposed dynamic bitrate selec- tion methods based on QoE maximization [19]-[22]. Zhang et al . [19] proposed a buffer management-based QoE model for HTTP adaptive bitrate streaming and formulated the adaptive request mechanisms as a constrained convex op- timization problem which is then solved by the Lagrange multiplier method. Gheibi et al . [20] proposed a QoE metric by considering the probability of interruption in media playback and the number of initial buffered packets (initial waiting time) for streaming media applications. However , QoE is not only influenced by buffer length, but also by the requested bitrate and bitrate switching frequency , etc. Xu et al . [21] modeled the user QoE as a combination of bitrate, starvation probability of playback buffer , and continuous playback time, and they proposed two bitrate switching algorithms based on the channel variation and buffer length. Rodr ´ ıguez et al . [22] proposed a non-refer ence QoE metric for DASH by taking initial buffer delay , temporal playback interruptions, and video resolution changes into account. In 3 addition, a Markov decision-based rate adaptation scheme for DASH aiming to maximize the user QoE under time- varying channel conditions was proposed by Zhou et al . [23] in which the video quality level, bitrate switching frequency and amplitude, buffer length, etc. are considered comprehensively . Similarly , Mart´ ın et al . [24] proposed a Q - Learning-based bitrate request method to efficiently control the selection of the segment quality by diminishing the quality switches and the occurrence of playback interrup- tion. Bokani et al . [25] proposed another Markov Decision Process-based rate adaptation method based on Q -Learning to gradually learn the optimal decisions in order to avoid playback interruption, which has been found as the most important factor affecting user QoE . It is worth noting that all of the abovementioned meth- ods are designed based on the assumption that multiple DASH users make their rate adaptation decisions separately . In practical applications, it is more common that multi-users request multimedia content simultaneously from a single server or a relay server of a cell. Since the export bandwidth of the server is limited, a natural problem concerns how to allocate the limited bandwidth jointly to users so as to improve the performance of the whole system, especially to guarantee the fairness of multi-users. In [26], Jiang et al . proposed an optimized bandwidth estimator based on the mean of the pr evious bandwidths for multi-users to increase bandwidth utilization and stabilize the buffer lengths of multi-users. Li et al . [27] demonstrated that the discr ete nature of the video bitrates leads to video bitrate oscilla- tion that negatively affects the video viewing experience and presented a probe-and-adapt bandwidth estimation approach to further increase the bandwidth utilization and stabilize the requested video bitrates of multi-users. Essaili et al . [28] proposed QoE -maximization based traf fic and resour ce management in a mobile network for multi-user adaptive HTTP streaming. However , a proxy is needed to intercept and rewrite the user HTTP r equests in this method, which increases the system’s complexity and the channel information for each user is based on the average channel statistics in the previous second that may not be accurate enough. 3 P RO P O S E D R A T E A D A P TA T I O N A L G O R I T H M As shown in Fig. 2, in a DASH-based video delivery system, multi-users compete the limited export bandwidth of the server , and the information (i.e., the requested bitrate of the last segment and the current buffer length) of users is unknown by each other . The DASH users first send their current buffer lengths to the server , and then request video segments based on the payoff variation information that is calculated by the server based on the server export band- width and the buffer information of each user . W e formulate the rate adaptation algorithm into a non-cooperative game as follows. The players in the game are the DASH users. The strategy of each player is the requested bitrate (denoted by r i for the i -th user). The payoff or profit for the i - th user (denoted by U i ) is its QoE determined by the requested video and accumulated buf fer . The commodity of the competition is the video segments encoded into differ ent DASH Serv er DASH Users Fig. 2. Scenario for adaptiv e HTTP video deliver y . bitrates. The solution of this game is Nash Equilibrium. Let G = { I , { R i } , { U i ( · ) }} denote the non-cooperative rate adaptation game where I = { 1 , 2 , · · · , N } is the index set for the users in the DASH system, and R i and U i ( · ) are the strategy space and utility function of the i -th user , re- spectively . Each user determines the required bitrate r i such that r i ∈ R i . Let the rate vector r = { r 1 , · · · , r i , · · · , r N } denote the outcome of the game in terms of the requested bitrates of all users. The resulting utility for the i -th user is U i ( r ) . W e will occasionally use U i ( r i , r − i ) to replace U i ( r ) to indicate the dependence among DASH users, where r − i denotes the vector that consists of elements of r without the i -th element, i.e., r − i = { r 1 , · · · , r i − 1 , r i +1 , · · · , r N } . In the following subsections, we first propose a novel QoE model for DASH users by taking the current buffer length, refer ence buffer length, and video quality into ac- count. Then, the existence of Nash Equilibrium of the non- cooperative game is theoretically proven. Finally , a dis- tributed iterative algorithm with stability analysis is pro- posed to find the Nash Equilibrium. 3.1 Modeling of D ASH User QoE In a DASH based video delivery system, the user QoE depends on both the qualities of received video segments and playback interruptions. 1) V ideo quality: The video quality is directly determined by the bitrate of the requested video. Although there are a lot of video quality-bitrate models, most of them can be uniformly repr esented as a logarithmic function of bitrate [29], i.e., q i ( r i ) = α i log(1 + β i r i ) , (1) where q i is the quality of the received video segment mea- sured in peak-signal-noise-ratio (PSNR), str ucture similarity index metric (SSIM) [30], etc., and α i and β i are parameters depending on video content. Therefore, without loss of generality , Eq. (1) is used to evaluate the quality of received video segments in the proposed QoE model. 2) Playback interruption: W e evaluate the influence of playback interruptions on user QoEs by explicitly modeling the relationship between the estimated buffer length and requested video bitrate. Usually , for the i -th user , the buffer variation can be calculated as the difference between the cumulated buffer length (i.e., the remaining video playback time) caused by the downloaded video segment and the 4 Fig. 3. Illustration of the relationship between the actual download time and estimated download time using T · P N j =1 r j /B W of each user. Here, three users with varied channel throughputs compete the fixed export bandwidth of a server , and the correlation coefficient is 0.8658. consumed buffer length caused by the played video content during the download time [19][31], i.e., ∆ b i ( r i ) = T − T · r i /B i W , (2) where ∆ b i ( r i ) is the buffer variation caused by the re- quested video bitrate r i , T is the length of a video segment, and B i W is the available channel bandwidth of the i -th user . Unfortunately , as aforementioned, the channel state of each user is unknown, resulting in Eq. (2) being worthless. But the export bandwidth of the server is known to users. When the summation of the requested video bitrates of all users is larger than the export bandwidth of the server , the down- load time of all users will increase since the throughput of the system increases. Therefor e, we use the total requested video bitrates of all the users over the export bandwidth of the server to estimate the download time for each user , and the average buf fer variation of all users in the system is ∆ b ( r ) = T − ω · T · X N i =1 r i /B W , (3) where ω is a coefficient, and B W is the export bandwidth of a server . Such a simple approximation can facilitate the following theoretical analysis with reasonable accuracy . For a single user , the relationship between the actual download time and the estimated download time may not exactly be linear , as shown in ” User1 ”, ” User2 ”, and ” User3 ” in Fig. 3. But for all three users, the overall relationship between the actual download time and estimated download time of all the users can be modeled using a pr oportional function with the correlation coefficient of 0.8658. Furthermore, after the curr ent video segment was down- loaded, the estimated buffer length of the i -th user consid- ering other users denoted as b est i ( r i , r − i ) can be derived by integrating ∆ b ( r ) over r i : b est i ( r i , r − i ) = Z ∆ b ( r )d r i = T · r i − ω · T · 1 2 r 2 i + r i X N j =1 j 6 = i r j ! /B W + b 0 = Φ( r i ) − ω · Ψ( r ) + b 0 , (4) where b 0 is a constant, denoting the initial average buffer length of all the users, Φ( r i ) = T · r i is the bene- fit gained from accumulated buffer , and Ψ( r ) = T ·  1 2 r 2 i + r i P N j =1 j 6 = i r j  /B W repr esents the system penalty (buffer consumption) caused by the requested bitrates of all the users. In or der to ensur e the buffer is equipped with an optimal state, i.e., the buffer length should be kept within a certain refer ence level, an adjustment factor (denoted as A f ) is adopted to modify the revenue function Φ( r i ) , i.e., Φ 0 ( r i ) = A f · Φ( r i ) = A f · T · r i . (5) Considering that a larger (resp. smaller) b curr indicates more aggressive (resp. defensive) behaviors in requesting video segments, A f is defined as a monotonically increasing function of the difference between the current buffer length b curr and the reference buffer length b ref [31]: A f = 2 · e p ( b curr − b ref ) 1 + e p ( b curr − b ref ) , (6) where p > 0 is a constant, b ref is the predefined refer ence buffer length, and b curr is the current known buffer length before the video segment was downloaded for a certain user , respectively . The differ ence between b ref and b curr is an in- dicator to control the requested bitrates. When b curr > b ref , A f is larger than 1, it indicates that the previously received video quality may be low . Then, the bitrates requested by users will be increased to achieve the optimal utility . When b curr < b ref , A f is smaller than 1. This indicates that the probability of playback interruption is large. Then, the bitrates requested by users will be decreased to achieve the optimal utility . Accordingly , the estimated buffer of the i -th user with respect to all the other users can be rewritten as b est i ( r i , r − i ) = Φ 0 ( r i ) − ω · Ψ( r ) + b 0 = 2 · e p ( b curr − b ref ) 1 + e p ( b curr − b ref ) · T · r i − ω · T · 1 2 r 2 i + r i X N j =1 j 6 = i r j ! /B W + b 0 . (7) Finally , we define the utility function of user i that considers the influence on other DASH users sharing the same network resour ces as the linear combination of the quality function in (1) and the buffer function in (7), i.e., U i ( r i , r − i ) = q i ( r i ) + µ · b est i ( r i , r − i ) = α i log(1 + β i r i ) + µ 2 · e p ( b curr − b ref ) 1 + e p ( b curr − b ref ) · T · r i ! − µ · ω · T · 1 2 r 2 i + r i X N j =1 j 6 = i r j ! /B W + µ · b 0 = α i log(1 + β i r i ) + µ 2 · e p ( b curr − b ref ) 1 + e p ( b curr − b ref ) · T · r i ! − ν · T · 1 2 r 2 i + r i X N j =1 j 6 = i r j ! /B W + µ · b 0 , (8) 5 where µ > 0 is a weight to balance the two parts, and ν = µ · ω . 3.2 Pr oof of the Existence of Nash Equilibrium The Nash Equilibrium of a game is a strategy pr ofile with the property in which no player can increase its utility by choosing a differ ent action when the other players’ actions are given [32]. A Nash Equilibrium exists for a game G if the following two conditions are met: a) the strategy space R i is a non-empty , convex, and compact subset of Euclidean space R N ; b) the utility U i ( r ) is continuous in r and (at least) quasi- concave in r i . For the proposed non-cooperative game, the strategy space is composed of the user requested bitrates with the range of [0 , r max ] , where r max is the maximum requested video bitrate of the i -th user . Thereby , there is no doubt that R i is a non-empty and compact subset of Euclidean space R N . According to the definition of the convex set [33], for any r x , r y ∈ R i and any ζ with 0 ≤ ζ ≤ 1 , we have 0 ≤ ζ r x ≤ ζ r max and 0 ≤ (1 − ζ ) r y ≤ (1 − ζ ) r max . Then, we can get 0 ≤ ζ r x + (1 − ζ ) r y ≤ r max . Therefor e, ζ r x + (1 − ζ ) r y ∈ R i , indicating R i is a convex set. Thus, condition a) is satisfied. For the utility function in (8), it is obviously a continuous function in terms of r . Besides, the second derivatives of U i ( r ) with respect to all the r i are    ∂ 2 U i ∂ r 2 i = − αβ 2 (1+ β r i ) 2 − ν · T B W , ∂ 2 U i ∂ r i ∂ r j = − ν · T B W | i 6 = j , (9) which are negative because α , β , ν , T , and B W are non- negative. Accordingly , the utility U i ( r ) is a strictly concave function of r i (for all i ) [33]. Therefor e, there must exist a Nash Equilibrium in the proposed rate adaptation game. For a non-cooperative game, the Nash Equilibrium can be achieved by jointly maximizing the utility functions of all players, i.e., the corresponding best response function of a player is defined as the strategy of the player with those of other players fixed: B i ( r − i ) = arg max r i ∈ R i U i ( r i , r − i ) . (10) When the Nash Equilibrium is achieved, the strategies of all players can be repr esented as r ∗ = { r ∗ 1 , r ∗ 2 , · · · , r ∗ N } , where r ∗ i = B i ( r ∗ − i ) is the optimal strategy of the i -th user and r ∗ − i = { r ∗ 1 , · · · , r ∗ i − 1 , r ∗ i +1 , · · · , r ∗ N , } is the set of the Nash Equilibrium of all users except user i . 3.3 Distrib uted Iterative Algorithm for Nash Equilibrium Theoretically , the Nash Equilibrium can be obtained by solving the following equations: ∂ U i ( r ) ∂ r i = α i β i 1 + β i r i + µ · T · 2 e p ( b curr − b ref ) 1 + e p ( b curr − b ref ) − ν · T · P N j =1 r j B W = 0 , ∀ i. (11) T aking a DASH system with only 2 users as an example, we have, ( Z 1 , 1 1+ β 1 r 1 + Z 2 , 1 − Z 3 ( r 1 + r 2 ) = 0 , Z 1 , 2 1+ β 2 r 2 + Z 2 , 2 − Z 3 ( r 1 + r 2 ) = 0 , (12) where        Z 1 , 1 = α 1 β 1 , Z 2 , 1 = µ · T · 2 e p ( b curr, 1 − b ref ) 1+ e p ( b curr, 1 − b ref ) , Z 1 , 2 = α 2 β 2 , Z 2 , 2 = µ · T · 2 e p ( b curr, 2 − b ref ) 1+ e p ( b curr, 2 − b ref ) , Z 3 = ν · T /B W . (13) Assume the two users request the same video and have the same channel condition (i.e., two identical users), we have Z 1 , 1 = Z 1 , 2 = Z 1 and Z 2 , 1 = Z 2 , 2 = Z 2 . Then, the Nash Equilibrium can be expressed as r ∗ 1 = r ∗ 2 = − (2 Z 3 − β i Z 2 ) + p (2 Z 3 + β i Z 2 ) 2 + 8 β i Z 1 Z 3 4 β i Z 3 . (14) From the above analysis, to determine the requested video bitrate for a certain user , the strategies of the other users must be available. However , such user strategies and information are unknown to each other in a practical DASH system. In order to adjust the requested bitrate r i for the i -th user , we propose to employ its own information (i.e., the requested bitrate of the last segment and the current buffer length) and communicate with the server to obtain the payoff variation that is induced by the varied down- load time. Therefor e, the requested video bitrate r i can be updated based on the sub-gradient method [34][35][36]: r i ( t + 1) = r i ( t ) + θ i r i ( t ) ∂ U i ( r ) ∂ r i ( t ) , (15) where θ i > 0 is the speed adjustment parameter (i.e., learning rate) of user i . In an actual system, the value of ∂ U i ( r ) /∂ r i ( t ) (i.e., the payoff variation information in Fig. 2) can be estimated by the server and transmitted to the user as, ∂ U i ( r ) ∂ r i ( t ) ≈ U + i ( r + ) − U − i ( r − ) 2 ε , (16) with ( r + = { r 1 ( t ) , · · · , r i ( t ) + ε, · · · , r N ( t ) } , r − = { r 1 ( t ) , · · · , r i ( t ) − ε, · · · , r N ( t ) } , (17) where ε is an especially small value (e.g., ε = 0 . 0001 ). When the Nash Equilibrium is achieved, we have r i ( t + 1) = r i ( t ) for any i , i.e., ( r ( t + 1) = r ( t ) , ∂ U i ( r ) ∂ r i ( t ) = 0 . (18) 3.4 Stability Analysis for the Distributed Iterative Algo- rithm The stability of the distributed strategy update algorithm (15) is analyzed by using the Routh-Hurvitz stability con- dition [37][38][39][40], which judges the distribution of the eigenvalues (denoted as λ i ) of the Jacobian matrix . That is, if all of the eigenvalues are inside a unit circle of the complex plane (i.e., | λ i | < 1 ), the Nash Equilibrium point is stable. T aking a DASH system with only two users as an example, the Jacobian matrix can be expressed as, J ( r 1 , r 2 ) = " ∂ r 1 ( t +1) ∂ r 1 ( t ) ∂ r 1 ( t +1) ∂ r 2 ( t ) ∂ r 2 ( t +1) ∂ r 1 ( t ) ∂ r 2 ( t +1) ∂ r 2 ( t ) # =  j 1 , 1 j 1 , 2 j 2 , 1 j 2 , 2  , (19) 6 where            j 1 , 2 = − θ 1 Z 3 r 1 j 2 , 1 = − θ 2 Z 3 r 2 j 1 , 1 = 1 + θ 1  − β 1 Z 1 , 1 r 1 (1+ β 1 r 1 ) 2 + Z 1 , 1 1+ β 1 r 1 + Z 2 , 1 − Z 3 (2 r 1 + r 2 )  j 2 , 2 = 1 + θ 2  − β 2 Z 1 , 2 r 2 (1+ β 2 r 2 ) 2 + Z 1 , 2 1+ β 2 r 2 + Z 2 , 2 − Z 3 ( r 1 + 2 r 2 )  (20) The two eigenvalues can be obtained by solving the charac- teristic equation: λ 2 − λ ( j 1 , 1 + j 2 , 2 ) + ( j 1 , 1 j 2 , 2 − j 1 , 2 j 2 , 1 ) = 0 , (21) whose solution is    λ 1 = ( j 1 , 1 + j 2 , 2 )+ √ ( j 1 , 1 − j 2 , 2 ) 2 +4 j 1 , 2 j 2 , 1 2 , λ 2 = ( j 1 , 1 + j 2 , 2 ) − √ ( j 1 , 1 − j 2 , 2 ) 2 +4 j 1 , 2 j 2 , 1 2 . (22) Assume the two users are identical (i.e., Z 1 , 1 = Z 1 , 2 = Z 1 = αβ and Z 2 , 1 = Z 2 , 2 = Z 2 ), and when the Nash equilibrium point is achieved (i.e., r ∗ 1 = r ∗ 2 ) and the buffer length is in a steady state (i.e., b curr = b ref and Z 2 , 1 = Z 2 , 2 = Z 2 = µ · T ), we can derive that j 1 , 1 = j 2 , 2 , j 1 , 2 = j 2 , 1 , λ 1 = j 1 , 1 − j 1 , 2 , and λ 2 = j 1 , 1 + j 1 , 2 . Therefor e, the condition to ensure the stability of the proposed algorithm is expressed as ( − 1 < j 1 , 1 − j 1 , 2 < 1 , − 1 < j 1 , 1 + j 1 , 2 < 1 . (23) For | j 1 , 1 − j 1 , 2 | < 1 , substituting Eq. (20) into (23), we have − 2 <θ 1  − β Z 1 r 1 (1 + β r 1 ) 2 + Z 1 1 + β r 1 + Z 2 − Z 3 (2 r 1 + r 2 )  + θ 1 Z 3 r 1 < 0 . (24) At the Nash Equilibrium point, since θ 1 = θ 2 = θ ∗ , and r 1 = r 2 = r ∗ , Eq. (24) can be rewritten as − 2 θ ∗ < − β Z 1 r ∗ (1 + β r ∗ ) 2 + Z 1 1 + β r ∗ + Z 2 − 2 Z 3 r ∗ < 0 . (25) Furthermore, Eq. (25) can be simplified as ( Z 1 + Z 2 (1 + β r ∗ ) 2 < 2 Z 3 r ∗ (1 + β r ∗ ) 2 , Z 1 +  Z 2 + 2 θ ∗  (1 + β r ∗ ) 2 > 2 Z 3 r ∗ (1 + β r ∗ ) 2 . (26) Substituting T = 2 , Z 1 , Z 2 , and Z 3 into (26), we can obtain ( αβ + 2 µ (1 + β r ∗ ) 2 < 4 ν B W r ∗ (1 + β r ∗ ) 2 , αβ +  2 µ + 2 θ ∗  (1 + β r ∗ ) 2 > 4 ν B W r ∗ (1 + β r ∗ ) 2 . (27) Similarly , for | j 1 , 1 + j 1 , 2 | < 1 , the stability condition is, ( αβ + 2 µ (1 + β r ∗ ) 2 < 8 ν B W r ∗ (1 + β r ∗ ) 2 , αβ +  2 µ + 2 θ ∗  (1 + β r ∗ ) 2 > 8 ν B W r ∗ (1 + β r ∗ ) 2 . (28) Since α , β , µ , ν , θ ∗ , and B W are all positive, we can conclude that the proposed distributed iterative updating algorithm is stable if the following conditions are satisfied: ( αβ + 2 µ (1 + β r ∗ ) 2 < 4 ν B W r ∗ (1 + β r ∗ ) 2 , αβ +  2 µ + 2 θ ∗  (1 + β r ∗ ) 2 > 8 ν B W r ∗ (1 + β r ∗ ) 2 . (29) When there are more than two users in the system, (30) must be satisfied for user i , Z 1 ,i 1 + β i r i + Z 2 ,i − Z 3   N X j =1 r j   = 0 , (30) Equation (30) of all users can be expressed by matrix format as Z 1 + ( 1 + r · β ) Z 2 = Z 3 ( r · 1 T )( 1 + r · β ) , (31) where                r = [ r 1 · · · r i · · · r N ] , 1 = [1 · · · 1 · · · 1] , Z 1 = [ Z 1 , 1 · · · Z 1 ,i · · · Z 1 ,N ] , β = diag ( β 1 , · · · , β N ) , Z 2 = diag ( Z 2 , 1 , · · · , Z 2 ,N ) . (32) The Jacobian matrix of the Nash Equilibrium for multi- users is given as J =     ∂ r 1 ( t +1) ∂ r 1 ( t ) · · · ∂ r 1 ( t +1) ∂ r N ( t ) . . . . . . . . . ∂ r N ( t +1) ∂ r 1 ( t ) · · · ∂ r N ( t +1) ∂ r N ( t )     . (33) Then, similar to the DASH system with only 2 users, the local stability condition can also be analyzed. Algorithm 1 : Distributed Iterative Algorithm of Rate Adaptation 1: Initially , all users request the bitrate r (1) = 0 . 1 Mbps to the server so as to quickly establish the predefined initial buffer length (e.g. 2s). 2: while n ≤ S ( S is the total number of requested segments) do 3: Each user sends the payoff request to the server; 4: The server sends the payoff information to corresponding user; 5: The users update the requested bitrate r ( n ) according to (15) and request video segments from the server; 6: The server sends the video segments to the users; 7: Update the buffer information b curr ( n ) ; 8: n = n + 1 ; 9: end while Algorithm 1 shows the detailed procedur e of the pro- posed method. The DASH users first request the bitrate of 0.1Mbps to quickly establish the initial buffer length. Then, the users send their buffer information to the server . Thirdly , the server calculates and sends the payoff variation information for each user based on the export bandwidth and the buffer lengths of users. Finally , the users update the requested bitrates and request video segments from the server . 4 S I M U L AT I O N R E S U LT S 4.1 Sim ulation Setup T o verify the performance of the proposed method, the Lib-DASH platform [41][42] is used. Users request video segments from the server , which is hosting an Apache HTTP W eb server [43]. And the server export bandwidth is controlled by DummyNet [44]. The video dataset includes BigBuckBunny [45][46], ElephantsDream [47], and SitaSings- theBlues [48][49]. Each video was encoded by FFMPEG [50] with 20 various bitrates from low to high, as shown in T ABLE 1. Fig. 4 shows the two parameters of the quality model in (1) (i.e., α and β ) for each video. The parameters α and β ar e obtained by fitting Eq. (1) using the actual qualities and bitrates of each segment. The lengths of each video segment and the initial buffer of each user are set as 2s. For the proposed method, all users are equipped with the same learning rate in (15), i.e., θ 1 = · · · = θ N = θ , 7 (a) (b) Fig. 4. Quality model parameters (a) α 1 , α 2 , α 3 and (b) β 1 , β 2 , β 3 of the video datasets in the simulation. for all i , to ensure the synchronization of the convergence of the distributed algorithm among all users in the system. Meanwhile, the initial requested bitrates of all users are set as 0.1 Mbps. T ABLE 1 Detailed Information of the T ested DASH Dataset The proposed algorithm is validated under four cases: Case 1 , two identical users ( without limitations on user channel throughputs ) request the same video content (i.e., BigBuckBunny ) with a fixed server export bandwidth; Case 2 , two identical users ( without limitations on user channel throughputs ) request the same video content (i.e., BigBuckBunny ) with a varied server export bandwidth; Case 3 , three users ( with fixed limitations on user chan- nel throughputs ) r equest different video contents (i.e., User1 , User2 , and User3 request BigBuckBunny , ElephantsDr eam , and SitaSingstheBlues , respectively) with a varied server export bandwidth; Case 4 , three users ( with random limitations on user channel throughputs ) request differ ent video contents with both fixed and varied server export bandwidth. Besides we also compare the Proposed method with three algorithms, i.e., the Quality First method ( QF ) [51], Buffer First method ( BF ) [51], and QoE-based Buffer-aware Resource Allocation method ( QBA ) [28]. 4.2 Results of Case 1 In this case, two users compete the limited server export bandwidth of size 6Mbps. The refer ence buffer length is set as 15s. Fig. 5 shows the r equested bitrates and buffer lengths of the two users by the proposed method with different (a) (b) Fig. 5. Results of case 1, in which two identical users compete the server export bandwidth of 6Mbps, and request the BigBuckBunny video sequence with lear ning rate θ = 50 , 100, 150, and 200. Here, µ = 0 . 003 , ν = 0 . 0041 , α = 2 . 15 , β = 0 . 0827 , r ∗ = 3 Mbps, and B W = 6 Mbps and the inequality relation of (29) is tenable. (a) dynamic behavior of requested bitrates, (b) actual buffer lengths of the two users. Note that the states of the two user s are identical . (a) (b) Fig. 6. Results of case 1, in which two identical users compete the server export bandwidth of 6Mbps, and request the BigBuckBunny video sequence with lear ning rate θ = 100 . Here, µ = 0 . 003 , ν = 0 . 0041 , α = 2 . 15 , β = 0 . 0827 , r ∗ = 3 Mbps, and B W = 6 Mbps and the inequality relation of (29) is tenable. (a) dynamic behavior of requested bitrates, (b) actual buff er lengths of the two users with the reference buff er lengths of 10s, 15s, and 20s respectiv ely . Note that the states of the two user s are identical . learning rates. W e can observe that the Nash Equilibrium is achieved at r ∗ = 3 Mbps, and the actual buffer length con- verges to 15-20s except for the case where the learning rate equals 50. When the learning rate increases (e.g., θ = 200 ), the requested bitrate varies severely; however , the reference buffer is achieved more accurately as shown in Fig. 5(b). T ABLE 2 compares the average r equested bitrate and bitrate switching frequency of the proposed method with differ ent learning rates ( θ ). W e can observe that the average requested bitrates are similar , while the minimum bitrate switching frequency is achieved when θ = 100 . T aking the learning rate of θ = 100 as an example, Fig. 6 shows that the Nash Equilibrium is achieved in a slower pace with the reference buffer increasing. Specifically , the converged buffer lengths ar e 13.32s, 17.93s, and 22.68s when the reference buffer lengths are set as 10s, 15s, and 20s, respectively . The reason is that the buffer needs more time to accumulate. T ABLE 2 Comparisons of Av erage Quality and A verage Number of Switches of the Proposed Method with Different Learning Rates under Case 1 Learning Rate θ 50 100 150 200 A verage Bitrate (Mbps) 2.822 2.858 2.862 2.862 A verage PSNR (dB) 44.602 44.700 44.683 44.654 A verage SSIM 0.996 0.997 0.997 0.997 A verage Number of Switches 12 11 17 67 8 4.3 Results of Case 2 W e verify the proposed method with a varied server ex- port bandwidth that is realized via three types of variations [52], i.e., persistent variation, staged variation, and short- term variation, as shown in Fig. 7. The persistent and staged bandwidth variations (both increment and decrement) that last for tens of seconds appear frequently in practice when the cross traffic in the path’s bottleneck varies significantly due to arriving or departing traffic of some users. A good rate adaptation method should adapt to such variations by decreasing or increasing the requested bitrate. The short- term variation that lasts for only a few seconds is usually caused by burst change of channel states. For such short- term variations, the user should be able to keep requested bitrate stable to avoid unnecessary bitrate variations. Fig. 8 shows the requested bitrates and buffer lengths of two users by the proposed method when the server export bandwidth exhibits persistent variation. W e can observe that the requested bitrate increases to the Nash Equilibrium rapidly , and the reference buffer is also reached before t = 50 s. When the server export bandwidth varies to 9 Mbps at the time of 100s, the requested bitrate with θ = 100 quickly increases to 4.5 Mbps, while the requested bitrate with θ = 50 firstly increases to 5 Mbps, which consumes about 5s playout buffer before dropping to 4.5 Mbps. When the available bandwidth decreases back to 6 Mbps at the time of 200s, the requested bitrate with θ = 100 drops to 3 Mbps dir ectly , while the requested bitrate with θ = 50 firstly drops to 3.5 Mbps. When the available bandwidth increases back to 9 Mbps at the time of 300s, the requested bitrate with θ = 50 steps to 4.5 Mbps, while the requested bitrate with θ = 100 incr eases to 4.5 Mbps dir ectly . W e can conclude that a larger learning rate can follow the bandwidth variation accurately and keep the buffer lengths more stable, while the requested bitrate is more stable for a smaller learning rate. Fig. 9 shows the requested bitrates and buffer lengths of two users by the proposed method when the server export bandwidth exhibits staged variation. Similarly , we can also observe that the requested bitrate with a smaller learning rate (e.g., θ = 50 ) is more stable and the bitrate fluctuations are smaller at the time of bandwidth changing (i.e., at 100s, 180s, 260s, and 340s). However , the buffer length is further away from the refer ence buffer length than the learning rate of θ = 100 . Nevertheless, the Nash Equilibrium can be achieved under both cases. For the case of short-term server export bandwidth variation, the requested bitrate with a smaller learning rate ( θ = 50 ) is more stable, as shown in Fig. 10. The requested bitrate with a smaller learning rate can avoid abrupt short- term changing by accumulating/consuming buffered video segments when encountering a small amplitude bandwidth variation, e.g., at the time of 100s and 260s, as shown in Fig. 10(a). But, the difference between the actual buffer length and the reference buffer length of θ = 50 is larger than that of the larger learning rate, i.e., θ = 100 , as shown in Fig. 10(b). (a) (b) (c) Fig. 7. Three kinds of ser ver bandwidth variations, (a) persistent varia- tion, (b) staged variation, (c) short-ter m variation. (a) (b) Fig. 8. Results of case 2, in which two identical users compete the server export bandwidth with persistent variation , and request the BigBuckBunn y video sequence with learning rate θ = 50 and 100 respectively . Here, µ = 0 . 003 , ν = 0 . 0041 . (a) dynamic behavior of requested bitrates, (b) actual buff er lengths of the two users with the reference buff er length of 15s. The initial ser ver export bandwidth is 6Mbps. Note that the states of the tw o users are identical . 4.4 Results of Case 3 This subsection presents the performance of the pro- posed algorithm under Case 3 , in which three users request three dif ferent videos (i.e., BigBuckBunny for User1 , Ele- phantsDream for User2 , and SitaSingstheBlues for User3 ) with a varied server export bandwidth. Note that the channel throughput of each user is limited up to 1.5 Mbps, which is unknown to the users, and the server export bandwidth is set as 6 Mbps. The reference buffer length is set as 15s. From Fig. 11, we can observe that the Nash Equilibrium (a) (b) Fig. 9. Results of case 2, in which tw o identical users compete the ser ver export bandwidth with staged variation , and request the BigBuc kBunny video sequence with lear ning rate θ = 50 and 100 respectively . Here, µ = 0 . 003 , ν = 0 . 0041 . (a) dynamic behavior of requested bitrates, (b) actual buff er lengths of the two users with the reference buff er length of 15s. The initial server e xpor t bandwidth is 6Mbps . Note that the states of the two user s are identical . 9 (a) (b) Fig. 10. Results of case 2, in which two identical users compete the server export bandwidth with short-term variation , and request the BigBuckBunn y video sequence with learning rate θ = 50 and 100 respectively . Here, µ = 0 . 003 , ν = 0 . 0041 . (a) dynamic behavior of requested bitrates, (b) actual buff er lengths of the two users with the reference buff er length of 15s. The initial ser ver export bandwidth is 6Mbps. Note that the states of the tw o users are identical . is achieved with r ∗ 1 = 1 . 5 Mbps, r ∗ 2 = 1 . 5 Mbps, and r ∗ 3 = 1 . 5 Mbps, for different server export bandwidth variation scenarios, and no playback interruption occurs. 4.5 Results of Case 4 In addition, we also compare the proposed method (denoted by Proposed ) with other methods (i.e., BF , QF , and QBA ) with random user channel thr oughput limitations that are also unknown for each user . Fig. 12 compares the requested bitrate and the buffer length of each user when the server export bandwidth is fixed to 6 Mbps. W e can observe that the BF method switches the video bitrate frequently (see Fig. 12(a)) in order to ensure the actual buffer length is close to the refer ence buffer length (see Fig. 12(b)). As shown in Figs. 12(c) and (d), the QF method first accumulates a certain buffer length, and then struggles to request the highest bitrates, resulting in frequent re-buf fering and bitrate switching. For the QBA method, its requested bitrates are more stable than those of BF and QF methods, but the buf fer lengths of the three users are not fair , i.e., the buffer length of User3 is much smaller than that of User1 and User2 , as shown in Figs. 12(e) and (f). The reason is that the QBA method does not take the fairness of users into consideration too much. From Figs. 12(g) and (h), it can be observed that the requested video bitrates by the Proposed method are stable for all of the three users, and their buffer lengths of the three users vary around the r eference buffer length (set as 15s). Besides, ther e is no re-buf fering (playback interruption) for the Proposed method. Similar to Fig. 12, the corresponding comparison results of the four methods with respect to the other three types of bandwidth variations, i.e., persistent variation , staged variation , and short-term variation , are given in Figs. 13, 14, and 15, respectively . Similar conclusions can be drawn, which consistently demonstrates the superiority of the pro- posed algorithm. Detailed numerical comparisons of the four methods are given in T ABLE 3, from which we can observe that the average received bitrate of the QF method is largest, but the bitrate fluctuations (see the standard deviation, average number of switches and average switching amplitude of received bitrates) are also the maximum, and there exist playback interruptions, while the bitrate fluctuation of the BF method is a little small but still large. It can also be (a) (b) (c) (d) (e) (f) (g) (h) Fig. 11. Results of case 3, in which three users (with fixed and limited throughputs of 1.5 Mbps) compete the ser ver export bandwidth (6 Mbps), and request BigBuckBunny , ElephantsDream , and SitaSings- theBlues , respectively , with lear ning rate θ = 50 . Here, µ = 0 . 003 , ν = 0 . 0041 . (a), (c), (e), and (g) show the requested bitrates for fixed, persistent variation, staged variation, and shor t-term variation of ser ver export bandwidth. (b), (d), (f), and (h) show the corresponding buff er length of each user . observed that the average video quality (by observing the average SSIMs) of the QF and BF methods is obviously lower than that of the QBA and the Proposed methods, and the quality variations (by observing the standard deviation of SSIM values of received video segments) of the QF and the BF methods are larger than those of the QBA and the Proposed methods. Moreover , although the average bitrates and SSIM values of the Pr oposed method are similar to those of the QBA method, it is obvious that the amplitudes of bitrate switching and the numbers of switches of Proposed method are smaller , which means that the performance of the Proposed method is the best. Last, we evaluate the performance of the four algo- rithms, i.e., BF , QF , QBA , and Proposed , by comparing their produced QoE values that are measured via two extensively 10 T ABLE 3 Detailed Comparisons of the Four Methods under Case 4 (The best results are bolded). (a) (b) (c) (d) (e) (f) (g) (h) Fig. 12. Results of case 4, in which three users (with random and lim- ited throughputs) compete the server expor t bandwidth ( fixed 6 Mbps), and request BigBuckBunny , ElephantsDream , and SitaSingstheBlues , respectively , with lear ning rate θ = 50 . Here, µ = 0 . 003 , ν = 0 . 0041 . (a), (c), (e), and (g) show the requested bitrates of BF , QF , QB A , and the Proposed methods; while (b), (d), (f), and (h) show the corresponding buff er length of each user. (a) (b) (c) (d) (e) (f) (g) (h) Fig. 13. Results of case 4, in which three users (with random and limited throughputs) compete the ser ver e xpor t bandwidth ( persistent variation ), and request BigBuckBunn y , ElephantsDream , and SitaS- ingstheBlues , respectively , with lear ning rate θ = 50 . Here, µ = 0 . 003 , ν = 0 . 0041 . (a), (c), (e), and (g) show the requested bitrates of BF , QF , QBA , and the Proposed methods; while (b), (d), (f), and (h) show the corresponding buff er length of each user . 11 (a) (b) (c) (d) (e) (f) (g) (h) Fig. 14. Results of case 4, in which three users (with random and limited throughputs) compete the ser ver export bandwidth ( staged variation ), and request BigBuckBunn y , ElephantsDream , and SitaS- ingstheBlues , respectively , with lear ning rate θ = 50 . Here, µ = 0 . 003 , ν = 0 . 0041 . (a), (c), (e), and (g) show the requested bitrates of BF , QF , QBA , and the Proposed methods; while (b), (d), (f), and (h) show the corresponding buff er length of each user. used QoE models [53][54]: QoE 1 = M X k =1 r [ k ] − ξ M − 1 X k =1 | r [ k + 1] − r [ k ] | − ψ M X k =1 max { 0 , T down [ k ] − b [ k ] } , (34) QoE 2 = M X k =1 q [ k ] − ϕ M − 1 X k =1 | q [ k + 1] − q [ k ] | − σ M − 1 X k =1 (max (0 , b ref − b [ k + 1])) 2 − η M X k =1 max { 0 , T down [ k ] − b [ k ] } , (35) where ξ = 1 , ψ = 6 , ϕ = 2 , σ = 0 . 001 and η = 2 are model parameters that are empirically defined in [53] and [54], M is the number of received segments, r [ k ] is the bitrate of the k th requested video segment, q [ k ] is the (a) (b) (c) (d) (e) (f) (g) (h) Fig. 15. Results of case 4, in which three users (with random and limited throughputs) compete the server e xpor t bandwidth ( short-term variation ), and request BigBuckBunn y , ElephantsDream , and SitaS- ingstheBlues , respectively , with lear ning rate θ = 50 . Here, µ = 0 . 003 , ν = 0 . 0041 . (a), (c), (e), and (g) show the requested bitrates of BF , QF , QBA , and the Proposed methods; while (b), (d), (f), and (h) show the corresponding buff er length of each user. corresponding SSIM value, T down [ k ] is the download time of the k th segment, and b [ k ] is the buffer length at the end time of the k th segment and b ref = 15 s. Note that we set η to 2 instead of 50 in [54] to ensure that the QoE values are positive. From T ABLE 4 and 5, it can be observed that, compared with the other three methods, the proposed algorithm always produces the highest average QoE with respect to differ ent types of bandwidth variations. Besides, the proposed method provides optimal QoE for most users, i.e., at least 2 out of 3 users. 5 E X P E R I M E N TA L R E S U L T F O R R E A L I S T I C A L L Y M O D E L E D N E T W O R K I N G S C E N A R I O S W e also established realistically modeled wireless and wired networking scenarios, in which 6 users requested dif- ferent videos from a single server . In the wireless network, they were connected by a movable router (2.4GHz, auto- matic frequency channel bandwidth, 802.11b/g/n mixed wireless mode, 1480 Byte maximum transmission unit con- figuration); yet in the wired network, the users connected to 12 T ABLE 4 Comparisons of the Four Methods under Case 4 in terms of the QoE metric in [53] (The best results are bolded) T ABLE 5 Comparisons of the Four Methods under Case 4 in terms of the QoE metric in [54] (The best results are bolded) the server via the campus network of Shandong University which includes many switches and r outers. T o ensure a large server export bandwidth (which can be constrained to 6Mbps by DummyNet ), the experiments were conducted at night. The initial buffer of each user was set as 20s in the experiments in order to calculate the initial playout delay . The performance of the proposed algorithm was verified under 2 cases: Case 1 , 6 users requested different video contents all the time; Case 2 , 6 users requested differ ent video contents at the beginning, and then some users leaved or joined the network. (a) (b) Fig. 16. Exper imental Results of realistically modeled wireless network, in which 6 users compete the ser ver e xpor t bandwidth (6 Mbps) with θ = 40 , µ = 0 . 003 , and ν = 0 . 0041 , (a) dynamic behavior of requested bitrates, (b) the actual buffer lengths of the 6 users with the ref erence buff er length of 20s. 5.1 Results of Case 1 The results of the wireless and wired networks are given in Figs. 16 and 17, respectively . W e can observe that the fluctuations of the r equested bitrates and buffer lengths are larger than those of the simulated networking scenario because of the complexity of the realistic networking envi- ronment. From T ABLE 6, we can observe that the average initial playout delays of the QF and the BF methods are similar , i.e., 3.61s and 2.70s, for the wireless networking environment and 0.88s and 1.47s for the wired networking environment. Similar to the results of the simulation, the average received bitrates of the Proposed method are not the lar gest but are comparable with the other methods. More importantly , we can see that there is no playback interruption under both the wireless and wired networking environment for the proposed algorithm, which is also demonstrated in Figs. 16(b) and 17(b). As shown in T A- BLE 6, although there is also no playback interruption for the BF method, the received bitrate fluctuations (average standard deviation) of all the 6 users are much larger than the Proposed method. From Figs. 16(a) and 17(a), we can see that the requested bitrates of the Proposed method fluctuate around 1Mbps (the total server export bandwidth is set as 6Mbps). Moreover , the average buffer length of the Proposed method is the largest, which means that the performance of the Pr oposed method can cope with network variations better . Because DASH achieves lossless transmission to im- prove the QoE of users at the cost of transmitting additional signaling information of TCP for HTTP sessions between the server and user , the overhead problem is common and inevitable in DASH. The signaling overhead is essentially determined by the number of HTTP sessions. Therefor e, we also investigated the signaling overhead of the information exchanges between servers and users in the proposed al- gorithm, as shown in T ABLE 7. W e have to clarify that the reported signaling overhead in our manuscript is calculated as the time ratio of the download time of over head informa- tion to that of the video segments . It can be observed that the average proportion of the signaling overhead induced by the information exchanges is about 30% of the whole download time. It is worth pointing out that by using additional HTTP sessions, the performance of a DASH system can be improved . 13 (a) (b) Fig. 17. Experimental Results of realistically modeled wired network, in which 6 users compete the ser ver e xpor t bandwidth (6 Mbps) with θ = 30 , µ = 0 . 003 , and ν = 0 . 0041 , (a) dynamic behavior of requested bitrates, (b) the actual buffer lengths of the 6 users with the ref erence buff er length of 20s. T ABLE 6 P erformance Comparisons of Different Methods 5.2 Results of Case 2 W e verified the performance of the proposed algorithm when users’ requests dynamically come and leave at differ - ent times (i.e., User5 leaves at about 600s and joins at 900s, User6 leaves at about 300s and joins at 1200s). Experimental results ar e shown in Figs. 18 (wireless) and 19 (wired). Fr om Figs. 18(a) and 19(a), we can observe that when User 5 and 6 leave, the requested bitrates of the remaining users increase to 1.5Mbps gradually , while the requested video bitrates of all the users gradually converge to 1Mbps when User 5 and 6 join again. Accordingly , the effectiveness of the proposed algorithm is demonstrated for the realistic network scenario with dynamic user leaving or joining. T ABLE 7 The Propor tion of Signaling Overhead Introduced b y the Information Exchanges between Server and Users User6 leaving User6 joining User5 leaving User5 joining (a) User6 leaving User6 joining User5 leaving User5 joining (b) Fig. 18. Exper imental Results of realistically modeled wireless network, in which users’ requests come and leav e at different time ( User5 leaving at about 600s and joining at 900s, User6 leaving at about 300s and joining at 1200s), with θ = 40 , µ = 0 . 003 , and ν = 0 . 0041 , and the server e xpor t bandwidth is 6Mbps, (a) dynamic behavior of requested bitrates, (b) the actual buffer lengths of the 6 users with the ref erence buff er length of 20s. User6 leaving User6 joining User5 leaving User5 joining (a) User6 leaving User6 joining User5 leaving User5 joining (b) Fig. 19. Exper imental Results of realistically modeled wired network, in which users’ requests come and leave at different time ( User5 leaving at about 600s and joining at 900s, User6 leaving at about 300s and joining at 1200s), with θ = 40 , µ = 0 . 003 , and ν = 0 . 0041 , and the server e xpor t bandwidth is 6Mbps, (a) dynamic behavior of requested bitrates, (b) the actual buffer lengths of the 6 users with the ref erence buff er length of 20s. 6 C O N C L U S I O N A N D D I S C U S S I O N In this paper , we have presented a novel non-cooperative game theory based algorithm to address the rate adaptation issue posed in a DASH system with single-server and multi- users. The proposed algorithm can not only guarantee user fairness but also improve user QoE . Moreover , no proxy is requir ed with our algorithm. W e have formulated the rate adaptation problem as a non-cooperative game by building a novel user QoE model that considers the received video quality , refer ence buffer length, and accumulated buffer lengths of users. W e have theoretically proven the existence of the Nash Equilibrium of our specific game, which can be found by our distributed iterative algorithm with stability analysis. Simulation and experimental results show that the quality and bitrates of received videos by the proposed algorithm are more stable than the state-of-the-art methods, while the actual buffer length of each user moves around the refer ence buffer all the time. Besides, there is no playback interruption for the proposed algorithm. Although the proposed rate adaptation algorithm can achieve impressive performance compar ed with existing algorithm, we believe it can be further improved by address- ing the following limitations: (1) Restriction on the QoE model. In order to guarantee the existence of the Nash Equilibrium, the QoE model must be designed as a continuous and quasi-concave function with respect to the bitrates of all the users. (2) Stability analysis of the scenarios with multi-users or users joining and leaving dynamically . In the proposed method, we used a distributed iterative algorithm to ob- tain the Nash Equilibrium. However , the stability of the distributed iterative algorithm is analyzed in detail for the scenario with only 2 users. When there are more than 2 users, the stability analysis will be very complex, and we only provide a sketch. Besides, for the scenario with new users joining/leaving dynamically , the stability analysis of the proposed method is not analyzed well. (3) Additional HTTP sessions. For the proposed method, additional HTTP sessions between users and servers are needed to achieve the Nash Equilibrium, which will intro- duce additional signaling overhead. Such signaling over- head may r esult in latency . Although this is a common drawback of DASH, it is also highly desirable to develop 14 new algorithms to reduce the additional HTTP sessions as much as possible. A C K N OW L E D G M E N T The authors would like to thank Institute of Information T echnology (ITEC) at Klagenfurt University for the valuable and basis work of DASH. They would also like to thank the editors and anonymous reviewers for their valuable comments. R E F E R E N C E S [1] C. Begen, T . Akgul, M. Baugher , “W atching V ideo over the W eb, Part I: Streaming Protocols,” IEEE Int. Comput. , vol. 15, no. 2, pp. 54-63, Mar . 2011. [2] A. C. Begen, T . Akgul, M. Baugher , “Applications, Standardiza- tion, and Open Issues,” IEEE Int. Comput ., vol. 15, no. 3, pp. 59-63, Apr . 2011. [3] ISO/IEC IS 23009-1: “Information T echnology - Dynamic Adap- tive Streaming over HTTP (DASH) - Part 1: Media Presentation Description and Segment Formats,” 2012. [4] T . Stockhammer , “Dynamic Adaptive Str eaming over HTTP - Stan- dards and Design Principles,” in Proc. ACM Multimedia Systems Conference , California, Feb. 2011, pp.133-144. [5] D. K. Krishnappa, D. Bhat and M. Zink, “DASHing Y ouT ube: An Analysis of Using DASH in Y ouT ube V ideo Service,” in Proc. IEEE Conf. Local Computer Networks , Sydney , NSW , 2013, pp. 407-415. [6] C. W ang, A. Rizk, M. Zink, “SQUAD: A Spectrum-Based Quality Adaptation for Dynamic Adaptive Streaming over HTTP ,” in Pr oc. International Conference on Multimedia Systems , Klagenfurt, Austria, May . 2016. [7] J. Nash, “Non-cooperative games,” Annals of Mathematics , pp. 286- 295, vol. 54, 1951. [8] J. Luo, I. Ahmad and Y . Sun, “Controlling the Bit Rate of Multi- Object V ideos with Noncooperative Game Theory ,” IEEE T rans. Multimedia , vol. 12, no. 2, pp. 97-107, Feb. 2010. [9] P . Goudarzi, “A Non-cooperative Quality Optimization Game for Scalable V ideo Delivery over MANET s,” Wir eless Networks , vol. 19, no. 5, pp. 755-770, 2013. [10] A. Cournot, Researches into the Mathematical Principles of the Theory of Wealth , NT Bacon T rans., 1927. [11] T . C. Thang, Q. D. Ho, J. W . Kang and A. T . Pham, “Adaptive Streaming of Audiovisual Content Using MPEG DASH,” IEEE T rans. Consumer Electron. , vol. 58, no. 1, pp. 78-85, Feb. 2012. [12] L. R. Romero, “A Dynamic Adaptive HTTP Streaming V ideo Ser- vice for Google Android,” M.S. Thesis, Royal Institute of T echnology ( KTH ), Stockholm, Oct. 2011. [13] T . C. Thang, H. T . Le, H. X. Nguyen, A. T . Pham, J. W . Kang and Y . M. Ro, “Adaptive V ideo Streaming over HTTP with Dynamic Resource Estimation,” Communications and Networks , vol. 15, no. 6, pp. 635-644, Dec. 2013. [14] C. Liu, I. Bouazizi, M. M. Hannuksela and M. Gabbouj, “Rate Adaptation for Dynamic Adaptive Streaming over HTTP in Con- tent Distribution Network,” Signal Process. Image Commun. , vol. 27, no. 4, pp. 288-311, 2012. [15] T .-Y . Huang, N. Handigol, B. Heller , N. McKeown, and R. Johari, “Confused, timid, and unstable: Picking a video streaming rate is hard,” in Proc. ACM Conf. Internet Meas. Conf. , Nov . 2012, pp. 225-238. [16] H. Mao, R. Netravali, and M. Alizadeh, “Neural adaptive video streaming with Pensieve,” in Proc . Conference of the ACM Special Interest Group on Data Communication ( SIGCOMM’17 ), pp. 197-210, Los Angeles, CA, USA, Aug. 21. [17] M. Li, C.-Y . Lee, “A Cost-Effective and Real-Time QoE Evaluation Method for Multimedia Streaming Services,” T elecommunication Systems , vol. 59, no. 3, pp. 317-327, Jul. 2015. [18] Z. Duanmu, K. Zeng, K. Ma, A. Rehman, and Z. W ang, “A quality- of-experience index for streaming video,” IEEE Journal of Selected T opics in Signal Processing , vol. 11, no. 1, pp. 154-166, Feb. 2017. [19] W . Zhang, Y . W en, Z. Chen and A. Khisti, “QoE-Driven Cache Management for HTTP Adaptive Bit Rate Streaming Over W ire- less Networks,” IEEE T rans. Multimedia , vol. 15, no. 6, pp. 1431- 1445, Oct. 2013. [20] A. Parandeh Gheibi, M. Medard, A. Ozdaglar , and S. Shakkottai, “A voiding Interruptions - A QoE Reliability Function for Stream- ing Media Applications,” IEEE J. Sel. Area Commun. , vol. 29, no. 5, pp. 1064-1074, May . 2011. [21] Y . Xu, Y . Zhou and D. M. Chiu, “Analytical QoE Models for Bit- Rate Switching in Dynamic Adaptive Streaming Systems,” IEEE T rans. Mobile Computing , vol. 13, no. 12, pp. 2734-2748, Dec. 2014. [22] D. Zegarra Rodr ´ ıguez, R. Lopes Rosa, E. Costa Alfaia, J. Issy Abrah ˜ a o and G. Bressan, “V ideo Quality Metric for Streaming Service Using DASH Standard,” IEEE T rans. Broadcast. , vol. 62, no. 3, pp. 628-639, Sept. 2016. [23] C. Zhou, C. W . Lin and Z. Guo, “mDASH: A Markov Decision- Based Rate Adaptation Approach for Dynamic HTTP Str eaming,” IEEE T rans. Multimedia , vol. 18, no. 4, pp. 738-751, Apr . 2016. [24] V . Mart ´ ın, J. Cabrera and N. Garc ´ ıa, “Design, Optimization and Evaluation of A Q-Learning HTTP Adaptive Streaming Client,” IEEE T rans. Consumer Electron. , vol. 62, no. 4, pp. 380-388, Nov . 2016. [25] A. Bokani, M. Hassan, S. Kanhere and X. Zhu, “Optimizing HTTP-Based Adaptive Str eaming in V ehicular Envir onment Using Markov Decision Process,” IEEE T rans. Multimedia , vol. 17, no. 12, pp. 2297-2309, Dec. 2015. [26] J. Jiang, V . Sekar and H. Zhang, “Improving Fairness, Efficiency , and Stability in HTTP-Based Adaptive V ideo Streaming With Festive,” IEEE/ACM T rans. Networking , vol. 22, no. 1, pp. 326-340, Feb. 2014. [27] Z. Li, X. Zhu, J. Gahm, R. Pan, H. Hu, A. C. Begen and D. Oran, “Probe and Adapt: Rate Adaptation for HTTP V ideo Streaming at Scale,” IEEE J. Select. Areas Commun. , vol. 32, no. 4, pp. 719-733, April 2014. [28] A. E. Essaili, D. Schroeder , E. Steinbach, D. Staehle and M. She- hada, “QoE-Based T raffic and Resour ce Management for Adaptive HTTP V ideo Delivery in L TE,” IEEE T rans. Circuits Syst. V ideo T echnol. , vol. 25, no. 6, pp. 988-1001, Jun. 2015. [29] Z. Su, Q. Xu, M. Fei and M. Dong, “Game Theoretic Resource Allocation in Media Cloud with Mobile Social Users,” IEEE T rans. Multimedia , vol. 18, no. 8, pp. 1650-1660, Aug. 2016. [30] S. W ang, A. Rehman, Z. W ang, S. Ma and W . Gao, “SSIM- Motivated Rate-Distortion Optimization for V ideo Coding,” IEEE T rans. Circuits Syst. V ideo T echnol. , vol. 22, no. 4, pp. 516-529, Apr . 2012. [31] G. T ian and Y . Liu, “T owards Agile and Smooth V ideo Adaptation in HTTP Adaptive Streaming,” IEEE/ACM T rans. Networking , vol. 24, no. 4, pp. 2386-2399, Aug. 2016. [32] P . D. Straffin, “Game Theory and Strategy ,” The Mathematical Association of America , W ashington, DC, 1993. [33] S. Boyd and L. V andenberghe, Convex Optimization , 6th ed. Cam- bridge, U.K.: Cambridge Univ . Press, 2008, pp. 67-71. [34] S. Boyd, X. Lin and A. Mutapcic, Subgradient Methods , Stanford University , Oct. 2003. [35] P . T siaflakis, I. Necoara, J. A. K. Suykens and M. Moonen, “Im- proved Dual Decomposition Based Optimization for DSL Dy- namic Spectrum Management,” IEEE T rans. Signal Processing , vol. 58, no. 4, pp. 2230-2245, Apr . 2010. [36] Z. Zhou, M. Dong, K. Ota, J. W u and T . Sato, “Energy Efficiency and Spectral Efficiency T radeoff in Device-to-Device (D2D) Com- munications,” IEEE Wirel. Commun. Letters , vol. 3, no. 5, pp. 485- 488, Oct. 2014. [37] D. Niyato and E. Hossain, “Competitive Spectrum Sharing in Cognitive Radio Networks: A Dynamic Game Approach,” IEEE T rans. Wireless Commun. , vol. 7, no. 7, pp. 2651-2660, Jul. 2008. [38] M. Sonis, “Once More on Henon Map: Analysis of Bifurcations” Chaos, Solitons and Fractals , vol. 7, no. 12, pp. 2215-2234, 1996. [39] H. N. Agiza, G.-I. Bischi, and M. Kopel, “Multistability in a Dynamic Cournot Game with Three Oligopolists,” Mathematics and Computers in Simulation , vol. 51, pp. 63-90, 1999. [40] E. Ahmed and H. N. Agiza, “Dynamics of a Cournot Game with n Competitors,” Chaos, Solitons & Fractals , vol. 9, pp. 1513-1517. [41] C. Mueller , S. Lederer , J. Poecher and C. T immerer , “Demo paper: Libdash - An Open Source Software Library for the MPEG-DASH Standard,” in Proc. IEEE Int. Conf. Multimedia and Expo Workshops ( ICMEW ), San Jose, CA, 2013, pp. 1-2. [42] C. Alberti, D. Renzi, C. T immerer , C. Mueller , S. Lederer , S. Bat- tista and M. Mattavelli, “Automated QoE Evaluation of Dynamic Adaptive Streaming over HTTP ,” in Proc. the 5th International Workshop on Quality of Multimedia Experience , IEEE , Los Alamitos, CA, USA, 2013, pp. 58-63. 15 [43] Apache HTTP Server Project, http://httpd.apache.org/, (last ac- cess: Jan. 2017). [44] L. Rizzo, “Dummynet: A Simple Approach to The Evaluation of Network Protocols,” ACM Computer Communication Review , vol. 27, no. 1, pp. 31-41, Jan. 1997. [45] Big Buck Bunny Movie, http://www .bigbuckbunny .org/, (last access: Jan. 2017). [46] C. Mueller , S. Lederer and C. T immerer , “An Evaluation of Dynamic Adaptive Streaming over HTTP in V ehicular Environ- ments,” in Proc. the Fourth Annual ACM SIGMM Workshop on Mobile V ideo , ACM , New Y ork, NY , USA, 2012, pp. 37-42. [47] Elephants Dream Movie, http://www .elephantsdream.org/, (last access: Jan. 2017). [48] Sita Sing the Blues Movie, http://www .sitasingstheblues.com/, (last access: Jan. 2017). [49] S. Lederer , C. M ¨ uller and C. T immerer , “Dynamic Adaptive Streaming over HTTP Dataset,” in Proc. the ACM Multimedia Systems Conference 2012 , Chapel Hill, North Carolina, 2012, Feb. 22-24. [50] S. Lederer , C. Mueller , C. T immerer , C. Concolato, J. L. Feuvre and K. Fliegel, “Distributed DASH Dataset,” in Proc. the 4th ACM Multimedia Systems Conference , ACM , New Y ork, NY , USA, 2013, pp. 131-135. [51] M. Dr ¨ a xler and H. Karl, “Cr oss-layer Scheduling for Multi-Quality V ideo Str eaming in Cellular W ireless Networks,” in Proc. Int. Wirel. Commun. and Mobile Comput . ( IWCMC ), Sardinia, 2013, pp. 1181- 1186. [52] S. Akhshabi, S. Narayanaswamy , A. C. Begen and C. Dovrolis, “An Experimental Evaluation of Rate-Adaptive V ideo Players over HTTP ,” Signal Processing Image Communication , vol. 27, no. 4, pp. 271-287, 2012. [53] X. Y in, V . Sekar and B. Sinopoli, “T oward a Principled Framework to Design Dynamic Adaptive Streaming Algorithms over HTTP .” in Proc. W orkshop on Hot T opics in Networks , ACM , Sept. 2014. [54] F . Chiariotti, S. D’Ar onco, L. T oni and P . Fr ossard, “Online Learn- ing Adaptation Strategy for DASH Clients,” in Proc. Multimedia Systems , ACM , Aug. 2016.