Structure-Aware Stochastic Control for Transmission Scheduling

1 Structure-Aware Stochastic Control for Transmission Scheduling Fangwen Fu and Mihaela van der Schaar Department of Electrical Engineering, University of California, Los Angeles, Los Angeles, CA, 90095 {fwfu, mihaela}@ee.ucla.edu ABSTRACT In this paper, we consider the problem of real-time transmission scheduling over time-varying channels. We first formulate the transmission scheduling problem as a Markov decision process (MDP) and systematically unravel the structural properties (e.g. concavity in the state-value function and monotonicity in the optimal scheduling policy) exhibited by the optimal solutions. We then propose an online learning algorithm which preserves these structural properties and achieves ε -optimal solutions for an arbitrarily small ε . The advantages of the proposed online method are that: (i) it does not require a priori knowledge of the traffic arrival and channel statistics and (ii) it adaptively approximates the state-value functions using piece-wise linear functions and has low storage and computation complexity. We also extend the proposed low-complexity online learning solution to the prioritized data transmission. The simulation results demonstrate that the proposed method achieves significantly better utility (or delay)-energy trade-offs when comparing to existing state-of-art online optimization methods. Keywords: Energy-efficient data transmission, Delay-sensitive communications, Markov decision processes, stochastic control, scheduling I. INTRODUCTION Wireless systems often operate in dynamic environments where they experience time-varying channel conditions (e.g. fading channel) and dynamic traffic arrivals. To improve the energy efficiency of such systems while meeting the delay requirements of the supported applications, the scheduling decisions (i.e. determining how much data should be transmitted at each time) should be adapted to the time-varying environment [1][9]. In other words, it is essential to design scheduling policies which consider the time-varying characteristics of the channels

2 as well as that of the applications (e.g. backlog in the transmission buffer, priorities of traffic, etc.). In this paper, we use optimal stochastic control to determine the transmission scheduling policy that maximizes the application utility given energy constraints.
The problem of energy-efficient scheduling for transmission over wireless channels has been intensively investigated in [1]-[15]. In [1], the trade-off between the average delay and the average energy consumption for a fading channel is characterized. The optimal energy consumption in the asymptotic large delay region (which corresponds to the case where the optimal energy consumption is close to the optimal energy consumption under queue stability constraints, as shown in Figure 1) is analyzed. In [8], joint source-channel coding is considered to improve the delay-energy trade-off. The structural properties of the solutions which achieve the optimal energy- delay trade-off are provided in [5][6][7]. It is proven that the optimal amount of data to be transmission increases as the backlog (i.e. buffer occupancy) increases, and decreases as the channel conditions degrade. It is also proven that the optimal state-value function (representing the optimal long-term utility starting from one state) is concave in terms of the instantaneous backlog.
Energy-efficient scheduling for traffic with individual delay deadlines is considered in [2][3][4]. In [2], the optimal scheduling policy is obtained using dynamic programming. In [3][4], optimality conditions are characterized for the optimal scheduling policies, and based on these, online heuristic scheduling policies are developed. Besides considering the time-varying channel conditions, the heterogeneous traffic features (e.g. different delay deadlines, importance and dependencies of packets) are considered in [14][15], where the optimal scheduling policies are developed by explicitly considering the impact of the heterogeneous data traffic.
We notice that the above solutions are characterized by assuming that the statistical knowledge of the underlying dynamics (e.g. channel state distribution, packet arrival distribution, etc.) is known. When the knowledge is unavailable, only heuristic solutions are provided, which cannot guarantee the optimal performance. In order to cope with the unknown environment, the stability-constrained optimization methods are developed in [10]-[13], where, instead of minimizing the queue delay, the queue stability is considered. The optimal energy consumption is achieved only for asymptotically large queue sizes (corresponding to asymptotic

3 large delays, i.e. in large delay region). These methods do not provide optimal energy consumption in the small delay region which is shown in Figure 1.

Figure 1. Illustration of large delay region and small delay region Other methods for copi

…(Full text truncated)…

This content is AI-processed based on ArXiv data.