Revisiting Transmission Scheduling in RF Energy Harvesting Wireless Communications

Revisiting T ransmission Scheduling in RF Energy Har vesting Wireless Communications Y u Luo South Dakota School of Mines and T echnology yu.luo@sdsmt.edu Lina Pu South Dakota School of Mines and T echnology lina.pu@sdsmt.edu Y anxiao Zhao South Dakota School of Mines and T echnology yanxiao.zhao@sdsmt.edu W ei W ang San Diego State University wwang@mail.sdsu.edu Qing Y ang University of North T exas qing.yang@unt.edu ABSTRA CT The transmission scheduling is a critical problem in radio frequency (RF) energy harvesting communications. Existing transmission strategies in an RF-based energy har vesting system is mainly based on a classic model, in which the data transmission is scheduled in a xed feasible energy tunnel. In this paper , we re-examine the classic energy har vesting model and show through the theoretical analysis and experimental results that the bounds of feasible energy tunnel are dynamic, which can be aected by the transmission schedul- ing due to the impact of residual energy on the harvested one. T o describe a practical energy harvesting process more accurately , a new model is proposed by adding a feedback lo op that reects the interplay between the energy har vest and the data transmission. Furthermore, to impr ove network performance, w e revisit the de- sign of an optimal transmission scheduling strategy based on the new model. T o handle the challenge of the endless feedback loop in the new model, a recursive algorithm is developed. The simulation results reveal that the new transmission scheduling strategy can balance the eciency of energy reception and energy utilization regardless of the length of energy packets, achieving improved throughput performance for wireless communications. KEY W ORDS RF energy harvesting model, optimal transmission scheduling, resid- ual energy , nonlinear charge. 1 IN TRODUCTION Due to the features of self-sustainability , pollution-free and p erpet- ual operation, energy har vesting becomes a promising technology to drive low-power devices in future wireless mobile networks [ 1 – 3 ]. An eective strategy to manage the arrived energy and to schedule the data transmission on an energy harvesting device (EHD) plays a crucial role to achieve a desired network performance in terms of throughput, transmission delay , and communication reliability . In recent years, many transmission scheduling strategies have been proposed for the radio frequency (RF) based energy harvesting system [ 4 – 7 ], where both the intermittency and the randomness of energy arrivals are taken into account for the power manage- ment. In those designs, the energy grabbed by an EHD has b een widely mo deled as a random process [ 8 – 12 ], which may depend on the activity of RF energy sources ( e.g., a TV tower or a cellular base station), but is rarely aected by the transmission sche duling on EHD. In this paper , we challenge this common cognition and conrm through theoretical analysis and experimental results that the transmission scheduling policy can aect the process of energy harvesting signicantly . Essentially , for a real RF energy har vesting system, there are two dierent concepts: the arrived energy and the har vested energy . The former is the ecient energy that reaches an EHD after consid- ering the propagation losses and the pow er conversion eciency , while the latter is the energy absorbed and conserved by the EHD’s battery . Due to the nonlinear charge characteristic of batteries, the harvested energy is not only determined by the arrived energy but also aected by the residual energy of the EHD [9, 13]. Although the nonlinear charge is a well-known characteristic of batteries, it is not taken into account in the commonly-used energy harvesting model for wireless communications. As illustrated in Fig. 1 ( a), the impact of residual energy of the EHD on the energy harvest process is not considered in the conventional model. With such an assumption, EHDs consuming energy in dierent manners will harvest an equivalent amount of energy as long as the arrived energy is the same. The existing work in optimal transmission scheduling strategy is to seek a curve within such a xed energy tunnel so that a pre-dened objective is optimized (e .g., maximizing throughput) [11, 12, 14]. + Harve sted energy Residual energy X Nonlinear charge + Classic energy harvesting model New feedback-based energy harvesting model Tr a n s mi s s i o n scheduling strategy Data transmission Tr a n s mi s s i o n scheduling strategy Data transmission Arrived energy Harve sted energy Energy from energy source Energy from energy source (a) (b) ( Modeled as random process ) ( Modeled as random process ) Figure 1: (a) The conventional energy harvesting model, and (b) the new feedback-base d model. Unfortunately , the above assumption may not b e true in real energy harvesting system. Since residual energy is aected by the data transmission, the harvested energy heavily depends on a data transmission strategy . In other words, an EHD cannot estimate what amount of energy it can har vest from an energy packet be- fore scheduling its transmissions, and we call it as the causality of energy harvest . The interplay between the transmission schedul- ing strategy and harvested energy has been completely neglecte d in the literature. It calls for a new energy harvesting model and re-examination of challenges identied in the existing resear ch. In this paper , we propose a new energy harvesting model inte- grating the causality of energy harvest, as illustrated in Fig. 1 (b). In the proposed mo del, an energy fee dback loop (the red line in Fig. 1 (b)) from a data transmission to the harvested energy is estab- lished that factors the nonlinear charge characteristics of a battery . What distinguishes the classic energy harvesting model is that the feasible energy tunnel is not xed: its bounds are aected by transmission strategies. Therefore , the formulation of an optimal transmission strategy based on the new mo del has to b e revised. With the new feedback-based model, we re-examine the design of an optimal oine transmission scheduling strategy , which faces grand challenge introduced by the feedback line: on the one hand, the design of an oine transmission scheduling strategy needs to know in advance the amount of energy an EHD can harvest; on the other hand, the transmission scheduling itself aects the energy harvesting process through the residual energy in a battery . In order to resolve the above challenge, which is referred to as the endless loop problem, we de velop a recursive algorithm by leveraging the inherent relationship among the residual energy , transmission power , and harvested energy . In the algorithm, the optimal transmission power in one epoch can b e represented by that in prior epochs. Conse quently , the design of an optimal trans- mission scheduling is converted to solving a nonlinear equation. As will b e veried in the recursive algorithm, both energies arriv ed in the past and that will arrive in the future have impacts on the optimal transmission scheduling at the current time. T o summarize, the contributions of our w ork are threefold. First, through the oretical analysis and experimental results, we verify the limitation of classic energy harvesting model that ignores the interplay between the transmission sche duling strategy and the energy harvesting process in a practical energy harvesting system. Second, a new fee dback-based model is propose d, in which the impact of data transmission sche duling on the energy har vest is described via the residual energy . Third, b oth an oine optimal and an online suboptimal transmission scheduling strategies are developed using the new feedback-based model. A recursive algo- rithm is designed to provide an upper b ound of throughput that an RF energy harvesting wireless communication system can achieve. Simulation results reveal that compared with existing transmission scheduling strategies, the proposed one can improve the system throughput signicantly without violating the causality of energy harvest. The remainder of this paper is organized as follows. The related work and an overview of the conventional transmission scheduling are introduced in Section 2 and Section 3, respectively . The nonlin- ear energy harvesting process is discusse d in Section 4. The oine optimal transmission scheduling strategy is developed in Section 5. The simulation results are shown in Section 6 and conclusions are drawn in Section 7. 2 RELA TED WORK Prior work presented in [ 10 ] proves that, with the assumption of innite battery size, the eciency of data transmission is maxi- mized when the transmission power remains constant between energy harvests. With such a conclusion, the transmission sched- uling at an EHD is simplied into a piecewise-linear optimization problem. Later on, the constraint of battery capacity is taken into account [ 11 , 12 ]. Researchers in [ 11 ] construct a feasible energy tunnel, where the upper bound and the lower b ound are determined by the constraints of energy causality and battery size, respectively . Geometrically , it is pointed out that to maximize the throughput, the aggregated energy consumption should be the tightest string in the energy feasibility tunnel. The directional water-lling algo- rithm applied in [ 12 ] is an alternative to optimize data transmission. In such an algorithm, the energy is considered as “water” , which can neither ow back nor excee d the maximal capacity of a bat- tery , and the algorithm aims at distributing the water equally over time. In [ 15 ], both the constraints of data and battery capacities are integrated to optimize the transmission scheduling strategy . T o manage the power more eciently , the energy that an EHD consumes on signal processing is investigated in [ 5 , 6 ]. As ana- lyzed in [ 5 ], due to the constant overhead of hardware, the energy eciency of an EHD is non-monotonic with respect to the sp ec- trum eciency . For the purpose of improving system throughput, a two-phase transmission scheduling policy is pro vided: the rst phase is to maximize the energy eciency through an on-o power allocation method; the spe ctrum eciency is optimized in the sec- ond phase with a non-de creasing power allocation strategy . The authors in [ 6 ] introduce a directional glue-p ouring algorithm to solve a similar problem. Akin to the directional water-lling algo- rithm, the “glue” in [ 6 ] is only allowed to ow forward and the equilibrium glue levels are then determined. In addition, due to the constant power consumption over time, a threshold of the transmis- sion power in the dir ectional glue-pouring algorithm needs to be calculated rst, and then the process of glue-pouring is performed so that the power level is always higher than the thr eshold. Recently , substantial research eorts have b een made on re- alistic scenarios, where battery imperfections are taken into ac- count [ 8 , 9 , 13 , 16 ]. The authors in [ 16 ] considers a realistic battery model where the capacity of batter y degrades over time and a con- stant energy leakage incurs. Such imp erfections modify the feasible energy tunnel: the distance between the upper and lower bounds monotonously decreases reecting the time-varying battery ca- pacity . The research conducted in [ 8 , 9 ] investigates the charge ineciencies caused by the nonlinear charge featur e of batteries. In the optimal policy design of data transmissions, howev er , the ob- tained energy is still modele d as an independent random variables, which neglects the impact of data transmission on energy harvest. The authors in [ 13 ] further consider the imperfect knowledge of the instant battery level and propose optimal transmission strategy under limited knowledge (e.g., battery low or hight). Through an extensive literature survey , we learn that the major- ity of existing transmission scheduling approaches are based on the classic energy har vesting model, in which the amount of energy replenished is assume d to b e random values independent of the way of consuming energy [ 5 – 12 , 15 , 16 ]. In our paper , we revise the classic mo del and show through experiments that the harvested energy and the transmission scheduling strategy inherently interact 2 with each other , which must be considered comprehensively for ecient power management. 3 CON VEN TIONAL SCHEDULING FOR DA T A TRANSMISSION In this se ction, we introduce the background knowledge about the conventional transmission scheduling and the corresponding formulation based on the classic harvesting model. When an EHD schedules the data transmission, the RF energy is usually considered as discrete energy packets with random sizes, e i , arrived at time, t i , as shown in Fig. 2(a). The initial energy of the battery is denoted by e 0 . Assume there are a number of N energy packets transmitted by an energy source in total. The time inter val between the successive energy arrivals is called epoch, the length of which is denoted by l i . t 2 t 1 t 3 t 4 t 5 t E e 1 e 2 e 3 e 4 e 5 l 1 l 2 l 3 l 4 l 5 (a) t E Z t 0 p ( x )d x C1 t i 0 , unless i = N + 1 . New Property 2. The new optimal transmission sche duling strategy will not fully charge an EHD , i.e., ∀ i ∈ Z + , i ≤ N + 1 : E r i + E h i < e m . The proof of New Property 1 can be found in Appendix .1. It em- phasizes that in the new optimal transmission strategy , an EHD will retain a positive amount of residual energy until the end of data transmission, i.e., E r N + 1 = 0 . This feature will b e used in Ap- pendix .2 to solv e the KKT conditions. New Property 2 is a result of the exponential component of the charging function presented in (3). The charging current approaches zer o when a battery’s voltage is close to the maximal value and it will take innite time to fully charge the EHD. Comparing with the Conventional Pr operty 1 and 2 introduced in Section 3, which require an EHD to fully charge or completely deplete the battery before the change of transmission power , the new properties of an optimal strategy t the real feature of energy harvesting module much better . Now , we solve the KKT conditions starting from the stationarity equations. Through the derivation performed in Appendix .2, the optimal transmission power of an EHD in the current epo ch can be represented by that in previous epo chs through the following iteration expression: p ∗ m + 1 =  1 + p ∗ m  ( X m + 1 ) − 1 , (19) where X m = 1 2 A 1 m A 3 m  E r m  − 1 2 + A 1 m A 4 m , and A i m ( i = 1 , 3 , 4 ) has been listed in (5). Given the iteration, a recursive algorithm, which is referred to as Algorithm 1, can be developed to calculate the optimal transmission scheduling. By executing commands b etween Step 2 and Step 7 of Algo- rithm 1, E r 2 to E r N + 1 are replaced by E r 1 iteratively . As a consequence, according to the New Property 1 that E r N + 1 = 0 , a nonlinear e quation, J ( E 1 r ) , with the single variable, E 1 r , is available at Step 8. Through Algorithm 1 Calculation of the optimal transmission power 1: Let E r 1 be the single variable. 2: Represent p ∗ 1 by E r 1 based on p ∗ 1 = E h 0 − E r 1 l 1 . 3: for i = 1 to N do 4: Represent E h i by E r 1 based on (4). 5: Represent p ∗ i + 1 by E r 1 based on (19). 6: Represent E r i + 1 by E r 1 based on (12). 7: end for 8: T o fully utilize all conserved energy by the end of data trans- missions, we set E r N + 1 = 0 , and then calculate E r 1 , which may have multiple solutions. 9: Calculate all potential p ∗ i , i = 1 , . . . , N + 1 , by substituting all solutions of E r 1 into Step 2 to Step 8. 10: Substituting all potential p ∗ i into P2 , the one that maximizes the objective function subject to the constraints is selected as the optimal strategy . using a numerical approach like Newton-Raphson, E 1 r can be cal- culated from J ( E 1 r ) . Eventually , all potential optimal transmission power , p ∗ i , in each epoch is determined iteratively according to the relationship between the transmission power , residual energy , and harvested energy describe d in Step 2 to Step 7 of the algorithm. It is worth noting that after the recursive process of Algorithm 1, J ( E 1 r ) contains the coecients from A 1 i to A 4 i , where i = 1 , . . . , N . According to (5), A x i is correlated with the length of energy packet i . This indicates that the optimal transmission power of an EHD in current ep och is aecte d not only by the energy received in previous epochs but also by the future energy arrivals, which poses a grand challenge on the development of an optimal online transmission scheduling strategy . In Section 6.3, we will have a short discussion on the design of a sub optimal online strategy by using the method of energy prediction. Although the new transmission scheduling strategy dev eloped in this section can only work in an oine manner , it is still valuable in real applications because: a) It provides an upper bound of the throughput that an RF energy harvesting communication system can reach. b) The new oine strategy can b e applied to an online scenario with energy prediction approaches. As reported in [ 19 ], the en- ergy densities of ambient RF signal on dierent frequency bands (680 MHz-3.5 GHz) are almost constants over time in the urban environment, providing a possibility to estimate the future energy arrivals accurately . 6 SIMULA TION RESULTS In this section, we evaluate the proposed oine optimal transmis- sion scheduling and the conventional strategies through simula- tions. As discussed in [ 14 ], conventional strategies assume the data transmission and the energy harvesting process are independent, which violates the causality of energy har vest introduce d in Sec- tion 5.1. In order to make a feasible comparison in simulations, 6 we generate the energy tunnel rst and then calculate the corre- sponding lengths of arrived energy based on the specic E h and E r , which is in reverse or der of a real energy harvesting process. 6.1 Channel Mo del and Simulation Settings In the simulation, we use a common A WGN channel model. The noise p ower denoted by N l is calculate d through N l = N 0 + 10 log 10 ( B ) , where B is the communication bandwidth and N 0 = − 174 dBm is the noise density . The distance b etween EHD and its intended receiver , d , is 10 ft, and the propagation loss of RF signals from EHD to the receiver is calculated through the free space path loss (FSPL) model, where FSPL (dB) = 20 log 10 ( d ) + 20 log 10 ( f ) − 147 . 55 . (20) According to the capacity of A W GN channel and propagation loss model, the power-rate function on a unit bandwidth is r ( t ) = log 2 [ 1 + p ( t ) − FSPL − N l ] , (21) where r ( t ) is the data transmission rate in bps and p ( t ) is the trans- mission power in dBm. The central frequency and bandwidth for data transmission are 2 . 4 GHz and 10 MHz, respectively . Moreover , the resistance of the EHD’s charging circuit, R , is 4 k Ω and the capacitance of the capacitor , C , is 50 mF. For comparison purpose, the performance of the maximal en- ergy harvesting strategy obtained through (6) and the conventional policy proposed in [ 11 ] are assessed. The maximal energy har vest- ing strategy aims at maximizing the energy harvest each time but ignores the eciency of energy utilization. T o prevent the conven- tional strategy from violating the causality of energy har vest, we implement the energy tunnel as follows: – Step 1: Generate a random energy tunnel for the conventional strat- egy , where ( E h i ) ′ represents the energy harvested from energy packet i . Apply the conventional strategy to schedule the data transmission 1 and calculate the corresponding ( E r i ) ′ before each energy harvest. – Step 2: Based on the sequences of  ( E h 1 ) ′ , . . . , ( E h N ) ′  and  ( E r 1 ) ′ , . . . , ( E r N + 1 ) ′  , the length of each energy packet, T e i for i = 1 , . . . , N , is calculated via (3). – Step 3: Given T e i , the new oine optimal strategy and the maximal energy harvesting policy are obtained thr ough Algorithm 1 and (6), respectively . In simulations, both ( E h i ) ′ and the length of each epoch obey a uniform distribution, the variance of which is set to one fth of the mean value. The evaluation results ar e the average of 30 indepen- dent tests. 6.2 Performance Evaluation Before analyzing the simulation results, we need to highlight again that the conventional strategy violates the causality of energy har- vest. Accor ding to the relationship between the harvested energy and data transmission represented in (8), the EHD cannot estimate the amount of energy it can har vest b efore scheduling its data trans- mission. This implies that the energy tunnel with a xed shape that the conventional strategy assumes does not exist. Therefor e, even 1 According to the New Property 2, an EHD cannot be charged to e m within a limit time. Therefore, to make it practical, we assume that in the conventional strategy , the EHD is fully charged when the energy reaches 0 . 99 × e m . if the conventional strategy shows a comparable performance with the new strategy in some circumstances, this strategy may not be feasible in a real system. One purpose of including the conventional strategy into comparison is to give insight into the tradeo between the energy harvest eciency and the energy utilization, which is discussed later . 10 3 0 10 4 0.3 10 5 Average throughput (bps) 10 6 0.2 10 7 1 0.1 2 0 7 T e (s) C (F) Figure 5: Throughput performance with respect to the aver- age length of energy packets, ¯ T e , and capacitor capacity , C . Fig. 5 shows the throughput performance of the new oine opti- mal strategy with respect to the average length of energy packets, ¯ T e , and capacitor capacities, C . With the increase of ¯ T e , we reduce the frequency of energy arrivals to maintain a constant energy density , which is dened by the average amount of energy arrived at EHDs per second. It is clear that the throughput monotonously decreases with the growing length of energy packets. The main reason for this phenomenon is that the eciency of energy harvest is reduced at the end of a long energy packet reception due to the nonlinear charge feature of EHDs discussed in Section 4. There- fore, if the energy density is a constant, EHDs prefer short energy packets to achieve a high throughput. T o improve the eciency on harvesting long energy packets, a supercapacitor with large capacitance can be used in the charging circuit. Howe ver , a supercapacitor with high capacity usually has a low voltage cell 2 and serial connections might b e required to drive the transmitter , which increases the cost and the size of the EHD . In Fig. 5, it can be observed that for any given ¯ T e , the improvement in throughput with respect to the increase of capacity is logarithmic. This indicates that the rise of throughput slows down quickly with a higher C . For this reason, EHDs should use the supercapacitor with a proper capacitance based on the budget of a project, the length of energy packet, and the quality of service (QoS) of an application. In order to well study the throughput performance of the three strategies, we split the results into three sub-gures, where Fig. 6 (a), (b) and (c) displays the scenarios in average with the short length, medium length and long length of energy packets, respectively . Additionally , to eliminate the eect of the capacitor’s capacitance on performance assessment, the average length of energy packets, ¯ T e , is divided by the time constant of the charging cir cuit, τ . The throughput degradation with the growth of ¯ T e / τ is consistent to the conclusions drawn from Fig. 5. 2 Standard supercapacitors with aqueous electrolyte and organic solvents are usually specied with a rated voltage of 2 . 1 – 2 . 3 V and 2 . 5 – 2 . 7 V , respe ctively . 7 10 -4 10 -3 10 -2 7 T e = = 1 2 3 4 5 Average throughput (bps) # 10 6 Conventional strategy Maximal energy harvesting New offline optimal strategy (a) Short energy packet 9 = # 10 0.85 0.9 0.95 1 1.05 1.1 1.15 1.2 1.25 Average throughput (bps) # 10 5 -2 0.6 0.7 0.8 Conventional strategy Maximal energy harvesting New offline optimal strategy (b) Me dium energy packet 10 -2 4 # 10 -2 10 -1 T e = = 3 4 5 6 7 8 Average throughput (bps) # 10 4 Conventional strategy Maximal energy harvesting New offline optimal strategy (c) Long energy packet Figure 6: A verage throughputs among dierent transmission strategies with respe ct to ¯ T e / τ . From Fig. 6 (a), it can be observed that with a small ¯ T e , the pro- posed strategy and the conventional one have signicantly higher throughput than the maximal energy harvesting policy , which ig- nores the eciency of energy utilization in the strategy design. This indicates that when the energy packets are of short length, it becomes more important to eciently utilize the marginal energy than to improve the energy harvest eciency . With the increase of ¯ T e / τ the throughput of the maximal energy harvesting strat- egy gradually approaches the other tw o policies. As illustrated in Fig. 6( b), if ¯ T e / τ stays in a moderate range, the new strategy oers the highest throughput amongst the three strategies, although the advantage is not signicant. As ¯ T e / τ further increases in Fig. 6(c), the throughput of the new oine and the maximal energy har vest- ing strategies become higher than that of the conventional policy . It implies that when the RF energy is str ong in the air , improving the eciency of energy harvest brings more b enet in terms of system throughput than improving the energy utilization. 4 # 10 -4 10 -3 7 T e = = 5.05 5.1 5.15 5.2 5.25 5.3 Average harvested energy (J) # 10 -3 Conventional strategy Maximal energy harvesting New offline optimal strategy Figure 7: Har vested energy with respect to ¯ T e / τ . Here, we show more details on how the eciency of energy harvest and the energy utilization are balanced in the new trans- mission strategy . In Fig. 7, we change the length of energy packets and compare the amount of energy har vested by the three transmis- sion strategies. As depicte d in Fig. 7, the maximal energy har vesting strategy acquired the highest energy among three policies by opti- mizing the residual energy through (6). When the energy packet is short, taking ¯ T e / τ = 2 . 6 × 10 − 4 as an example, the maximal energy harvesting policy harvested 0 . 18% more energy at the cost of 55% throughput degradation in Fig. 6(a) compared to the new optimal strategy . When the length of energy packets increases, the dier- ence of the amount of energy har vested by the conv entional one and the other two strategies becomes signicant, resulting in a low throughput as shown in Fig. 6(c). This result validates that the new optimal strategy can balance the eciencies on energy harvest and utilization dynamically based on the arrived energy . This feature makes the proposed strategy achieve better performance than the maximal energy harvesting policy , which sacrices eciencies on energy utilization, and the conventional strategy , which ignores the eciency of energy harvest. T o summarize, according to the above analysis, it could be ob- tained that if the energy packet is short, improving the energy utilization plays a more important role in throughput optimization compared to enhancing the energy harvest eciency . By contrast, with long energy packets, high eciency of energy har vest be- comes a dominant factor in the design of a transmission scheduling strategy . The proposed oine optimal transmission scheduling is validated to have well balanced the energy utilization and energy harvest eciency . 6.3 Discussion of Online Policy Recall that Algorithm 1 reveals a property of the optimal oine strategy: the optimal transmission power of an EHD in current epoch is aected not only by the energy r eceived before but also by future energy arrivals. How ever , their impacts may be inequivalent. In particular , a nearby energy packet causes a much heavier impact on the current transmission strategy than a remote one. Therefore, if an EHD can predict the length and the arrival time of energy packets that will be received in a near future, an online strategy with near optimal performance can be expecte d. Now , we investigate the eect of future energy arrivals on the optimal transmission power in the curr ent epoch with a medium length of energy packets. Denote the current epoch as i , and then the epoch that d epochs way from the current one is ( i + d ) . Assume the original duration of the ( i + d ) t h epoch is ¯ L ; the length of energy 8 packet arrived at the ( i + d ) t h epoch is ¯ T e . In Fig. 8, we change ¯ L and ¯ T e to ¯ L + ∆ L and ¯ T e + ∆ ¯ T e , respectively , and then present the dierence of optimal transmission power before and after the variation of ¯ L and ¯ T e . 1 2 3 4 5 6 d 0 5 10 15 20 25 30 Difference of transmission power (%) " L = 0 : 4 # L " L = 0 : 6 # L " L = 0 : 8 # L " T e = 0 : 4 # T e " T e = 0 : 6 # T e " T e = 0 : 8 # T e Figure 8: Impact of future energy arrivals on the transmis- sion scheduling in the current epoch. As demonstrated in Fig. 8, if the interval between energy arrivals or the length of an incoming energy packet changes, i.e ., d = 1 , an EHD needs to modify its transmission power signicantly so that it can better adapt to the changes of ¯ T e and ¯ L . However , if an energy packet is ve ep ochs away , i.e., d = 5 , the impacts of varying ¯ T e and ¯ L on the optimal transmission power in the current epoch are less than 2% , which is negligible in a real application. The obser vations from Fig. 8 implies that developing an online strategy with near optimal throughput is promising if an EHD can predict the time of arrival and the length of next few energy packets to some extent. The above expectation is also validated by the simulation results of Fig. 6, which demonstrate the following two critical observations: Observation 1. With medium or long energy packets (i.e., ¯ T e / τ ≥ 10 − 3 ), the optimal transmission strategy approaches the maximal energy har vesting policy . The maximal energy harvesting strategy can be considered as an online policy when the average length of energy packets is greater than 10 − 3 τ . The throughput can be nearly optimized by retaining a specic amount of residual energy that depends on the length of the next incoming energy packet, as presented in (6). Therefore, it can be implemented online with one-step prediction of energy arrivals. The length and arrival time of neighb oring energy packets are usually highly correlated when the EHD is p ower ed by a de dicated energy source, e.g., RF identication (RFID) system [ 21 , 22 ]. In this scenario, the prediction model, such as a Markov chain or an adaptive lter [ 23 ], can be applied to estimate the information of incoming energy packet. When the energy source is ambient RF signal, the length and the time that an energy packet arrives at the EHD are random. The average length and the time interval of past energy packets can be treated as an estimation to the next energy replenishment. Observation 2. With short energy packets (i.e., ¯ T e / τ < 10 − 3 ), the optimal transmission strategy approaches the policy that maximizes the eciency of energy utilization. The strategy that maximizes the eciency of energy utilization has similar feature to existing transmission scheduling, where the battery will be drained or nearly fully charged before the change of transmission power . In this case, a state-of-the-art of the online solutions can be found in [14]. 7 CONCLUSIONS In this paper , a new feedback-based model has be en proposed for RF energy harvesting communications. T aking the charge char- acteristic of an energy harvesting circuit into account, the new model reveals the impact of data transmission on harvested energy , which introduces a new constraint called the causality of energy harvest for the design of energy har vesting strategy . With such the constraint the feasible energy tunnel is not xed; its b ounds change with dierent transmission strategies dynamically . Base d on the new energy harvesting model, the problem of se eking oine optimal transmission strategy has b een reformulated and solved by developing a r ecursive algorithm. According to simulation r e- sults, the new transmission scheduling strategy is able to balance eciencies between energy harvest and energy utilization. The design of an online policy with the new energy harvesting model is also briey discusse d. From the discussion, it can be realized that a near-optimal online strategy is available if the length and the arrival time of energy packets that will be received in the near future can be predicted in a certain level. .1 Proof of New Property 1 Proof. W e prove the New Property 1 by contradiction, assuming in an optimal transmission scheduling, the EHD’s batter y can be fully depleted at least once before the ( N + 1 ) t h epoch. Let ST a be such a strategy , which consumed all stored energy at t i , i , N + 1 . Referring to Lemma 2 in [ 10 ], it can be easily proved that the optimal transmission power will not change within one epoch. Therefore, the transmission power of ST a in epochs i and i + 1 can be represented by p i and p i + 1 , respectively . The overall throughput of ST a in two epochs is denoted by Z a , where Z a = l i 2 log 2 ( 1 + p i ) + l i + 1 2 log 2 ( 1 + p i + 1 ) . (22) Assume in strategy ST b , the transmission power in epo ch i is p i − ∆ p i , where ∆ p i ∈ ( 0 , p i ) . Consequently , the residual energy at t i is E r i = ∆ p i l i . Denote the dierence between the energy harvested in ST a and ST b at t i by ∆ E h i , which can be calculated by substituting E r i into (3), i.e., ∆ E h i = Q ( E r i ) − Q ( 0 ) = A 1 i ( ∆ p i l i ) 1 2  A 3 i + A 4 i ( ∆ p i l i ) 1 2  . (23) As analyzed in Section 4.1, A 4 i is negative, but A 1 i and A 3 i are positive values; hence we could always nd a small ∆ p i that makes ∆ E h i > 0 . Therefore, ST b can choose higher transmission power by ∆ p i + 1 , where ∆ p i + 1 = ∆ p i l i + ∆ E h i l i + 1 . (24) The overall throughput of ST b in epochs i and i + 1 will be Z b = l i 2 log 2 ( 1 + p i − ∆ p i ) + l i + 1 2 log 2 ( 1 + p i + 1 + ∆ p i + 1 ) . (25) Through some simple calculations, it can b e obtained that the derivative of Z b − Z a with respect to ∆ p i is continuous and positive 9 innite at ∆ p i = 0 ; meanwhile, Z b = Z a at ∆ p i = 0 . Accordingly , a small ∆ p i could always be found to make Z b − Z a > 0 , i.e., Z b > Z a , which indicates that ST a is not optimal. Hence, the battery cannot be fully deplete d before the last ep och in an optimal scheduling strategy . □ .2 Derivation of KKT Conditions T o solve the stationarity equations in the KKT conditions, we need to simplify ∇ p ∗ m   m j = 1 ˜ Q j  through the following steps, where ˜ Q j = E h j is the power-harvest function: Y m ( m ) = ∇ p ∗ m    m  j = 1 ˜ Q j  p ∗ 1 , . . . , p ∗ j     = ∇ p ∗ m  ˜ Q m  p ∗ 1 , . . . , p ∗ m   = ∇ p ∗ m  E r m   1 2 A 1 m A 3 m  E r m  − 1 2 + A 1 m A 4 m  . (26) Based on (12), we have that ∇ p ∗ m  E r m  = ∇ p ∗ m    m − 1  j = 0 E h j − m  j = 1 p ∗ j l j    = ∇ p ∗ m    E h 0 + m − 1  j = 1 ˜ Q m  p ∗ 1 , . . . , p ∗ j     − l m = − l m . (27) Based on the denition of X i in (19), it could be obtained that Y m ( m ) = − l m  1 2 A 1 m A 3 m  E r m  − 1 2 + A 1 m A 4 m  = − l m X m = l m [ 1 − ( X m + 1 )] . (28) Similarly , we have that Y m ( m + 1 ) = ∇ p ∗ m    m + 1  j = 1 E h j    = ∇ p ∗ m  ˜ Q m + 1  p ∗ 1 , . . . , p ∗ m + 1   + Y m ( m ) = X m + 1  ∇ p ∗ m  ˜ Q m  p ∗ 1 , . . . , p ∗ m   − l m  + Y m ( m ) = l m [ 1 − ( X m + 1 + 1 ) ( X m + 1 )] . (29) Eventually , Y m ( i ) can be represented by X m through: Y m ( i ) = l m       1 − i  j = m  X j + 1        . (30) According to the properties introduced in Section 5.2, E r i > 0 unless i = N + 1 and E r i + E h i < e m . Hence in complementary slackness, we have that λ j = µ j = 0 for j = 1 , . . . , N . Then ∇ p ∗ m L = l m 2 ln2 ( 1 + p ∗ m ) + λ N + 1    ∇ p ∗ m    N  j = 1 ˜ Q j  p ∗ 1 , . . . , p ∗ j     − l m    = l m 2 ln2 ( 1 + p ∗ m ) + λ N + 1 ( Y m ( N ) − l m ) = l m       1 2 ln2 ( 1 + p ∗ m ) − λ N + 1 N  j = m  X j + 1        . (31) T o satisfy the stationarity of KKT conditions, we have that p ∗ m = 1 2 λ N + 1 ln2 N  j = m  X j + 1  − 1 , (32) Then we can get iteration between p ∗ m + 1 and p ∗ m in (19). REFERENCES [1] Kaibin Huang and Xiangyun Zhou. Cutting the last wires for mobile communica- tions by micro wave power transfer . IEEE Communications Magazine , 53(6):86–93, 2015. [2] Howard H Y ang, Jemin Lee, and T ony QS Quek. Heterogeneous cellular network with energy har vesting-based D2D communication. IEEE Transactions on Wireless Communications , 15(2):1406–1419, 2016. [3] Vincent Liu, Aaron Parks, V amsi T alla, Shyamnath Gollakota, David W etherall, and Joshua R Smith. Ambient backscatter: wireless communication out of thin air. In Proceedings of the Association for Computing Machinery’s Special Interest Group on Data Communications (SIGCOMM) , pages 39–50. ACM, 2013. [4] Y u Luo, Lina Pu, Y anxiao Zhao, Guodong W ang, and Min Song. Optimal energy requesting strategy for RF-based energy harvesting wireless communications. In Proceedings of the International Conference on Computer Communications (INFO- COM) , pages 1–9. IEEE, 2017. [5] Jie Xu and Rui Zhang. Throughput optimal policies for energy harvesting wireless transmitters with non-ideal circuit power. IEEE Journal on Selected Areas in Communications , 32(2):322–332, 2014. [6] Oner Orhan, Deniz Gunduz, and Elza Erkip. Energy harvesting broadband communication systems with processing energy cost. IEEE Transactions on Wireless Communications , 13(11):6095–6107, 2014. [7] Pranav Sakulkar and Bhaskar Krishnamachari. Online learning schemes for power allo cation in energy harvesting communications. arXiv preprint arXiv:1607.02552 , 2016. [8] Kaya T utuncuoglu, A ylin Y ener , and Sennur Ulukus. Optimum policies for an energy har vesting transmitter under energy storage losses. IEEE Journal on Selected A reas in Communications , 33(3):467–481, 2015. [9] Maria Gorlatova, A ya W allwater, and Gil Zussman. Networking low-power energy harvesting devices: measurements and algorithms. IEEE Transactions on Mobile Computing , 12(9):1853–1865, 2013. [10] Jing Y ang and Sennur Ulukus. Optimal packet scheduling in an energy harvesting communication system. IEEE Transactions on Communications , 60(1):220–230, 2012. [11] Kaya T utuncuoglu and A ylin Y ener . Optimum transmission policies for battery limited energy harvesting nodes. IEEE Transactions on Wireless Communications , 11(3):1180–1189, 2012. [12] Omur Ozel, K aya T utuncuoglu, Jing Y ang, Sennur Ulukus, and A ylin Y ener . Trans- mission with energy harvesting nodes in fading wireless channels: optimal p oli- cies. IEEE Journal on Selected A reas in Communications , 29(8):1732–1743, 2011. [13] Alessandro Biason and Michele Zorzi. On the eects of batter y imperfections in an energy harvesting device. In Proceedings of International Conference on Computing, Networking and Communications (ICNC) , pages 1–7. IEEE, 2016. [14] Sennur Ulukus, A ylin Y ener , Elza Erkip, Osvaldo Simeone, Michele Zorzi, Pulkit Grover , and Kaibin Huang. Energy harvesting wireless communications: a review of recent advances. IEEE Journal on Selected Ar eas in Communications , 33(3):360– 381, 2015. [15] Xiaojing Chen, W ei Ni, Xin W ang, and Yichuang Sun. Provisioning quality-of- service to energy harvesting wireless communications. IEEE Communications Magazine , 53(4):102–109, 2015. [16] Bertrand Devillers and Deniz Gündüz. A general framework for the optimization of energy har vesting communication systems with battery imperfections. Journal of Communications and Networks , 14(2):130–139, 2012. 10 [17] Manuel Piñuela, Paul D Mitcheson, and Stepan Lucyszyn. Ambient RF energy har- vesting in urban and semi-urban environments. IEEE Transactions on Microwave Theory and Techniques , 61:2715–2726, 2013. [18] Hamid Jabbar , Y oung S Song, and Taiky eong T ed Jeong. Rf energy har vesting system and circuits for charging of mobile devices. IEEE Transactions on Consumer Electronics , 56(1), 2010. [19] D Bouchouicha, F Dup ont, M Latrach, and L V entura. Ambient RF energy harvesting. In Proceedings of the International Conference on Renewable Energies and Power Quality (ICREPQ) , pages 1–4, 2010. [20] Powercast Corporation. Lifetime power energy har vesting development kit for wireless sensors user’s manual , July 2015. [21] Raghu Das and Peter Harrop. RFID forecasts, players and opportunities 2014– 2024. IDT echEx report , 2014. [22] Mandeep Kaur , Manjeet Sandhu, Neeraj Mohan, and Par vinder S Sandhu. RFID technology principles, advantages, limitations & its applications. International Journal of Computer and Electrical Engineering , 3(1):151–157, 2011. [23] Nicolo Michelusi, Kostas Stamatiou, and Michele Zorzi. Transmission policies for energy harvesting sensors with time-correlated energy supply . IEEE Transactions on Communications , 61(7):2988–3001, 2013. 11