Distributed Power Control for Delay Optimization in Energy Harvesting Cooperative Relay Networks

📝 Abstract

We consider cooperative communications with energy harvesting (EH) relays, and develop a distributed power control mechanism for the relaying terminals. Unlike prior art which mainly deal with single-relay systems with saturated traffic flow, we address the case of bursty data arrival at the source cooperatively forwarded by multiple half-duplex EH relays. We aim at optimizing the long-run average delay of the source packets under the energy neutrality constraint on power consumption of each relay. While EH relay systems have been predominantly optimized using either offline or online methodologies, we take on a more realistic learning-theoretic approach. Hence, our scheme can be deployed for real-time operation without assuming acausal information on channel realizations, data/energy arrivals as required by offline optimization, nor does it rely on precise statistics of the system processes as is the case with online optimization. We formulate the problem as a partially observable identical payoff stochastic game (PO-IPSG) with factored controllers, in which the power control policy of each relay is adaptive to its local source-to-relay/relay-to-destination channel states, its local energy state as well as to the source buffer state information. We derive a multi-agent reinforcement learning algorithm which is convergent to a locally optimal solution of the formulated PO-IPSG. The proposed algorithm operates without explicit message exchange between the relays, while inducing only little source-relay signaling overhead. By simulation, we contrast the delay performance of the proposed method against existing heuristics for throughput maximization. It is shown that compared with these heuristics, the systematic approach adopted in this paper has a smaller sub-optimality gap once evaluated against a centralized optimal policy armed with perfect statistics.

💡 Analysis

We consider cooperative communications with energy harvesting (EH) relays, and develop a distributed power control mechanism for the relaying terminals. Unlike prior art which mainly deal with single-relay systems with saturated traffic flow, we address the case of bursty data arrival at the source cooperatively forwarded by multiple half-duplex EH relays. We aim at optimizing the long-run average delay of the source packets under the energy neutrality constraint on power consumption of each relay. While EH relay systems have been predominantly optimized using either offline or online methodologies, we take on a more realistic learning-theoretic approach. Hence, our scheme can be deployed for real-time operation without assuming acausal information on channel realizations, data/energy arrivals as required by offline optimization, nor does it rely on precise statistics of the system processes as is the case with online optimization. We formulate the problem as a partially observable identical payoff stochastic game (PO-IPSG) with factored controllers, in which the power control policy of each relay is adaptive to its local source-to-relay/relay-to-destination channel states, its local energy state as well as to the source buffer state information. We derive a multi-agent reinforcement learning algorithm which is convergent to a locally optimal solution of the formulated PO-IPSG. The proposed algorithm operates without explicit message exchange between the relays, while inducing only little source-relay signaling overhead. By simulation, we contrast the delay performance of the proposed method against existing heuristics for throughput maximization. It is shown that compared with these heuristics, the systematic approach adopted in this paper has a smaller sub-optimality gap once evaluated against a centralized optimal policy armed with perfect statistics.

📄 Content

0018-9545 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2016.2610444, IEEE Transactions on Vehicular Technology IEEE Transactions on Vehicular Technology

1  Abstract— We consider cooperative communications with energy harvesting (EH) relays, and develop a distributed power control mechanism for the relaying terminals. Unlike prior art which mainly deal with single-relay systems with saturated traffic flow, we address the case of bursty data arrival at the source cooperatively forwarded by multiple half-duplex EH relays. We aim at optimizing the long-run average delay of the source packets under the energy neutrality constraint on power consumption of each relay. While EH relay systems have been predominantly optimized using either offline or online methodologies, we take on a more realistic learning-theoretic approach. Hence, our scheme can be deployed for real-time operation without assuming acausal information on channel realizations, data/energy arrivals as required by offline optimization, nor does it rely on precise statistics of the system processes as is the case with online optimization. We formulate the problem as a partially observable identical payoff stochastic game (PO-IPSG) with factored controllers, in which the power control policy of each relay is adaptive to its channel and energy states as well as to the state of the source buffer. We equip each relay with a reinforcement learning procedure, and prove that the parallel execution of this procedure is convergent to (at least) a locally optimal solution of the formulated PO-IPSG. The proposed algorithm operates without explicit message exchange between the relays, while inducing only little source-relay signaling overhead. By simulation, we contrast the delay performance of the proposed method against existing heuristics for throughput maximization. It is shown that compared with these heuristics, the systematic approach adopted in this paper has a smaller sub-optimality gap once evaluated against a centralized optimal policy armed with perfect statistics.

Index Terms— bursty traffic, cooperative relaying, energy harvesting, power control, reinforcement learning, stochastic game, wireless communication.

Copyright (c) 2016 IEEE. Personal use of this material is permitted. However, permission to use this material for any other purposes must be obtained from the IEEE by sending a request to pubs-permissions@ieee.org. V. Hakami is with the Department of Computer Engineering, Iran University of Science and Technology (IUST), Tehran 16846-13114, Iran (email: vhakami@iust.ac.ir).
M. Dehghan is with the Department of Computer Engineering and Information Technology, Amirkabir University of Technology (AUT), Tehran 15916-34311, Iran (e-mail: dehghan@aut.ac.ir). I. INTRODUCTION OOPERATIVE relaying is a promising paradigm which results in broader coverage and in combating the wireless channel impairments. Relay-assisted transmission mitigates the need to use a high power at the transmitter, leading to prolonged battery life and lower level of interference [1]. Relays in wireless networks can be classified as decode-and- forward (DaF) relays, which decode and possibly re-encode the information before forwarding it, and amplify-and-forward (AaF) relays, which forward an amplified version of the signal without hard decoding. AaF relays compared with other types which require signal detection, are less complicated, have lower implementation cost, and are thus utilizable widely [4]. While cooperative relaying results in higher network capacity, in forwarding to the destination a representation of the signal it has received from the source, a relay consumes its own energy. Since replacing batteries for such devices is either impracticable or costly in several scenarios, recent advances in energy harvesting devices [5] have paved the way for self- sustainable relays [6] that power themselves from theoretically unlimited energy sources that are present in their surrounding environment (e.g., in the form of solar, vibration, thermoelectricity, etc.). However, the harvested energy rates are typically quite low with sporadic arrivals in random limited amounts, and it is thus desirable to accumulate the harvested energy by storing it in a buffer such as a rechargeable battery for subsequent usage. In practice, the energy buffer is restricted in size, and thus EH relays may face power outage whenever the energy consumption rate is higher than the harvesting rate. Hence, there is a need for novel power-use policies which exploit

This content is AI-processed based on ArXiv data.