UAV-Assisted Resilience in 6G and Beyond Network Energy Saving: A Multi-Agent DRL Approach

This paper investigates the unmanned aerial vehicle (UAV)-assisted resilience perspective in the 6G network energy saving (NES) scenario. More specifically, we consider multiple ground base stations (GBSs) and each GBS has three different sectors/cells in the terrestrial networks, and multiple cells are turned off due to NES or incidents, e.g., disasters, hardware failures, or outages. To address this, we propose a Multi-Agent Deep Deterministic Policy Gradient (MADDPG) framework to enable UAV-assisted communication by jointly optimizing UAV trajectories, transmission power, and user-UAV association under a sleeping ground base station (GBS) strategy. This framework aims to ensure the resilience of active users in the network and the long-term operability of UAVs. Specifically, it maximizes service coverage for users during power outages or NES zones, while minimizing the energy consumption of UAVs. Simulation results demonstrate that the proposed MADDPG policy consistently achieves high coverage ratio across different testing episodes, outperforming other baselines. Moreover, the MADDPG framework attains the lowest total energy consumption, with a reduction of approximately 24% compared to the conventional all GBS ON configuration, while maintaining a comparable user service rate. These results confirm the effectiveness of the proposed approach in achieving a superior trade-off between energy efficiency and service performance, supporting the development of sustainable and resilient UAV-assisted cellular networks.

💡 Research Summary

The paper addresses the dual challenge of energy efficiency and resilience in upcoming 6G networks by leveraging unmanned aerial vehicles (UAVs) as aerial base stations (ABS) to compensate for sleeping ground base stations (GBSs). The authors consider a realistic scenario where each GBS consists of three sectors, some of which may be switched off either for network energy‑saving (NES) purposes or due to unexpected incidents such as disasters or hardware failures. In such “outage” zones, users must be served by UAVs that can quickly reposition themselves and provide line‑of‑sight connectivity.

To jointly optimize UAV trajectories, transmit power, and user‑UAV association, the authors formulate a partially observable Markov decision process (POMDP) and adopt a multi‑agent deep deterministic policy gradient (MADDPG) framework with centralized training and decentralized execution (CTDE). Each UAV is modeled as an independent agent equipped with its own actor‑critic pair. During training, the critic of each agent receives the global state (including all users’ locations, rate demands, and cell on/off states) and the actions of all agents, enabling coordinated learning. At execution time, each UAV acts only on its local observation, which contains its own position, power, neighboring UAV states, the five nearest users’ positions and rate requirements, and the current on/off status of all cells.

The system model includes a detailed 3‑D distance‑based Rician fading channel, SINR calculation that accounts for inter‑UAV interference, and a propulsion‑plus‑communication energy model for UAVs. The reward function balances a coverage term (the fraction of users in sleeping cells that are successfully served with the required QoS) against an energy penalty (transmit power plus propulsion cost). Hyper‑parameters ω₁ and ω₂ weight these two objectives.

Training enhancements comprise prioritized experience replay (PER) to sample high‑TD‑error transitions more frequently, and Ornstein‑Uhlenbeck (OU) noise to generate temporally correlated exploration actions suitable for continuous UAV dynamics. The authors also enforce realistic constraints on UAV speed, transmit power, and total network power.

Simulation experiments are based on synthetic traffic patterns derived from a real 5G dataset, ensuring that sudden demand surges resemble realistic usage. Over 25 000 training episodes, the average episodic reward stabilizes around 0.032, with a noticeable improvement after the learning‑rate decay at episode 5 000. The proposed MADDPG policy consistently outperforms baseline methods—including a single‑agent DDPG, independent Q‑learning, and a static‑trajectory scheme—both in terms of coverage ratio and total energy consumption. Specifically, the MADDPG solution achieves roughly a 24 % reduction in total network energy compared with a configuration where all GBSs remain active, while maintaining a comparable user service rate.

The paper’s contributions are threefold: (1) it is the first to study UAV‑assisted resilience for 6G NES using a multi‑agent DRL approach; (2) it integrates realistic traffic models, detailed energy consumption, and interference‑aware SINR calculations into the learning environment; (3) it demonstrates that CTDE‑MADDPG can learn cooperative policies that balance coverage and energy efficiency in a dynamic, partially observable setting.

Limitations include the assumption of a fixed UAV altitude, a bounded 2‑D operating area, and simplified battery/propulsion models. Centralized training also requires global state information, which may become a bottleneck for very large‑scale deployments. Future work is suggested on multi‑altitude UAVs, asynchronous learning, incorporation of flight‑restriction zones, and real‑world field trials.

In summary, the study presents a compelling DRL‑driven framework that enables UAVs to dynamically restore service in sleeping‑cell scenarios while substantially cutting energy consumption, thereby advancing the vision of sustainable and resilient 6G networks.