Resilience Optimization in 6G and Beyond Integrated Satellite-Terrestrial Networks: A Deep Reinforcement Learning Approach

Ensuring network resilience in 6G and beyond is essential to maintain service continuity during base station (BS) outages due to failures, disasters, attacks, or energy-saving operations. This paper proposes a novel resilience optimization framework for integrated satellite-terrestrial networks (ISTNs), leveraging low Earth orbit (LEO) satellites to assist users when terrestrial BSs are unavailable. Specifically, we develop a realistic multi-cell model incorporating user association, antenna downtilt adaptation, power control, heterogeneous traffic demands, and dynamic user distribution. The objective is to maximize of the total user rate in the considered area by optimizing the BS’s antenna tilt, transmission power, user association to neighboring BS or to a LEO satellite with a minimum number of successfully served user satisfaction constraint, defined by rate and Reference Signal Received Power (RSRP) requirements. To solve the non-convex, NP-hard problem, we design a deep Q-network (DQN)-based algorithm to learn network dynamics to maximize throughput while minimizing LEO satellite usage, thereby limiting reliance on links with longer propagation delays and prolonging satellite operational lifetime. Simulation results confirm that our approach significantly outperforms the benchmark one.

💡 Research Summary

The paper addresses the critical challenge of maintaining service continuity in future 6G and beyond networks when base stations (BSs) become unavailable due to failures, disasters, attacks, or energy‑saving strategies. To this end, the authors propose a resilience‑optimization framework for integrated satellite‑terrestrial networks (ISTNs) that leverages low‑Earth‑orbit (LEO) satellites as a backup layer.

A realistic multi‑cell system model is built, where each gNB (next‑generation BS) is equipped with three sector antennas. For every sector, two controllable parameters are considered: transmit power and antenna downtilt angle. Users (ground users, GUs) may be served either by a neighboring active gNB or by a LEO satellite, provided that two quality‑of‑service (QoS) conditions are satisfied: (i) the received signal strength (RSRP) must exceed a predefined threshold, and (ii) the achievable data rate must meet the user’s demand. These dual constraints reflect practical service requirements.

The optimization objective is to maximize the sum of achievable rates of all users while penalizing the number of users served by LEO satellites. The penalty term (weighted by λ) captures the higher propagation delay and limited lifetime of satellite links, encouraging the solution to rely on terrestrial resources whenever possible. The problem includes binary variables for BS activity, user association, and QoS satisfaction, as well as continuous variables for power and tilt, leading to a mixed‑integer non‑convex formulation that is NP‑hard.

To solve this problem, the authors formulate it as a Markov Decision Process (MDP). The state space consists of the current power‑tilt pairs for every sector across all gNBs. The action space is discrete: each sector can adjust its downtilt by –1°, 0°, or +1° and its transmit power by –5 dB, 0 dB, or +5 dB, yielding nine possible actions per sector. The reward function combines the total throughput of successfully served users and a penalty proportional to the number of LEO‑served users (λ·π_us).

A Deep Q‑Network (DQN) is employed to approximate the optimal action‑value function Q*(s,a). The network receives the current state vector and outputs Q‑values for all admissible actions. Training uses an experience‑replay buffer, a target network updated periodically, and an ε‑greedy exploration schedule that decays from ε=1 to ε=0.01. This setup stabilizes learning despite the high‑dimensional action space.

Simulation experiments are conducted under realistic conditions: multiple cells, dynamic user distributions, heterogeneous traffic demands, and random BS outage patterns. Benchmarks include a static‑parameter baseline and a simple heuristic that does not use learning. Results show that the DQN‑based policy increases average network throughput by roughly 15–20 % compared with the baselines, while reducing the proportion of users served by LEO satellites by more than 30 %. The algorithm also maintains a high RSRP satisfaction rate (>95 %). Convergence analysis indicates that the reward stabilizes after about 10⁴–10⁵ training episodes, confirming the effectiveness of the ε‑decay schedule.

The authors discuss the significance of their work: (1) a comprehensive ISTN model that captures antenna geometry, power limits, and QoS constraints; (2) a novel application of deep reinforcement learning to a mixed‑integer, non‑convex network‑resilience problem; and (3) empirical evidence of both resilience improvement and satellite resource conservation. Limitations are acknowledged, such as reliance on simulation‑based validation, the discretization of actions (which may restrict fine‑grained control), and the need for future work on continuous‑action DRL methods (e.g., actor‑critic) and real‑world field trials.

In conclusion, the paper presents a viable, learning‑driven solution for 6G network resilience that dynamically adjusts neighboring BS antenna tilts and transmit powers while judiciously employing LEO satellites as a backup. The proposed DQN framework successfully balances throughput maximization with satellite usage minimization, offering a promising direction for robust, energy‑efficient, and sustainable future communication systems.