ORCHID: Fairness-Aware Orchestration in Mission-Critical Air-Ground Integrated Networks

In the era of 6G Air-Ground Integrated Networks (AGINs), Unmanned Aerial Vehicles (UAVs) are pivotal for providing on-demand wireless coverage in mission-critical environments, such as post-disaster rescue operations. However, traditional Deep Reinforcement Learning (DRL) approaches for multi-UAV orchestration often face critical challenges: instability due to the non-stationarity of multi-agent environments and the difficulty of balancing energy efficiency with service equity. To address these issues, this paper proposes ORCHID (Orchestration of Resilient Coverage via Hybrid Intelligent Deployment), a novel stability-enhanced two-stage learning framework. First, ORCHID leverages a GBS-aware topology partitioning strategy to mitigate the exploration cold-start problem. Second, we introduce a Reset-and-Finetune (R&F) mechanism within the MAPPO architecture that stabilizes the learning process via synchronized learning rate decay and optimizer state resetting. This mechanism effectively suppresses gradient variance to prevent policy degradation, thereby ensuring algorithmic resilience in dynamic environments. Furthermore, we uncover a counter-intuitive efficiency-fairness synergy: contrary to the conventional trade-off, our results demonstrate that the proposed Max-Min Fairness (MMF) design not only guarantees service for cell-edge users but also achieves superior energy efficiency compared to Proportional Fairness (PF), which tends to converge to suboptimal greedy equilibria. Extensive experiments confirm that ORCHID occupies a superior Pareto-dominant position compared to state-of-the-art baselines, ensuring robust convergence and resilient connectivity in mission-critical scenarios.

💡 Research Summary

The paper tackles the pressing challenge of orchestrating multiple unmanned aerial vehicles (UAVs) to provide reliable, on‑demand wireless coverage in mission‑critical Air‑Ground Integrated Networks (AGINs) envisioned for 6G, especially in disaster‑relief scenarios where terrestrial infrastructure may be compromised. Existing multi‑agent deep reinforcement learning (MARL) solutions suffer from two fundamental problems: (1) non‑stationarity caused by simultaneous policy updates of many agents, which leads to unstable training and policy collapse, and (2) a perceived trade‑off between system‑wide energy efficiency and user fairness, where fairness‑oriented schemes are thought to sacrifice throughput.

ORCHID (Orchestration of Resilient Coverage via Hybrid Intelligent Deployment) is introduced as a two‑stage learning framework that directly addresses these issues. In the first stage, a ground‑base‑station (GBS)‑aware topology partitioning is performed using a modified K‑Means++ algorithm that incorporates both the fixed GBS location and the spatial density of ground users. User locations are modeled with a Thomas Cluster Process (TCP) to reflect realistic hotspot‑centric distributions typical of emergency situations. This partitioning provides a principled warm‑start for UAV deployment, dramatically reducing the cold‑start exploration cost and bias that plague naïve random initializations.

The second stage embeds a Reset‑and‑Finetune (R&F) mechanism within the Multi‑Agent Proximal Policy Optimization (MAPPO) architecture. At predefined intervals, the optimizer’s internal states (e.g., momentum buffers, adaptive learning‑rate statistics) are reset, while a synchronized exponential decay of the learning rate is applied across all agents. This dual action suppresses gradient variance, prevents the accumulation of stale momentum that can push policies into sub‑optimal equilibria, and enables fine‑grained adjustments in later training phases. Empirical results show that policies trained with R&F avoid the abrupt performance drops observed in vanilla MAPPO or MADDPG, achieving smoother, more reliable convergence.

A central contribution is the systematic comparison between Max‑Min Fairness (MMF) and Proportional Fairness (PF) objectives. Contrary to the conventional belief that MMF inevitably reduces overall efficiency, the authors integrate the MMF criterion directly into the loss function alongside an energy‑efficiency term, forming a multi‑objective optimization problem. Simulations reveal that the MMF‑driven policy not only guarantees service to cell‑edge users (eliminating coverage holes) but also attains 4–7 % higher normalized energy efficiency (NEE) than the PF baseline. This “efficiency‑fairness synergy” demonstrates that, with an appropriate learning architecture, fairness and efficiency can be simultaneously maximized.

The system model explicitly distinguishes the terrestrial tier (a fixed macro GBS with unlimited power) from the aerial tier (a fleet of rotary‑wing UAVs constrained by battery capacity). UAV dynamics follow discrete‑time kinematics within a prescribed altitude corridor, and the communication channel incorporates realistic line‑of‑sight probabilities and distance‑dependent path loss. The joint trajectory‑design and power‑control problem is formalized as maximizing a composite objective comprising NEE, average user throughput, and Jain’s Fairness Index (JFI), subject to UAV energy budgets and QoS constraints for users not covered by the GBS.

Extensive experiments evaluate ORCHID against state‑of‑the‑art baselines: MADDPG, standard MAPPO, and heuristic optimization approaches (e.g., static K‑Means placement). Across three key metrics—NEE, average rate, and JFI—ORCHID consistently occupies a Pareto‑dominant region, delivering a 6.8 % NEE gain while preserving or improving fairness and throughput. Moreover, convergence speed improves by roughly 30 % relative to baselines, and policy variance remains low even under dynamic user distributions and channel fluctuations, confirming the framework’s suitability for real‑time, safety‑critical deployments.

In conclusion, ORCHID offers a comprehensive solution that (i) mitigates MARL non‑stationarity through GBS‑aware initialization and synchronized optimizer resets, (ii) demonstrates that Max‑Min Fairness can coexist with superior energy efficiency, and (iii) provides a scalable, robust orchestration strategy for UAV swarms in next‑generation 6G AGINs. The work paves the way for practical, fairness‑aware UAV coordination in emergency communications, where guaranteeing service to every user is as critical as conserving limited aerial resources.