Decentralized Spatial Reuse Optimization in Wi-Fi: An Internal Regret Minimization Approach

Spatial Reuse (SR) is a cost-effective technique for improving spectral efficiency in dense IEEE 802.11 deployments by enabling simultaneous transmissions. However, the decentralized optimization of SR parameters – transmission power and Carrier Sensing Threshold (CST) – across different Basic Service Sets (BSSs) is challenging due to the lack of global state information. In addition, the concurrent operation of multiple agents creates a highly non-stationary environment, often resulting in suboptimal global configurations (e.g., using the maximum possible transmission power by default). To overcome these limitations, this paper introduces a decentralized learning algorithm based on regret-matching, grounded in internal regret minimization. Unlike standard decentralized ``selfish’’ approaches that often converge to inefficient Nash Equilibria (NE), internal regret minimization guides competing agents toward Correlated Equilibria (CE), effectively mimicking coordination without explicit communication. Through simulation results, we showcase the superiority of our proposed approach and its ability to reach near-optimal global performance. These results confirm the not-yet-unleashed potential of scalable decentralized solutions and question the need for the heavy signaling overheads and architectural complexity associated with emerging centralized solutions like Multi-Access Point Coordination (MAPC).

💡 Research Summary

The paper tackles the problem of spatial reuse (SR) in dense IEEE 802.11 deployments, where each Basic Service Set (BSS) must jointly select a transmit power and a carrier‑sensing threshold (CST) to enable concurrent transmissions. Traditional decentralized solutions rely on external‑regret minimization (e.g., multi‑armed bandits, Q‑learning) that aim to maximize an individual’s cumulative reward. In multi‑agent settings these approaches often converge to inefficient Nash equilibria (NE), typically characterized by all BSSs using the maximum power or overly conservative CST, which severely limits overall throughput.

To overcome this limitation, the authors propose a decentralized learning algorithm based on internal regret minimization, specifically the regret‑matching procedure introduced by Hart and Mas‑Colell. Internal regret measures the loss incurred by not swapping a played action j with an alternative action k in the same context. By minimizing this swap‑regret, agents are driven toward a Correlated Equilibrium (CE), a state where agents’ strategies can be implicitly coordinated without any explicit signaling. CE generally yields higher social welfare than NE because it allows joint action profiles that are mutually beneficial.

The SR problem is formalized as a multi‑agent multi‑armed bandit game. Each BSS’s action set consists of discrete (power, CST) pairs. At each time slot t, agents receive bandit feedback: the actual normalized throughput for the chosen action and no direct information about unchosen actions. To estimate the missing rewards, the authors introduce a heuristic that combines an estimated airtime factor (capturing contention, fairness penalties, and hidden‑node starvation) with an estimated data rate derived from the expected RSSI and the corresponding MCS table. The airtime estimate includes a contention term proportional to the number of neighboring nodes whose RSSI exceeds the candidate CST, and a fairness term that penalizes actions that would starve hidden nodes. An SINR‑based capture‑effect indicator further filters out actions that would likely lead to packet loss.

Algorithm 1 details the learning loop. Each BSS maintains a swap‑regret matrix Q and a preference vector π. In every slot the agent selects the action with the highest current preference (a pure‑strategy choice to avoid instability). After observing the actual reward, the algorithm updates Q for all possible swaps using a decay factor λ to discount stale information. The preference vector is then refreshed proportionally to the positive entries of Q, normalized by a parameter μ that guarantees all preferences remain non‑negative and sum to one. A stickiness factor is also employed to reduce excessive switching.

Simulation results compare the proposed internal‑regret approach against external‑regret bandits, Q‑learning, cooperative bandits, and a centralized Multi‑Access Point Coordination (MAPC) scheme. The internal‑regret method consistently achieves higher average throughput (≈20 % improvement over external‑regret baselines) while using lower transmit powers and reducing collision rates. It converges to a CE within a few hundred slots, demonstrating that decentralized agents can implicitly coordinate to exploit SR without any extra signaling overhead. The authors also discuss sensitivity to the accuracy of the reward estimator: under‑estimation can trap the system in sub‑optimal equilibria, while over‑estimation may cause excessive exploration and transient unfairness. Nonetheless, the estimator need not be perfect; its role is to provide a reasonable heuristic that bridges the gap between fully decentralized and fully centralized control.

The paper’s contributions are threefold: (1) introducing internal regret minimization to the Wi‑Fi SR domain, (2) designing a practical reward‑estimation mechanism that respects CSMA/CA dynamics, and (3) showing that decentralized learning can approach the performance of heavyweight MAPC solutions. Limitations include dependence on the estimator’s fidelity and the focus on only two control variables (power and CST). Future work is suggested on extending the framework to additional dimensions (channel selection, BSS coloring, frame aggregation), improving real‑time RSSI/MCS mapping, and adapting decay parameters dynamically for highly mobile or bursty traffic scenarios.

In summary, the study demonstrates that regret‑matching with internal regret minimization offers a powerful, scalable, and low‑overhead pathway to achieve near‑optimal spatial reuse in dense Wi‑Fi networks, effectively delivering the benefits of coordinated operation while preserving the decentralized nature of IEEE 802.11.