Grover's Search-Inspired Quantum Reinforcement Learning for Massive MIMO User Scheduling

The efficient user scheduling policy in the massive Multiple Input Multiple Output (mMIMO) system remains a significant challenge in the field of 5G and Beyond 5G (B5G) due to its high computational complexity, scalability, and Channel State Information (CSI) overhead. This paper proposes a novel Grover’s search-inspired Quantum Reinforcement Learning (QRL) framework for mMIMO user scheduling. The QRL agent can explore the exponentially large scheduling space effectively by applying Grover’s search to the reinforcement learning process. The model is implemented using our designed quantum-gate-based circuit, which imitates the layered architecture of reinforcement learning, where quantum operations act as policy updates and decision-making units. Moreover, the simulation results demonstrate that the proposed method achieves proper convergence and significantly outperforms classical Convolutional Neural Networks (CNN) and Quantum Deep Learning (QDL) benchmarks.

💡 Research Summary

The paper addresses the pressing challenge of user scheduling in massive Multiple‑Input Multiple‑Output (mMIMO) systems, a cornerstone of 5G and beyond‑5G networks. Classical approaches—whether heuristic, conventional machine‑learning, or deep‑learning based—suffer from exponential growth in computational load and channel state information (CSI) overhead as the number of antennas and users scales. To overcome these limitations, the authors propose a novel Quantum Reinforcement Learning (QRL) framework that integrates Grover’s quantum search algorithm directly into the reinforcement‑learning loop.

The system model assumes a single‑cell downlink with A = X·Y rectangular antenna array at the base station (BS) serving T single‑antenna users. The received signal, ergodic sum‑rate, and proportional‑fairness (PF) objective are formulated in standard MIMO notation, with Rician fading employed to capture both line‑of‑sight and non‑line‑of‑sight components. The scheduling decision is encoded as a binary vector θ(t)∈{0,1}^T, where θ_i(t)=1 indicates that user i is scheduled in the current slot.

The core contribution is a three‑layer quantum circuit that mimics the layered architecture of conventional reinforcement learning:

1. Hadamard (Superposition) Layer – T qubits (one per user) are initialized to |0⟩ and transformed by Hadamard gates into a uniform superposition over all 2^T possible scheduling vectors.
2. Oracle Layer – For each candidate policy the instantaneous sum‑rate is computed classically and fed back as a reward. If the reward exceeds a predefined threshold τ, the corresponding quantum basis state is marked by a phase flip using a multi‑controlled Z gate (preceded by Pauli‑X preprocessing). This step implements the “evaluation” phase of reinforcement learning.
3. Diffusion (Amplitude‑Amplification) Layer – A Grover diffusion operator (2|U⟩⟨U| – I) reflects the state amplitudes about the uniform mean, thereby amplifying the probability of the marked high‑reward policies.

The algorithm iterates the oracle‑diffusion sequence G times per training batch, updating a set of trainable parameters K that control the amplitude‑amplification strength. After a prescribed number of epochs, the quantum state is measured, collapsing to a concrete scheduling vector θ(t) that is then logged as the policy for that time slot. The training loop follows a standard batch‑wise reinforcement‑learning schedule, with learning rate η, batch size B, and epoch count N.

Simulation results are presented for a BS equipped with 32 antennas (A=32) and SNR = 20 dB. The authors evaluate three dimensions of scalability:

• Convergence – Starting from an average sum‑rate of 22 bps/Hz, the QRL agent steadily improves over 500 epochs, reaching ≈32 bps/Hz. The learning curve shows rapid early gains and later stabilization, indicating effective exploration‑exploitation balance.
• User‑Number Scaling – When the number of users varies from 2 to 10, QRL’s sum‑rate rises sharply, achieving ≈20 bps/Hz at T=10, whereas a classical Convolutional Neural Network (CNN) and a Quantum Neural Network (QNN) benchmark reach only ≈15.8 bps/Hz and ≈17.2 bps/Hz respectively. This corresponds to a 25 %–30 % relative improvement.
• Antenna‑Number Scaling – With a fixed user count (T=6) and antenna numbers ranging from 6 to 16, QRL’s performance climbs from ≈8.2 bps/Hz to ≈14.7 bps/Hz, again outperforming QNN and CNN across the board. The gap widens as the antenna array grows, demonstrating that QRL better exploits the spatial degrees of freedom.

Additional experiments varying SNR confirm that QRL’s advantage persists across different channel conditions, with the most pronounced gains observed at moderate to high SNR where the reward landscape is smoother.

The authors argue that the quadratic speed‑up offered by Grover’s search translates into a practical reduction of CSI traversal time and computational burden, enabling near‑optimal scheduling even in extremely large combinatorial spaces. They also acknowledge that the current work is confined to simulation; real quantum hardware introduces decoherence, gate errors, and limited qubit counts that could impede direct deployment. Future directions include integrating error‑correction techniques, developing hybrid quantum‑classical architectures, and testing on near‑term quantum processors.

In summary, the paper presents a compelling case that quantum‑enhanced reinforcement learning, powered by Grover’s search, can substantially improve user‑scheduling throughput and scalability in massive MIMO systems, pointing toward a viable pathway for quantum computing to impact next‑generation wireless networks.