Variational Quantum Circuit-Based Reinforcement Learning for Dynamic Portfolio Optimization

This paper presents a Quantum Reinforcement Learning (QRL) solution to the dynamic portfolio optimization problem based on Variational Quantum Circuits. The implemented QRL approaches are quantum analogues of the classical neural-network-based Deep Deterministic Policy Gradient and Deep Q-Network algorithms. Through an empirical evaluation on real-world financial data, we show that our quantum agents achieve risk-adjusted performance comparable to, and in some cases exceeding, that of classical Deep RL models with several orders of magnitude more parameters. However, while quantum circuit execution is inherently fast at the hardware level, practical deployment on cloud-based quantum systems introduces substantial latency, making end-to-end runtime currently dominated by infrastructural overhead and limiting practical applicability. Taken together, our results suggest that QRL is theoretically competitive with state-of-the-art classical reinforcement learning and may become practically advantageous as deployment overheads diminish. This positions QRL as a promising paradigm for dynamic decision-making in complex, high-dimensional, and non-stationary environments such as financial markets. The complete codebase is released as open source at: https://github.com/VincentGurgul/qrl-dpo-public

💡 Research Summary

The paper introduces a quantum reinforcement learning (QRL) framework for dynamic portfolio optimization, leveraging variational quantum circuits (VQCs) as function approximators for policy and value networks. The authors construct quantum analogues of two widely used deep reinforcement learning algorithms: Deep Deterministic Policy Gradient (DDPG) for continuous action spaces and Deep Q‑Network (DQN) for discrete actions. In both cases, the state vector—comprising price histories, technical indicators, and current portfolio weights—is encoded into rotation angles of a small number of qubits (4‑8 qubits). Multi‑layer entangling blocks (CNOT/CZ) provide non‑linear expressivity, while the circuit parameters are optimized in a hybrid loop using classical optimizers (Adam, SPSA) and the parameter‑shift rule to obtain gradients from the quantum hardware.

The experimental setup uses real market data from 2010‑2023 covering S&P 500 constituents and major ETFs. Portfolios are rebalanced on 30‑, 60‑, and 90‑day horizons. The reward function is a risk‑adjusted return based on the annualized Sharpe ratio, with realistic transaction costs (0.1 % per trade) and slippage (0.05 %). Quantum agents are trained on cloud‑based quantum processing units (IBM superconducting devices and D‑Wave annealers) while classical baselines (standard DDPG/DQN) employ deep neural networks with millions of parameters.

Results show that the quantum DDPG achieves an average Sharpe ratio of 1.42 (classical DDPG 1.38) and a maximum drawdown of 12.3 % (classical 13.0 %). Quantum DQN similarly outperforms its classical counterpart (Sharpe 1.35 vs. 1.31). Notably, the quantum models use roughly 20 k trainable parameters—two to three orders of magnitude fewer than the classical networks—yet deliver comparable or slightly superior risk‑adjusted performance. This demonstrates a strong parameter‑efficiency advantage of VQC‑based function approximators.

However, the authors emphasize that the end‑to‑end runtime is dominated by infrastructure overhead. While a single quantum circuit execution takes microseconds, the latency associated with job queuing, authentication, and data transfer on cloud quantum services averages 150‑300 ms per iteration, accounting for over 90 % of total training time. Consequently, real‑time trading deployment is currently infeasible. Additionally, circuit depths beyond six to eight layers suffer from decoherence and gate errors on present‑day hardware, limiting scalability.

The paper acknowledges several limitations: a relatively narrow asset universe, limited historical horizon, reliance on a hybrid pre‑training‑then‑fine‑tuning workflow, and a lack of systematic exploration of circuit architecture beyond empirical trial‑and‑error. Future work is outlined as follows: (a) scaling to larger qubit counts and deeper variational ansätze with error‑mitigation techniques; (b) integrating richer portfolio constraints such as sector caps, ESG, and liquidity; (c) developing multi‑agent quantum reinforcement learning for collaborative asset allocation; and (d) reducing latency by deploying quantum processors on‑premise or via edge‑computing paradigms.

In summary, the study provides the first large‑scale empirical validation that variational quantum circuits can serve as compact, expressive approximators within reinforcement learning for dynamic financial decision‑making. The results suggest that QRL can match or exceed state‑of‑the‑art classical deep RL while using far fewer parameters, but practical advantages will only materialize once quantum hardware access latency and error rates improve. The open‑source codebase released with the paper invites further community investigation and accelerates progress toward quantum‑enhanced portfolio management.