Quantum feedback control with a transformer neural network architecture

Attention-based neural networks such as transformers have revolutionized various fields such as natural language processing, genomics, and vision. Here, we demonstrate the use of transformers for quantum feedback control through both a supervised and reinforcement learning approach. In particular, due to the transformer’s ability to capture long-range temporal correlations and training efficiency, we show that it can surpass some of the limitations of previous control approaches, e.g.~those based on recurrent neural networks trained using a similar approach or policy based reinforcement learning. We numerically show, for the example of state stabilization of a two-level system, that our bespoke transformer architecture can achieve near unit fidelity to a target state in a short time even in the presence of inefficient measurement and Hamiltonian perturbations that were not included in the training set as well as the control of non-Markovian systems. We also demonstrate that our transformer can perform energy minimization of non-integrable many-body quantum systems when trained for reinforcement learning tasks. Our approach can be used for quantum error correction, fast control of quantum states in the presence of colored noise, as well as real-time tuning, and characterization of quantum devices.

💡 Research Summary

The paper introduces a transformer‑based architecture for quantum feedback control, addressing the limitations of recurrent neural networks (RNNs) in handling long‑range temporal dependencies and non‑Markovian dynamics. The authors develop a custom encoder‑decoder pair, named QuantumEncoder and QuantumDecoder, that processes the initial quantum state together with the continuous measurement record. The encoder embeds these inputs into a high‑dimensional latent space using multi‑head self‑attention, while the decoder, equipped with causally masked attention, autoregressively predicts the optimal control parameter λₜ for the next time step. Training is performed with two complementary strategies: supervised learning, where locally optimal control sequences generated by the PaQS algorithm serve as labels, and reinforcement learning, where the transformer directly learns a policy πθ(λₜ₊₁|rₜ) that maximizes a reward based on state fidelity or energy minimization.

The authors first demonstrate the approach on a two‑level system (TLS) with Hamiltonian Ĥ(λₜ)=ℏε/2 σ̂_z+ℏλₜ/2 σ̂_x under continuous weak measurement (jump operator ĉ=√κ σ̂_-). Despite being trained only on unbiased dynamics (ε=0) and ideal measurement efficiency, the transformer successfully stabilizes the TLS to the target superposition |ψ_target⟩=(|0⟩+i|1⟩)/√2 even when the bias is switched on (ε=0.5) and the measurement efficiency is reduced to η=0.7. The fidelity reaches near‑unit values within a short time, outperforming random control and matching or surpassing the PaQS baseline. Moreover, inference time for a full 100‑step trajectory is about 0.23 s on a standard laptop, roughly two orders of magnitude faster than the gradient‑based PaQS solver (≈19 s), highlighting the transformer’s suitability for real‑time control despite higher memory consumption.

To test generalization, the authors extend the scenario to a non‑Markovian setting by coupling the TLS to a harmonic oscillator (reaction coordinate) with Hamiltonian Ĥ(λₜ)=ℏε/2 σ̂_z+ℏλₜ/2 σ̂_x+ℏΩ â†â+ℏg σ̂_z(â+â†). The oscillator is continuously monitored, producing non‑Markovian back‑action on the qubit when κ≲g. Using transfer learning, the transformer is fine‑tuned on a small dataset of optimal λₜ values for this enlarged system. Benchmarks against a vanilla RNN and a GRU‑RNN show that while the latter two perform slightly better for short histories (≤60 time steps), the transformer clearly outperforms them when the context window is extended to 2000 measurement samples. This superiority stems from the transformer’s ability to attend to arbitrarily long sequences without suffering from vanishing gradients, a known issue for recurrent architectures.

Finally, the paper tackles many‑body state preparation via reinforcement learning. An open N‑qubit mixed‑field Ising chain is driven by a control field, and the reward is defined as the negative of the final energy (or equivalently the overlap with the ground state). The transformer learns a policy that drives the system close to its ground state faster and to lower energy than conventional policy‑gradient methods, demonstrating its capacity to capture complex many‑body correlations.

Key contributions of the work are:

1. Long‑range temporal modeling: Self‑attention enables the network to incorporate the full measurement history, essential for non‑Markovian quantum dynamics.
2. Hybrid learning framework: Combining supervised learning (using analytically or numerically optimal controls) with reinforcement learning provides flexibility for tasks lacking explicit optimal solutions.
3. Robustness to imperfections: The transformer maintains high fidelity under reduced measurement efficiency and Hamiltonian perturbations not seen during training.
4. Computational efficiency: Inference is orders of magnitude faster than iterative solvers, making the approach viable for real‑time experimental deployment, albeit with higher memory demands.
5. Transferability: Fine‑tuning on related tasks (e.g., reaction‑coordinate models) requires only modest additional data, indicating strong generalization.

Overall, the study showcases that transformer‑based neural networks constitute a powerful, scalable, and versatile tool for quantum feedback control, opening pathways toward practical quantum error correction, rapid state preparation, and adaptive device characterization in noisy, realistic quantum hardware.