Group-Theoretic Reinforcement Learning of Dynamical Decoupling Sequences

Dynamical decoupling seeks to mitigate phase decoherence in qubits by applying a carefully designed sequence of effectively instantaneous electromagnetic pulses. Although analytic solutions exist for pulse timings that are optimal under specific noise regimes, identifying the optimal timings for a realistic noise spectrum remains challenging. We propose a reinforcement learning (RL)-based method for designing pulse sequences on qubits. Our novel action set enables the RL agent to efficiently navigate this inherently non-convex optimization landscape. The action set, derived from Thompson’s group $F$, is applicable to a broad class of sequential decision problems whose states can be represented as bounded sequences. We demonstrate that our RL agent can learn pulse sequences that minimize dephasing without requiring explicit knowledge of the underlying noise spectrum. This work opens the possibility for real-time learning of optimal dynamical decoupling sequences on qubits which are dephasing-limited. The model-free nature of our algorithm suggests that the agent may ultimately learn optimal pulse sequences even in the presence of unmodeled physical effects, such as pulse errors or non-Gaussian noise.

💡 Research Summary

The paper tackles the problem of designing dynamical decoupling (DD) sequences that suppress dephasing in a single qubit without requiring prior knowledge of the underlying noise spectrum. Traditional analytic DD protocols such as spin‑echo, CPMG, and Uhrig DD (UDD) are optimal only for specific noise models (hard or soft cut‑offs). In realistic settings the noise can be a mixture of Gaussian and non‑Gaussian components, may drift over time, and experimental constraints limit the number of pulses and the number of fidelity measurements that can be performed. To address these challenges the authors cast the DD optimization as a deterministic Markov decision process (MDP) in which the state is the current pulse‑time vector and an action is a deterministic transformation of the entire vector.

A key innovation is the choice of the action set. The authors employ the generators of Thompson’s group F—specifically the two elementary piecewise‑linear homeomorphisms x₀ and x₁, their inverses, and the identity. Thompson’s group F consists of order‑preserving homeomorphisms of the unit interval whose breakpoints lie on dyadic rationals and whose slopes are powers of two. This group is dense in the space of all order‑preserving homeomorphisms, meaning that any continuous monotone mapping u that would transform an arbitrary initial pulse schedule s₀ into the optimal schedule s* can be approximated arbitrarily well by a finite composition of these generators. By rescaling from