Safely Learning Controlled Stochastic Dynamics

We address the problem of safely learning controlled stochastic dynamics from discrete-time trajectory observations, ensuring system trajectories remain within predefined safe regions during both training and deployment. Safety-critical constraints of this kind are crucial in applications such as autonomous robotics, finance, and biomedicine. We introduce a method that ensures safe exploration and efficient estimation of system dynamics by iteratively expanding an initial known safe control set using kernel-based confidence bounds. After training, the learned model enables predictions of the system’s dynamics and permits safety verification of any given control. Our approach requires only mild smoothness assumptions and access to an initial safe control set, enabling broad applicability to complex real-world systems. We provide theoretical guarantees for safety and derive adaptive learning rates that improve with increasing Sobolev regularity of the true dynamics. Experimental evaluations demonstrate the practical effectiveness of our method in terms of safety, estimation accuracy, and computational efficiency.

💡 Research Summary

The paper tackles the challenging problem of learning the dynamics of controlled stochastic systems while guaranteeing safety throughout both the data‑collection phase and subsequent deployment. The authors consider continuous‑time stochastic differential equations (SDEs) of the form
dX(t)=b(X(t),u(t,X(t)))dt + a(X(t),u(t,X(t)))dW(t),
with controls parameterized by a low‑dimensional vector θ∈D. Safety is defined via a scalar function g(x)≥0 that partitions the state space into safe and unsafe regions, while a second function h(x)≥0 delineates a “reset” region from which the system can be returned to its initial distribution.

The core contribution is a safe exploration‑exploitation algorithm that expands an initially known safe‑resettable set using kernel‑based confidence bounds. The method rests on three mild assumptions: (A1) existence of at least one known safe control (the set S₀), (A2) existence of at least one known reset‑capable control (the set R₀), and (A3) Sobolev regularity of the state‑density map p(θ,t,x). The Sobolev smoothness ν>½·max(n,m+1) (where n is the state dimension and m the control‑parameter dimension) enables the use of Matérn kernels whose smoothness matches the underlying function class.

Algorithmically, the procedure iterates over the following steps:

1. Initialization – Using S₀ and R₀ the authors define an initial safe‑resettable region Γ₀⊂D×