Distributionally Robust Regret Optimal Control Under Moment-Based Ambiguity Sets

In this paper, we consider a class of finite-horizon, linear-quadratic stochastic control problems, where the probability distribution governing the noise process is unknown but assumed to belong to an ambiguity set consisting of all distributions whose mean and covariance lie within norm balls centered at given nominal values. To address the distributional ambiguity, we explore the design of causal affine control policies to minimize the worst-case expected regret over all distributions in the given ambiguity set. The resulting minimax optimal control problem is shown to admit an equivalent reformulation as a tractable convex program that corresponds to a regularized version of the nominal linear-quadratic stochastic control problem. While this convex program can be recast as a semidefinite program, semidefinite programs are typically solved using primal-dual interior point methods that scale poorly with the problem size in practice. To address this limitation, we propose a scalable dual projected subgradient method to compute optimal controllers to an arbitrary accuracy. Numerical experiments are presented to benchmark the proposed method against state-of-the-art data-driven and distributionally robust control design approaches.

💡 Research Summary

This paper addresses a significant challenge in stochastic control: designing controllers when the exact probability distribution of the system noise is unknown. The authors focus on finite-horizon, linear-quadratic systems and propose a novel distributionally robust regret optimal control framework.

The core problem is formulated as a minimax optimization: find a causal affine control policy that minimizes the worst-case expected regret across all possible disturbance distributions within a prescribed ambiguity set. Regret is defined as the excess cost incurred by a causal controller compared to an ideal, non-causal (clairvoyant) controller that knows the entire disturbance sequence in advance. This regret-based objective aims to mitigate the excessive conservatism often associated with traditional worst-case cost minimization.

The key innovation lies in the definition of the ambiguity set. Instead of using Wasserstein distance balls common in prior work, the authors construct a moment-based ambiguity set. This set contains all distributions whose mean vector lies within a Euclidean norm ball and whose covariance matrix lies within a Schatten p-norm ball, both centered around given nominal estimates. This approach captures uncertainty in the first two moments while accommodating temporally correlated and non-Gaussian disturbances.

A major theoretical contribution is the exact convex reformulation of the seemingly intractable minimax problem. Through a series of analytical steps (Lemma 1, Theorem 2), the authors prove that the problem is equivalent to a tractable convex program. This program resembles a regularized linear-quadratic stochastic control problem. The objective function consists of: (i) the expected regret under the nominal distribution, (ii) a spectral norm penalty term (∥C(K)∥∞) arising from uncertainty in the mean, and (iii) a Schatten q-norm penalty term (∥C(K)∥q, where 1/p+1/q=1) arising from uncertainty in the covariance. The optimal controller has an insightful feedback-feedforward structure: φ*(w) = K*(w - μ̂) + K°μ̂, where the feedforward term K°μ̂ represents the expected action of the optimal non-causal controller, and the feedback term K* compensates for deviations from the nominal mean.

While the convex program can be reformulated as a Semidefinite Program (SDP), solving large-scale SDPs with interior-point methods is computationally prohibitive. To overcome this, the authors develop a scalable Dual Projected Subgradient Algorithm. This algorithm leverages the separable structure of the regularization terms in the dual problem, requiring only relatively simple matrix projections at each iteration. It can compute optimal controllers to arbitrary accuracy, even for large-scale problems.

Numerical experiments demonstrate the practical value of the proposed framework. When controllers are designed using finite training samples from the true (unknown) disturbance distribution, the distributionally robust regret-optimal controllers consistently achieve superior out-of-sample performance compared to several state-of-the-art data-driven and distributionally robust control design methods. The parameters defining the ambiguity set (radii r1, r2, norm order p) provide intuitive knobs for balancing robustness against performance, which can be tuned via cross-validation.

In summary, this paper makes substantial contributions by: (1) introducing a flexible moment-based ambiguity set for distributionally robust control, (2) deriving an exact and interpretable convex reformulation for the regret minimization problem, (3) providing a computationally efficient algorithm for large-scale implementation, and (4) empirically validating the framework’s advantages over existing approaches. It successfully bridges robust control theory with practical algorithmic design.