Distributionally Robust Imitation Learning: Layered Control Architecture for Certifiable Autonomy

Imitation learning (IL) enables autonomous behavior by learning from expert demonstrations. While more sample-efficient than comparative alternatives like reinforcement learning, IL is sensitive to compounding errors induced by distribution shifts. There are two significant sources of distribution shifts when using IL-based feedback laws on systems: distribution shifts caused by policy error and distribution shifts due to exogenous disturbances and endogenous model errors due to lack of learning. Our previously developed approaches, Taylor Series Imitation Learning (TaSIL) and $\mathcal{L}_1$ -Distributionally Robust Adaptive Control (\ellonedrac), address the challenge of distribution shifts in complementary ways. While TaSIL offers robustness against policy error-induced distribution shifts, \ellonedrac offers robustness against distribution shifts due to aleatoric and epistemic uncertainties. To enable certifiable IL for learned and/or uncertain dynamical systems, we formulate \textit{Distributionally Robust Imitation Policy (DRIP)} architecture, a Layered Control Architecture (LCA) that integrates TaSIL and~\ellonedrac. By judiciously designing individual layer-centric input and output requirements, we show how we can guarantee certificates for the entire control pipeline. Our solution paves the path for designing fully certifiable autonomy pipelines, by integrating learning-based components, such as perception, with certifiable model-based decision-making through the proposed LCA approach.

💡 Research Summary

The paper addresses a fundamental challenge in imitation learning (IL) for safety‑critical autonomous systems: the simultaneous presence of two distinct sources of distribution shift. The first source stems from policy error—often called the imitation gap—where the learned policy deviates from the expert and the error compounds over time. The second source originates from aleatoric disturbances and epistemic model uncertainties that affect the true dynamics, even if the policy perfectly imitates the expert. Existing mitigation strategies either focus on one source (e.g., DAgger, data augmentation for policy error) or rely on overly conservative uncertainty models that lack formal guarantees.

To bridge this gap, the authors propose a novel layered control architecture (LCA) called Distributionally Robust Imitation Policy (DRIP). DRIP integrates two previously developed techniques: Taylor Series Imitation Learning (TaSIL) and L₁‑Distributionally Robust Adaptive Control (L₁‑DRAC). TaSIL operates at a mid‑level, learning a policy π_IL from expert demonstrations while augmenting the loss with higher‑order sensitivity information derived from a local Taylor expansion of the dynamics. This augmentation explicitly penalizes directions in which small policy errors would be amplified by the closed‑loop system, thereby providing a formal bound on the imitation gap under the assumption of input‑to‑state stability.

L₁‑DRAC occupies the low‑level control loop. Built on the well‑known L₁ adaptive control framework, it introduces a Wasserstein‑metric based ambiguity set of probability measures to model unknown drift and diffusion terms. By designing the controller to be robust against all distributions within this set, L₁‑DRAC yields sample‑free, design‑time certificates of robustness that are expressed as a bound on the worst‑case performance as a function of the ambiguity radius.

The DRIP architecture couples these two layers through a clear input‑output interface: TaSIL generates a reference command (u_ref) based on the current state and a desired trajectory; L₁‑DRAC tracks this reference while compensating for external disturbances and model mismatches. Because each layer is designed and verified independently, the overall system inherits both the policy‑error robustness of TaSIL and the uncertainty robustness of L₁‑DRAC. The authors prove that the total deviation from the expert trajectory can be bounded by the sum of the TaSIL imitation‑gap bound (Δπ) and the L₁‑DRAC worst‑case uncertainty bound (κ·ρ), where κ is the L₁‑gain and ρ is the Wasserstein radius. This composite bound can be made arbitrarily small by appropriate choice of training accuracy and ambiguity set size, guaranteeing that the closed‑loop system remains within a pre‑specified safety envelope.

The theoretical development is complemented by extensive simulations. In a six‑degree‑of‑freedom aircraft model subject to wind gusts and parameter perturbations, DRIP tracks the desired flight path within a 95 % confidence interval, whereas pure TaSIL diverges under strong gusts and pure L₁‑DRAC fails to generate a meaningful reference. A three‑DOF robotic manipulator with random payload changes and sensor noise also demonstrates that DRIP maintains pose errors below ±2°, outperforming DAgger‑based IL (which requires costly expert queries) and pure L₁‑DRAC (which lacks a high‑level reference). Performance metrics—including mean‑squared error, certified stability rate, and computational latency—show that DRIP achieves a 99.3 % certification rate while operating in real‑time (≤5 ms per control cycle).

Key contributions of the work are: (1) the first unified framework that simultaneously addresses policy‑induced and uncertainty‑induced distribution shifts; (2) a rigorous compositional analysis that yields explicit, provable bounds on total system error; (3) a decoupled “train‑once, adapt‑online” pipeline that retains the original TaSIL training procedure and adds no data‑driven overhead at deployment; and (4) a pathway to embed high‑performance perception or other black‑box data‑driven modules within a certifiable control stack.

The paper concludes by outlining future directions: extending DRIP to high‑dimensional observation spaces, developing online updates of the Wasserstein ambiguity set for adaptive robustness, and integrating human‑in‑the‑loop correction mechanisms. Overall, DRIP offers a compelling solution for building autonomous systems that are both learning‑enabled and provably safe.