Reservoir Predictive Path Integral Control for Unknown Nonlinear Dynamics

Neural networks have found extensive application in data-driven control of nonlinear dynamical systems, yet fast online identification and control of unknown dynamics remain central challenges. To meet these challenges, this paper integrates echo-state networks (ESNs)–reservoir computing models implemented with recurrent neural networks–and model predictive path integral (MPPI) control–sampling-based variants of model predictive control. The proposed reservoir predictive path integral (RPPI) enables fast learning of nonlinear dynamics with ESNs and exploits the learned nonlinearities directly in MPPI control computation without linearization approximations. This framework is further extended to uncertainty-aware RPPI (URPPI), which achieves robust stochastic control by treating ESN output weights as random variables and minimizing an expected cost over their distribution to account for identification errors. Experiments on controlling a Duffing oscillator and a four-tank system demonstrate that URPPI improves control performance, reducing control costs by up to 60% compared to traditional quadratic programming-based model predictive control methods.

💡 Research Summary

The paper tackles the longstanding challenge of simultaneously identifying and controlling unknown nonlinear dynamical systems in real time. Traditional model‑based approaches either assume a known linear model or rely on offline training of high‑capacity neural networks, both of which become impractical when the plant exhibits strong nonlinearities and data must be processed online. To bridge this gap, the authors propose a novel framework called Reservoir Predictive Path Integral (RPPI) control, which couples an Echo‑State Network (ESN) with Model Predictive Path Integral (MPPI) control, and an uncertainty‑aware extension named URPPI.

Key components

1. Echo‑State Network as a fast online surrogate model

   • The ESN consists of a fixed random reservoir (weights (W_{res}) and input weights (W_{in})) and a trainable linear read‑out (W_{out}).
   • Only (W_{out}) is updated online using Recursive Least Squares (RLS), giving a constant per‑step computational cost (O(\hat N^2)) independent of the number of past samples, where (\hat N) is the reservoir dimension.
   • RLS also yields a precision matrix (P_t) (inverse covariance) that quantifies the posterior uncertainty of (W_{out}).

2. Model Predictive Path Integral (MPPI) control

   • MPPI samples a large set of candidate input sequences, propagates each through the ESN surrogate, evaluates a user‑defined cost, and computes a weighted average of the inputs using exponential cost‑based weights.
   • This sampling‑based approach avoids any linearization of the dynamics and can handle arbitrary, possibly non‑convex, cost functions.

3. Uncertainty‑aware URPPI

   • URPPI augments MPPI by injecting Gaussian perturbations into the read‑out weights during trajectory sampling, with covariance given by the RLS‑derived (\Sigma_w = P_t^{-1}).
   • The controller therefore minimizes the expected cost (\mathbb{E}_{w\sim\mathcal N(\mu_w,\Sigma_w)}