Consistency of support vector machines for forecasting the evolution of an unknown ergodic dynamical system from observations with unknown noise

We consider the problem of forecasting the next (observable) state of an unknown ergodic dynamical system from a noisy observation of the present state. Our main result shows, for example, that support vector machines (SVMs) using Gaussian RBF kernels can learn the best forecaster from a sequence of noisy observations if (a) the unknown observational noise process is bounded and has a summable $\alpha$-mixing rate and (b) the unknown ergodic dynamical system is defined by a Lipschitz continuous function on some compact subset of $\mathbb{R}^d$ and has a summable decay of correlations for Lipschitz continuous functions. In order to prove this result we first establish a general consistency result for SVMs and all stochastic processes that satisfy a mixing notion that is substantially weaker than $\alpha$-mixing.

💡 Research Summary

The paper addresses the challenging problem of predicting the next observable state of an unknown ergodic dynamical system when only noisy observations of the current state are available. Classical forecasting methods typically assume that the underlying system is known or that the observation noise is independent and often Gaussian. In many real‑world settings, however, the system dynamics are hidden and the noise may exhibit temporal dependence. To bridge this gap, the authors formulate two realistic assumptions.

First, the observation noise process ({\varepsilon_t}) is assumed to be bounded and to possess a summable (\alpha)-mixing rate, i.e. (\sum_{k=1}^{\infty}\alpha(k)<\infty). This condition guarantees that statistical dependence between distant noise terms decays sufficiently fast, while boundedness prevents extreme outliers. Second, the dynamical system is modeled by a Lipschitz‑continuous map (F) defined on a compact subset (\mathcal{X}\subset\mathbb{R}^d). Moreover, for any pair of Lipschitz functions (g,h) the correlation decay (\sum_{k=1}^{\infty}|\operatorname{Cov}(g\circ F^k,h)|<\infty) holds. This captures the ergodic nature of the system and ensures that long‑range memory is limited.

Under these conditions the authors propose to learn the optimal predictor using a Support Vector Machine (SVM) with a Gaussian radial basis function (RBF) kernel. The learning problem is cast as regularized empirical risk minimization:

\