A joint optimization approach to identifying sparse dynamics using least squares kernel collocation

We develop an all-at-once modeling framework for learning systems of ordinary differential equations (ODE) from scarce, partial, and noisy observations of the states. The proposed methodology amounts to a combination of sparse recovery strategies for the ODE over a function library combined with techniques from reproducing kernel Hilbert space (RKHS) theory for estimating the state and discretizing the ODE. Our numerical experiments reveal that the proposed strategy leads to significant gains in terms of accuracy, sample efficiency, and robustness to noise, both in terms of learning the equation and estimating the unknown states. This work demonstrates capabilities well beyond existing and widely used algorithms while extending the modeling flexibility of other recent developments in equation discovery.

💡 Research Summary

**
The paper introduces a novel framework called Joint SINDy (JSINDy) for discovering sparse ordinary differential equation (ODE) models from limited, noisy, and possibly indirect measurements. Traditional sparse identification methods such as SINDy operate in two separate stages: first a denoising or derivative‑estimation step that reconstructs the state trajectory, and second a regression step that fits a sparse combination of candidate functions to the estimated derivatives. When data are scarce, noisy, or only a subset of the state variables is observed, the first stage becomes unreliable and the errors propagate to the second stage, leading to poor model recovery.

JSINDy addresses this problem by formulating state estimation and model identification as a single, joint optimization problem. The authors define a loss function consisting of four terms: (i) a data‑misfit term that penalizes the discrepancy between the estimated trajectory and the measurements, (ii) a collocation term that enforces consistency between the estimated trajectory and the ODE defined by the candidate model, (iii) an RKHS regularization term that promotes smoothness of the trajectory, and (iv) a quadratic regularization on the coefficient matrix. The trajectory is represented in a vector‑valued reproducing kernel Hilbert space (RKHS) built from a Matérn kernel plus a constant term, guaranteeing sufficient differentiability (up to order p) for the collocation constraints.

A key innovation is the use of an explicit feature‑selection operator S(x) that, given a current trajectory estimate, selects a subset of active library functions. Rather than imposing an ℓ₁ penalty, the algorithm alternates between (a) solving a nonlinear least‑squares problem (via the Levenberg–Marquardt method) on the currently active set of features, and (b) updating the active set using the newly estimated trajectory. This “fixed‑point” iteration mirrors the alternating structure of many dynamics‑informed learning methods but provides a clear theoretical justification: if S is a consistent sparsifier, the iterates converge to a fixed point that satisfies both the data and the ODE constraints.

The representer theorem reduces the infinite‑dimensional RKHS problem to a finite‑dimensional parameterization of the trajectory as a linear combination of kernel evaluations at selected collocation points. Consequently, the joint optimization becomes a tractable nonlinear least‑squares problem in the coefficients of the kernel expansion and the sparse coefficient matrix θ.

The authors conduct extensive numerical experiments on a variety of benchmark systems, including low‑dimensional linear and nonlinear ODEs, higher‑order dynamics, and chaotic models such as the Lorenz system. They systematically vary three challenging aspects: (1) extremely low sampling rates (time steps an order of magnitude larger than the system’s natural time scale), (2) partial observations through arbitrary linear measurement matrices, and (3) additive Gaussian noise with standard deviations up to 0.2. Across all scenarios, JSINDy outperforms standard SINDy, ODR‑BINDy, PINN‑SR, and other recent physics‑informed sparse regression methods. The mean‑squared error in state reconstruction is reduced by 40 %–80 % relative to the baselines, and the active library terms are recovered with >95 % accuracy even when only a subset of the state variables is measured.

The paper also provides theoretical results: (i) a representer theorem guaranteeing that the optimal trajectory lies in the finite‑dimensional span of kernel sections, (ii) existence of solutions for both the continuous and discretized problems under mild regularity conditions, and (iii) sufficient conditions for convergence of the alternating algorithm when the feature‑selection operator is monotone.

In conclusion, JSINDy unifies three powerful ideas—RKHS‑based collocation for smooth trajectory estimation, explicit sparsity enforcement via a feature‑selection operator, and alternating Levenberg–Marquardt optimization—into a single framework that is robust to the very data limitations that cripple traditional two‑step methods. The approach naturally extends to higher‑order ODEs, accommodates arbitrary linear measurement operators, and remains effective under severe noise and undersampling. Future directions suggested by the authors include Bayesian extensions for uncertainty quantification, adaptive kernel learning, and real‑time online implementation for streaming data.