A State Space Approach for Piecewise-Linear Recurrent Neural Networks for Reconstructing Nonlinear Dynamics from Neural Measurements

The computational properties of neural systems are often thought to be implemented in terms of their network dynamics. Hence, recovering the system dynamics from experimentally observed neuronal time series, like multiple single-unit (MSU) recordings or neuroimaging data, is an important step toward understanding its computations. Ideally, one would not only seek a state space representation of the dynamics, but would wish to have access to its governing equations for in-depth analysis. Recurrent neural networks (RNNs) are a computationally powerful and dynamically universal formal framework which has been extensively studied from both the computational and the dynamical systems perspective. Here we develop a semi-analytical maximum-likelihood estimation scheme for piecewise-linear RNNs (PLRNNs) within the statistical framework of state space models, which accounts for noise in both the underlying latent dynamics and the observation process. The Expectation-Maximization algorithm is used to infer the latent state distribution, through a global Laplace approximation, and the PLRNN parameters iteratively. After validating the procedure on toy examples, the approach is applied to MSU recordings from the rodent anterior cingulate cortex obtained during performance of a classical working memory task, delayed alternation. A model with 5 states turned out to be sufficient to capture the essential computational dynamics underlying task performance, including stimulus-selective delay activity. The estimated models were rarely multi-stable, but rather were tuned to exhibit slow dynamics in the vicinity of a bifurcation point. In summary, the present work advances a semi-analytical (thus reasonably fast) maximum-likelihood estimation framework for PLRNNs that may enable to recover the relevant dynamics underlying observed neuronal time series, and directly link them to computational properties.

💡 Research Summary

The paper presents a principled method for recovering the underlying nonlinear dynamics of neural systems from observed neuronal time series by fitting a piecewise‑linear recurrent neural network (PLRNN) within a state‑space modeling framework. PLRNNs combine linear dynamics with a rectified‑linear (ReLU‑like) activation that introduces a set of thresholds, allowing the network to approximate a broad class of nonlinear flows while keeping the number of parameters manageable. The authors formulate the latent dynamics as a stochastic difference equation and the observations as a linear mapping of the latent state corrupted by Gaussian noise, thus casting the problem into a classic hidden‑Markov / state‑space setting.

To estimate both the latent state distribution and the model parameters, they employ an Expectation‑Maximization (EM) algorithm. Because the piecewise‑linear transition makes the exact posterior intractable, the E‑step uses a global Laplace approximation: the full log‑joint density over the entire time series is expanded to second order around its mode, yielding a multivariate Gaussian approximation to the posterior. This approach avoids costly particle‑filter or MCMC sampling while preserving accuracy, as demonstrated in the simulation studies. The M‑step then updates the linear parameters (the recurrent weight matrix, input matrix, and observation matrix) via closed‑form least‑squares solutions, and the threshold parameters via constrained nonlinear optimization. A spectral radius regularization term is added to keep the recurrent matrix stable.

The methodology is first validated on synthetic data. In the first toy example, a known three‑dimensional chaotic system is approximated by a PLRNN; the recovered network reproduces the system’s fixed points, limit cycles, and overall phase portrait. In the second example, the authors test robustness to observation noise and to misspecification of the latent dimensionality, showing that the Laplace‑EM scheme still converges to a model that captures the essential dynamics. Compared with particle‑based EM, the proposed method achieves comparable log‑likelihoods but reduces computation time by an order of magnitude.

The core contribution lies in the application to real neurophysiological recordings. The authors analyze multi‑single‑unit (MSU) data from the anterior cingulate cortex (ACC) of rats performing a delayed alternation working‑memory task. The task consists of a stimulus presentation, a delay period, and a choice phase. Model order selection using the Bayesian Information Criterion (BIC) indicates that a five‑dimensional latent space provides the best trade‑off between fit and complexity. The fitted PLRNN exhibits stimulus‑selective delay activity: during the delay, the latent trajectory remains near a particular state that encodes the previously presented stimulus, and then diverges to a different state at the choice point depending on the required response.

Dynamic analysis of the fitted network reveals that the system is not strongly multistable; instead, it operates close to a bifurcation point where eigenvalues of the Jacobian are near the unit circle, producing slow dynamics that can sustain information over the delay without requiring multiple attractors. This finding aligns with theoretical proposals that working memory may rely on critical slowing rather than discrete attractor states. Moreover, the estimated recurrent weight matrix and thresholds provide a mechanistic interpretation of how ACC neurons could implement such a computation, linking the abstract dynamical description directly to circuit‑level parameters.

In summary, the paper advances a semi‑analytical, maximum‑likelihood estimation framework for piecewise‑linear RNNs that (i) integrates noise in both latent dynamics and observations, (ii) leverages a global Laplace approximation to make EM computationally tractable, and (iii) demonstrates the ability to uncover interpretable, task‑relevant dynamics from real neural recordings. The approach bridges the gap between black‑box deep learning models and classical dynamical‑systems analysis, offering a scalable tool for neuroscientists seeking to map observed neural activity onto underlying computational mechanisms. Future extensions could incorporate spiking observation models, hierarchical latent structures, or online inference for real‑time applications.