Cognitive computation with autonomously active neural networks: an emerging field

The human brain is autonomously active. To understand the functional role of this self-sustained neural activity, and its interplay with the sensory data input stream, is an important question in cognitive system research and we review here the present state of theoretical modelling. This review will start with a brief overview of the experimental efforts, together with a discussion of transient vs. self-sustained neural activity in the framework of reservoir computing. The main emphasis will be then on two paradigmal neural network architectures showing continuously ongoing transient-state dynamics: saddle point networks and networks of attractor relics. Self-active neural networks are confronted with two seemingly contrasting demands: a stable internal dynamical state and sensitivity to incoming stimuli. We show, that this dilemma can be solved by networks of attractor relics based on competitive neural dynamics, where the attractor relics compete on one side with each other for transient dominance, and on the other side with the dynamical influence of the input signals. Unsupervised and local Hebbian-style online learning then allows the system to build up correlations between the internal dynamical transient states and the sensory input stream. An emergent cognitive capability results from this set-up. The system performs online, and on its own, a non-linear independent component analysis of the sensory data stream, all the time being continuously and autonomously active. This process maps the independent components of the sensory input onto the attractor relics, which acquire in this way a semantic meaning.

💡 Research Summary

The paper reviews the current state of theoretical modeling aimed at understanding the functional role of the brain’s autonomously generated activity and its interaction with incoming sensory streams. It begins by summarizing experimental evidence that the human brain exhibits continuous, self‑sustained neural dynamics even in the absence of external stimuli. This intrinsic activity is argued not to be mere background noise but a crucial substrate for cognition, providing a pre‑existing dynamical scaffold onto which sensory information can be mapped.

The authors then critique conventional reservoir computing, which typically relies on a fixed, high‑dimensional dynamical “reservoir” that passively transforms inputs. While useful for certain tasks, such reservoirs lack the ongoing internal transients that characterize biological neural tissue. To bridge this gap, the review focuses on two novel network architectures that generate continuously evolving transient states: saddle‑point networks and networks of attractor relics.

In a saddle‑point network, the system’s state wanders near high‑dimensional saddle points—regions where stable and unstable directions coexist. The dynamics linger in quasi‑stationary configurations but are highly susceptible to small perturbations, allowing rapid re‑orientation in response to sensory input. This mirrors the brain’s ability to maintain a baseline activity while remaining ready to react.

Networks of attractor relics take a different approach. Traditional attractors are strong basins of attraction that pull the system into fixed points. By deliberately weakening these basins, the authors create “relics” that retain a memory of the original attractor’s structure but no longer dominate the dynamics indefinitely. Multiple relics coexist and compete for transient dominance through competitive neural dynamics. When an external stimulus arrives, it perturbs the current dominant relic, reducing its stability and allowing another relic to rise. This competition yields a continuously shifting internal landscape that is both self‑generated and input‑driven.

A key contribution of the paper is the integration of local, Hebbian‑style online learning with the competitive dynamics of attractor relics. Whenever a relic becomes transiently dominant, the coincident sensory input strengthens the synaptic connections associated with that relic. Over time, each relic becomes selectively tuned to a particular independent component of the sensory stream. Importantly, this learning occurs without supervision and in real time, effectively performing a nonlinear independent component analysis (ICA) on the incoming data while the network remains autonomously active.

The authors argue that this architecture resolves the apparent dilemma between maintaining a stable internal dynamical state and being sensitive to external perturbations. Stability is provided by the structured competition among relics, while sensitivity emerges from the continual reshaping of the competition by incoming signals. Consequently, the system exhibits emergent cognitive capabilities: it autonomously extracts statistically independent features from raw sensory data, assigns semantic meaning to internal states, and does so continuously without external control.

In summary, the review positions self‑active neural networks—particularly those built from attractor relics with competitive dynamics and Hebbian learning—as a promising computational framework for modeling the brain’s autonomous activity and its role in cognition. By demonstrating how such networks can perform unsupervised, online ICA and develop meaningful internal representations, the paper provides a theoretical foundation for future artificial intelligence systems that aim to emulate the brain’s capacity for continuous, self‑directed information processing.