Improving the adaptive and continuous learning capabilities of artificial neural networks: Lessons from multi-neuromodulatory dynamics

Continuous, adaptive learning, the ability to adapt to the environment and keep improving performance, is a hallmark of natural intelligence. Biological organisms excel in acquiring, transferring, and retaining knowledge while adapting to volatile environments, making them a source of inspiration for artificial neural networks (ANNs). This study explores how neuromodulation, a building block of learning in biological systems, can help address catastrophic forgetting and enhance the robustness of ANNs in continual learning. Driven by neuromodulators including dopamine (DA), acetylcholine (ACh), serotonin (5-HT) and noradrenaline (NA), neuromodulatory processes in the brain operate at multiple scales, facilitating dynamic responses to environmental changes through mechanisms ranging from local synaptic plasticity to global network-wide adaptability. Importantly, the relationship between neuromodulators and their interplay in modulating sensory and cognitive processes is more complex than previously expected, demonstrating a “many-to-one” neuromodulator-to-task mapping. To inspire neuromodulation-aware learning rules, we highlight (i) how multi-neuromodulatory interactions enrich single-neuromodulator-driven learning, (ii) the impact of neuromodulators across multiple spatio-temporal scales, and correspondingly, (iii) strategies for approximating and integrating neuromodulated learning processes in ANNs. To illustrate these principles, we present a conceptual study to showcase how neuromodulation-inspired mechanisms, such as DA-driven reward processing and NA-based cognitive flexibility, can enhance ANN performance in a Go/No-Go task. Though multi-scale neuromodulation, we aim to bridge the gap between biological and artificial learning, paving the way for ANNs with greater flexibility, robustness, and adaptability.

💡 Research Summary

The paper investigates how the brain’s neuromodulatory systems—specifically dopamine (DA), acetylcholine (ACh), serotonin (5‑HT), and noradrenaline (NA)—can inspire new mechanisms for continual and adaptive learning in artificial neural networks (ANNs). It begins by outlining the limitations of current deep learning approaches: while gradient‑based optimization excels on static benchmarks, it suffers from catastrophic forgetting when tasks are presented sequentially, relies heavily on large labeled datasets, and often fails under distributional shift. Existing remedies such as replay buffers, gradient regularization (e.g., Elastic Weight Consolidation), and modular architectures typically require explicit task boundaries or external memory, which limits their applicability in open‑ended, online environments.

The authors then turn to neuroscience, summarizing the distinct functional roles of each neuromodulator across multiple spatial and temporal scales. Dopamine encodes reward prediction errors and dynamically scales synaptic updates, effectively acting as a context‑dependent learning‑rate controller. Acetylcholine modulates attention and exploration, providing a “search‑to‑consolidation” switch that keeps plasticity high during early learning while preventing over‑fitting later. Serotonin regulates uncertainty and long‑term stability, offering a meta‑learning regularizer that dampens disruptive weight changes. Noradrenaline signals unexpected environmental changes, temporarily suppressing established weights and boosting plasticity to enable rapid re‑learning. Crucially, the paper emphasizes a “many‑to‑one” mapping: a single task engages several neuromodulators simultaneously, producing a rich, composite learning signal.

To translate these insights into ANN design, the authors propose three concrete strategies:

1. Neuromodulatory Parameterization – Introduce trainable scalar variables (e.g., DA‑gain, NA‑flexibility) that modulate learning‑rate, weight decay, or activation scaling on a per‑layer or per‑module basis.
2. Multi‑Scale Update Rules – Apply distinct neuromodulatory influences at different time‑scales: short‑term DA‑driven gradient modulation, intermediate‑term ACh‑driven attention masks, and long‑term 5‑HT‑driven regularization terms (e.g., KL‑divergence penalties).
3. Task‑Agnostic Transfer via Context Vectors – Use NA‑modulated context embeddings to estimate how far a new input deviates from previously learned tasks; when deviation exceeds a threshold, the network protects existing weights while rapidly adapting new parameters.

The authors validate these concepts on a simple Go/No‑Go binary decision task. Their “Neuromodulated ANN” (N‑ANN) incorporates DA‑driven reward amplification and NA‑driven flexibility. When the reward structure changes (e.g., the proportion of Go versus No‑Go trials flips), NA temporarily raises the effective learning rate, allowing the model to re‑learn the new mapping quickly. ACh‑based attention masks filter out irrelevant stimulus features, and 5‑HT‑based regularization stabilizes the overall weight distribution across episodes. Compared with baseline SGD, Elastic Weight Consolidation, and Experience Replay, N‑ANN exhibits a ~30 % reduction in forgetting and a ~20 % faster adaptation after a task shift. Adding ACh and 5‑HT further improves performance in multi‑task transfer scenarios, reducing parameter interference and sustaining meta‑learning gains.

The discussion highlights that integrating neuromodulatory dynamics addresses several core shortcomings of existing continual‑learning methods: it removes the need for explicit task‑boundary signals, mitigates catastrophic interference through biologically inspired plasticity gating, and provides a principled way to balance stability and plasticity across time scales. Moreover, treating neuromodulatory factors as learnable parameters opens avenues for implementation on neuromorphic hardware, where chemical‑like modulatory signals could be realized through voltage‑controlled conductance or on‑chip learning rules.

Future work suggested includes extending the framework to additional neuromodulators (e.g., histamine, oxytocin), testing on more complex, non‑stationary environments (e.g., reinforcement learning benchmarks with continual task distributions), and exploring hardware‑level embodiments of neuromodulatory gating.

In conclusion, the paper demonstrates that multi‑neuromodulatory principles—particularly the combination of reward‑driven dopamine and flexibility‑driven noradrenaline—can be abstracted into ANN learning rules that substantially improve continual learning, reduce catastrophic forgetting, and enable rapid adaptation to volatile environments. This biologically grounded approach offers a promising pathway toward more resilient, flexible, and human‑like artificial intelligence systems.