Salience-Affected Neural Networks

We present a simple neural network model which combines a locally-connected feedforward structure, as is traditionally used to model inter-neuron connectivity, with a layer of undifferentiated connections which model the diffuse projections from the human limbic system to the cortex. This new layer makes it possible to model global effects such as salience, at the same time as the local network processes task-specific or local information. This simple combination network displays interactions between salience and regular processing which correspond to known effects in the developing brain, such as enhanced learning as a result of heightened affect. The cortex biases neuronal responses to affect both learning and memory, through the use of diffuse projections from the limbic system to the cortex. Standard ANNs do not model this non-local flow of information represented by the ascending systems, which are a significant feature of the structure of the brain, and although they do allow associational learning with multiple-trial, they simply don’t provide the capacity for one-time learning. In this research we model this effect using an artificial neural network (ANN), creating a salience-affected neural network (SANN). We adapt an ANN to embody the capacity to respond to an input salience signal and to produce a reverse salience signal during testing. This research demonstrates that input combinations similar to the inputs in the training data sets will produce similar reverse salience signals during testing. Furthermore, this research has uncovered a novel method for training ANNs with a single training iteration.

💡 Research Summary

The paper introduces a novel neural‑network architecture called the Salience‑Affected Neural Network (SANN) that explicitly incorporates a global, diffuse projection from a limbic‑like system into a conventional locally‑connected feed‑forward network. Traditional artificial neural networks (ANNs) excel at modeling local, task‑specific processing but ignore the brain’s non‑local information flow—particularly the widespread modulatory signals originating in the limbic system that bias cortical activity according to affect and salience. To address this gap, the authors add a “diffuse layer” that receives a scalar salience signal alongside each input and distributes it uniformly (or near‑uniformly) to all neurons in the subsequent layer. This layer is meant to abstract the diffuse neuromodulatory projections (e.g., from the amygdala or basal forebrain) that globally alter neuronal excitability in the cortex.

During training, the salience signal modulates the weight‑update rule. In a standard gradient‑descent step Δw = η·δ·x, the authors multiply the learning rate η by the salience factor s, yielding Δw = η·s·δ·x. Consequently, patterns presented with high salience produce larger weight changes, mimicking the biological phenomenon where emotionally or attentively salient events are learned more rapidly and robustly. Conversely, low‑salience inputs receive attenuated updates, requiring multiple exposures for comparable performance.

A second key contribution is the notion of a “reverse salience” output during inference. After training, the network not only produces the usual task‑specific prediction (e.g., class label) but also computes a reverse‑salience value for each test sample. This value is derived from the inner product between the current activation vector and the stored salience‑weighted weight matrix, effectively measuring how strongly the current input matches patterns that were previously associated with high salience. In practice, inputs that are similar to highly salient training examples generate high reverse‑salience scores, providing a quantitative read‑out of the memory’s affective strength.

The authors validate the model on simple datasets, including a reduced MNIST subset and synthetic 2‑D patterns. They assign salience values in the range