A Backpropagation-Free Feedback-Hebbian Network for Continual Learning Dynamics

Feedback-rich neural architectures can regenerate earlier representations and inject temporal context, making them a natural setting for strictly local synaptic plasticity. Existing literature raises doubt about whether a minimal, backpropagation-free feedback-Hebbian system can already express interpretable continual-learning-relevant behaviors under controlled training schedules. In this work, we introduce a compact prediction-reconstruction architecture with a dedicated feedback pathway providing lightweight, locally trainable temporal context for continual adaptation. All synapses are updated by a unified local rule combining centered Hebbian covariance, Oja-style stabilization, and a local supervised drive where targets are available. With a simple two-pair association task, learning is characterized through layer-wise activity snapshots, connectivity trajectories (row and column means of learned weights), and a normalized retention index across phases. Under sequential A to B training, forward output connectivity exhibits a long-term depression (LTD)-like suppression of the earlier association, while feedback connectivity preserves an A-related trace during acquisition of B. Under an alternating sequence, both associations are concurrently maintained rather than sequentially suppressed. Architectural controls and rule-term ablations isolate the role of dedicated feedback in regeneration and co-maintenance, alongside the role of the local supervised term in output selectivity and unlearning. Together, the results show that a compact feedback pathway trained with local plasticity can support regeneration and continual-learning-relevant dynamics in a minimal, mechanistically transparent setting.

💡 Research Summary

The paper introduces a minimal neural architecture that can exhibit continual‑learning‑relevant dynamics without any form of back‑propagation. The model consists of two forward layers and two dedicated feedback layers, each a 10 × 10 weight matrix matching the ten input/output sites of a toy association task. The forward pathway learns a two‑step mapping from input to target output, while the feedback pathway reconstructs the activity of the first forward layer and adds it to the next time step, thereby providing a lightweight temporal context.

All synapses are updated by a single local rule (Eq. 1) that combines three terms: (i) a centered Hebbian covariance term that strengthens co‑active deviations from running averages, (ii) an Oja‑style normalization term (β) that prevents runaway growth by constraining row norms, and (iii) a supervised drive (t − y)x that is active only when a target is supplied. The rule is applied independently to each weight matrix; no weight transport, global error signals, or replay buffers are required.

Two training regimes are examined on a two‑pair association task (pair A: input 3 → outputs 8, 9; pair B: input 7 → outputs 5, 6). In the sequential regime, the network first learns A for ten epochs and then learns B for another ten epochs. During B training the forward output weights show a strong LTD‑like depression of the A‑specific connections (R ≈ ‑0.8), while the feedback weights retain an A‑related trace (R ≈ 0.6). Thus the system demonstrates “learning‑without‑forgetting”: the output behavior switches to B, yet the earlier association persists in the feedback pathway.

In the interleaved regime, A and B samples alternate every trial for ten epochs. Both forward and feedback connections maintain elevated strengths for both pairs (R ≈ 0.4–0.5), indicating co‑maintenance of the two mappings, akin to classical conditioning where alternating presentations lead to simultaneous retention.

Ablation studies removing the feedback pathway or the supervised term show that (a) without feedback the forward weights completely erase the earlier association, and (b) without the supervised drive learning slows dramatically and output accuracy drops. These results highlight the crucial role of a dedicated feedback channel for providing temporal context and of the local supervised term for rapid, selective learning.

Overall, the work demonstrates that a compact prediction‑reconstruction network trained with a strictly local Hebbian‑Oja rule can reproduce key continual‑learning primitives—selective unlearning, retention of prior traces, and concurrent maintenance under alternating exposure—without any global credit assignment. The findings offer a mechanistically transparent model that bridges biological plausibility and practical continual‑learning algorithms.