Continual Learning through Control Minimization

Catastrophic forgetting remains a fundamental challenge for neural networks when tasks are trained sequentially. In this work, we reformulate continual learning as a control problem where learning and preservation signals compete within neural activity dynamics. We convert regularization penalties into preservation signals that protect prior-task representations. Learning then proceeds by minimizing the control effort required to integrate new tasks while competing with the preservation of prior tasks. At equilibrium, the neural activities produce weight updates that implicitly encode the full prior-task curvature, a property we term the continual-natural gradient, requiring no explicit curvature storage. Experiments confirm that our learning framework recovers true prior-task curvature and enables task discrimination, outperforming existing methods on standard benchmarks without replay.

💡 Research Summary

Continual learning suffers from catastrophic forgetting when tasks are learned sequentially. This paper proposes a novel formulation that casts continual learning as a control‑minimization problem, integrating learning and preservation signals directly into neural activity dynamics.
The authors first convert any parameter‑space regularizer R(θ) into a neuron‑specific preservation signal γ. For each neuron k with presynaptic activity ϕ_k, the signal is γ_k = ϕ_kᵀ∇{θ_k}R(θ). When instantiated with the diagonal Fisher matrix of a previous task (the classic EWC regularizer), the signal becomes γ_k = β·ϕ_kᵀF{DA,k}(θ_k−θ*_A,k), where β controls preservation strength. This formulation ensures that only synapses that were important for earlier tasks and are currently active incur a cost.
Next, the network state ϕ evolves according to the differential equation τ·ẋ = −ϕ + e·ψ + γ⊙f(ϕ,θ), where ψ is a learning signal that drives the system toward low loss, f is the feed‑forward mapping, and ⊙ denotes element‑wise multiplication. Because both ψ and γ act multiplicatively on the same neurons, the learning signal must expend additional effort to overcome preservation in directions that would interfere with prior knowledge. In the absence of either signal, the dynamics reduce to ordinary feed‑forward computation.
The learning objective is the least‑control principle: minimize ‖ψ‖₂ subject to reaching a loss‑minimizing equilibrium (∇_ϕ L(ϕ)=0). The optimal ψ* is obtained by running the controlled dynamics with fixed parameters until convergence. Parameters are then updated by descending H(θ)=‖ψ*(θ)‖₂², i.e., θ←θ−η∇_θ H(θ). This two‑stage process forces the network to gradually relax toward parameter configurations that require less control effort to achieve equilibrium.
A key theoretical contribution is the “continual‑natural gradient” property. The authors prove (Theorem 3.1) that, for small learning rates and linearization around the previous task optimum θ*A, the weight update for a new sample x_B satisfies Δθ≈−η·˜F_A⁻¹∇θ L_B(x_B). Here ˜F_A is an implicit approximation of the full Fisher information matrix of the previous task, emerging from the network dynamics despite only storing its diagonal. Thus the method captures second‑order interactions without explicitly computing or storing a curvature matrix.
The paper further analyzes class‑incremental learning, where a single output head must handle classes learned at different times. Standard regularization methods cannot filter the sample‑dependent interference term G{A←B} because they add a static gradient after the interference has already been formed. In contrast, the preservation signal raises the cost of moving in the subspace V_A (the column space of the previous task Fisher). The optimal learning signal naturally avoids V_A, leading to an update Δθ_EFC ∝ ˜F_A⁻¹G⊥B + O(˜λ_min⁻¹) (Theorem 3.2). Consequently, components aligned with prior task curvature are strongly attenuated, while orthogonal components pass through, guaranteeing descent on the class‑incremental objective without explicit identification of harmful gradients.
Empirically, the authors evaluate Equilibrium Fisher Control (EFC) on standard benchmarks such as MNIST‑Permutation, CIFAR‑100 Split, and TinyImageNet. Without any replay buffer, EFC consistently outperforms Elastic Weight Consolidation, Synaptic Intelligence, Memory Aware Synapses, and other regularization‑based baselines. The recovered curvature ˜F_A shows a high Pearson correlation (>0.85) with the true Fisher, confirming that the dynamics indeed encode full curvature information. Moreover, task‑wise accuracy remains stable across task switches, demonstrating effective mitigation of forgetting and superior task discrimination.
In summary, this work introduces a principled control‑theoretic framework for continual learning that (1) transforms regularization penalties into activity‑based preservation signals, (2) couples learning and preservation within neural dynamics, (3) implicitly recovers the full prior‑task Fisher at equilibrium (the continual‑natural gradient), (4) provides theoretical guarantees for curvature‑aware updates and task discrimination, and (5) validates these claims experimentally, achieving state‑of‑the‑art performance without replay. The approach opens new avenues for integrating dynamical systems theory with lifelong learning.