경험 재생에서 깊은 망각과 얕은 망각의 비대칭: 작은 버퍼는 특징 공간을 유지하지만 분류 경계는 왜곡한다

A persistent paradox in continual learning (CL) is that neural networks often retain linearly separable representations of past tasks even when their output predictions fail. We formalize this distinction as the gap between deep (feature-space) and shallow (classifier-level) forgetting. We reveal a critical asymmetry in Experience Replay: while minimal buffers successfully anchor feature geometry and prevent deep forgetting, mitigating shallow forgetting typically requires substantially larger buffer capacities. To explain this, we extend the Neural Collapse framework to the sequential setting. We characterize deep forgetting as a geometric drift toward out-of-distribution subspaces and prove that any non-zero replay fraction asymptotically guarantees the retention of linear separability. Conversely, we identify that the “strong collapse” induced by small buffers leads to rank-deficient covariances and inflated class means, effectively blinding the classifier to true population boundaries. By unifying CL with out-of-distribution detection, our work challenges the prevailing reliance on large buffers, suggesting that explicitly correcting these statistical artifacts could unlock robust performance with minimal replay. Tasks Task onset Good buffer boundary Good population boundary Area of Bufferoptimal decision boundaries The data is OOD, there is no class information in the features Classes are separable Classes are still separable but the the decision boundary is misaligned Shallow Forgetting Figure 1 : Evolution of decision boundaries and feature separability. PCA evolution of two Cifar10 classes (1% replay). Replay samples are highlighted with a black edge. While features retain separability across tasks (low deep forgetting), the classifier optimization becomes underdetermined: multiple “buffer-optimal” boundaries (dashed brown) perfectly classify the stored samples but largely fail to align to the true population boundary (dashed green), resulting in shallow forgetting.

💡 Research Summary

This paper tackles a long‑standing paradox in continual learning (CL): neural networks often preserve linearly separable feature representations of past tasks even when their output predictions have completely deteriorated. The authors formalize this phenomenon by distinguishing deep forgetting (loss of separability in the feature space) from shallow forgetting (the classifier’s decision boundary no longer aligns with the true population boundary despite still separable features). Using Experience Replay (ER) as the primary CL mechanism, they uncover a striking asymmetry: a minimal replay buffer is sufficient to anchor the geometry of the feature space and thus prevent deep forgetting, whereas mitigating shallow forgetting typically demands a substantially larger buffer.

Core Contributions

1. Sequential Neural Collapse Theory – The paper extends the Neural Collapse framework, originally defined for a single‑task setting, to the sequential CL scenario. It proves that any non‑zero replay fraction ρ > 0 guarantees that, as training proceeds, class means remain at a bounded distance from each other and class covariances collapse onto a low‑dimensional subspace. Consequently, linear separability of the features is asymptotically preserved regardless of buffer size.

2. Strong Collapse and Rank‑Deficient Covariances – When the buffer is tiny, the replayed samples dominate the statistics used to update the classifier. This induces a “strong collapse”: class means become overly inflated, and covariance matrices become rank‑deficient. The classifier therefore learns buffer‑optimal decision boundaries that perfectly separate the stored samples but are misaligned with the true population boundary, manifesting shallow forgetting.

3. Empirical Demonstration – On CIFAR‑10, CIFAR‑100, and TinyImageNet, the authors show that with as little as 1 % replay (≈50 samples for CIFAR‑10) the feature space remains linearly separable across tasks (deep forgetting ≈ 0). However, the decision boundary drifts dramatically, producing multiple plausible boundaries that classify the buffer perfectly yet fail on the full data distribution. Only when the buffer reaches roughly 10 % of the dataset do the learned boundaries converge toward the true ones.

4. Statistical Corrections – To bridge the gap without inflating memory, the paper proposes three corrective mechanisms:

   • Mean correction – Adjust stored class means using moving averages or estimated population means to counteract bias.
   • Covariance regularization – Add a small isotropic term (εI) or perform low‑rank reconstruction to avoid rank deficiency.
   • Dynamic replay ratio – Increase the replay fraction adaptively when a new task arrives, preventing abrupt shifts of the buffer‑optimal boundary.

5. Link to Out‑of‑Distribution (OOD) Detection – The authors reinterpret the statistical artifacts created by a small buffer as OOD signals. By integrating Mahalanobis‑based OOD scoring into sample selection or weighting, the replay buffer can be made more representative, thereby reducing shallow forgetting.

Implications

The work challenges the prevailing belief that large replay buffers are a prerequisite for robust CL. It demonstrates that deep forgetting can be eliminated with virtually any non‑zero buffer, while shallow forgetting is a statistical artifact that can be corrected. This insight is especially valuable for memory‑constrained platforms such as edge devices, where storing a large buffer is infeasible. By explicitly addressing the mean inflation and rank‑deficiency induced by tiny buffers, practitioners can achieve performance comparable to large‑buffer baselines with a fraction of the memory footprint.

Future Directions

The authors suggest extending the analysis to non‑linear classifiers (e.g., transformers), exploring buffer‑selection policies that jointly optimize OOD detection and replay utility, and implementing hardware‑efficient regularization schemes for on‑device continual learning.

In summary, the paper provides a rigorous theoretical foundation for the deep/shallow forgetting dichotomy, empirically validates the asymmetry in ER, and offers practical, low‑memory solutions that could reshape how continual learning systems are designed for real‑world, resource‑limited environments.