Graceful Forgetting in Generative Language Models

Recently, the pretrain-finetune paradigm has become a cornerstone in various deep learning areas. While in general the pre-trained model would promote both effectiveness and efficiency of downstream tasks fine-tuning, studies have shown that not all knowledge acquired during pre-training is beneficial. Some of the knowledge may actually bring detrimental effects to the fine-tuning tasks, which is also known as negative transfer. To address this problem, graceful forgetting has emerged as a promising approach. The core principle of graceful forgetting is to enhance the learning plasticity of the target task by selectively discarding irrelevant knowledge. However, this approach remains underexplored in the context of generative language models, and it is often challenging to migrate existing forgetting algorithms to these models due to architecture incompatibility. To bridge this gap, in this paper we propose a novel framework, Learning With Forgetting (LWF), to achieve graceful forgetting in generative language models. With Fisher Information Matrix weighting the intended parameter updates, LWF computes forgetting confidence to evaluate self-generated knowledge regarding the forgetting task, and consequently, knowledge with high confidence is periodically unlearned during fine-tuning. Our experiments demonstrate that, although thoroughly uncovering the mechanisms of knowledge interaction remains challenging in pre-trained language models, applying graceful forgetting can contribute to enhanced fine-tuning performance.

💡 Research Summary

The paper tackles the problem of negative transfer that arises when pre‑trained language models retain knowledge that is irrelevant or even harmful to a downstream fine‑tuning task. While “graceful forgetting”—the deliberate removal of such knowledge—has been explored in vision and non‑autoregressive models, its application to generative language models (GLMs) remains largely unexplored due to ambiguous knowledge boundaries and architectural incompatibilities.

To bridge this gap, the authors propose Learning With Forgetting (LWF), a three‑stage framework designed specifically for GLMs.

1. Eliciting Self‑Knowledge – Because the original pre‑training corpus is typically unavailable, LWF uses the target forgetting dataset (D_F) as a set of prompts fed to the base model. The model’s own generated responses constitute a synthetic “self‑knowledge” dataset (D_{self}), which approximates the model’s internal representation of the knowledge to be forgotten. This step works even with unlabeled data.
2. Evaluating Forgetting Confidence – For each generated sample (x\in D_{self}), the method quantifies how much it conflicts with the learning task (D_L). Starting from a Bayesian formulation, the authors approximate the posterior (P(D_L|x)) by the surrogate (P(\theta^(x)|D_L)), where (\theta^(x)=\arg\max_\theta P(\theta|x)) is obtained via a single‑step gradient update. Assuming a Gaussian posterior over parameters centered at the optimal fine‑tuning parameters (\theta^_L), the forgetting confidence is defined as (-\log P(\theta^(x)|D_L)). Practically, this reduces to a Fisher‑weighted squared distance:

\