SA-CAISR: Stage-Adaptive and Conflict-Aware Incremental Sequential Recommendation

Sequential recommendation (SR) aims to predict a user’s next action by learning from their historical interaction sequences. In real-world applications, these models require periodic updates to adapt to new interactions and evolving user preferences. While incremental learning methods facilitate these updates, they face significant challenges. Replay-based approaches incur high memory and computational costs, and regularization-based methods often struggle to discard outdated or conflicting knowledge. To overcome these challenges, we propose SA-CAISR, a Stage-Adaptive and Conflict-Aware Incremental Sequential Recommendation framework. As a buffer-free framework, SA-CAISR operates using only the old model and new data, directly addressing the high costs of replay-based techniques. SA-CAISR introduces a novel Fisher-weighted knowledge-screening mechanism that dynamically identifies outdated knowledge by estimating parameter-level conflicts between the old model and new data, selectively removing obsolete knowledge while preserving compatible historical patterns. This dynamic balance between stability and adaptability allows our method to achieve state-of-the-art performance in incremental SR. Specifically, SA-CAISR improves Recall@20 by 2.0% on average across datasets, while reducing memory usage by 97.5% and training time by 46.9% compared to the best baseline. This efficiency allows real-world systems to rapidly update user profiles with minimal computational overhead, ensuring more timely and accurate recommendations.

💡 Research Summary

Sequential recommendation (SR) systems must constantly adapt to evolving user preferences and newly emerging items. Traditional incremental learning approaches fall into two categories: replay‑based methods that store a subset of historical interactions for rehearsal, and regularization‑based methods that constrain parameter updates using importance estimates from the previous model. Replay‑based approaches achieve strong stability but incur prohibitive memory, computational, and privacy costs. Regularization‑based approaches are lightweight but treat the old model as a fixed teacher, lacking mechanisms to detect and discard knowledge that conflicts with the latest data, which can lead to over‑constraining or catastrophic forgetting under distribution shift.

The paper introduces SA‑CAISR (Stage‑Adaptive and Conflict‑Aware Incremental Sequential Recommendation), a buffer‑free framework that operates solely with the old model and newly arrived interactions. Its core contribution is a Fisher‑weighted knowledge‑screening mechanism. For each incremental stage, the method computes Fisher information on the fresh data by approximating the Hessian of the log‑likelihood. This yields a per‑parameter sensitivity score that quantifies how much the new data would be affected by changes to each weight. By comparing this sensitivity with the corresponding parameter values in the old model, a “conflict score” is derived. Parameters whose conflict scores exceed a dynamically set threshold are masked, effectively removing outdated or contradictory knowledge from the update process. Parameters with low conflict scores are left untouched, preserving compatible historical patterns. The screening is stage‑adaptive: Fisher information and conflict scores are recomputed at every update, allowing the model to continuously re‑evaluate which parts of its knowledge remain relevant as user behavior drifts.

To prevent the loss of useful historical representations, SA‑CAISR augments the standard cross‑entropy loss with an InfoNCE‑based consistency loss. After masking, the old model becomes a “purified teacher” that contains only the compatible knowledge. The updated model is trained to align its latent representations with those of the purified teacher using a contrastive objective, ensuring that the new model inherits stable, high‑quality historical signals while still learning fresh preferences. The final loss is a weighted sum of (i) the cross‑entropy loss on new interactions and (ii) the InfoNCE consistency loss, with a hyper‑parameter λ controlling the trade‑off.

Extensive experiments on four public benchmarks (Beauty, Sports, Toys, Yelp) compare SA‑CAISR against state‑of‑the‑art replay‑based methods (e.g., ADER, ReLoop) and regularization‑based methods (e.g., Elastic Weight Consolidation, Learning without Forgetting, CL4SRec). Across all datasets, SA‑CAISR improves Recall@20 by an average of 2.0 percentage points, and also yields gains of 2.1 pp in MRR@20 and NDCG@20. Importantly, it reduces memory consumption by 97.5 % relative to replay‑based baselines and cuts training time by 46.9 % on average, demonstrating that the proposed knowledge‑screening is both effective and efficient. The method remains robust under abrupt trend shifts, where traditional regularization methods either over‑constrain the model or fail to discard obsolete patterns.

The authors also discuss the relationship to concurrent work FGGM, which uses Fisher‑guided gradient masking for continual learning in static classification tasks. While both share the idea of Fisher‑based importance, FGGM does not address the sequential, user‑item interaction dynamics nor the need for a contrastive consistency term. SA‑CAISR tailors Fisher‑guided masking to the SR setting, incorporates stage‑adaptive thresholds, and introduces a purified teacher for contrastive alignment, thereby handling user preference drift and new item emergence more effectively.

In summary, SA‑CAISR offers three key innovations: (1) a buffer‑free design that eliminates storage and privacy concerns; (2) a dynamic, Fisher‑based conflict detection that selectively prunes outdated knowledge at the parameter level; and (3) a contrastive consistency loss that preserves compatible historical representations. These contributions enable real‑time, large‑scale recommendation services to update models with minimal overhead while maintaining or improving recommendation quality, striking an optimal balance between stability and plasticity in continual learning for sequential recommendation.