Introduction

Sequential recommendation (SR) aims to predict the next items a user is likely to prefer based on their interaction history. Recent neural SR models  employ various architectures to effectively capture sequential dependencies among items. However, they still face inherent challenges such as the cold-start and data sparsity problems , which limit their generalizability and overall performance.

Cross-domain sequential recommendation (CDSR) has emerged as a promising research direction . CDSR aims to improve recommendation accuracy by either jointly training models across multiple domains  or by transferring knowledge from data-rich domains to sparser ones . However, existing CDSR works face three fundamental limitations. (1) User/Item overlap dependency: knowledge transfer typically relies on the presence of overlapping users or items across domains. However, such overlap is extremely limited in practice. We observe that only 16 users and 0 items are shared across eight Amazon domains, reflecting the real-world nature of independently operated domains; (2) Data isolation: in real-world scenarios, access to raw user data is often restricted due to organizational boundaries or privacy regulations . User logs contain sensitive information and cannot be shared across domains due to privacy restrictions, making domain-specific training data inaccessible; (3) Low scalability: joint training across multiple domains incurs substantial computational overhead, making it impractical for large-scale deployment. Consequently, most prior work has been limited to integration of only two or three domains, leaving scalable multi-domain integration largely unresolved.

We suggest that model merging  offers an effective alternative for building universal recommender systems. Model merging integrates fine-tuned parameters from multiple domain- or task-specific models into a single unified model. This paradigm provides several advantages that directly address the key limitations of CDSR: (1) It eliminates the need for overlapping users or items across domains; (2) It naturally preserves user privacy, as only model parameters, not sensitive interaction data, are required; (3) It achieves high scalability by avoiding the computational burden of cross-domain joint training.

In this paper, we explore the feasibility of applying model merging to CDSR under a new, realistic problem setting termed data-isolated CDSR. This setting is motivated by practical real-world constraints, where user interaction data can be used only to train domain-specific models and cannot be shared across domains or accessed afterward. Unlike conventional CDSR, which often relies on strong and impractical assumptions (i.e., overlapping users or items), data-isolated CDSR allows domains to be disjoint. Moreover, while privacy-preserving CDSR typically requires access to domain-specific interaction data during model optimization, data-isolated CDSR constructs a universal cross-domain recommender system without accessing any user interaction data, thereby providing a stronger guarantee of user privacy.

Under this setting, however, directly applying existing model merging methods is non-trivial for two key reasons. First, since interaction data are not shared across domains, test-time adaptation schemes, commonly used in the model merging paradigm to optimize merging weights, cannot be applied. Second, even if test data were accessible, leveraging test sequences in sequential recommendation would violate the core assumptions of model merging. While the model merging paradigm explicitly prohibits using training data, these assumptions do not hold in sequential recommender systems. In such systems, test sequences are not independent of the training data but are generated from the same evolving user behavior. Thus, using test-time user interaction sequences during the merging process would inevitably expose training information (Figure 1). Consequently, leveraging test data for model merging is fundamentally incompatible with the data-isolated CDSR setting.

To this end, we propose MergeRec, a novel framework tailored for data-isolated CDSR. MergeRec comprises three key components: (1) merging initialization, (2) pseudo-user data construction, and (3) collaborative merging optimization. First, we synthesize an initial merged model using training-free merging methods based on task vectors, defined as the parameter difference between a fine-tuned model and its corresponding pre-trained model, to capture domain-specific knowledge . Next, we construct pseudo-user data by treating each item in every domain as an individual sequence. Despite its simplicity, MergeRec enables the construction of meaningful samples for merging domains without relying on real user data, effectively simulating cold-start users across domains. Finally, we refine domain-specific merging weights through a recommendation-oriented merging objective.

To design an effective objective function for merging recommender systems, we argue that an ideal merged model should satisfy two fundamental requirements. First, it should be able to decode users’ multiple intents, which are often reflected in domain-specific sequential patterns. Second, the unified model should exhibit strong ranking ability, accurately prioritizing items with the highest click probability within each domain context. We point out that existing adaptive merging methods, i.e., AdaMerging, address only the latter aspect and are therefore insufficient for merging recommender systems (Section 3).

To overcome this limitation, we propose a joint objective that combines: (1) a distillation loss, which leverages the prediction distributions of fine-tuned models as soft labels, and (2) a recommendation loss, which treats the top-1 predicted item from each fine-tuned model for a pseudo-user in its corresponding domain as a hard label. The distillation loss transfers collaborative filtering (CF)  signals from the fine-tuned models to the merged model, and the recommendation loss guides the merged model to accurately rank items according to their likelihood of being clicked.

Extensive experiments demonstrate that MergeRec not only preserves the strengths of the individual source models but also generalizes effectively to unseen domains. Compared with existing merging methods and strong baselines, including fine-tuned and joint learning models, MergeRec consistently achieves superior performance. Specifically, MergeRec outperforms joint learning and AdaMerging by average gains of 8.72% and 17.21% on Recall@10, respectively. These results highlight that model merging can be a scalable and efficient paradigm for building universal recommender systems.

Our contribution can be summarized as follows:

• Thorough empirical analysis: We provide empirical evidence demonstrating that entropy-based optimization, though effective in computer vision and natural language processing, fundamentally fails to capture the multi-intent behavioral patterns inherent in recommender systems.

• The first model merging framework for recommender systems: We propose MergeRec, a task vector-based model merging framework tailored for recommender systems. MergeRec comprises three key components: (1) training-free merging initialization, (2) privacy-preserving pseudo-user data construction, and (3) a recommendation-oriented merging objective.

• Comprehensive evaluation: Through extensive experiments across eight Amazon benchmark datasets and four backbone architectures, we demonstrate that MergeRec consistently outperforms existing model merging baselines. Notably, MergeRec exhibits superior generalizability to unseen domains and robust performance under data-scarce conditions.

Preliminaries

Cross-domain Sequential Recommendation

Let $`\mathcal{D}=\{D_1, D_2, ..., D_K\}`$ denote the set of all recommendation domains, where $`D_k`$ denotes the $`k`$-th domain. Each domain $`D_k`$ consists of a set of items $`\mathcal{I}_k`$ and users $`\mathcal{U}_k`$. For an arbitrary user $`u \in \mathcal{U}_k`$, the interaction history is represented as an ordered sequence of items based on timestamps: $`u = \left[ i_{1},\, i_{2},\, \ldots,\, i_{|u|} \right]`$, where $`|u|`$ denotes the number of interactions of user $`u`$. CDSR models aim to predict and rank items in $`\mathcal{I}_k`$ by estimating the probability that user $`u`$ will interact with each item next, conditioned on the user’s past interactions:

\begin{equation}
    \mathbf{\theta}^* = \arg\max_\mathbf{\theta} P \bigl(i = i_{|u|+1} \mid u, \mathbf{\theta}\bigr),
\end{equation}

where $`\mathbf{\theta}`$ denotes the parameters of the CDSR model.

Text-based Sequential Recommendation

Text-based SR  leverages pre-trained language models (PLMs) to encode item-level textual information. By representing both users and items through textual descriptions, this approach enables recommendations for previously unseen (i.e., cold-start) items.

Formally, the textual representation of an item $`t_i`$ is constructed from its attribute descriptions (e.g., title, brand, and category). The textual representation of a user $`t_u`$ is defined as the concatenation of the textual representations of all items the user has interacted with:

\begin{equation}
    t_{u} = \left[ t_{i_{1}};\, t_{i_{2}};\, \ldots;\, t_{i_{|u|}} \right],
\end{equation}

where $`;`$ denotes the concatenation operator.

Let $`f(\cdot \mid \mathbf{\theta_k})`$ denote a PLM-based encoder with parameters $`\mathbf{\theta}_k \in \mathbb{R}^P`$ fine-tuned on domain $`D_k`$, where $`P`$ is the total number of model parameters. Given the textual inputs $`t_u`$ and $`t_i`$, the encoder produces a user representation vector $`\mathbf{r}_u \in \mathbb{R}^d`$ and an item representation vector $`\mathbf{r}_i \in \mathbb{R}^d`$ by extracting the final hidden state representations:

\begin{equation}
    \mathbf{r}_u = f\left( t_u \mid \mathbf{\theta}_k \right), ~~ \mathbf{r}_i = f\left( t_i \mid \mathbf{\theta}_k \right),
\end{equation}

where $`d`$ denotes the dimension of the final hidden representations.

The recommendation score $`\hat{y}_{ui}`$ between user $`u`$ and item $`i`$ is computed as the cosine similarity between their representation vectors:

\begin{equation}
    \hat{y}_{ui} = \cos(\mathbf{r}_u, \mathbf{r}_i).
\end{equation}

The model parameters $`\mathbf{\theta}_k`$ are optimized using a cross-entropy objective:

\begin{equation}
    \mathbf{\theta}_k^{*} = \arg\min_{\mathbf{\theta}_k} \, \sum_{u \in \mathcal{U}_k} \left( \log{\hat{y}_{k, ui^+}} + \sum_{i^- \in \mathcal{I}_k} \log{(1 - \hat{y}_{k, ui^-})} \right),
\end{equation}

where $`i^+`$ denotes the next item in the user sequence, and $`i^-`$ represents negative items in the domain $`\mathcal{I}_k`$ excluding $`i^+`$. Note that the domain-specific parameters $`\mathbf{\theta}_k`$ are initialized from the pre-trained base model parameters $`\mathbf{\theta}_{base} \in \mathbb{R}^P`$.

Proposed Method: MergeRec

As illustrated in Figure 2, we design the MergeRec framework to address the practical constraint that no interaction data can be shared across domains, termed data-isolated CDSR. MergeRec consists of three key components: (1) Merging Initialization, which consolidates multiple domain-specific fine-tuned models into a single unified model that integrates knowledge across domains; (2) Pseudo-user Data Construction, which synthesizes meaningful merging samples without relying on real user interactions; and (3) Collaborative Merging Optimization, which jointly optimizes a recommendation loss and a knowledge distillation loss to enable recommendation-aware parameter integration. Through this design, MergeRec effectively preserves domain-specific CF signals while ensuring strong generalizability across multiple domains.

Data-isolated CDSR

We formalize a new and realistic setting for cross-domain sequential recommendation, termed data-isolated CDSR. As shown in Table [tab:research_categorization], this setting is characterized by two key requirements: (i) it assumes no overlap across domains, (ii) it prohibits access to user interaction data during cross-domain model construction, in contrast to conventional CDSR. Under data-isolated CDSR, domains may be entirely disjoint in both users and items (e.g., $`\mathcal{U}_k \cap \mathcal{U}_{k'}=\emptyset`$ and $`\mathcal{I}_k \cap \mathcal{I}_{k'}=\emptyset`$ for $`k\neq k'`$), reflecting real-world environments that are independently operated. Moreover, this setting enforces a strict data isolation constraint: raw interaction logs are accessible only within each domain for training domain-specific models and cannot be shared across domains. Consequently, a cross-domain recommender system must be constructed without accessing any domain-specific interaction data. Instead, we assume access only to $`K`$ domain-specific fine-tuned models $`\{\theta_k\}_{k=1}^K`$, while the datasets used to train these models remain completely inaccessible. The goal of data-isolated CDSR is to produce a single universal sequential recommender system that can operate across multiple domains.

Merging Initialization

Problem definition. Let $`f_{\theta_k}(u_k) \rightarrow \hat{y}_k`$ denote an SR model fine-tuned on the private data of domain $`D_k = \{\mathcal{U}_k, \mathcal{I}_k\}`$. For an arbitrary user interaction sequence $`u_k \in \mathcal{U}_k`$, the model outputs a click probability vector $`\hat{\mathbf{y}}_k \in \mathbb{R}^{|\mathcal{I}_k|}`$ over candidate items. Without loss of generality, we assume that the model parameters are composed of $`L`$ layers, i.e., $`\theta = \left\{ \theta^1, \theta^2, ..., \theta^L \right\}`$.

Task vector. A task vector represents the parameter shift required to adapt a pre-trained model to a specific downstream task . In our context, each task corresponds to recommendation within a specific domain. Accordingly, the task vector captures domain-specific knowledge, enabling a pre-trained model to specialize in that domain.

Formally, the task vector $`\boldsymbol{\tau}_k \in \mathbb{R}^P`$ for domain $`k`$ is defined as the difference between the parameters of the fine-tuned model $`\mathbf{\theta}_k`$ and those of the original pre-trained base model $`\mathbf{\theta}_{base}`$:

\begin{equation}
\boldsymbol{\tau}_k = \mathbf{\theta}_k - \mathbf{\theta}_{base}.
\end{equation}

where $`P`$ denotes the total number of model parameters.

Domain-wise merging. Domain-wise merging integrates multiple fine-tuned models by combining their task vectors, each weighted by a domain-specific scalar $`w_k`$, and adding them to the base model parameters. The merging weights $`\mathbf{w}=\{w_1, ..., w_K\}`$ can be either uniformly assigned  or adaptively learned to reflect domain characteristics . Intuitively, domains containing more distinctive knowledge may receive higher weights, while those sharing similar CF signals may be down-weighted. Formally, domain-wise merging is defined as:

\begin{equation}
 \label{eq:domain_wise_merging}
    \theta_{merge}=\theta_{base}+\sum_{k=1}^{K}{w_k \cdot \boldsymbol{\tau}_k}.
\end{equation}

In this work, we learn the domain-specific weights $`\mathbf{w}`$ in a data-driven manner.

Layer-wise merging. Since different layers in deep neural networks capture different levels of abstraction , applying a single scalar weight per domain may be insufficient to control inter-domain interference. To enable fine-grained integration, we assign independent merging weights to each layer for every domain.

Let $`\theta_k = \{\theta_k^1, \dots, \theta_k^L\}`$ denote the parameters of the fine-tuned model for domain $`k`$. The corresponding layer-wise task vector is defined as $`\boldsymbol{\tau}_k = \{\theta_k^1 - \theta_{\text{base}}^1, ..., \theta_k^L - \theta_{\text{base}}^L\}`$. The layer-wise merging is then defined as:

\begin{equation}
 \label{eq:layer_wise_merging}
    \theta_{merge}=\left\{ \theta_{base}^l+\sum_{k=1}^{K}{w_k^l \cdot \tau_k^l} \right\}_{l=1}^L.
\end{equation}

The layer-specific merging weights $`\mathbf{w}_k = \{w_k^1, \dots, w_k^L\}`$ are similarly learned in a data-driven manner.

Pseudo-user Data Construction

User logs in recommender systems typically contain sensitive personal information and cannot be shared across domains, making it challenging to construct data for learning merging weights. To address this, we propose a novel pseudo-user data construction strategy that represents each item in a domain as a single-item interaction sequence. Our design is grounded in the idea that CF knowledge is encapsulated within domain-specific models and can be transferred without relying on the real user data on which they were trained. By leveraging pseudo-users as surrogate inputs, our approach enables learning merging weights without access to domain-specific fine-tuning data while strictly preserving data isolation.

Formally, we construct the pseudo-user set for domain $`k`$ as:

\begin{equation}
    \mathcal{\tilde{U}}_{k} = \left \{ [i] \mid i \in \mathcal{I}_k \right \}.
\end{equation}

These synthesized samples emulate plausible cold-start users in each domain and thus provide meaningful signals for model merging. Although each pseudo-user sequence contains no explicit sequential context, it serves as a probe to elicit rich CF knowledge encoded in the corresponding domain-specific model $`f_{\theta_k}`$. We therefore employ each domain-specific model as a teacher, whose conditional distribution $`P_{\theta_k}(\cdot \mid [i])`$ captures the local co-consumption structure around item $`i`$, i.e., next-item likelihoods. By distilling the merged model to align with these teacher distributions, we effectively transfer domain-specific CF signals. We observe that even single-item pseudo-user sequences are sufficient for effective model merging (Section 5), providing a practical and privacy-preserving foundation for collaborative merging optimization. While extending pseudo-user sequences to longer contexts may further enrich the transferred signals, we leave this promising direction for future work.

style="width:100.0%" />

Collaborative Merging Optimization

Inadequacy of entropy-based optimization. We examine a representative adaptive merging method, AdaMerging , from the perspective of recommender systems. Figure 3 illustrates the training dynamics of cross-entropy loss and prediction entropy for both AdaMerging and our proposed MergeRec over training iterations on eight datasets, evaluated using test user sequences. The cross-entropy loss reflects alignment with the recommendation objective, whereas prediction entropy measures the confidence of the merged model’s predictions. As shown in Figure 3 (a), AdaMerging successfully reduces prediction entropy during training but fails to achieve a corresponding decrease in cross-entropy loss. This limitation stems from its exclusive focus on entropy minimization, which merely amplifies confidence in the top-1 predicted item. However, users often exhibit multi-intent behavioral patterns rather than a single dominant intent , making entropy-based optimization alone insufficient to capture the rich CF signals learned by domain-specific fine-tuned models.

To overcome this, we introduce a distillation loss that extends beyond entropy-based optimization by explicitly aligning the merged model with teacher distributions derived from domain-specific models. MergeRec (Figure 3 (b)) simultaneously reduces both cross-entropy loss and prediction entropy, demonstrating more consistent and effective optimization toward the recommendation objective.

Joint objective function. We posit that an ideal merged model for cross-domain recommendation should simultaneously satisfy two essential aspects. First, it should effectively capture diverse user intents reflected in behavioral patterns within each domain and retrieve items relevant to those intents. This requires successfully transferring domain-specific CF knowledge from fine-tuned models to the merged model. Second, the merged model should exhibit strong discriminative capability to accurately identify the items that users are most likely to click on within each domain. To jointly address these requirements, we propose the following optimization function:

\begin{equation}
\mathcal{L} = \mathcal{L}_{\text{Rec}} + \lambda \cdot \mathcal{L}_{\text{KD}},
\end{equation}

where $`\lambda`$ is a hyperparameter that balances the two losses. In Eqs. [eq:domain_wise_merging] and [eq:layer_wise_merging], we optimize only the merging weights $`w`$, while keeping the base model parameters $`\theta_{base}`$ and the task vector $`\tau`$ fixed. Since only $`K`$ or $`K \times L`$ domain-specific weights are optimized, MergeRec provides a computationally efficient solution.

The knowledge distillation loss $`\mathcal{L}_{\text{KD}}`$ integrates domain-specific CF knowledge encoded in fine-tuned models by aligning the predictions of the merged model with those of the corresponding domain-specific models. For a pseudo-user sequence $`u \in \mathcal{\tilde{U}}_k`$ in domain $`k`$, we minimize the Kullback-Leibler (KL) divergence between the prediction $`\hat{y}_{merge}`$ produced by the merged model $`\theta_{merge}`$ and the prediction $`\hat{y}_{k}`$ produced by the fine-tuned model $`\theta_{k}`$:

\begin{equation}
    \begin{split}
    \mathcal{L}_{\text{KD}} &= \sum_{k=1}^{K} \sum_{u \in \mathcal{\tilde{U}}_{k}} \text{KL}(\hat{p}_{merge,u} \parallel \hat{p}_{k,u}) \\
    &= \sum_{k=1}^{K} \sum_{u \in \mathcal{\tilde{U}}_{k}} \sum_{i \in \mathcal{I}_k} \hat{p}_{merge,ui} \log{\frac{\hat{p}_{merge, ui}}{\hat{p}_{k,ui}}},
    \end{split}
\end{equation}

where $`\hat{p}_{*}=\text{softmax}(\hat{y}_{*} / T)`$, and $`T`$ denotes the temperature hyperparameter which is empirically set to $`1`$ in our experiments.

The recommendation loss $`\mathcal{L}_{\text{Rec}}`$ encourages the merged model to accurately identify items aligned with user intent by assigning high scores to the next item and low scores to others. Since real user sequences are unavailable, we leverage the top-1 predicted item $`\tilde{i}^+`$, obtained by feeding pseudo-user data into the corresponding fine-tuned model, as a positive pseudo-label:

\begin{equation}
    \begin{split}
    \mathcal{L}_{\text{Rec}} = \sum_{k=1}^{K} \sum_{u \in \mathcal{\tilde{U}}_{k}} \left( \log{\hat{y}_{merge,u\tilde{i}^+}} + \sum_{i^- \in \mathcal{I}_k} \log{(1 - \hat{y}_{merge,ui^-})} \right).
    \end{split}
\end{equation}

Experimental Setup

Datasets. To simulate a cross-domain recommendation environment, we use eight categories from the Amazon dataset^1 2: Arts, Beauty, Instruments, Office, Pantry, Scientific, Sports, and Toys. Following existing work , we adopt a 5-core setting, i.e., users and items with fewer than five interactions are removed. Detailed dataset statistics are provided in Appendix 8.

Baselines. We compare MergeRec with the following methods:

• Zero-shot: Directly applies pre-trained text-based SR models without fine-tuning on a specific domain.

• Fine-tuning: Fine-tunes pre-trained models using domain-specific interaction data.

• Joint Learning: Trains a unified model on aggregated multi-domain datasets with shared parameters.

• Task Arithmetic : Constructs a cross-domain model by linearly adding task vectors to a pre-trained model.

• TIES : Reduces noise and conflicts between task vectors by selecting parameters with large variance and aligning their signs.

• AdaMerging : Learns adaptive merging weights in an unsupervised manner by minimizing the prediction entropy of the merged model.

We evaluate all methods on RecFormer-base/large , a representative text-based SR model, and BLaIR-base/large , a language model post-trained on a recommendation corpus, as backbone architectures. For a fair comparison under the data-isolation setting, AdaMerging is adapted to use the same pseudo-user data as MergeRec, treating it as unlabeled inputs for entropy-based optimization. Implementation details are provided in Appendix 9.

Evaluation protocol. Following , we adopt the leave-one-out strategy to split the train, validation, and test datasets. For each user, the most recently interacted item is used for testing, the second most recently interacted item for validation, and the rest for training. Note that the training data is used only for fine-tuning and joint learning. We evaluate recommendation performance using Recall@10 (R@10) and NDCG@10 (N@10). Following , we normalize the performance of each method by that of its corresponding fine-tuned model. The normalized results are reported in Table [tab:overall] and Figures 4, 5, and 7.

Experimental Results

Overall Performance

Table [tab:overall] shows the normalized R@10 of MergeRec and seven baseline methods across eight datasets and four backbone models, where the performance of the fine-tuned model on each dataset is normalized to 100%. The corresponding normalized N@10, unnormalized R@10, N@10 results are provided in Appendix 10.

MergeRec consistently achieves the best average performance across all datasets and backbone models for both the domain-wise and layer-wise variants. Specifically, MergeRec outperforms Joint Learning and AdaMerging with average gains of 8.72% and 17.21%, respectively. This indicates that MergeRec simultaneously enhances the ranking discriminative ability of the merged model and effectively transfers domain-specific CF knowledge from fine-tuned models. Meanwhile, AdaMerging performs substantially worse across all datasets and backbone models, suggesting that merely amplifying prediction confidence is insufficient to capture the diverse CF signals present across multiple domains. Furthermore, MergeRec surpasses training-free model merging methods (i.e., Task Arithmetic and TIES) by average gains of 9.90% and 3.04%, respectively.

Several model merging methods, i.e., MergeRec, Weight Averaging, and TIES, consistently outperform Joint Learning. These results demonstrate that model merging can effectively capture complementary domain knowledge and improve recommendation quality without relying on cross-domain training data. It further highlights the practical advantages of model merging, as it not only reduces computational overhead but also enables synergistic knowledge transfer across domains without end-to-end re-training.

Cross-domain merging is particularly beneficial for data-scarce domains. On the Scientific dataset, MergeRec achieves improvements of 1.44% for RecFormer-base (Domain-wise) and 5.18% for BLaIR-large (Layer-wise) compared to their respective fine-tuned counterparts. These improvements can be attributed to the limited number of users in the Scientific domain, where the merged model benefits more substantially from cross-domain knowledge transferred from other domains.

Overall, these results demonstrate that MergeRec provides a robust and scalable solution for cross-domain model merging in recommender systems, delivering consistent and significant performance gains across diverse domains and backbone architectures.

Model Merging on Scarce Training Data

Collecting sufficient data is often challenging in the early stages of recommender systems. Under such data-scarce conditions, model merging can offer a promising solution for improving model generalization by leveraging knowledge from data-rich domains. We investigate whether model merging can improve recommendation performance in domains with limited data.

To simulate this scenario, we divide the eight domains into two groups: five source domains (Arts, Beauty, Pantry, Sports, Toys) and three target domains (Instruments, Office, Scientific). We then vary the degree of scarcity in the target domain by randomly sampling $`k\%`$ of users ($`k = 1, 5, 10`$) from the whole user set to construct data-scarce training sets. Each fine-tuned model trained on a data-scarce target domain is subsequently merged with the fine-tuned models trained from the five source domains.

Table 1 compares MergeRec with five merging methods under different levels of data scarcity using the RecFormer-base backbone. MergeRec consistently outperforms the corresponding fine-tuned models, demonstrating its strong ability to transfer knowledge across domains even under extreme data scarcity. Moreover, MergeRec consistently surpasses AdaMerging across all scarcity levels, indicating that our recommendation-oriented optimization captures transferable CF patterns more effectively than the entropy minimization approach. These results confirm that MergeRec can reliably transfer CF signals from data-rich to data-scarce domains.

Performance on Unseen Domains

To examine whether model merging remains effective under extreme conditions where no interaction data are available for the target domain, we merge models trained on five source domains (Arts, Beauty, Pantry, Sports, Toys) and evaluate their performance on three unseen target domains (Inst., Office, Sci.). Note that neither interaction data nor domain-specific models from the target domains are used in constructing the merged model.

Table 2 shows the performance of four merging methods on unseen domains. MergeRec consistently outperforms the other methods, achieving improvements of up to 3.92% over Task Arithmetic. These results demonstrate the strong generalizability of MergeRec and highlight its effectiveness in transferring knowledge to entirely unseen domains without relying on any target-domain data.

style="width:90.0%" />

style="width:100.0%" />

style="width:90.0%" />

Further Analysis

Effect of the number of merged domains. To assess the robustness of MergeRec, we analyze how recommendation performance varies with the number of merged models, as shown in Figure 4. Task Arithmetic and TIES exhibit competitive performance when merging a small number of domains (four or five), but their performance deteriorates as more domains are included, eventually falling behind MergeRec. AdaMerging consistently performs poorly across all settings, indicating that its entropy-based optimization strategy fails to effectively adapt to recommendation tasks. In contrast, MergeRec maintains stable performance regardless of the number of merged domains. Notably, its average performance improves as more domains are integrated, highlighting its strong ability to effectively consolidate and leverage knowledge across an increasing number of diverse domains.

Performance across user and item groups. To further analyze the sources of performance improvements, we partition the test data into multiple groups based on user history length and target item popularity. Figure 5 shows the performance of three merging methods across five sequence-length groups and five item popularity ranges. All methods are evaluated on the eight datasets, and the results are averaged and normalized by the performance of the corresponding fine-tuned model within each group.

As shown in Figure 5 (a), AdaMerging exhibits severe performance degradation as the sequence length increases. This result indicates that AdaMerging struggles to capture CF signals in longer sequences, as it does not learn inter-item relationships during optimization. In contrast, MergeRec consistently achieves the best performance across all sequence lengths by effectively transferring inter-item relationships through the proposed objective function. Figure 5 (b) shows that MergeRec outperforms other methods across all item popularity ranges. Notably, in the least popular item group (1–10), MergeRec achieves substantial performance gains over competing methods, highlighting its ability to leverage cross-domain knowledge to recommend less popular items.

Domain-specific weight dynamics. To understand how the model adapts to different domains during merging, we analyze the training trajectories of the domain-wise merging weights $`\mathbf{w}`$ across eight domains using the RecFormer-base model. Figure 6 illustrates the evolution of each domain’s weight during training. We observe that the weights gradually converge to distinct values for each domain, indicating that the model learns to differentiate the relative importance of individual domains. Notably, domains with larger scales (e.g., Arts and Office) converge to higher weight coefficients. These results suggest that domains exhibiting more complex CF patterns are assigned greater emphasis, enabling the merged model to better preserve domain-specific knowledge. The corresponding results for other backbone models are reported in Appendix 11.

style="width:95.0%" />

Hyperparameter Sensitivity

Figure 7 depicts the effect of the trade-off hyperparameter $`\lambda`$ on recommendation performance. The results are averaged across all eight datasets and normalized by the performance of the corresponding fine-tuned models. We find that performance generally improves as $`\lambda`$ increases, except for domain-wise merging with the RecFormer-base backbone. This suggests that placing greater emphasis on the knowledge distillation loss is typically more effective. This can be attributed to the model learning the full item-prediction distribution from domain-specific teacher models, which enables the merged model to capture richer and more informative CF signals. In contrast, methods that focus solely on amplifying top-1 prediction confidence provide a more limited supervisory signal.

Related Work

Cross-Domain Sequential Recommendation

CDSR  has emerged as an effective approach to alleviate data sparsity and cold-start problems in single-domain recommender systems . Existing CDSR approaches can be categorized into three types: single-target CDSR, dual-target CDSR, and multi-target CDSR. Additionally, privacy-preserving cross-domain recommendation (CDR) has gained increasing attention in privacy-sensitive and federated learning scenarios.

Single-target CDSR. Single-target CDSR  is the most extensively studied setting, aiming to improve recommendation performance in a data-scarce target domain by transferring knowledge from data-rich source domains with abundant user-item interactions or auxiliary information. C$`^2`$DSR  jointly models intra-sequence and inter-sequence item relationships through graph neural networks and self-attention mechanisms. To enhance self-attention modules, MAN  introduces both local and global attention modules to capture domain-specific and cross-domain information. Recently, LLM4CDSR  leverages large language models to generate semantic item representations from textual attributes and hierarchical user profiles from interaction sequences, facilitating cross-domain knowledge transfer.

Dual- and multi-target CDSR. Dual- and multi-target CDSR  aims to simultaneously improve recommendation performance across multiple domains. DTCDR  adopts a multi-task learning framework that bidirectionally transfers user preference representations across domains via shared embeddings. Recent studies  have highlighted the negative transfer problem, where knowledge from other domains can fail to provide beneficial contributions. To address this, CGRec  and SyNCRec  introduce adaptive loss weighting based on estimated transfer gaps between single-domain and cross-domain sequential recommendation tasks. However, these approaches fundamentally rely on overlapping users or items to enable knowledge transfer and do not account for the privacy constraints commonly required in real-world scenarios.

Privacy-preserving CDR. Privacy-preserving CDR  aims to improve recommendation performance under settings where access to raw data from individual domains is restricted or entirely unavailable. An early study  separates personalized and transferable components to enable privacy-compliant recommendations. P2M2-CDR  disentangles domain-common and domain-specific embeddings while applying local differential privacy to perturb shared representations. FedGCDR  further adopts federated graph learning with differential privacy-based knowledge extraction and graph expansion to mitigate the negative transfer problem. However, these approaches typically rely on overlapping users across domains and require cross-domain coordination during training, where domain-specific data are used alongside data from other domains (e.g., exchanged model parameters or gradients). Therefore, they do not fully satisfy the data isolation constraint.

In this paper, we pioneer the application of model merging to data-isolated multi-target CDSR, providing a scalable solution for integrated multi-domain recommendations without requiring direct access to domain-specific user interaction data, thereby preserving user privacy.

Model Merging

Model merging  aims to improve the generalization of domain-specific fine-tuned models by consolidating knowledge from multiple models trained on diverse domains or tasks into a single model, typically without requiring access to training data. The simplest approaches  perform parameter averaging across fine-tuned models that share the same backbone pre-trained model.

Beyond these early methods, task vector-based model merging  has been proposed to enable more effective knowledge consolidation. Task vectors are defined as the parameter differences between each fine-tuned model and the pre-trained model. They can be interpreted as directions that encode domain- or task-specific adaptations. Task Arithmetic  shows that task vectors can be combined to build multi-task models, or negated to attenuate (or remove) task-specific knowledge. TIES  selects parameters with large task-induced changes and resolves sign conflicts to reduce interference. AdaMerging  extends linear task-vector composition by learning domain- or layer-wise merging weights through entropy minimization on unlabeled test samples. Although model merging has demonstrated strong effectiveness in computer vision and natural language processing, its potential for recommender systems has remained largely unexplored.

Conclusion

In this work, we addressed the fundamental limitations of existing CDSR under realistic constraints, where user interaction data cannot be shared across domains. To this end, we introduced MergeRec, a novel framework that applies task vector-based model merging to a new problem setting, termed data-isolated CDSR. MergeRec consists of three key components: (1) Merging initialization constructs an initial merged model using training-free task vectors based merging. (2) Pseudo-user data construction synthesizes virtual interaction sequences from domain items, allowing CF signals to be extracted without exposing sensitive user data. (3) Collaborative merging optimization jointly optimizes a recommendation loss and a knowledge distillation loss, facilitating the transfer of domain-specific CF patterns while preserving ranking effectiveness. Extensive experiments confirmed that MergeRec consistently outperforms existing model merging baselines and significantly improves generalization, including on unseen and data-scarce domains, highlighting the potential of model merging as a scalable, privacy-preserving solution for building universal recommender systems.

Ethical Use of Data

This paper utilizes publicly available datasets (Amazon product review datasets) that contain no personally identifiable information and require no Institutional Ethics Review Board approval. All experiments use anonymized interaction sequences released for academic research purposes.

Dataset Statistics

Table 3 summarizes the statistics of each domain, including the number of users, items, interactions, and dataset density. The density is calculated as $`\frac{\#~\text{Interactions}}{\#~\text{Users} \times \#~\text{Items}}`$.

Implementation Details

All methods, including MergeRec and baselines, are implemented in PyTorch. For RecFormer-base , we use the official pre-trained checkpoint³, while for RecFormer-large, we pre-train the model following the protocol described in the original paper. For BLaIR-base⁴ and BLaIR-large⁵ , we use the official pre-trained checkpoints available on HuggingFace. Fine-tuning is performed with in-batch negative sampling and a batch size of 64. For merging baselines, we adopt their hyperparameter configurations for the validation-less setting, i.e., no training, validation, or test data are used. Specifically, we set $`w_1 = w_2 = \dots = w_K = 0.4`$ for Task Arithmetic , and $`w=1`$ for TIES . We use the top 20% of the parameters for TIES. For AdaMerging  and MergeRec, all coefficients are initialized to $`0.2`$ and optimized for 500 steps using the Adam optimizer with a learning rate of 0.001 and a batch size of 16. For MergeRec, we set $`\lambda = 1{,}000`$ to balance the scale of the two loss functions. All reported performance metrics represent averages computed across five random seeds. For the significance test, we assume that deterministic merging approaches (i.e., Weight Averaging, Task Arithmetic, and TIES) have identical performance values across all five runs.

Overall Performance on Other Metrics

Table [tab:overall_ndcg] shows the normalized N@10 of MergeRec and seven baseline methods across eight datasets on four backbone models, with the performance of the fine-tuned model trained on each dataset set to 100%. In addition, we report the unnormalized R@10 and N@10 results in Table [tab:overall_recall_abs] and Table [tab:overall_ndcg_abs], respectively. We observe similar trends for N@10 (Table [tab:overall_ndcg]) as for R@10 (Table [tab:overall]). (i) MergeRec achieves the best performance on average across all datasets and backbone models. This shows that MergeRec effectively transfers domain-specific knowledge and improves model discrimination, whereas AdaMerging is less effective across the board. (ii) Merging methods like MergeRec, Weight Averaging, and TIES consistently outperform joint learning, demonstrating that parameter merging can efficiently transfer knowledge across domains and reduce computational cost without cross-domain training data. (iii) Cross-domain merging is especially helpful for domains with limited data, such as Scientific, where MergeRec shows notable improvements over fine-tuned models due to more effective knowledge transfer.

Domain-Specific Weight Dynamics on Other Backbones

We further analyze the evolution of domain-specific weights $`\mathbf{w}`$ on the remaining three backbone models: RecFormer-large and BLaIR-base/large. Consistent with our observations for RecFormer-base, we find that domains with larger data scales (e.g., Arts and Office) tend to converge to higher weight coefficients. This trend suggests that these models also prioritize capturing more complex collaborative patterns in large-scale domains, thereby allocating more representational capacity to preserve domain-specific knowledge. In contrast to this general trend, the merging weight for the Pantry dataset in the BLaIR model converges to negative values. This behavior may be attributed to the absence of category information in the Pantry dataset, which differs from other domains and can lead to misalignment during model merging.