Empowering Contrastive Federated Sequential Recommendation with LLMs

Federated sequential recommendation (FedSeqRec) aims to perform next-item prediction while keeping user data decentralised, yet model quality is frequently constrained by fragmented, noisy, and homogeneous interaction logs stored on individual devices. Many existing approaches attempt to compensate through manual data augmentation or additional server-side constraints, but these strategies either introduce limited semantic diversity or increase system overhead. To overcome these challenges, we propose \textbf{LUMOS}, a parameter-isolated FedSeqRec architecture that integrates large language models (LLMs) as \emph{local semantic generators}. Instead of sharing gradients or auxiliary parameters, LUMOS privately invokes an on-device LLM to construct three complementary sequence variants from each user history: (i) \emph{future-oriented} trajectories that infer plausible behavioural continuations, (ii) \emph{semantically equivalent rephrasings} that retain user intent while diversifying interaction patterns, and (iii) \emph{preference-inconsistent counterfactuals} that serve as informative negatives. These synthesized sequences are jointly encoded within the federated backbone through a tri-view contrastive optimisation scheme, enabling richer representation learning without exposing sensitive information. Experimental results across three public benchmarks show that LUMOS achieves consistent gains over competitive centralised and federated baselines on HR@20 and NDCG@20. In addition, the use of semantically grounded positive signals and counterfactual negatives improves robustness under noisy and adversarial environments, even without dedicated server-side protection modules. Overall, this work demonstrates the potential of LLM-driven semantic generation as a new paradigm for advancing privacy-preserving federated recommendation.

💡 Research Summary

The paper introduces LUMOS, a novel framework for federated sequential recommendation (FedSeqRec) that leverages on‑device large language models (LLMs) to generate semantic augmentations of user interaction histories. Traditional FedSeqRec suffers from severe data scarcity, noise, and homogeneity because each client only holds a short, sparse sequence of item IDs. Existing remedies such as manual data augmentation or server‑side constraints either add little semantic diversity or increase communication and privacy overhead. LUMOS tackles these problems without altering the standard federated learning protocol.

Core Idea
Each client is assumed to have access to a pre‑trained LLM (e.g., GPT‑Neo, LLaMA) locally or via a privacy‑preserving API. Conditioned on the user’s original interaction sequence (S_u), the LLM is prompted to produce three complementary synthetic sequences:

1. Future‑oriented view ((S^F_u)) – a short continuation that predicts plausible next items, effectively extending the user’s trajectory.
2. Paraphrased intent‑preserving view ((S^P_u)) – a re‑expression of the same preferences using synonyms, slight re‑ordering, or alternative but semantically equivalent items.
3. Preference‑inconsistent counterfactual view ((S^N_u)) – a deliberately mismatched sequence (e.g., items from opposite categories) that serves as a hard negative.

All four sequences (original, future, paraphrase, counterfactual) are encoded by the same sequential backbone (GRU4Rec, SASRec, etc.) to obtain representations (h_u, h^F_u, h^P_u, h^N_u).

Tri‑view Contrastive Learning
LUMOS defines a contrastive objective that treats the original representation as an anchor, the future and paraphrase embeddings as positive views, and the counterfactual embedding as the negative view. Using an InfoNCE‑style loss, each client minimizes

\