Collaborative User Prompt for Personalized Generative Recommendation

Large Language Models (LLMs) have become powerful foundations for generative recommender systems, framing recommendation tasks as text generation tasks. However, existing generative recommendation methods often rely on discrete ID-based prompts or task-specific soft prompts, which overlook the valuable collaborative signals shared among users with similar interests. To address this limitation, this paper presents a compositional framework that integrates a user’s individual preferences with collective preferences from similar users to build personalized soft prompts. Specifically, an attention-based mechanism fuses embeddings from users with similar interests, creating a richer representation that captures multiple facets of user preferences. This design dynamically emphasizes shared interests while preserving individual user preferences. Experiments on three real-world datasets demonstrate the effectiveness of the proposed approach across sequential recommendation, top-n recommendation, and explanation generation tasks, underscoring the advantages of incorporating collaborative signals through an attention-based compositional strategy.

💡 Research Summary

The paper addresses a critical limitation of current large‑language‑model (LLM) based generative recommender systems: most existing approaches either use a single global continuous prompt for all users or construct a personal prompt solely from a user’s own interaction history. Both strategies ignore the collaborative signals that arise from users with similar tastes—a cornerstone of traditional recommender algorithms such as matrix factorization and neighborhood models. To bridge this gap, the authors propose a novel framework called Personalized Prompt Distillation (PeaPOD) that fuses individual preferences with group‑level information into a unified soft prompt that can be fed to an LLM for any downstream recommendation task.

Core Methodology

1. User Embedding Initialization – The authors first train probabilistic matrix factorization (PMF) on the user‑item feedback matrix to obtain dense user embeddings (dimension d_u). Because PMF captures collaborative similarity, these embeddings already encode group‑level signals.
2. Attention‑Based Prompt Construction – For a target user u, the model retrieves the top‑n most similar users based on cosine similarity in the PMF space. The target user embedding is projected to a query vector Q, while the neighbor embeddings are projected to key (K) and value (V) vectors via learned linear maps (W_Q, W_K, W_V). Multi‑head scaled dot‑product attention then computes
   Attn(Q,K,V) = softmax(QKᵀ / √d_m) V,
   producing an attention‑weighted summary z that dynamically emphasizes the most relevant collaborative information. A final linear layer maps z to a prompt vector p of length d_p. Multi‑head attention allows each head to focus on different aspects of similarity (e.g., genre, temporal patterns).
3. Integration with Task‑Specific Prompts – The collaborative prompt p is concatenated with existing task‑specific continuous prompts and any discrete prompt tokens (augmented with whole‑word embeddings for IDs). This composite input is fed to a seq2seq LLM (e.g., T5‑based) that has been fine‑tuned on three recommendation tasks.
4. Training Regime – The authors adopt a task‑alternated training schedule: each mini‑batch contains data from only one of the three tasks (sequential recommendation, top‑N recommendation, explanation generation). This preserves input length uniformity and speeds up training without sacrificing performance. The loss is standard negative log‑likelihood; inference uses beam search to generate item IDs or natural‑language explanations.

Experimental Evaluation
Experiments are conducted on three Amazon datasets (Sports, Beauty, Toys) covering a range of sparsity levels and user/item counts. The authors compare PeaPOD against strong baselines including P5, VIP5, POD, and recent user‑prompt methods that lack collaborative augmentation. Evaluation metrics include HR@10, NDCG@10 for top‑N recommendation, MRR for sequential recommendation, and BLEU/ROUGE for explanation generation. Across all datasets and tasks, PeaPOD consistently outperforms baselines, achieving statistically significant gains (often 2–5% absolute improvement). Ablation studies show that removing the attention‑based collaborative component or reducing the number of neighbor users degrades performance, confirming the importance of both individual and group signals.

Key Contributions

• Introduction of a compositional prompt distillation pipeline that converts collaborative‑filtering embeddings into soft prompts for LLMs.
• Exploitation of multi‑head attention to dynamically weight neighbor user information conditioned on the target user, enabling richer, context‑aware personalization.
• Demonstration of a single unified prompt that can be reused across heterogeneous recommendation tasks, simplifying system design while delivering state‑of‑the‑art results.
• Release of code and processed data to facilitate reproducibility and future research.

Implications and Future Directions
PeaPOD illustrates that collaborative knowledge can be seamlessly injected into LLM‑based recommenders without altering the underlying language model architecture. This opens avenues for extending the approach with richer side information (e.g., item metadata, temporal context), adaptive neighbor selection in real‑time, and scaling to billions of users via approximate nearest‑neighbor search. Moreover, the attention‑based prompt composition could be generalized to other domains where personalization benefits from both individual and community signals, such as personalized dialogue systems or adaptive tutoring.

In summary, the paper presents a well‑motivated, technically sound, and empirically validated solution to the personalization challenge in generative recommendation, establishing a new baseline for future work that seeks to blend collaborative filtering insights with the expressive power of large language models.