Collaborative Deep Learning for Recommender Systems

Collaborative filtering (CF) is a successful approach commonly used by many recommender systems. Conventional CF-based methods use the ratings given to items by users as the sole source of information for learning to make recommendation. However, the ratings are often very sparse in many applications, causing CF-based methods to degrade significantly in their recommendation performance. To address this sparsity problem, auxiliary information such as item content information may be utilized. Collaborative topic regression (CTR) is an appealing recent method taking this approach which tightly couples the two components that learn from two different sources of information. Nevertheless, the latent representation learned by CTR may not be very effective when the auxiliary information is very sparse. To address this problem, we generalize recent advances in deep learning from i.i.d. input to non-i.i.d. (CF-based) input and propose in this paper a hierarchical Bayesian model called collaborative deep learning (CDL), which jointly performs deep representation learning for the content information and collaborative filtering for the ratings (feedback) matrix. Extensive experiments on three real-world datasets from different domains show that CDL can significantly advance the state of the art.

💡 Research Summary

The paper introduces Collaborative Deep Learning (CDL), a hierarchical Bayesian framework that jointly learns deep item representations from textual content and collaborative filtering (CF) from user‑item rating data. Traditional CF methods suffer when rating matrices are sparse, and hybrid approaches such as Collaborative Topic Regression (CTR) attempt to alleviate this by coupling a probabilistic topic model (LDA) with probabilistic matrix factorization (PMF). However, CTR’s latent topics can be weak when the auxiliary content is itself sparse or low‑dimensional.

CDL addresses this limitation by replacing the shallow topic model with a Stacked Denoising Autoencoder (SDAE) formulated in a Bayesian manner. Each item’s bag‑of‑words vector is corrupted to produce a noisy input; the SDAE learns to reconstruct the clean vector while compressing the data through multiple nonlinear layers. The middle layer (the encoder output) yields a dense latent representation for each item. This representation is then combined with a Gaussian “offset” vector to form the final item latent vector v_j = γ_j + encoder_output_j. User latent vectors u_i are drawn from zero‑mean Gaussians as in standard PMF. Ratings are modeled as Gaussian observations R_ij ~ N(u_i^T v_j, C_ij^{-1}), where C_ij encodes confidence (high for observed interactions, low otherwise).

Training proceeds by maximizing the joint log‑posterior of all variables. An EM‑style algorithm is employed: the E‑step updates user and item vectors via coordinate ascent, while the M‑step updates SDAE weights and biases using back‑propagation on a loss that combines reconstruction error and rating prediction error. Hyper‑parameters λ_w, λ_s, λ_u, λ_v, and λ_n control regularization of weights, noise variance, and the relative strength of content versus collaborative signals. In the limit λ_s → ∞ the SDAE becomes a deterministic encoder‑decoder, and the model reduces to two neural networks sharing the same corrupted input layer—one for reconstruction, one for rating prediction. By varying the ratio λ_n/λ_v the model can interpolate between a pure CTR (content fixed) and a pure CF (content ignored) regime; experiments show that intermediate values give the best performance.

The authors evaluate CDL on three real‑world datasets: CiteULike‑a (academic article bookmarks), Netflix (movie ratings), and a news‑article collection. Metrics include Recall@M and NDCG@M. Compared baselines are standard CF, CTR, and several recent deep‑learning‑based hybrids. CDL consistently outperforms all baselines, especially in extreme sparsity settings where only 1 % of possible ratings are observed, achieving roughly 5–10 % relative improvement over CTR. Moreover, the learned encoder weights are transferable to other text‑related tasks (e.g., classification, clustering), demonstrating the model’s broader utility.

Key contributions are: (1) a principled Bayesian integration of deep representation learning and collaborative filtering; (2) a novel use of CF as a target for deep networks rather than simple classification or reconstruction; (3) an EM‑style MAP estimation that can be interpreted as a Bayesian generalization of back‑propagation; (4) the first hierarchical Bayesian model that bridges state‑of‑the‑art deep learning architectures (SDAE, DBM, RNN, CNN) with recommender systems; and (5) extensive empirical evidence that the tightly coupled approach yields significant gains over both loosely coupled hybrids and pure CF or content‑only methods. The paper concludes that CDL provides a flexible, extensible foundation for future recommender systems that can incorporate additional side information (e.g., user profiles, temporal dynamics) while maintaining robust performance under data sparsity.