Infinite Mixed Membership Matrix Factorization

Rating and recommendation systems have become a popular application area for applying a suite of machine learning techniques. Current approaches rely primarily on probabilistic interpretations and extensions of matrix factorization, which factorizes a user-item ratings matrix into latent user and item vectors. Most of these methods fail to model significant variations in item ratings from otherwise similar users, a phenomenon known as the “Napoleon Dynamite” effect. Recent efforts have addressed this problem by adding a contextual bias term to the rating, which captures the mood under which a user rates an item or the context in which an item is rated by a user. In this work, we extend this model in a nonparametric sense by learning the optimal number of moods or contexts from the data, and derive Gibbs sampling inference procedures for our model. We evaluate our approach on the MovieLens 1M dataset, and show significant improvements over the optimal parametric baseline, more than twice the improvements previously encountered for this task. We also extract and evaluate a DBLP dataset, wherein we predict the number of papers co-authored by two authors, and present improvements over the parametric baseline on this alternative domain as well.

💡 Research Summary

The paper introduces the infinite Mixed Membership Matrix Factorization (iM³F) model, an extension of the previously proposed Mixed Membership Matrix Factorization (M³F) framework for recommender systems. While M³F improves upon standard probabilistic matrix factorization by adding a contextual bias term that captures “mood” or “context” through a fixed number of user and item topics, it requires the number of topics to be set a priori. iM³F removes this limitation by placing a non‑parametric Bayesian prior over the topic distributions, allowing the data to dictate how many user and item topics are needed.

The authors employ a hierarchical Dirichlet process (HDP) to model an infinite set of shared topics. At the top level, a global Dirichlet process draws a set of “dishes” (topics) that are common to all users and items. At the lower level, each user and each item draws its own Dirichlet process (with the global DP as base measure) that defines a distribution over these shared dishes. This construction yields a Chinese Restaurant Franchise (CRF) representation: rating residuals are customers, tables correspond to user‑specific topic proportions, and dishes correspond to the actual topic parameters (biases).

The generative process proceeds as follows. Latent user and item factors (a_u, b_j) are drawn from Gaussian priors with Wishart‑distributed precision matrices. For each rating r_{uj}, a user‑topic assignment z_{U,uj} and an item‑topic assignment z_{M,uj} are sampled from the CRF. Conditional on these assignments, user‑topic bias c_{k u} and item‑topic bias d_{i j} are drawn from zero‑mean Gaussians. The observed rating is then generated as a Gaussian whose mean equals a global offset χ₀ plus the two bias terms plus the inner product a_u·b_j.

Inference is performed via Gibbs sampling. The authors derive closed‑form conditional posteriors for all continuous variables using conjugacy (Gaussian‑Wishart). The key non‑conjugate part—sampling the topic assignments—leverages the CRF representation: first sample table assignments for each residual, then sample dish assignments for each table, integrating out the random measures. The algorithm naturally creates new tables (and thus new topics) when the data warrants, and removes unused tables, achieving an automatic model order selection.

Empirical evaluation is conducted on two heterogeneous datasets. On MovieLens 1M, iM³F reduces root‑mean‑square error (RMSE) by 0.0065 relative to the parametric M³F baseline, which corresponds to roughly twice the improvement previously reported for M³F over its own baseline. A second experiment uses a DBLP‑derived co‑authorship count prediction task; iM³F again outperforms M³F, confirming that the approach generalizes beyond movie rating data. The authors attribute these gains to the model’s ability to capture the “Napoleon Dynamite” effect—high variance in ratings for specific items among otherwise similar users—through flexible, data‑driven contextual topics.

Beyond performance, the paper highlights that the non‑parametric treatment of topics is orthogonal to existing work on learning the latent dimensionality D of the factor vectors. Consequently, iM³F can be combined with non‑parametric approaches for D, yielding a fully flexible Bayesian recommender. The authors suggest future directions such as integrating online variational inference for streaming environments and extending the HDP to jointly infer both the number of latent factors and the number of topics.

In summary, iM³F provides a principled Bayesian framework that automatically discovers the appropriate number of contextual topics, integrates them with latent factor models, and delivers measurable improvements on real‑world recommendation and relational prediction tasks.