Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determination

CANDECOMP/PARAFAC (CP) tensor factorization of incomplete data is a powerful technique for tensor completion through explicitly capturing the multilinear latent factors. The existing CP algorithms require the tensor rank to be manually specified, however, the determination of tensor rank remains a challenging problem especially for CP rank. In addition, existing approaches do not take into account uncertainty information of latent factors, as well as missing entries. To address these issues, we formulate CP factorization using a hierarchical probabilistic model and employ a fully Bayesian treatment by incorporating a sparsity-inducing prior over multiple latent factors and the appropriate hyperpriors over all hyperparameters, resulting in automatic rank determination. To learn the model, we develop an efficient deterministic Bayesian inference algorithm, which scales linearly with data size. Our method is characterized as a tuning parameter-free approach, which can effectively infer underlying multilinear factors with a low-rank constraint, while also providing predictive distributions over missing entries. Extensive simulations on synthetic data illustrate the intrinsic capability of our method to recover the ground-truth of CP rank and prevent the overfitting problem, even when a large amount of entries are missing. Moreover, the results from real-world applications, including image inpainting and facial image synthesis, demonstrate that our method outperforms state-of-the-art approaches for both tensor factorization and tensor completion in terms of predictive performance.

💡 Research Summary

The paper introduces a fully Bayesian formulation of CANDECOMP/PARAFAC (CP) tensor factorization designed to handle incomplete data while automatically determining the CP rank. Traditional CP methods require the user to pre‑specify the rank, a task that is notoriously difficult, especially in the presence of missing entries. Moreover, existing approaches typically provide only point estimates of the factor matrices and ignore uncertainty quantification.

To overcome these limitations, the authors construct a hierarchical probabilistic model. The observed tensor Y is modeled as a noisy version of a latent tensor X, where X follows an exact CP decomposition with R rank‑one components. Each factor matrix A⁽ⁿ⁾ (size Iₙ × R) is assigned a zero‑mean Gaussian prior with a shared precision matrix Λ = diag(λ₁,…,λ_R). The hyper‑parameters λ_r themselves follow Gamma hyper‑priors, which act as sparsity‑inducing Automatic Relevance Determination (ARD) priors. A Gamma hyper‑prior is also placed on the noise precision τ. This hierarchy makes every component of the model latent, allowing the inference process to prune irrelevant rank‑one terms automatically.

Exact Bayesian inference is intractable, so the authors adopt a Variational Bayesian (VB) framework with a mean‑field factorization q(Θ) = ∏ₙ q(A⁽ⁿ⁾) q(λ) q(τ). Closed‑form update equations are derived for each factor: the factor matrices have multivariate Gaussian posteriors whose means involve Khatri‑Rao products of the expectations of the other modes, while λ and τ retain Gamma posteriors. The updates only require summations over observed entries, leading to a computational complexity that scales linearly with the number of observed elements (O(|Ω|·R·N)). The algorithm starts with a generous upper bound for R; as the VB iterations proceed, many λ_r grow large, effectively driving the corresponding columns of all factor matrices to zero and thereby revealing the true CP rank.

Predictive distributions for missing entries are obtained by integrating the likelihood over the variational posterior, yielding both point predictions and associated uncertainties.

Extensive experiments validate the approach. On synthetic tensors with varying observation ratios, the method accurately recovers the ground‑truth rank and achieves lower reconstruction error than competing techniques such as CPWOPT, geomCG, HaLRTC, and STDC, even when the initial R is over‑estimated. Real‑world tests on image inpainting and facial image synthesis demonstrate superior PSNR/SSIM scores and visually more coherent restorations. Importantly, the Bayesian model avoids over‑fitting when many components are unnecessary, a problem that plagues deterministic CP algorithms.

The paper’s contributions are threefold: (1) a principled Bayesian CP model that jointly performs factorization and rank selection without manual tuning; (2) an efficient variational inference scheme with linear scalability; and (3) provision of predictive uncertainty for missing data. Limitations include reliance on mean‑field approximations (which may miss multimodal posteriors) and the need to set weak hyper‑priors (c₀, d₀, a₀, b₀) empirically. Future work suggested by the authors involves stochastic variational methods, non‑Gaussian noise models, and more sophisticated hyper‑parameter learning. Overall, the study offers a compelling Bayesian alternative for tensor completion and low‑rank tensor analysis.