Infinite Tucker Decomposition: Nonparametric Bayesian Models for Multiway Data Analysis

Tensor decomposition is a powerful computational tool for multiway data analysis. Many popular tensor decomposition approaches—such as the Tucker decomposition and CANDECOMP/PARAFAC (CP)—amount to multi-linear factorization. They are insufficient to model (i) complex interactions between data entities, (ii) various data types (e.g. missing data and binary data), and (iii) noisy observations and outliers. To address these issues, we propose tensor-variate latent nonparametric Bayesian models, coupled with efficient inference methods, for multiway data analysis. We name these models InfTucker. Using these InfTucker, we conduct Tucker decomposition in an infinite feature space. Unlike classical tensor decomposition models, our new approaches handle both continuous and binary data in a probabilistic framework. Unlike previous Bayesian models on matrices and tensors, our models are based on latent Gaussian or $t$ processes with nonlinear covariance functions. To efficiently learn the InfTucker from data, we develop a variational inference technique on tensors. Compared with classical implementation, the new technique reduces both time and space complexities by several orders of magnitude. Our experimental results on chemometrics and social network datasets demonstrate that our new models achieved significantly higher prediction accuracy than the most state-of-art tensor decomposition

💡 Research Summary

The paper introduces InfTucker, a non‑parametric Bayesian framework that extends the classical Tucker decomposition of tensors into an infinite‑dimensional feature space. By placing a latent Gaussian process (GP) or a Student‑t process (t‑process) prior on an infinite core tensor and mapping each factor matrix through a countably infinite feature map, the model captures highly nonlinear interactions among the modes while retaining the multilinear structure of Tucker.

Two observation models are supported: a Gaussian likelihood for continuous entries and a probit likelihood for binary entries. For the probit case, the authors adopt the Albert‑Chib data‑augmentation scheme, introducing latent Gaussian variables Z; for the t‑process, a Gamma‑distributed scale variable η is mixed with a Gaussian to obtain heavy‑tailed robustness. Missing values are naturally handled by marginalizing over unobserved entries.

Learning proceeds via a variational Expectation‑Maximization (EM) algorithm. In the E‑step, the posterior over the latent variables (Z, the latent tensor M, and the scale η) is approximated by a fully factorized distribution q. The variational distribution of M is a multivariate normal with mean μ and covariance Υ, where Υ = (E