Scalable Tensor Factorizations for Incomplete Data

The problem of incomplete data - i.e., data with missing or unknown values - in multi-way arrays is ubiquitous in biomedical signal processing, network traffic analysis, bibliometrics, social network analysis, chemometrics, computer vision, communication networks, etc. We consider the problem of how to factorize data sets with missing values with the goal of capturing the underlying latent structure of the data and possibly reconstructing missing values (i.e., tensor completion). We focus on one of the most well-known tensor factorizations that captures multi-linear structure, CANDECOMP/PARAFAC (CP). In the presence of missing data, CP can be formulated as a weighted least squares problem that models only the known entries. We develop an algorithm called CP-WOPT (CP Weighted OPTimization) that uses a first-order optimization approach to solve the weighted least squares problem. Based on extensive numerical experiments, our algorithm is shown to successfully factorize tensors with noise and up to 99% missing data. A unique aspect of our approach is that it scales to sparse large-scale data, e.g., 1000 x 1000 x 1000 with five million known entries (0.5% dense). We further demonstrate the usefulness of CP-WOPT on two real-world applications: a novel EEG (electroencephalogram) application where missing data is frequently encountered due to disconnections of electrodes and the problem of modeling computer network traffic where data may be absent due to the expense of the data collection process.

💡 Research Summary

The paper addresses the pervasive problem of missing entries in multi‑way (tensor) data and proposes a scalable algorithm, CP‑WOPT (CP Weighted OPTimization), for performing CANDECOMP/PARAFAC (CP) decomposition under such conditions. The authors begin by highlighting the importance of tensor analysis in fields such as biomedical signal processing, network traffic monitoring, bibliometrics, and computer vision, where data collection often yields incomplete observations. Traditional CP methods either ignore missing values or rely on Expectation‑Maximization (EM) based alternating least squares (ALS), both of which become computationally prohibitive for large, sparse tensors.

The core contribution is a reformulation of the CP decomposition as a weighted least‑squares problem that explicitly incorporates a binary mask W indicating observed entries. The objective function minimizes the Frobenius norm of the element‑wise product between the mask and the residual tensor, thereby restricting the loss to known data points only. To solve this problem efficiently, the authors adopt a first‑order optimization framework, specifically a limited‑memory BFGS (L‑BFGS) algorithm, which requires only gradient information. Crucially, the gradient can be computed solely over the observed entries, reducing the computational complexity to O(|Ω| R), where |Ω| is the number of known elements and R is the CP rank. The implementation leverages sparse data structures (e.g., COO format) and block‑wise matrix‑vector products to keep memory usage modest even for tensors of size 1000 × 1000 × 1000 with only 0.5 % density.

Extensive experiments validate the method’s robustness and scalability. Synthetic tests vary the missing‑data ratio from 10 % to 99 % and add Gaussian noise; CP‑WOPT consistently achieves lower reconstruction error and faster convergence than CP‑ALS, CP‑OPT, and nuclear‑norm based tensor completion. In a large‑scale scenario (1 billion entries, five million observed), CP‑WOPT completes factorization within tens of minutes using less than 8 GB of RAM, demonstrating true big‑data capability.

Two real‑world applications illustrate practical impact. First, electroencephalogram (EEG) recordings often suffer from electrode disconnections; CP‑WOPT successfully reconstructs the missing channels, preserving spectral features essential for neurological analysis. Second, in computer network traffic monitoring, collecting exhaustive flow statistics is expensive; the authors deliberately sample a small subset of router measurements and use CP‑WOPT to infer the full traffic tensor, achieving prediction accuracy above 95 % while drastically reducing measurement overhead.

The paper concludes that CP‑WOPT offers a memory‑efficient, high‑performance solution for CP decomposition with missing data, opening avenues for extensions to other tensor models (e.g., Tucker), dynamic streaming tensors, and more complex missing‑data patterns. Its ability to handle extremely sparse, massive tensors makes it a valuable tool for modern data‑intensive scientific and engineering domains.