Accelerated Canonical Polyadic Decomposition by Using Mode Reduction

Canonical Polyadic (or CANDECOMP/PARAFAC, CP) decompositions (CPD) are widely applied to analyze high order tensors. Existing CPD methods use alternating least square (ALS) iterations and hence need to unfold tensors to each of the $N$ modes frequently, which is one major bottleneck of efficiency for large-scale data and especially when $N$ is large. To overcome this problem, in this paper we proposed a new CPD method which converts the original $N$th ($N>3$) order tensor to a 3rd-order tensor first. Then the full CPD is realized by decomposing this mode reduced tensor followed by a Khatri-Rao product projection procedure. This way is quite efficient as unfolding to each of the $N$ modes are avoided, and dimensionality reduction can also be easily incorporated to further improve the efficiency. We show that, under mild conditions, any $N$th-order CPD can be converted into a 3rd-order case but without destroying the essential uniqueness, and theoretically gives the same results as direct $N$-way CPD methods. Simulations show that, compared with state-of-the-art CPD methods, the proposed method is more efficient and escape from local solutions more easily.

💡 Research Summary

The paper addresses a fundamental bottleneck in canonical polyadic (CP) decomposition of high‑order tensors: the repeated unfolding of an N‑way tensor (N > 3) required by alternating‑least‑squares (ALS) algorithms. Unfolding each mode incurs heavy memory traffic and computational cost, especially when N is large or the tensor dimensions are massive. To eliminate this overhead, the authors propose a “mode‑reduction” strategy that first transforms the original N‑dimensional tensor into a third‑order tensor, performs CP decomposition on this reduced representation, and finally restores the factor matrices of the original tensor via a Khatri‑Rao product projection.

Core methodology

Mode grouping and reduction – The N modes are partitioned into two disjoint groups. Within each group the factor matrices are combined using the Khatri‑Rao product, yielding two composite modes. Together with the remaining single mode, a 3‑way tensor (\mathcal{Y}) is formed:
\