High-Dimensional Partial Least Squares: Spectral Analysis and Fundamental Limitations

Partial Least Squares (PLS) is a widely used method for data integration, designed to extract latent components shared across paired high-dimensional datasets. Despite decades of practical success, a precise theoretical understanding of its behavior in high-dimensional regimes remains limited. In this paper, we study a data integration model in which two high-dimensional data matrices share a low-rank common latent structure while also containing individual-specific components. We analyze the singular vectors of the associated cross-covariance matrix using tools from random matrix theory and derive asymptotic characterizations of the alignment between estimated and true latent directions. These results provide a quantitative explanation of the reconstruction performance of the PLS variant based on Singular Value Decomposition (PLS-SVD) and identify regimes where the method exhibits counter-intuitive or limiting behavior. Building on this analysis, we compare PLS-SVD with principal component analysis applied separately to each dataset and show its asymptotic superiority in detecting the common latent subspace. Overall, our results offer a comprehensive theoretical understanding of high-dimensional PLS-SVD, clarifying both its advantages and fundamental limitations.

💡 Research Summary

This paper provides a rigorous random‑matrix‑theoretic analysis of Partial Least Squares (PLS) in the high‑dimensional regime, focusing on the PLS‑SVD variant that extracts latent components by a single singular value decomposition of the normalized cross‑covariance matrix (S_{XY}= \frac{1}{\sqrt{pq}}X^{\top}Y). The authors model the two data matrices (X\in\mathbb{R}^{n\times p}) and (Y\in\mathbb{R}^{n\times q}) as a sum of three parts: a common low‑rank signal (TP^{\top}) and (TR^{\top}) (shared scores (T) and loadings (P,R)), individual low‑rank structures (M) and (N), and independent Gaussian noise (E) and (F). They assume a high‑dimensional asymptotic where (n,p,q\to\infty) with ratios (n/p\to\beta_{p}>0) and (n/q\to\beta_{q}>0).

The analysis proceeds by studying the symmetric matrices (K=\frac{1}{pq}Y^{\top}XX^{\top}Y) and (\tilde K=\frac{1}{pq}X^{\top}YY^{\top}X), whose non‑zero eigenvalues equal the squared singular values of (S_{XY}). The authors first establish deterministic equivalents for the resolvents of these matrices (Theorem 1), which serve as the foundation for all subsequent results. Using these equivalents they derive the limiting spectral distribution of the singular values (Proposition 2), a generalization of the Marčenko–Pastur law that holds even when signal components are present.

A central contribution is the identification of phase‑transition thresholds for spike detection. For each type of signal—common ((PR^{\top})) or individual ((M,N))—they define a critical signal‑to‑noise ratio (\tau_c) (equation 14). When the actual ratio (\tau) exceeds (\tau_c), the corresponding singular value separates from the bulk and becomes an outlier (Propositions 3 and 5). Closed‑form expressions for the asymptotic locations of these outliers are provided as functions of the underlying true singular values and the aspect ratios (\beta_{p},\beta_{q}).

Beyond eigenvalue behavior, the paper rigorously quantifies eigenvector alignment. Proposition 4 shows that when individual structures dominate, the leading PLS singular vectors align with these spurious directions rather than with the true shared signal, revealing a fundamental limitation. Proposition 6 demonstrates a systematic “skewing” of the estimated common components: even when a common spike is detected, the recovered singular vectors are biased versions of the true loadings, and this bias persists unless the signal strength tends to infinity.

The authors also compare PLS‑SVD with applying principal component analysis (PCA) separately to (X) and (Y). Proposition 10 proves that, under the same asymptotic regime, PLS‑SVD always detects at least as many common spikes as separate PCA and does so with a strictly larger spectral gap, establishing a clear theoretical advantage of PLS for multimodal integration.

Finally, the paper situates its signal‑plus‑noise model within the broader context of data‑integration frameworks such as JIVE, and discusses how the results extend to other PLS variants that share the same initial SVD step. The authors highlight two fundamental drawbacks of high‑dimensional PLS—spurious alignment with individual components and persistent skewing of common components—and suggest that future work should develop regularization or filtering strategies to mitigate these effects.

In summary, this work delivers the first comprehensive high‑dimensional theory for PLS‑SVD, delivering deterministic equivalents, bulk‑spike separation thresholds, precise eigenvector consistency formulas, and a formal superiority proof over separate PCA. These contributions deepen our understanding of why PLS works well in practice, delineate its limits, and provide a solid foundation for designing improved multimodal integration methods.