Improved Analysis of the Accelerated Noisy Power Method with Applications to Decentralized PCA

We analyze the Accelerated Noisy Power Method, an algorithm for Principal Component Analysis in the setting where only inexact matrix-vector products are available, which can arise for instance in decentralized PCA. While previous works have established that acceleration can improve convergence rates compared to the standard Noisy Power Method, these guarantees require overly restrictive upper bounds on the magnitude of the perturbations, limiting their practical applicability. We provide an improved analysis of this algorithm, which preserves the accelerated convergence rate under much milder conditions on the perturbations. We show that our new analysis is worst-case optimal, in the sense that the convergence rate cannot be improved, and that the noise conditions we derive cannot be relaxed without sacrificing convergence guarantees. We demonstrate the practical relevance of our results by deriving an accelerated algorithm for decentralized PCA, which has similar communication costs to non-accelerated methods. To our knowledge, this is the first decentralized algorithm for PCA with provably accelerated convergence.

💡 Research Summary

This paper presents a refined theoretical analysis of the Accelerated Noisy Power Method (ANPM), an algorithm designed for principal component analysis (PCA) when only inexact matrix‑vector products are available. The authors address a key limitation of prior work: existing convergence guarantees for ANPM require extremely restrictive bounds on the magnitude of the perturbation (noise) added to each matrix‑vector multiplication. In particular, the analysis by Xu (2023) demands that the noise norm be bounded by a factor proportional to ε · μ, where μ grows as the inverse square‑root of the eigengap Δ_k = (λ_k − λ_{k+1})/λ_k. For ill‑conditioned problems with small Δ_k, this condition becomes impractically stringent.

The authors propose a new set of noise conditions that are essentially identical to those required for the non‑accelerated Noisy Power Method (NPM) introduced by Hardt and Price (2014). Specifically, for each iteration t they require
‖U_{⊥k}^T Ξ_t‖_2 ≤ c (λ_k − 2√β) ε and
‖U_k^T Ξ_t‖_2 ≤ c (λ_k − 2√β) cos θ_k(U_k, X_t),
with a modest constant c = 1/32. The first bound controls the component of the noise that lies in the subspace spanned by the eigenvectors below the k‑th, preventing the iterates from drifting too far from the target subspace. The second bound limits the noise component within the top‑k subspace, ensuring that the geometric decay term in the error recursion is not overwhelmed by noise. Importantly, these conditions scale linearly with the eigengap, rather than exponentially as in prior accelerated analyses.

The algorithm itself proceeds as follows: starting from an orthonormal matrix X_0 ∈ St(d,k), each iteration computes a “momentum‑adjusted” matrix‑vector product Y_{t+1} = A X_t − β X_{t‑1} R_{t‑1}^{‑1} + Ξ_t, followed by a QR factorization Y_{t+1}=X_{t+1} R_{t+1}. The momentum parameter β is chosen so that λ_k > 2√β ≥ λ_{k+1}. Under this choice the unnormalized iterates Z_t = X_t R_t…R_1 can be expressed as Z_t = p_t(A) X_0, where p_t is a degree‑t Chebyshev polynomial of the first kind scaled to have leading coefficient 1/2. This polynomial minimizes the maximum absolute value on the interval