On the rank-one approximation of symmetric tensors

The problem of symmetric rank-one approximation of symmetric tensors is important in Independent Components Analysis, also known as Blind Source Separation, as well as polynomial optimization. We analyze the symmetric rank-one approximation problem for symmetric tensors and derive several perturbation results. Given a symmetric rank-one tensor obscured by noise, we provide bounds on the accuracy of the best symmetric rank-one approximation for recovering the original rank-one structure, and we show that any eigenvector with sufficiently large eigenvalue is related to the rank-one structure as well. Further, we show that for high-dimensional symmetric approximately-rank-one tensors, the generalized Rayleigh quotient is mostly close to zero, so the best symmetric rank-one approximation corresponds to a prominent global extreme value. We show that each iteration of the Shifted Symmetric Higher Order Power Method (SS-HOPM), when applied to a rank-one symmetric tensor, moves towards the principal eigenvector for any input and shift parameter, under mild conditions. Finally, we explore the best choice of shift parameter for SS-HOPM to recover the principal eigenvector. We show that SS-HOPM is guaranteed to converge to an eigenvector of an approximately rank-one even-mode tensor for a wider choice of shift parameter than it is for a general symmetric tensor. We also show that the principal eigenvector is a stable fixed point of the SS-HOPM iteration for a wide range of shift parameters; together with a numerical experiment, these results lead to a non-obvious recommendation for shift parameter for the symmetric rank-one approximation problem.

💡 Research Summary

The paper investigates the symmetric rank‑one approximation problem for symmetric tensors, a task that underlies Independent Component Analysis (ICA), Blind Source Separation (BSS), and homogeneous polynomial optimization. The authors model a noisy rank‑one tensor as
 A = λ a⊗m + E,
where a is a unit vector, λ is a scalar, and E is a symmetric noise tensor.

First, they derive perturbation bounds for the principal eigenpair of A. Using the quantity β(E) = (m − 1)·max_{‖x‖=1}ρ(E x^{m‑2}), they prove that the principal eigenvalue λₚ satisfies
 |λ| − β(E)^{m‑1} ≤ |λₚ| ≤ |λ| + β(E)^{m‑1},
and that the angle θ between a and the corresponding eigenvector xₚ obeys
 |cos mθ| ≥ 1 − 2β(E)/(|λ|(m − 1)).
Thus, when the noise is small (β(E) → 0), the recovered eigenpair is close to the original rank‑one structure.

A second result shows that any unit vector x for which the generalized Rayleigh quotient |A x^{m}| exceeds ε^{m}+β(E)^{m‑1} must satisfy |aᵀx| ≥ ε. Consequently, large Rayleigh values reveal alignment with the true rank‑one direction.

The third theorem addresses high‑dimensional tensors. For a random unit vector x, the probability that |aᵀx| exceeds ε is bounded by 1/(nε²). Hence, in large dimensions the Rayleigh quotient is typically near zero, making the principal eigenpair a prominent global extremum.

The paper then focuses on the Shifted Symmetric Higher‑Order Power Method (SS‑HOPM), whose iteration is
 x_{k+1} = (A x_k^{m‑1}+αx_k)/‖A x_k^{m‑1}+αx_k‖,
with shift parameter α. Existing theory requires α > β̂(A) (a crude bound) for convergence. The authors improve this by analyzing stability of negative‑stable eigenpairs (those corresponding to local maxima of the Rayleigh quotient). They prove that a negative‑stable eigenpair (xₚ,λₚ) is a stable fixed point of SS‑HOPM provided
 −λₚ + (m − 1)λ·|sin θ·cos^{m‑2}θ| + β(E)² < α.
When the noise is small and θ is small, this reduces to the simple condition −λ/2 < α, which is far less restrictive than α > β̂(A). For general eigenpairs with larger θ, the bound becomes α > λ(m/2 − 1). Thus, a much wider interval of shift values guarantees stability of the principal eigenvector while potentially destabilizing spurious eigenvectors.

The authors also examine the unshifted case (α = 0) on a pure rank‑one tensor (E = 0). They show that any initial vector not orthogonal to a converges to a in a single iteration; if orthogonal, the method still moves toward a under mild conditions (γ = aᵀx₁, γ^{m‑2}>0). This demonstrates global convergence of SS‑HOPM for exact rank‑one tensors.

Numerical experiments validate the theory. Across various dimensions, orders, and noise levels, choosing α within the interval (−λ/2, λ(m/2 − 1)) yields higher recovery rates and faster convergence than the conservative α > β̂(A) rule. The experiments also illustrate that the principal eigenpair indeed stands out as a global extremum in high dimensions, confirming the probabilistic analysis.

In summary, the paper provides (1) tight perturbation bounds for noisy symmetric rank‑one tensors, (2) a statistical characterization of the Rayleigh quotient in high dimensions, and (3) an expanded, practically useful range for the SS‑HOPM shift parameter that ensures convergence to the desired principal eigenvector. These contributions advance both the theoretical understanding and practical algorithms for tensor decomposition, ICA, BSS, and related polynomial optimization problems.