Guaranteed Nonconvex Low-Rank Tensor Estimation via Scaled Gradient Descent

Tensors, which give a faithful and effective representation to deliver the intrinsic structure of multi-dimensional data, play a crucial role in an increasing number of signal processing and machine learning problems. However, tensor data are often accompanied by arbitrary signal corruptions, including missing entries and sparse noise. A fundamental challenge is to reliably extract the meaningful information from corrupted tensor data in a statistically and computationally efficient manner. This paper develops a scaled gradient descent (ScaledGD) algorithm to directly estimate the tensor factors with tailored spectral initializations under the tensor-tensor product (t-product) and tensor singular value decomposition (t-SVD) framework. With tailored variants for tensor robust principal component analysis, (robust) tensor completion and tensor regression, we theoretically show that ScaledGD achieves linear convergence at a constant rate that is independent of the condition number of the ground truth low-rank tensor, while maintaining the low per-iteration cost of gradient descent. To the best of our knowledge, ScaledGD is the first algorithm that provably has such properties for low-rank tensor estimation with the t-SVD. Finally, numerical examples are provided to demonstrate the efficacy of ScaledGD in accelerating the convergence rate of ill-conditioned low-rank tensor estimation in a number of applications.

💡 Research Summary

This paper tackles the fundamental problem of low‑rank tensor estimation under the modern t‑product and tensor singular value decomposition (t‑SVD) framework. The authors parameterize a target tensor X∈ℝ^{n₁×n₂×n₃} of tubal rank r as X = L ∗_Φ Rᴴ, where L∈ℝ^{n₁×r×n₃} and R∈ℝ^{n₂×r×n₃} are factor tensors and Φ is an invertible linear transform (typically the discrete Fourier transform). This factorization preserves the intrinsic low‑rank structure in the spectral domain and avoids the information loss associated with matricization‑based approaches.

The core algorithmic contribution is a Scaled Gradient Descent (ScaledGD) method. Standard gradient descent updates L and R by subtracting a step size η times the gradient of the least‑squares loss. ScaledGD pre‑conditions each update with the inverse of the small r×r×n₃ tensors (Rᴴ∗_Φ R)^{-1} and (Lᴴ∗_Φ L)^{-1}. Because these inverses are computed on the low‑dimensional factor space, the per‑iteration overhead is modest (O(r³ n₃) operations) while the search direction is dramatically improved. The scaling neutralizes the effect of the condition number κ of the ground‑truth tensor, allowing large step sizes and guaranteeing linear convergence that does not depend on κ.

Theoretical analysis proceeds in two stages. First, a tailored spectral initialization is shown to produce factors (L₀,R₀) that lie within a small Frobenius‑norm ball around the true factors, provided the sampling pattern satisfies an incoherence condition and the number of observed entries exceeds a near‑optimal threshold (roughly O(μ r log n) where μ is the incoherence parameter). Second, once the iterates enter a region where the objective is locally strongly convex and smooth, the scaled updates satisfy a contraction inequality
‖