A Gradient Descent Algorithm on the Grassman Manifold for Matrix Completion

We consider the problem of reconstructing a low-rank matrix from a small subset of its entries. In this paper, we describe the implementation of an efficient algorithm called OptSpace, based on singular value decomposition followed by local manifold optimization, for solving the low-rank matrix completion problem. It has been shown that if the number of revealed entries is large enough, the output of singular value decomposition gives a good estimate for the original matrix, so that local optimization reconstructs the correct matrix with high probability. We present numerical results which show that this algorithm can reconstruct the low rank matrix exactly from a very small subset of its entries. We further study the robustness of the algorithm with respect to noise, and its performance on actual collaborative filtering datasets.

💡 Research Summary

The paper addresses the low‑rank matrix completion problem, where only a small subset of entries of an unknown matrix M is observed and the goal is to recover the full matrix under the assumption that its rank r is much smaller than its dimensions. Traditional approaches such as nuclear‑norm minimization provide convex relaxations but become computationally prohibitive for large‑scale data, while non‑convex methods like alternating least squares are highly sensitive to initialization. To overcome these limitations, the authors propose OptSpace, a two‑stage algorithm that combines a fast spectral initialization with a Riemannian gradient descent on the Grassmann manifold.

In the first stage, the observed entries are scaled by the factor (n^{2}/|\Omega|) to obtain an unbiased estimator of M. A truncated singular value decomposition (SVD) is then performed, retaining only the top r singular values and the corresponding left and right singular vectors. This yields an initial low‑rank approximation (M_{0}=U_{r}\Sigma_{r}V_{r}^{T}). Theoretical results from random matrix theory guarantee that, when the number of observed entries satisfies (|\Omega|\ge C,nr\log n) for a suitable constant C, the spectral estimate is within a small Frobenius‑norm distance of the true matrix with high probability.

The second stage refines this estimate by solving the optimization problem
\