Matrix Completion from Noisy Entries

Given a matrix M of low-rank, we consider the problem of reconstructing it from noisy observations of a small, random subset of its entries. The problem arises in a variety of applications, from collaborative filtering (the `Netflix problem’) to structure-from-motion and positioning. We study a low complexity algorithm introduced by Keshavan et al.(2009), based on a combination of spectral techniques and manifold optimization, that we call here OptSpace. We prove performance guarantees that are order-optimal in a number of circumstances.

💡 Research Summary

The paper addresses the fundamental problem of recovering a low‑rank matrix from a small, randomly sampled subset of its entries when those observations are corrupted by additive noise. This setting models many practical applications such as collaborative filtering (e.g., the Netflix problem), structure‑from‑motion in computer vision, and sensor‑network localization, where only a fraction of pairwise measurements are available and each measurement is noisy.

The authors focus on a computationally efficient algorithm originally proposed by Keshavan, Montanari, and Oh (2009), which they refer to as OptSpace. OptSpace consists of three main stages: (1) Trimming, which discards rows and columns that are observed far more often than the average, thereby enforcing an incoherence condition on the remaining data; (2) Spectral Initialization, where the trimmed observation matrix is scaled by the sampling probability and its top‑r singular vectors are extracted to form an initial estimate of the left and right subspaces; and (3) Manifold Optimization, in which the subspace estimates are refined by performing gradient descent on the product of two Grassmann manifolds, minimizing the squared error over the observed entries. The gradient steps are carefully re‑orthogonalized to maintain numerical stability.

The theoretical contribution is twofold. First, under a standard μ‑incoherence assumption on the true matrix and assuming independent Gaussian noise with variance σ², the authors prove that if the number of observed entries |Ω| satisfies |Ω| ≥ C·μ·n·r·log n (for a universal constant C), then with high probability the spectral initialization lands within a small neighborhood of the true subspaces. Second, they show that the subsequent manifold optimization converges linearly to a point (\hat M) whose Frobenius error obeys
\