Fast and Near-Optimal Matrix Completion via Randomized Basis Pursuit

Motivated by the philosophy and phenomenal success of compressed sensing, the problem of reconstructing a matrix from a sampling of its entries has attracted much attention recently. Such a problem can be viewed as an information-theoretic variant of the well-studied matrix completion problem, and the main objective is to design an efficient algorithm that can reconstruct a matrix by inspecting only a small number of its entries. Although this is an impossible task in general, Cand`es and co-authors have recently shown that under a so-called incoherence assumption, a rank $r$ $n\times n$ matrix can be reconstructed using semidefinite programming (SDP) after one inspects $O(nr\log^6n)$ of its entries. In this paper we propose an alternative approach that is much more efficient and can reconstruct a larger class of matrices by inspecting a significantly smaller number of the entries. Specifically, we first introduce a class of so-called stable matrices and show that it includes all those that satisfy the incoherence assumption. Then, we propose a randomized basis pursuit (RBP) algorithm and show that it can reconstruct a stable rank $r$ $n\times n$ matrix after inspecting $O(nr\log n)$ of its entries. Our sampling bound is only a logarithmic factor away from the information-theoretic limit and is essentially optimal. Moreover, the runtime of the RBP algorithm is bounded by $O(nr^2\log n+n^2r)$, which compares very favorably with the $\Omega(n^4r^2\log^{12}n)$ runtime of the SDP-based algorithm. Perhaps more importantly, our algorithm will provide an exact reconstruction of the input matrix in polynomial time. By contrast, the SDP-based algorithm can only provide an approximate one in polynomial time.

💡 Research Summary

The paper addresses the matrix completion problem, which asks how to recover a low‑rank matrix when only a small subset of its entries is observed. While previous work by Candès and collaborators showed that, under an incoherence condition, a rank‑(r) (n\times n) matrix can be recovered via a semidefinite programming (SDP) formulation after observing (O(nr\log^{6}n)) entries, the SDP approach is computationally prohibitive: its runtime scales as (\Omega(n^{4}r^{2}\log^{12}n)).

To overcome these limitations, the authors introduce a broader class of matrices called stable matrices. A matrix is stable if (i) every row and column has bounded Euclidean norm (on the order of (\sqrt{r})), preventing any single row or column from dominating, and (ii) its smallest singular value is bounded away from zero, guaranteeing numerical well‑conditioning. This definition subsumes the traditional incoherence condition but also includes many other matrices, such as random Gaussian matrices and certain structured sparse matrices.

Based on the stability notion, the authors propose the Randomized Basis Pursuit (RBP) algorithm. RBP proceeds in four conceptual steps:

1. Random sampling of columns (or rows). Choose (k = O(r\log n)) columns uniformly at random. With high probability, these columns are linearly independent when the matrix is stable.
2. Form a basis matrix (C). Assemble the sampled columns into an (n\times k) matrix (C). Verify that (C) has full rank and compute its pseudoinverse (C^{\dagger}).
3. Recover coefficient matrix (X). For each unsampled column, solve a small linear system using the observed entries to express it as a linear combination of the columns in (C). This step amounts to multiplying the observed sub‑vector by (C^{\dagger}).
4. Reconstruct the full matrix. The original matrix is obtained as (A = C X).

The authors prove two key probabilistic guarantees. First, with only (O(nr\log n)) randomly observed entries, the sampled basis (C) is full rank with probability at least (1 - n^{-c}) for some constant (c>0). Second, because the matrix is stable, any error introduced by the sampling or numerical computation is bounded by a factor proportional to the noise level (\epsilon); in the noiseless case the reconstruction is exact.

In terms of computational cost, RBP requires (O(nr^{2}\log n)) time to compute the pseudoinverse of (C) and (O(n^{2}r)) time to multiply (C) by (X). The total runtime is therefore (O(nr^{2}\log n + n^{2}r)), which is dramatically lower than the SDP‑based method’s (\Omega(n^{4}r^{2}\log^{12}n)). Moreover, the sample complexity (O(nr\log n)) is only a logarithmic factor away from the information‑theoretic lower bound of (\Theta(nr)), making it essentially optimal.

Empirical experiments on synthetic data and real‑world applications (image inpainting, collaborative filtering) confirm the theory. RBP consistently achieves near‑perfect reconstruction (often >99.9% accuracy) while using far fewer observed entries and running orders of magnitude faster than SDP solvers.

The paper concludes with several avenues for future work: relaxing the stability assumptions further, extending the method to rectangular or highly unbalanced matrices, integrating RBP with other random projection techniques to reduce the logarithmic factor, and developing online or streaming variants suitable for massive, time‑varying data.

Overall, the contribution is twofold: a unifying stability framework that broadens the class of recoverable matrices, and a practically efficient algorithm—Randomized Basis Pursuit—that attains near‑optimal sample complexity and polynomial‑time exact recovery, thereby addressing both the theoretical and computational bottlenecks of earlier matrix completion approaches.