The Power of Convex Relaxation: Near-Optimal Matrix Completion

In mathematical terms, the problem may be posed as follows: we have a data matrix M ∈ R n 1 ×n 2 which we would like to know as precisely as possible. Unfortunately, the only information available about M is a sampled set of entries M ij , (i, j) ∈ Ω, where Ω is a subset of the complete set of entries [n 1 ] × [n 2 ]. (Here and in the sequel, [n] denotes the list {1, . . . , n}.) Clearly, this problem is ill-posed for there is no way to guess the missing entries without making any assumption about the matrix M .

An increasingly common assumption in the field is to suppose that the unknown matrix M has low rank or has approximately low rank. In a recommendation system, this makes sense because often times, only a few factors contribute to an individual’s taste. In [7], the authors showed that this premise radically changes the problem, making the search for solutions meaningful. Before reviewing these results, we would like to emphasize that the problem of recovering a low-rank matrix from a sample of its entries, and by extension from fewer linear functionals about the matrix, comes up in many application areas other than collaborative filtering. For instance, the completion problem also arises in computer vision. There, many pixels may be missing in digital images because of occlusion or tracking failures in a video sequence. Recovering a scene and inferring camera motion from a sequence of images is a matrix completion problem known as the structure-from-motion problem [9,23]. Other examples include system identification in control [19], multi-class learning in data analysis [1][2][3], global positioning-e.g. of sensors in a network-from partial distance information [5,21,22], remote sensing applications in signal processing where we would like to infer a full covariance matrix from partially observed correlations [25], and many statistical problems involving succinct factor models.

This paper is concerned with the theoretical underpinnings of matrix completion and more specifically in quantifying the minimum number of entries needed to recover a matrix of rank r exactly. This number generally depends on the matrix we wish to recover. For simplicity, assume that the unknown rank-r matrix M is n × n. Then it is not hard to see that matrix completion is impossible unless the number of samples m is at least 2nr -r 2 , as a matrix of rank r depends on this many degrees of freedom. The singular value decomposition (SVD)

where σ 1 , . . . , σ r ≥ 0 are the singular values, and the singular vectors u 1 , . . . , u r ∈ R n 1 = R n and v 1 , . . . , v r ∈ R n 2 = R n are two sets of orthonormal vectors, is useful to reveal these degrees of freedom. Informally, the singular values σ 1 ≥ . . . ≥ σ r depend on r degrees of freedom, the left singular vectors u k on (n -1) + (n -2) + . . . + (n -r) = nr -r(r + 1)/2 degrees of freedom, and similarly for the right singular vectors v k . If m < 2nr -r 2 , no matter which entries are available, there can be an infinite number of matrices of rank at most r with exactly the same entries, and so exact matrix completion is impossible. In fact, if the observed locations are sampled at random, we will see later that the minimum number of samples is better thought of as being on the order of nr log n rather than nr because of a coupon collector’s effect. In this paper, we are interested in identifying large classes of matrices which can provably be recovered by a tractable algorithm from a number of samples approaching the above limit, i.e. from about nr log n samples. Before continuing, it is convenient to introduce some notations that will be used throughout: let P Ω : R n×n → R n×n be the orthogonal projection onto the subspace of matrices which vani

…(Full text truncated)…