Strongly Convex Programming for Exact Matrix Completion and Robust Principal Component Analysis

Strongly Con v ex Programming for Ex act Matri x Completion and Robust Principal Comp onen t Analys is Hui Zhang ∗ Jian-F eng Cai † Lizhi Cheng ‡ Jub o Zh u § Abstract The common task in matrix completion (MC) and robust principle co mpone nt analy sis (RPCA) is to reco ver a low-rank matrix from a giv en data ma trix. These problems gained great attent ion f ro m v arious areas in applied s ciences recently , esp ecially after the publication o f the pioneering w or ks o f Cand` es et al.. One fundamen tal r esult in MC and RPCA is tha t nuclear norm based conv ex optimizations lead to t he exact low-rank matrix reco very under suitable conditions. In this pap er, we ex tend this result by showing that s trongly conv ex optimizations can g uarantee the exact low-rank matrix re cov ery as well. The res ult in this pap er not o nly provides sufficient conditions under whic h the strongly co nv ex models lead to the exact lo w-ra nk matrix recovery , but also guides us on ho w to c ho ose suitable para meters in prac tica l algorithms. 1 In tro duction There is a rapidly gro wing in terest in the reco v ery of an u nkno wn lo w-rank or appr o ximately lo w- rank matrix f rom a sampling of its en tries. This p roblem o ccurs in many areas of app lied sciences ranging from mac hine learning [1] and con trol [27] to computer vision [36], and it has regained great atten tion since the p ublication of the pioneering works [6, 9, 30]. Under different settings, lo w- rank matrix reco v ery problems can b e solve d by n uclear n orm based conv ex optimizations without an y loss of accuracy if the underlying matrix is indeed of lo w rank [6 – 9, 30–33]. Th ese fu ndamen tal results h a v e had a g reat impact in engineering and applied sciences. Recen tly , they hav e sh o wn their tremendous p o w ers in man y practical applications such as video restoration [19, 20], cognitiv e radio net w orks [25], d ecomp osing bac kground topic fr om k eyw ords [24], lo w-rank textures capture [40], and 4D computed tomography [17 ]. In order to design efficient algo rithms for lo w-rank matrix reco v ery p roblems, instead of dir ectly solving th e original con v ex optimizations, we sometimes us e their strongly con v ex ap p ro ximations; see, e.g., [3, 37]. In this p ap er, w e will sho w that these strongly con vex programmings guarantee the exact low-rank matrix reco v ery as w ell. Our result not only pro vides su fficien t conditions under whic h the str ongly con v ex mo dels lead to the exact lo w-rank matrix reco v ery , but also guid e us on ho w to c ho ose suitable p arameters in practical algorithms. ∗ Department of Mathematics and Sy stems Science, Col lege of Science, National U niversi ty of Defense T ec hn ology , Changsha, Hu nan, 410073, P .R .China. Correspond ing aut hor. Email: h.zhang19 84@163.com † Department of Mathematics, Un iver sity of Iow a, Iow a City , IA 52242, USA. Email: jianfen g-cai@uiowa.edu ‡ Department of Mathematics and Sy stems Science, Col lege of Science, National U niversi ty of Defense T ec hn ology , Changsha, Hu nan, 410073, P .R .China. § Department of Mathematics and Sy stems Science, Col lege of Science, National U niversi ty of Defense T ec hn ology , Changsha, Hu nan, 410073, P .R .China. 1 1.1 Con v ex programming In th is p ap er, w e mainly f o cus on t w o sp ecific lo w-rank matrix reco v ery pr oblems, n amely , the low- rank matrix completion (MC) [3, 6, 7 ] and the r obust principle comp onen t analysis (RPCA) [9 , 37]. Let M ∈ R n 1 × n 2 b e an unkno wn lo w-rank matrix. In MC, we w ould like to reco v er M from its partially kno wn en tries { m ij , ( i, j ) ∈ Ω } , where Ω $ [ n 1 ] × [ n 2 ] is the set of indices of kn own en tries. Here [ n ] stands for the s et { 1 , . . . , n } . Define P Ω as [ P Ω X ] ij =  x ij , ( i, j ) ∈ Ω , 0 , otherwise . Then the sampled d ata in MC can b e represented as P Ω M . The goal of MC is to r eco ver the unknown lo w-rank matrix M from P Ω M . Sin ce the matrix we are seeking is of lo w rank, a n atural w a y to solv e MC is to find the lo west rank matrix among the feasible set { X ∈ R n 1 × n 2 : P Ω X = P Ω M } . This leads to th e follo wing optimization problem minimize : rank( X ) sub ject to : P Ω X = P Ω M . (1) Unfortunately , this problem is kn own to b e NP-hard and the ob jectiv e is non-con v ex. A p opular alternativ e is to r elax (1) to its nearest con ve x problem minimize : k X k ∗ sub ject to : P Ω X = P Ω M . (2) Here k X k ∗ is the su m mation of the sin gu lar v alues of X . Note that k X k ∗ is the b est con v ex lo w er b ound of the rank function on the set of matrices whose top singular v alue is b ound ed b y one [15]. It was sho wn that, und er suitable assump tions, the u nderlying lo w-rank matrix M can b e exactly reco v ered with high probabilit y if Ω is uniformly randomly dra wn from all subsets of [ n 1 ] × [ n 2 ] with the same cardinality . In RPCA, the sampled data matrix, denoted b y D , is M with a small p ortion of i ts en tries b eing corru pted. T he goal is to restore the low-rank matrix M from its p artially corrupted data matrix D . Since only a small p ortion ent ries are corr u pted, the noise matrix D − M is a sparse matrix. Therefore, D consists of t wo comp onents: a lo w-rank matrix (the underlyin g matrix M ) and a sp arse m atrix (the noise). Thus, the RPCA p roblem can b e solv ed by minimize : rank( L ) + λ k S k 0 sub ject to : D = L + S, (3) where k S k 0 stands for the n umber of nonzero en tries of S . This problem is NP-hard and non- con v ex. Again, one can exploit the con ve x r elaxatio n tec hniqu e used in the MC prob lem. Notice that the conv ex lo w er b ound of the zero-norm fun ction k · k 0 in the in fi nit y-norm unit ball is the 1-norm k · k 1 (the summ ation of absolute v alues). Therefore, the conv ex relaxation of (3) is [9, 12] minimize : k L k ∗ + λ k S k 1 sub ject to : D = L + S. (4) where λ is a parameter balancing the lo w-rank and sparse comp onents. I t w as sho wn in [9, 12] that the con vex optimization can r eco v er the lo w-rank matrix M exactly under suitable assum p tions. 2 1.2 Strongly con v ex progr amming There exist man y efficient algo rithm s to solv e the conv ex optimization problems (2) and (4) ev en when the scale of the problems is large up to 10 5 × 10 5 , including [2 , 21, 26, 35, 41] f or (2) and [9, 22, 23, 37] for (4 ). Ho wev er, instead of solving the conv ex minimizations (2) a nd (4) directly , some of the aforemen tioned algorithms solve th eir appro ximations in vol ving str ongly c onvex ob jecti ves. In particular, the singu lar v alue thresholding (SVT) algorithm [2] in MC u ses Uza w a’s algorithm to solv e minimize : k X k ∗ + 1 2 τ k X k 2 F sub ject to : P Ω X = P Ω M , (5) and the iterativ e thresholding(IT) algorithm [37] in RPCA s olv es 1 minimize : k L k ∗ + 1 2 τ k L k 2 F + λ k S k 1 + 1 2 τ k S k 2 F sub ject to : D = L + S. (6) where τ is some p ositiv e p enalty parameter. One of the m ain adv an tages of using strongly con vex programmings is that a broader r ange of existing optimization metho ds in the literature can be applied to the MC and RPCA p roblems. F or example, it is w ell kno wn that the con v ex conjugate of a str on gly con ve x function is d ifferen tiable [34]. T h erefore, the Lagrange dual of the strongly con vex programmings (5) and (6) are differen tiable, and hence smo oth con v ex optimizatio n metho ds can b e applied to th e dual. In fact, the SVT algorithm [2] and the IT algo rithm [37] are the gradient algorithms app lied to the Lagrange dual of (5) and (6) resp ectiv ely (a.k.a. Uza w a’s algo rithm ). Additionally , one can exploit Nestro v’s optimal sc heme for s mo oth optimization [28] and ev en quasi-Newton metho d (e.g. L-BF GS [38]), j ust to name a f ew. Ho w ev er, the con v ex conjugate of k X k ∗ or k X k 1 is not different iable so th at the Lagrange dual of (2) or (4) is not smo oth. That give s us a p art of reason why some algorithms exploit strongly con v ex approximati ons to solv e (2) and (4). On th e other hand, it f ollo ws from [2 , 37] and standard con v ex optimization theory th at, when τ tends to infinit y , the strongly con vex optimization (5) and (6) b eco me (2) and (4) r esp ectiv ely . Th erefore, in order to get the exact lo w-ran k matrix reco very , w e hav e to c ho ose an infin ite τ in the S VT and IT algorithms, which is impractical. F ortu n ately , it has b een observ ed empirically in [2, 37] that a finite τ is enough for the purp ose of the exact low-rank matrix reco v ery . So a n atural theoretical question is whether (5) and (6) with a fi nite τ lead to the exact lo w-rank matrix reco v ery in MC an d RPCA. F uthermore, th e empirical con v ergence sp eed of th e SVT and IT algorithms dep end on τ . Th e sm aller τ is, the faster the algorithm con v erges. Therefore, it is int eresting to find a lo w er b oun d of τ for the exact low-rank matrix reco v ery . These t w o questions are answered p ositiv ely in this pap er. The related literature can b e traced bac k to the linearized Bregman iteration (LB I) al gorithm [3– 5, 29, 38, 39], whic h can appro ximately bu t efficien tly s olve the basis p ursuit problem [13 ] min x {k x k 1 : Ax = b } in compressed s en sing [10, 11, 14]. An inte resting phenomenon ab out the LBI alg orithm wa s disco v ered in [3, 4, 38]: the LBI algorithm conv erges to a str on gly con ve x optimization min x {k x k 1 + 1 The pap er [37] contains a critical error on the theoretical analysis, whic h h ad b een disco v- ered and remo ved b y Emmanel Cand` es of S t andford. F ortunately , the corrected vers ion does not change the original convex mo del and has no effect on its iterative thresholding algorithm. See http://boo ks.nips.cc/pape rs/files/nips22/NIPS2009_0116_correction.pdf . 3 1 2 τ k x k 2 : Ax = b } whose solution is the same as the basis pursuit p roblem when the parameter τ b ey ond a finite s caler u n der some suitable conditions. In other wo rds , u nder some suitable conditions, a conv ex pr ogramming is equ iv alent to a strongly conv ex p rogramming when τ is large enough. In [42], this idea is extend ed to th e MC problem. Unfortun ately , th e b oun d is ve ry rough, and the m etho d used there has many limitations and can not b e extended to the RPCA p roblem. In [16], a generic exact r egularizati on result is given, but the conditions are n ot easy to c hec k for b oth MC and RPCA p roblems. 1.3 Con tributions and organization In th e pap er, w e pro v e that, if τ exceed some v alue, then un der some suitable conditions strongly con v ex programming (5) and (6) can reco v er lo w-rank matrices exactly in MC and RPCA pr oblems resp ectiv ely . Th e explicit lo wer b ounds of τ are also giv en, and the low er b ounds greatly improv e the result in [42]. The significance of the pap er is t wo -folded. Firstly , sufficien t conditions under whic h the str on gly con v ex programming leads to exact lo w-rank matrix reco v ery are deriv ed. T his, in tu rn, allo ws us to exploit a b r oader range of existing optimiza tion method in the literature. Seco nd ly , when w e prefer to minimizing the s tr ongly con v ex ob jectiv es for designing fast algorithms, we o nly n eed to set a finite parameter τ b ey ond some v alue determined b y the gi ven data matrix, whic h le ad to faster con ve rgence o f the algorithms compared with the one using τ close to in finit y . In other w ords, it pro vides some guid ance on ho w to c ho ose su itable parameters in p ractical algorithms. The remainder of the p ap er is organized as follo ws. In Section 2, we p ro vide a brief summary of preliminaries including notations, assump tions, and some existing results. In Section 3, we show that strongly con ve x programming (5) and (6) lead to th e exact lo w rank matrix r eco v ery , and we giv e exp licit expressions of the lo we r b ounds of τ . Conclusion and further works are discussed in Section 4. 2 Preliminaries In this section, w e giv e s ome notations and existing results that will b e u s ed later in this pap er. 2.1 Notations Let X , Y ∈ R n 1 × n 2 b e t wo matrices. The Euclidean inner pro duct in R n 1 × n 2 is defined as h X , Y i := trace( X ∗ Y ), wh ere X ∗ stands for the transp ose X . Then the Euclidean norm of X , denoted b y k X k F , is k X k F := p h X, X i , and it is also kno wn as the F rob enius norm. W e denote the i -th nonzero singular v alue of matrix X by σ i ( X ), where 1 ≤ i ≤ r ank( X ). The F orb en iu s norm also equals to the Eu clidean norm of the ve ctor of singular v alues, i.e., k X k F =  P i σ 2 i ( X )  1 / 2 . The sp ectral norm of X , denoted b y k X k , is k X k := σ 1 ( X ), the largest singular v alue of X . The nucle ar norm of X is the s u mmation of its singular v alues, i.e. k X k ∗ := P i σ i ( X ). The maxim um en try of X (in absolute v alue) is denoted by k X k ∞ := max i,j | X ij | . The l 1 norm of a matrix viewed as a long v ector is den oted b y k X k 1 := P i,j | X ij | . It can b e easily v erified that  k X k ∞ ≤ k X k F ≤ k X k 1 ≤ n 1 n 2 k X k ∞ , k X k ≤ k X k F ≤ k X k ∗ ≤ p rank( X ) · k X k F ≤ rank( X ) · k X k . (7) 4 The dual relationship is also imp ortan t in our deriv atio n. More pr ecisely , w e hav e  k X k = sup k Y k ∗ ≤ 1 h X, Y i , k X k ∗ = sup k Y k≤ 1 h X, Y i , k X k 1 = sup k Y k ∞ ≤ 1 h X, Y i , k X k ∞ = sup k Y k 1 ≤ 1 h X, Y i . (8) It follo ws directly that |h X , Y i| ≤ k X k · k Y k ∗ and |h X , Y i | ≤ k X k 1 · k Y k ∞ . W e denote M ∈ R n 1 × n 2 to b e the unkno wn lo w-rank matrix that is to b e reco vered in the MC and RPCA p r oblems. Rec all that the av ailable data in the MC problem is P Ω M , where Ω is the index set o f kno wn entries. In the RPC A pr oblem, th e a v ailable data is D . W e will denote the sparse comp onen t, or the noise, by S 0 = D − M . Without ambiguit y , Ω is used as th e su pp ort of S 0 in the RPC A p roblem. Let n = max { n 1 , n 2 } . W e will alwa y s us e r := rank( M ), and denote the singular v alue decom- p osition of M by M = U Σ V ∗ = r X i =1 σ i u i v ∗ i , where σ 1 , . . . , σ r are th e p ositiv e singular v alues of M , and U = [ u 1 , . . . , u r ] an d V = [ v 1 , . . . , v r ] are the matrices of the left- and right -singular v ectors. Let T b e the linear s u bspace of R n 1 × n 2 T := { U X ∗ 1 + X 2 V ∗ , X 1 ∈ R n 2 × r , X 2 ∈ R n 1 × r } , (9) and T ⊥ b e its orthogonal completion. Define P T and P T ⊥ b e th e orthogonal p ro jectio ns on to T and T ⊥ resp ectiv ely . Let P U and P V represent the pro jectio n op erators onto the spaces sp an n ed b y the columns of U and V resp ectiv ely , i.e. P U = U U ∗ and P V = V V ∗ . The op erato r P T can b e written as P T Y = P U Y + Y P V − P U Y P V . (10) T o sa v e notations, Ω also denotes the li near s p ace of the m atrices supp orted on Ω, so that the pro jection on to Ω is P Ω in (1). It is well known that an y sub gradien t [34] of the n uclear n orm fu nction at M is of the form U V ∗ + W , (11) where W ∈ R n 1 × n 2 is a matrix s atisfying P T W = 0 , k W k ≤ 1 . (12) The subgrad ient of th e l 1 norm fun ction at S 0 is sgn( S 0 ) + F , (13) where F satisfies P Ω F = 0 and k F k ∞ ≤ 1, and sgn( S 0 ) is the matrix whose entries are the signs of those of S 0 . Finally , we shall also m anipulate some linear op erators wh ich act on the space R n 1 × n 2 . W e will use calligraphic letters for these op erato rs as in A ( X ). In particular, I : R n 1 × n 2 → R n 1 × n 2 denotes the identit y op erator and I  A means that A − I is symmetric p ositiv e semidefinite, and of course I ≺ A means that A − I is s y m metric p ositiv e defin ite. W e introdu ce the op erator norm, denoted b y kAk and d efined as kAk = su p {k X k F ≤ 1 } kA X k F . 5 2.2 Existing results In th is sub s ection, we p resen t some existing results f or exact lo w-rank matrix r eco v ery via conv ex optimization. F or this, we need s ome assumptions on the unkn own lo w-rank matrix M and the set Ω. Let us first giv e assumptions on M . It is observed in [6, 8] that, for the MC problem, it is imp ossible to rec ov er a matrix whic h is equal to zero in nearly all of its en tries un less all of t he matrix en tries are observed. F or the RPCA problem, the authors in [9, 12] observed if the lo w-rank comp onen t is also sparse, then it is imp ossible to decide whether it is lo w-rank or spars e. These observ atio ns lead to the introd uction of the incoherence assumptions that w ill b e introd uced in th e follo wing. There are some other conditions to c haracterize the lo w-rank sparse decomp osition, e.g., the un certain t y pr in ciple and rank-sparsit y incoherence in [12]. Assumption 1 (Incoherence A ssu mption) . We say that the matrix M ob eys the inc oher ent as- sumption with p ar ameter µ , if the fol lowing statements ar e valid: max i k U ∗ e (1) i k 2 ≤ µr n 1 , max i k V ∗ e (2) i k 2 ≤ µr n 2 (14) and k U V ∗ k ∞ ≤ r µr n 1 n 2 . (15) wher e e (1) i and e (2) i ar e the i -th ve ctor in the c ano nic al b asis of R n 1 and R n 2 r esp e ctively. A s tronger assu mption, namely the strong in coheren t c ond ition, can also guarantee exact matrix completion via con vex programming [7]. It has b een sho wn that the strong assump tion imp lies the w eak er one in [43], but for th e easy p r esen tation of the existing results, we shall in tro duce b oth of them. Assumption 2 (Strong Incoherence Assu m ption) . We say that the matrix M ob eys the str ong inc oher ent assumption with p ar ameter µ 1 , if, for any µ ≤ µ 1 , the ine quality (15) holds true and the fol lowing statements ar e valid: for al l p airs ( a, a ′ ) ∈ [ n 1 ] × [ n 1 ] and ( b, b ′ ) ∈ [ n 2 ] × [ n 2 ] , it holds that ( |h e (1) a , U U ∗ e (1) a ′ i − r n 1 1 a = a ′ | ≤ µ √ r n 1 ; |h e (2) b , V V ∗ e (2) b ′ i − r n 2 1 b = b ′ | ≤ µ √ r n 2 . (16) Next we mo v e to the assumptions on Ω, the samplin g pattern in the MC p roblem and the sparse p attern of the sparse matrix in RPCA. Th e theorems stated in [6, 9] alw a ys assume that Ω is un iformly randomly dr a wn f rom the set { Λ : | Λ | = m, Λ ⊂ [ n 1 ] × [ n 2 ] } , wh ere | · | stand s for the cardinalit y and m = | Ω | . Ho w ev er, as p oin ted out in [9 ], it is a little more conv enien t to w ork with the Bernoulli mo del Ω = { ( i, j ) : δ i,j = 1 } , w h ere the δ i,j ’s are i.i.d. Bernoulli v ariable taking v alue one with probabilit y ρ an d zero w ith probabilit y 1 − ρ . Because o ur pro ofs will hea vily rely on the dual certificates constru cted u n der the Bernoulli mo del, we c ho ose this pro xy random mo del to the uniform sampling, denoted by Ω ∼ B er ( ρ ), in our pap er for simplicit y . W e w ould lik e to p oin t out th at the conv enience brought b y the Bernoulli mo del d o es n ot weak en the formal theorems; for more details see [9]. No w we are ready to present t wo main results for exact m atrix completion and robust principle comp onen t analysis via con ve x optimization. 6 Theorem 1 ( [7, T h eorem 1.2]) . L et M ∈ R n 1 × n 2 b e a fixe d matrix ob eying the str ong inc oher enc e assumption with p ar ameter µ . L et n := max( n 1 , n 2 ) . Supp ose we observe m entries of M w ith lo c ations sample d uniformly at r and om and ther e i s a p ositive numeric al c onsta nt C such that m ≥ C µ 2 nr log 6 n, (17) Then M is the unique solution to c onvex pr o gr amming (2) with pr ob ability at le ast 1 − n − 3 . Theorem 2 ( [9, Th eorem 1.1]) . Supp ose M ∈ R n 1 × n 2 ob eys the inc oher enc e assumption with p ar ameter µ , and that the supp ort set of S 0 is unif ormly distribute d amo ng al l sets of c ar dinality m . Write n (1) := max( n 1 , n 2 ) and n (2) := min( n 1 , n 2 ) . Pr ovide d that rank( M ) ≤ ρ r n (2) µ − 1 (log n (1) ) − 2 , and m ≤ ρ s n 1 n 2 . (18) wher e ρ r and ρ s ar e p ositive numeric al c onstants. Then ther e is a p ositive numeric al c onstant c with pr ob ability at le ast 1 − cn − 10 (1) such that ( M , S 0 ) is the u nique solution to c onvex pr o gr amming (4) with λ = 1 / √ n (1) . 3 Main results In t his section, we presen t th e main resu lts of this pap er. More precisely , we will sho w that, if τ is la rge e nough, then the low r an k matrix M can b e exactly rec ov ered by the strongly con v ex programming (5) f or the MC problem and (6 ) for the RPCA problem. A compu table and explicit lo w er boun d of τ will b e giv en. Th e r esults are p ro v ed in te rms of dual certificates (also kno wn as L agrangian multiplier) in combination w ith a relaxation method in tro duced in [18 ] and studied in [9]. 3.1 Matrix comp letion In this section, w e discuss the strongly con vex programming for the matrix completion problem. Recall that, in th e MC problem, w e w ould lik e to reco v er an unkn o wn lo w-rank matrix M fr om th e sampling of its p artial ent ries P Ω M . W e will sh o w that M is the unique solution of the strongly con v ex minimization (5) with domin ant probabilit y , pro vided that τ exceeds than a sp ecified finite n umb er. W e will also giv e an explicit lo wer b ound of τ u sing only the sampling P Ω M . W e first give a su fficien t condition for M b eing the uniqu e solution of (5). It follo ws from the standard conv ex optimization theory and [7]. Theorem 3. A ssume that ther e is a matrix Y t hat satisfies (a) Y = P Ω Y , (b) P T Y = 1 τ M + U V ∗ , (c) kP T ⊥ Y k ≤ 1 . Then M is the unique solution of the str ongly c onvex pr o gr am ming (5 ). Pr o of. The Lagrangian fu n ction asso ciated with the strongly conv ex programming (5) is L ( X, Y ) := k X k ∗ + 1 2 τ k X k 2 F − h Y , P Ω X − P Ω M i , (19) 7 where Y is the the Lagrangian m ultiplier. By the strong c onv exit y of the ob jectiv e and t he firs t order optimalit y of the Lagrangian function, M is the un ique solution if and only if there is a dual matrix Y satisfying 0 ∈ ∂ k M k ∗ + 1 τ M − P Ω Y . (20) Since w e only u s e the entries of Y on Ω from the expression ab o v e, it is more con ve nient to assume Y = P Ω Y . W e ha v e kn o wn that ∂ k M k ∗ = U V ∗ + W with P T W = 0 and k W k ≤ 1 . Thus the sufficien t and n ecessary condition b ecomes: there exists W such that W = Y − 1 τ M − U V ∗ (21) with Y = P Ω Y , P T W = 0, and k W k ≤ 1. Therefore, it su ffices to find a matrix Y ob eying    Y = P Ω Y P T ( Y − 1 τ M − U V ∗ ) = 0 kP T ⊥ ( Y − 1 τ M − U V ∗ ) k ≤ 1 . (22) By the expression of P T and the fact U ∗ U = V ∗ V = I r , it is not hard to see that P T ( U V ∗ ) = U V ∗ and P T M = M , and hence P T ⊥ ( U V ∗ ) = 0 and P T ⊥ M = 0. S o (22) is equiv alen t to    Y = P Ω Y P T Y = 1 τ M + U V ∗ kP T ⊥ Y k ≤ 1 , (23) whic h concludes the pro of. Next, w e shall construct a v ali d dual certificate Y ob eying the conditions listed in the theorem ab o v e. W e need th e f ollo wing lemma fr om [7 ]. Lemma 1 ( [7, Corollary 3. 7], [8 , Theorem 6]) . Under th e assumptions in The or em 1, we have, with pr ob ability at le ast 1 − n − 3 , kP T ⊥ Λ k ≤ 1 2 , wher e Λ := P Ω P T ( P T P Ω P T ) − 1 U V ∗ , (24) and p 2 I  P T P Ω P T  3 p 2 I , wher e p := m/ ( n 1 n 2 ) . (25) It is ob vious th at the matrix Λ in Lemma 1 satisfies Λ = P Ω Λ and P T Λ = U V ∗ . No w w e are going to constru ct a v alid d u al certificate for exact matrix completion via strongly con v ex programming. Theorem 4. A ssume τ ≥ kP T ⊥ P Ω P T ( P T P Ω P T ) − 1 M k 1 − kP T ⊥ Λ k , (26) wher e Λ is define d in L emma 1. Then, under the assumptions in The or em 1, M is the unique solution to the str ongly c onvex pr o gr amming (5) with pr ob ability at le ast 1 − n − 3 . 8 Pr o of. W e p r o v e the theorem b y constructing a matrix Y whic h satisfies cond itions in Theorem 3 with high pr obabilit y . In deed, if w e defin e Y = Λ + 1 τ P Ω P T ( P T P Ω P T ) − 1 M with Λ defined as in Lemma 1, then the conditions in Theorem 3 are satisfied w ith pr obabilit y at least 1 − n − 3 . By Lemma 1, with probabilit y at least 1 − n − 3 , we ha v e kP T ⊥ Λ k ≤ 1 2 . W e ve rify (a)(b)(c) in Theorem 3 u nder the eve nts that kP T ⊥ Λ k ≤ 1 2 . First of all, τ > 0. Therefore, (5 ) is a strongly con v ex programming. Next, b y the construction of Y , it is ob vious that (a) and (b) in T heorem 3 are true. It remains to sho w (c). In fact, (26) implies that 1 τ kP T ⊥ P Ω P T ( P T P Ω P T ) − 1 M k ≤ 1 − kP T ⊥ Λ k . Since kP Ω ⊥ Λ k ≤ 1 / 2, we ha v e kP T ⊥ Y k ≤ kP T ⊥ Λ k + 1 τ kP T ⊥ P Ω P T ( P T P Ω P T ) − 1 M k ≤ 1 . Therefore, (c) is verified. The low er b ound of τ in Theorem 4 is ob viously finite. Ho w eve r, it is hard to estimate in practice, as the qu an tities in (26) are hard to estimate b efore w e get M . I n the follo wing, we present a computable lo wer b ound of τ , using only the given d ata P Ω M . Our starting p oin t is Lemma 1, wh ic h guarant ee (24) and (25) with v ery high probabilit y . Notice that kP Ω P T Z k 2 F = hP Ω P T Z, P Ω P T Z i = h Z , P T P Ω P T Z i . (27) W e ha ve kP Ω P T ( P T P Ω P T ) − 1 M k 2 F = h ( P T P Ω P T ) − 1 M , M i ≤ 2 p k M k 2 F , and kP Ω P T M k 2 F ≥ p 2 k M k 2 F . With these tw o inequalities, we can easily get kP T ⊥ P Ω P T ( P T P Ω P T ) − 1 M k 1 − kP T ⊥ Λ k ≤ 2 kP T ⊥ P Ω P T ( P T P Ω P T ) − 1 M k F ≤ 2 kP Ω P T ( P T P Ω P T ) − 1 M k F ≤ 2 r 2 p k M k F ≤ 4 p kP Ω P T M k F = 4 p kP Ω M k F . (28) This b ound can b e computed b y using only the samp lin g P Ω M . W e summ arize the resu lt int o the follo wing corollary . Corollary 1. Assume τ ≥ 4 p kP Ω M k F , (29) wher e p = m n 1 n 2 is the sampling r atio. Then, u nder the assumpt ions in The or em 1, M is the uniqu e solution to the str ongly c onvex pr o gr amming (5) with pr ob ability at le ast 1 − n − 3 . 9 3.2 The Robus t Principle Comp onent Analysis problem Next, w e discuss the RPCA case. Recall that, in the RPCA pr oblem, we would lik e to reco v er the unknown lo w-rank matrix M from the sampling D of its co mp lete entries w ith a small fr action of them corrup ted. W e denote the noise in D by S 0 := D − M . Since only a small fraction of en tries are corru pted, S 0 is a sparse matrix. Let Ω b e the supp orted set of S 0 . In this section, w e shall show that ( M , S 0 ) is the uniqu e s olution of the strongly con ve x programming (6) with high probabilit y , pro vided τ exceeding a fin ite num b er. An explicit lo wer b oun d of τ will b e giv en a s w ell. Similar to the MC case, we first p resen t a theorem th at states a sufficient condition f or ( M , S 0 ) b eing the u nique solution of ( 6). Theorem 5. Assume kP Ω P T k ≤ 1 / 2 . Supp ose that ther e exists a p air ( W, F ) and a matrix B ob eying U V ∗ + W + 1 τ M = λ (sgn( S 0 ) + F + P Ω B ) + 1 τ S 0 (30) with P T W = 0 , k W k ≤ β , P Ω F = 0 , k F k ∞ ≤ β , kP Ω B k F ≤ α, wher e α, β ar e p ositive p ar am eters satisfying      β ≤ 1 , α + β ≤ 1 /λ, λ ≤ (1 − β ) / 2 α. (31) Then ( M , S 0 ) is the u nique solution of the str ongly c onvex pr o gr amming (6 ). Pr o of. The main idea of the pr o of follo ws the arguments of [9, Lemm a 2.4]. W e consider a feasible p erturb ation of the form ( M + H , S 0 − H ) and sh o w that the ob jecti ve increases unless H = 0. Let U V ∗ + W 0 + 1 τ M b e an arbitrary su bgradien t of k L k ∗ + 1 2 τ k L k 2 F at M , and so w e hav e P T W 0 = 0 and k W 0 k ≤ 1. Similarly , let λ (sgn( S 0 ) + F 0 ) + 1 τ S 0 b e an arbitrary subgradient of λ k S k 1 + 1 2 τ k S k 2 F at S 0 , and we hav e P Ω F 0 = 0 and k F 0 k ∞ ≤ 1. By the defi nition of subgradient, it holds f ( M + H , S 0 − H ) ≥ f ( M , S 0 ) + h U V ∗ + W 0 + 1 τ M , H i − h λ (sgn( S 0 ) + F 0 ) + 1 τ S 0 , H i . (32) By the construction in [9], we can alw a ys c ho ose W 0 in (32) su ch that h W 0 , H i = kP T ⊥ H k ∗ , an d F 0 suc h that h F 0 , H i = −kP Ω ⊥ H k 1 . Th er efore, f ( M + H , S 0 − H ) ≥ f ( M , S 0 ) + kP T ⊥ H k ∗ + kP Ω ⊥ H k 1 + h U V ∗ + 1 τ M − λ sgn( S 0 ) − 1 τ S 0 , H i = f ( M , S 0 ) + kP T ⊥ H k ∗ + kP Ω ⊥ H k 1 + h U V ∗ + 1 τ M − λ sgn( S 0 ) − 1 τ S 0 − λ P Ω B , H i + λ hP Ω B , H i = f ( M , S 0 ) + kP T ⊥ H k ∗ + kP Ω ⊥ H k 1 + h− W + λF , H i + λ hP Ω B , H i , (33) 10 where in the last equalit y w e hav e u s ed the assum ption (30). By the triangle inequalit y and the assumptions on W and F , w e derive that |h− W + λF , H i| ≤ |h W , H i| + λ |h F , H i| = |h W , P T ⊥ H i| + λ |h F , P Ω ⊥ H i| ≤ k W k · kP T ⊥ H k ∗ + λ k F k ∞ · kP Ω ⊥ H k 1 ≤ β ( kP T ⊥ H k ∗ + λ kP Ω ⊥ H k 1 ) . (34) By the Cauch y-Sc h wa rz inequalit y , it holds |hP Ω B , H i| = |hP Ω B , P Ω H i| ≤ α kP Ω H k F . (35) F u rthermore, we ha ve kP Ω H k F ≤ kP Ω P T H k F + kP Ω P T ⊥ H k F ≤ 1 2 k H k F + kP T ⊥ H k F ≤ 1 2 kP Ω H k F + 1 2 kP Ω ⊥ H k F + kP T ⊥ H k F , (36) whic h imp lies kP Ω H k F ≤ kP Ω ⊥ H k F + 2 kP T ⊥ H k F ≤ kP Ω ⊥ H k 1 + 2 kP T ⊥ H k ∗ . (37) Com bining all the inequalities ab o v e together, we get f ( M + H , S 0 − H ) − f ( M , S 0 ) ≥ (1 − β − 2 αλ ) k P T ⊥ H k ∗ + (1 − λβ − λα ) kP Ω ⊥ H k 1 . (38) This, together with (31), imp lies that ( M , S 0 ) is a s olution to (6). The uniqueness follo ws from the strong con vexit y of the ob jectiv e in (6). Before w e deriv e a low er b ound of τ for the RPCA problem, we n eed the follo wing lemmas. Lemma 2 ( [9, Lemmas 2.8]) . Assume that Ω ∼ B er ( ρ ) with p ar ameter ρ ≤ ρ s for some ρ s > 0 . Then ther e exists a matrix W L in the form of W L = P T ⊥ Y satisfying: under the other assumptions of The or em 2, (a) k W L k < 1 / 4 , (b) kP Ω ( U V ∗ + W L ) k F < λ/ 4 , (c) kP Ω ⊥ ( U V ∗ + W L ) k ∞ < λ/ 4 . Lemma 3 ( [9, Lemmas 2.9]) . Assume that S 0 is supp orte d on a set Ω sa mple d as i n L emma 2 , and that the signs of S 0 ar e i .i.d. symmetric (and ind ep endent of Ω ). Then ther e exists a matrix W S in the f orm of W S = P T ⊥ Z satisfying: under the other assumptions of The or em 2, (a) k W S k < 1 / 4 , (b) kP Ω ⊥ W S k ∞ < λ/ 4 , The construction of W L and W S can b e found in [9]. No w, we sho w that ( M , S 0 ) is the unique solution to the strongly con ve x p r ogramming (6) w ith h igh probabilit y . The idea is to ve rify that W L + W S is a v alid du al certificate w h en the p arameter τ go es b eyond some finite v alue. More precisely , we c hec k the conditions in T heorem 5. 11 Theorem 6. A ssume τ ≥ 2 kP Ω ⊥ D k ∞ + λ kP Ω ( M − S 0 ) k F λ (1 − λ ) (39) Then, under the assumptions of The or em 2, ( M , S 0 ) is the uniq ue solution to the str ongly c onvex pr o gr amming (6) with pr ob ability at le ast 1 − cn − 10 (1) , wher e c is the numeric al c onstant in The or em 2. Pr o of. W e pr o v e the theorem b y c h ec king the conditions in T h eorem 5. Due to the rela tion b et w een F , W and B in (30), it suffices to s ho w th at, there exists a matrix W ob eying        P T W = 0 , k W k ≤ β , kP Ω ⊥ ( U V ∗ + W + 1 τ M − 1 τ S 0 ) k ∞ ≤ β λ, kP Ω ( U V ∗ + W − λ sgn( S 0 ) + 1 τ M − 1 τ S 0 ) k F ≤ αλ, (40) where α, β satisfies (31) . Let W = W L + W S with W L and W S defined in Lemmas 2 and 3 resp ect ive ly . W e sho w that (40) holds tru e. F or simplicit y , we d enote γ := kP Ω ⊥ ( M − S 0 ) k ∞ , δ := kP Ω ( M − S 0 ) k F . W e c ho ose α = ǫ 2 λ and β = 1 − ǫ , where 0 ≤ ǫ < 1 2 is a parameter determined later. Then, (31) is satisfied fo r an y λ ≤ 1; note that the final choic e λ = 1 / √ n (1) ≤ 1. By the expressions of W S and W L , it is obvious that W ∈ T ⊥ . Since k W L k < 1 / 4 and k W S k < 1 / 4, we ha v e k W k ≤ k W L k + k W S k < 1 / 2 ≤ β . F or the third inequalit y in (40), w e hav e kP Ω ⊥ ( U V ∗ + W + 1 τ M − 1 τ S 0 ) k ∞ ≤ kP Ω ⊥ ( U V ∗ + W L ) k ∞ + kP Ω ⊥ W S k ∞ + 1 τ kP Ω ⊥ ( M − S 0 ) k ∞ ≤ λ 4 + λ 4 + 1 τ kP Ω ⊥ ( M − S 0 ) k ∞ = λ 2 + γ τ . (41) F or the last inequalit y in (40), we n otice that P Ω ( W S ) = λ sgn( S 0 ) as shown in [9 ], and therefore w e h a v e kP Ω ( U V ∗ + W − λ sgn( S 0 ) + 1 τ M − 1 τ S 0 ) k F = kP Ω ( U V ∗ + W − W S + 1 τ M − 1 τ S 0 ) k F = kP Ω ( U V ∗ + W L + 1 τ M − 1 τ S 0 ) k F ≤ kP Ω ( U V ∗ + W L ) k F + 1 τ kP Ω ( M − S 0 )) k F = λ 4 + δ τ . (42) In order that the last t w o inequalities in (40) are satisfied, we hav e to c ho ose a τ such that λ 2 + γ τ ≤ (1 − ǫ ) λ, and λ 4 + δ τ ≤ ǫ 2 , whic h imp lies that τ ≥ max ( γ ( 1 2 − ǫ ) λ , δ ( ǫ 2 − λ 4 ) ) . (43) 12 T o get a tigh ter lo wer b ound of τ , w e c ho ose ǫ to minimize the maxim um in the righ t hand side of (43). Since γ ( 1 2 − ǫ ) λ monotonically increases and δ ( ǫ 2 − λ 4 ) monotonically decreases as ǫ increases from λ 2 to 1 2 , it is easy to see that the maxim um is attained when γ ( 1 2 − ǫ ) λ = δ ( ǫ 2 − λ 4 ) , w hic h yields a solution ǫ = δ 2 + γ 4 γ 2 λ + δ that alwa y s falls in [ λ 2 , 1 2 ] if λ ≤ 1. S u bstituting it in to (43), w e obtain an optimal low er b ound of τ τ ≥ 2 γ + λδ λ (1 − λ ) . In viewing of P Ω ⊥ S 0 = 0 and therefore P Ω ⊥ ( M − S 0 ) = P Ω ⊥ D , we get (39) . In the ab ov e deriv atio n, w e hav e used only assertions (a)(b)(c) in Lemma 2 and (a)(b) in Lemma 3. The prob ab ilities th at these assertions hold true and kP Ω P T k ≤ 1 / 2 are shown in the pro of of [9, Theorem 1.1], u nder the same assumptions of this theorem. Th erefore, the remaining of the pr o of of th is theorem follo ws directly from th at of [9, Th eorem 1.1]. It is easy to see that the lo w er b ound of τ is a fi nite num b er. Ho w ev er, th e exact lo w er b ound is very hard to get, b ecause w e can only manipulate th e giv en data matrix D . In the follo wing, w e giv e a metho d to estimate the lo w er b ound. F or th is, w e ha ve to get upp er b ounds for the norms in vo lve d in (39) . F or the first n orm, w e simply u se kP Ω ⊥ D k ∞ ≤ k D k ∞ . F or the second norm, by u sing the facts P Ω ⊥ M = P Ω ⊥ D and kP Ω P T k ≤ 1 / 2, we ha v e kP Ω M k 2 F = kP Ω P T M k 2 F ≤ 1 4 k M k 2 F = 1 4 ( kP Ω M k 2 F + kP Ω ⊥ D k 2 F ) , whic h imp lies kP Ω M k F ≤ √ 3 3 kP Ω ⊥ D k F . Therefore, w e obtain kP Ω ( M − S 0 ) k F = k 2 P Ω M − P Ω D k F ≤ kP Ω D k F + 2 kP Ω M k F ≤ kP Ω D k F + 2 √ 3 3 kP Ω ⊥ D k F . By using the C auc h y-Sch w arz in equalit y , we finally ha v e kP Ω ( M − S 0 ) k F ≤ v u u t 1 2 + 2 √ 3 3 ! 2 · q kP Ω D k 2 F + kP Ω ⊥ D k 2 F = √ 15 3 k D k F . Therefore, in pr actice, we can c ho ose τ ≥ 2 k D k ∞ + λ √ 15 3 k D k F λ (1 − λ ) , to guarantee th e exact sparse lo w-rank matrix decomp osition b y solving the strongly con v ex opti- mization (6). Similar to Corollary 1 for exact matrix completion, we h a v e an analog lo w er b ound estimate based on th e obs erv ed data matrix for exact lo w-rank and sparse matrices decomp osition. 13 Corollary 2. Assume τ ≥ 2 k D k ∞ + λ √ 15 3 k D k F λ (1 − λ ) . Then, under the assumptions of The or em 2, ( M , S 0 ) is the uniq ue solution to the str ongly c onvex pr o gr amming (6) with pr ob ability at le ast 1 − cn − 10 (1) , wher e c is the numeric al c onstant in The or em 2. 4 Conclusion In th is pap er, w e ha v e sho wn that str ongly co nv ex optimizations can lead to exact low-rank m atrix reco v ery in b oth the matrix completion pr oblem and robust prin ciple comp onen t a nalysis under suitable conditions. Explicit lo w er b ounds for the parameters τ inv olv ed are giv en. Th ese results are complemen tary to the results in [6, 7, 9], w here con v ex optimizations are sho wn to lead to the exact lo w-rank matrix reco v ery . W e would like to p oint out that the com bination of the MC and RPCA problems, i.e., matrix completion from grossly corru pted data [9] h as b een mo d eled as minimize : k L k ∗ + λ k S k 1 sub ject to : P Ω ( L + S ) = Y . (44) With the tec hnique discus s ed in this pap er, one can mo dif y (44) to its strongly con v ex counterpart that can guarantee the exact reco v ery as well. Note that all mo dels here assum e that the observed data are exact. Ho w ev er, in an y r eal w orld application, one only hav e observ ed d ata corrupted at least by a sm all amoun t of noise. Th erefore, a p ossible dir ection f or fur ther stud y is to consid er the case of noisy data [8] in matrix completio n and robus t prin ciple comp on ent analysis. Ac kno wledgemen t The work of Hui Zhang h as b een supp orted b y the C hinese Sc holar Council d uring his visit to Rice Univ ersit y . The w ork of Lizhi Cheng an d Jub o Zhu has b een supp orted b y the National Science F oun dation of Ch ina (No.610721 18 and No. 61002024). References [1] A. Ar gyriou, T. E vgeniou, and M. Pontil, Multi-task feature lear ning, Adv ances in neural infor mation pro cessing systems (NIPS), 2007 . [2] J.-F. Ca i, E.J. Cand` es, and Z. Shen, A sing ular v alue thresholding alg orithm for matrix co mpletion, SIAM J. On Optimization, 20(20 10), pp. 195 6–198 2. [3] J.-F. Cai, S. Osher, and Z. Shen, Linearized Bregman Iter ations for C o mpressed Sensing, Math. Co mp., 78(200 9), pp. 1515 –1536 . [4] J.-F. Cai, S. O sher, a nd Z. Shen, Conv erge nc e of the Lineariz e d Breg man Iteration for ℓ 1 -Norm Mini- mization, Math. Comp., 78(200 9), pp. 21 27–21 36. 14 [5] J.-F. Cai, S. Osher and Z. Shen, Linearized Br e g man Itera tions for F r ame-Based Image Deblurring, SIAM J. Imaging Sciences, 2(2009 ), pp. 226 –252. [6] E.J. Cand` es, B . Rech t, Exa ct matrix completion via con vex optimization. F oundations of Computational Mathematics, 9(6)(2009 ), pp. 717 -772. [7] E.J. Cand ` es , T. T a o, The Pow er of Con vex Relaxatio n: Near-Optimal Matrix Completion, IEEE T ra ns- actions on Information Theory , 56(201 0), pp. 2 0 53–20 86. [8] E.J. Cand` es, Y. P lan, Matrix completion with noise, Pro ceeding of the IEEE , 98 (2 010), pp. 92 5–936 . [9] E.J. Cand` es, X. Li, Y. Ma, and J. W right, Robust principal comp onent analysis?, Jo ur nal of ACM, 58(1)(201 1), pp. 1–37 . [10] E.J . Cand` es, T. T ao , Deco ding by linear progr a mmingming, IE EE T ra nsactions on Informatio n Theor y , 51(12)(20 06), pp. 420 3–421 5. [11] E.J . Cand ` es , J. Romber g, and T. T ao, Robust uncertaint y principles: exact sig nal reconstruction from highly incomplete frequency information. IEE E T r ans. Inform. Theory , 52(2)(200 6), pp. 489 – 509. [12] V. Chandras ekharan, S. Sanghavi, P . P ar illo, and A. Wilsky , Rank-s parsity incoherence for matrix decomp osition, SIAM Journal on Optimization, 21(2 )(2011), pp. 57 2–596 . [13] S.S. Chen, D.L. Donoho, and M.A. Saunders, At omic decompos ition b y ba s is pursuit, SIAM J.Sci.Comput., 20(1998 ), pp. 33 – 61. [14] D.L. Donoho , Compressed s ensing, IEEE T rans. Inform. Theory , 52(4 )(2006), pp. 12 89–13 06. [15] M. F azel, Ma tr ix rank minimizatio n with applications. Ph.D. thesis, Stanford University (2002). [16] M. P . F riedlander , and P . Tseng, Exa ct r egulariza tio n of convex progra ms, SIAM J. O ptim., 18(2007 ), pp. 1326– 1350. [17] H. Gao , J .-F. Cai, Z. Shen, and H. Zha o, Robust principal comp onent a nalysis-bas e d four-dimensional computed tomography , Physics in Medicine and Biology , 56(1 1)(2011), pp. 318 1–319 8. [18] D. Gross, Recovering lo w-rank matrices from few co e ficien ts in any basis, IE EE T rans. Infor m. Theor y , 57(201 1), pp. 1548 –1566 . [19] H. Ji, S. Huang, Z. Shen, and Y. Xu, Ro bust video r estoration by joint sparse and low rank matrix approximation, SIAM Jo ur nal o n Imag ing Science s , 4 (4)(2011), pp. 1122– 1142. [20] H. Ji, C. Liu, Z.W. Shen, and Y. Xu, Robust Video Denoising using Low Ra nk Matrix Completion, IEEE Conf. on Computer Vision and Pattern Recognitio n (CVPR), Sa n F rancisco, (2010), pp. 1791 – 1798. [21] Z. Liu, L. V andenber ghe, Interior-p oint metho d for n uclear norm approximation with application to system identification. SIAM Jo urnal on Matr ix Analysis and Applications, 3 1(3)(2009 ), pp. 1 235–1 256. [22] Z. Lin, A. Ganes h, J. W right, L. W u, M. Chen, and Y. Ma, F as t c onv ex optimiza tion a lgorithms for exact recov ery of a co r rupted low-rank matrix, T echnical Repor t UILU-E NG-09-221 4, (2010). [23] Z. Lin, M. Chen, L. W u, and Y. Ma , The augmented Lagr ange m ultiplier metho d for exact r ecov ery of corrupted low-rank matrices , T e chnical Repo rt UILU-ENG-09-22 15, (2010). [24] K. Min, Z.D. Zha ng , J. W ritht, and Y. Ma, Decomp osing background topic from keyw ords b y principle comp onent purs uit, Pro c eeding of the 19th A CM in terational conference on Information and k nowledge, (2010), pp. 269–2 78. [25] J. Meng, W. Yin, E. Houssain, and Z. Han, Co lla bo rative Spectrum Sens ing from Sparse O bserv ations using Ma trix Completio n for Co gnitive Radio Netw ork s , In P ro ceedings of ICASSP , (2010 ), pp. 3114– 3117. 15 [26] S. Ma, D. Goldfarb, and L. Chen, Fixed p oint a nd Bregman iter ative metho ds for ma trix ra nk mini- mization, Mathematical progr amming Ser ie s A, 12 8 (1)(2011), pp. 3 21–35 3. [27] M. Mesba hi, and G.P . Papav ass ilop oulos, On the r ank minimization problem ov er a po sitive semidefinite linear matrix inequality , IE E E T r ansactions on Automatic Control, 42(2)(1 9 97), pp. 239–2 43. [28] Y. Nesterov, Smo oth minimization o f no n-smo oth functions, Math. Prog ram., Serie A, 103 (2005), pp. 127–1 52. [29] S. Os her, Y. Mao, B. Dong, a nd W. Yin, F ast linearized Breg man iteration for compressive se nsing and sparse denoising, Comm. in Math. Sciences, 1(201 0), pp. 93 –111. [30] B. Rech t, M. F a z el, and P . Parrilo, Gua ranteed minimum ra nk so lutions of matrix equations via nuclear norm minimization. SIAM Review, 52(3)(20 10), pp. 471 –501. [31] B. Rech t, W. Xu, and B. Hassibi, Necessary and sufficient conditions for success of the n uclear norm heuristic for rank minimization. In: Pro ceeding s of the 47th IEEE Conference on Decision and Control, (2008), pp. 3065– 3070. [32] B. Recht , W. Xu, and B. Hass ibi, Null space conditions and thr esholds for ra nk minimization, Mathe- matics progra mming, (2010), pp. 1–2 8. [33] B. Rech t, A simpler approach to matrix completion. J. Machine Learning Res., Pr eprint av aila ble at arXiv:091 0.0651 [34] R. T. Ro ck afella r , Con vex Analysis , Pr inceton University Pres s , Pr inceton, NJ, 1970. [35] K.C. T oh, S. Y un, An accelerated proximal g radient algo rithm for nuclear norm regularize d least squa r es problems,Pacific Journa l of Optimiza tio n, 6 (2009), pp. 615–6 40. [36] C. T o masi, and T. K anade, Shap e and mo tio n from image strea ms under orthogr aphy: a factor ization metho d, In terna tional Jo urnal of Computer Vision, 9(2)(199 2 ), pp. 13 7–154 . [37] J. W right, A. Ganesh, S. Rao , and Y. Ma, Robust principal comp onent analys is : Exact r ecov ery of corrupted low-rank matrices via conv ex optimiza tion. submitted to J o urnal of the ACM (20 09), av ailable at arXiv:0905 .0233 . [38] W. Yin, Analysis a nd gener a lizations of the lineariz e d Breg man method, SIAM J ournal on Image Science, 3(2)(2010), pp. 856– 8 77. [39] W. Yin, S. Osher, D. Goldfarb, and J . Da r bo n, Bregman iterative algorithms for l 1 minimization with applications to compressed sensing, SIAM J. Imaging Sciences, 1 (2008), pp. 14 3–168 . [40] Z.D. Zha ng, X. Lia ng , A. Ganesh, and Y. Ma, TIL T: T r ansform Inv aria n t Low-rank T extures, Computer Vision-ACCV, (2010 ), pp. 314 –328. [41] H. Zhang, L.Z. Cheng, P ro jected Landw eb er iter ation for matr ix completion, Journal of Computational and Applied Mathematics, 235 (2 010), pp. 59 3–601 . [42] H. Zhang, L.Z. Cheng, and W. Zhu, A low er b ound guara n teeing ex act matrix completion via singular v alue thres holding algorithm, Applied and Computational Harmonic Analysis, 31 (2011), pp. 4 5 4–459 . [43] H. Zhang, L.Z. Cheng, a nd W. Z h u, Nuclear nor m r egulariza tion with lo w-r ank co nstraint for ma tr ix completion, Inv erse P roblems, 26(2010 ) 115 009(15pp). 16