Necessary and Sufficient Null Space Condition for Nuclear Norm Minimization in Low-Rank Matrix Recovery

Necessary and Sufficien t Null Space Co ndit ion for Nuclear Norm Minim i zation in Lo w-R ank Matrix Reco v ery Jirong Yi a ∗ W eiyu Xu c † F ebruary 15, 2018 Abstract Lo w-rank matrix reco very has found many applications in science and engineering su c h as mac hin e learning, signal processing, collab orative filtering, system iden tification, and Euclidean em b ed ding. But the low -rank matrix recov ery problem is an N P hard problem and thus c hal- lenging. A commonly used heuristic approac h is th e nuclear norm minimization. In [ 12 , 14 , 15 ], the authors established the n ecessary and sufficient null space conditions for nuclear norm min- imization to reco ver every p ossible lo w-rank matrix with rank at most r (the strong null space condition). I n addition, in [ 12 ], Oymak et al. established a null space condition for successful reco very of a given lo w-rank matrix (the weak null space condition) using nuclear norm mini- mization, and deri ved th e p h ase transition for the nuclear norm min imization. In this pap er, w e show that the weak null sp ace condition in [ 12 ] is only a sufficient condition for successful matrix recov ery using nuclear n orm minimization, and is not a necessary condition as claimed in [ 12 ]. In this p ap er, we further give a wea k n u ll space cond ition for low-rank matrix recov ery , whic h is both necessary an d sufficient for the success of nuclear norm minimization. At the core of our deriv ation are an ineq ualit y for characterizing the n u clear norms of blo ck matrices, and the cond itions for equalit y to hold in that inequality . Keywords: Null space condition, r ank minimization, nuclear no rm minimization, nuclear norm, blo ck matrices. 1 INTR ODUCTION The problem of reconstructing matrices from limited observ ations has attracted significant attention in many areas such as machine lear ning [ 1 , 4 , 5 ], computer vision [ 17 ], and phaseless signal recovery [ 3 ]. V ery often w e dea l with the pr oblem of finding a low-rank matrix consistent with existing observ ations, known as the rank minimization pr o blem (RMP) [ 8 , 13 ]. Let X ∈ R n 1 × n 2 be the ground tr uth low-rank matrix, and we o bserve X through a linear mapping A : R n 1 × n 2 → R m . Sup p os e the measuremen t result is b = A ( X ), then the RMP can be ∗ a Departmen t of Electrical and Computer Engineering, Universit y of Iow a, Io wa City , IA 52242. † c Departmen t of Electrical and Computer Engineering, Universit y of Iow a, Iow a Ci t y , IA 52242. Email: w eiyu-xu@uiowa.e du 1 formulated as follows: minimize rank( Y ) (1) sub ject to A ( Y ) = b. (2) Because the RMP is an NP -hard problem, resear chers often rela x it to the nu cle a r no r m minimiza- tion (NNM): minimize k Y k ∗ sub ject to A ( Y ) = b, (3) where k · k ∗ is the nu cle a r no r m, na mely the sum of the singular v alues of a matrix. Ov er the years, there have b een a large volume of research results on nuclear norm minimization ( 5 ), including deriving reco very pe r formance guarantees [ 4 , 5 , 13 ] and des igning numerical metho ds for solving it [ 1 , 9 – 11 ]. The proper ties of A play an im p or tant role in establishing the recov ery guarant ees of n uclear norm minimization. The r e s tricted isometry pr op erty (RIP) fo r A is o ften used to prov e per formance guarantees of the nuclear nor m minimization [ 4 , 13 ]. How ever, the RIP is o nly a suffcien t but not a necess ary conditio n for the s uccess o f nuclear norm minimization [ 15 ]. In [ 14 , 15 ], the authors characterized the necessa ry and sufficient n ull space condition for suc- cessful reconstr uction of every g round tr uth matr ix with rank no more than r (the stro ng null spa ce condition). In [ 12 ], Oymak et a l. o bta ined a more concise a nd verifiable neces sary and sufficient n ull space condition in the strong s ense. The authors established a null space condition fo r successful recov er y o f a given low-rank matrix (the weak null spa ce co ndition) in [ 12 ]. Compared with the strong null space condition, the weak null spac e co ndition is a conditio n on the null space of A such that a particular ground truth ra nk- r matrix can be successfully r ecov ered using n uclear norm minimization. In [ 12 ], it w a s claimed tha t the weak null space condition discov ered in [ 12 ] w as b o th a necessary a nd sufficient condition for re cov ering a particular rank- r matrix using nuclear nor m minimization. How ever, in this pap er , we show that the weak null space condition pro p o sed in [ 12 ] is not a nec essary condition for the success of nuclear no rm minimization in low-rank matrix recov ery . F urthermo re, we provide a new deriv ation which gives a true neces s ary and sufficient weak null space condition for the nuclear norm minimization. A t the cor e of our deriv ation are a n inequality for characterizing the n uclea r norms of blo ck matrices, and the conditions for eq uality to ho ld in that ineq uality . This paper is orga nized as follows. In Section 2 , we formulate the matrix recovery pro blem a s a nu cle a r norm minimization problem. W e provide a counterexample to illustr a te that the weak n ull space condition in [ 12 ] is not a necessar y condition. In Section 3 , we give formal statements of our main theorems and their pr o ofs. W e a lso g ive key lemmas needed for proving the ma in results. W e present the pro ofs of these lemmas in App endix. Notations : In this pap er, we denote the nuclear norm of a matrix X by k X k ∗ = P i =1 σ i ( X ), where σ i ( X ) is the i -th largest singular v alue of X . The b old 0 is used to denote all-zero vectors or a ll-zero matrices, and its dimension dep ends on the co nt ex t. The trace o f a matrix X is denoted by T r ( X ), a nd the transp o se and conjugate tra nsp ose of a matr ix X ar e denoted by X T and X ∗ resp ectively . 2 2 PRELIMINARIES Let X ∈ R n 1 × n 2 ( n 1 ≤ n 2 ) b e of rank r ≤ min { n 1 , n 2 } , A : R n 1 × n 2 → R m be a linear measrue- men t op erator , a nd b = A ( X ) ∈ R m be the measur ement vector. W e co ns ider the following r a nk minimization problem: minimize rank( Y ) sub ject to A ( Y ) = b. (4) Recently , in [ 12 ], Oyma k et al. prop osed the following weak null space condition as a “necessar y” and sufficient condition for the nuclear norm minimizatio n ( 5 ): minimize k Y k ∗ sub ject to A ( Y ) = b. (5) W eak null space conditi on from [ 12 ]: Let X ∈ R n × n be a matrix with rank r , a nd let its singular v alue decomp osition (SVD) b e X = U Σ V T with Σ ∈ R r × r . Then X will be the unique solution to ( 5 ) if and only if for all nonz e ro W ∈ N ( A ), we hav e T r( U T W V ) + k ¯ U T W ¯ V k ∗ > 0 , (6) where ¯ U a nd ¯ V are chosen such that [ U ¯ U ] and [ V ¯ V ] are unitar y , and N ( A ) is the null space of A . How ever, we discov er that the ab ove weak null space condition fr om [ 12 ] is only a sufficient but not neces sary condition for the succes s of nuclear norm minimization. Here we pr esent a s imple counterexample wher e ( 6 ) is violated, but the nuclear norm minimization still succeeds in recov er ing the gr ound tr uth matrix X . T o simplify presentation, we first use a counterexample in the field of r eal num b ers (wher e every element in the null space is a real-num b ered matrix ) to illustrate the idea. Building on this real-num b ered example, we further give a counterexample in the field of complex num b ers. Suppo se X =  − 1 0 0 0  , Q =  1 1 1 1  . (7) W e also a ssume that the linear ma pping A is suc h that tQ ( t 6 = 0 ) is the only type of no nzero elements in the null s pace of A . Then the so lution to ( 5 ) must b e of the for m X + tQ , where t is any real num b er. Let the singular v alue deco mpo sition o f X be X = U Λ V ∗ , where U =  1 0  , V =  − 1 0  . (8) Define ¯ U and ¯ V as ¯ U =  0 1  , ¯ V =  0 1  . (9) One ca n chec k that, fo r this exa mple, −| T r( U ∗ QV ) | + k ¯ U ∗ Q ¯ V k ∗ = 0 . (10) 3 How ever, w e will show that || X + tQ || ∗ > 1 , ∀ t 6 = 0 , (11) implying that X is the unique solution to ( 5 ). In fact, we calculate B = ( X + tQ )( X + tQ ) T =  ( − 1 + t ) 2 + t 2 ( − 1 + t ) t + t 2 ( − 1 + t ) t + t 2 2 t 2  , (12) and then the singular v alue s of X + tQ ar e the squar e ro ots of the eigenv alues of B . The eigenv alues of B can b e obtained by solving for λ using det( B − λI ) = 0 , (13) where I is the ident ity ma tr ix and det( · ) is the determinant. This results in λ = a ( t ) + b ( t ) ± p ( a ( t ) − b ( t )) 2 + 4 c ( t ) 2 , (14) where a ( t ) = ( − 1 + t ) 2 + t 2 , b ( t ) = 2 t 2 , c ( t ) = (( − 1 + t ) t + t 2 ) 2 . (15) Thu s the tw o eig env a lues of B a re λ 1 = 4 t 2 − 2 t + 1 + √ 16 t 4 − 16 t 3 + 8 t 2 − 4 t + 1 2 , (16) λ 2 = 4 t 2 − 2 t + 1 − √ 16 t 4 − 16 t 3 + 8 t 2 − 4 t + 1 2 , (17) and the singular v alues of X + tQ are σ 1 = p λ 1 = s 4 t 2 − 2 t + 1 + √ 16 t 4 − 16 t 3 + 8 t 2 − 4 t + 1 2 , (18) σ 2 = p λ 2 = s 4 t 2 − 2 t + 1 − √ 16 t 4 − 16 t 3 + 8 t 2 − 4 t + 1 2 . (19) After s o me alg ebra, we get k X + tQ k ∗ = σ 1 + σ 2 = ( √ 4 t 2 + 1 , t ≥ 0 , 1 − 2 t, t < 0 . (20) This means k X + tQ k ∗ is alwa ys grea ter than 1 for t 6 = 0, showing X is the uniq ue s o lution to the nu cle a r no r m minimization. But −| T r( U ∗ QV ) | + k ¯ U ∗ Q ¯ V k ∗ ≯ 0 . Now we give a coun terexa mple in the field of complex nu mber s, where the null space of A contains complex-num b ered matrices. Supp ose that we have the sa me matrices X and Q . Then the so lution to ( 5 ) must b e of the for m X + tQ , where t is any complex num be r . Without lo ss of 4 generality , let us take t = − ae − ıθ , wher e a ≥ 0 is a nonnegative rea l n umber , θ is a ny rea l nu mber betw ee n 0 and 2 π , and ı = √ − 1. W e fur ther denote B = ( X + tQ )( X + tQ ) ∗ . Then by calculating the eig env a lues of B , we obtain that k X + tQ k ∗ = σ 1 + σ 2 = p 4 a 2 + 2 a (1 + cos( θ )) + 1 (21) = s 4  a + 1 + cos( θ ) 4  2 + 1 − (1 + co s( θ )) 2 4 . (22) So k X + tQ k ∗ > 1, if a 6 = 0 (namely t 6 = 0), implying tha t the n uclea r norm minimiza tion can uniquely recov er s X even though − | T r( U ∗ QV ) | + k ¯ U ∗ Q ¯ V k ∗ ≯ 0 . Our co unterexample ra ises the following question: what is a necessar y and sufficient weak null space condition for the succes s o f nuclear norm minimization? In the next s ection, we will answer this q uestion. 3 NECESSAR Y AND SUFFICIENT NU LL SP ACE CON- DITION In this section, we give a nec e s sary and s ufficie nt n ull spa c e c o ndition for successful recovery of the ground tr uth matrix using nuclear norm minimiza tion. O ur main results a re stated in Theo rems 3.1 and 3.5 . T o s implify our presentations, in Theorem 3.1 and its proof, we o nly consider the case of X being a s quare real-num b er e d matrix. In Theorem 3.5 , w e gener alize our results to complex-num b ered non-squar e ma trices witho ut pro of. Theorem 3. 1. Consider a r ank- r matrix X ∈ R n × n , and let its singular value de c omp osition b e X = U X Λ X V ∗ X . Su pp ose t hat we observe b = A ( X ) , and u se the nucle ar norm minimization to r e c over the matr ix . F or any matrix Q ∈ R n × n , we define a m atr ix Q ′ =  A ′ B ′ C ′ D ′  ∈ R n × n satisfying Q = [ U X U X ] × Q ′ × [ V X V X ] ∗ , (23) wher e [ U X U X ] and [ V X V X ] ar e un itary matric es in R n × n , A ′ ∈ R r × r and D ′ ∈ R ( n − r ) × ( n − r ) . We let the singular value de c omp osition of D ′ b e D ′ = U D ′ Λ D ′ V ∗ D ′ . We further define a matrix Q ′′ =  A ′′ B ′′ C ′′ D ′′  ∈ R n × n such t hat Q ′ =  I r × r 0 0 0 U D ′ U D ′  Q ′′  I r × r 0 0 0 V D ′ V D ′  ∗ , (24) wher e [ U D ′ U D ′ ] and [ V D ′ V D ′ ] ar e unitary matric es in R ( n − r ) × ( n − r ) , A ′′ ∈ R r × r and D ′′ ∈ R ( n − r ) × ( n − r ) . Then X is the unique solution to nu cle ar norm minimization if and only if every nonzer o element Q ∈ R n × n fr om the nul l s p ac e of A satisfies one of t he fol lowing two c onditions: (1) T r( U T X QV X ) + k U X T QV X k ∗ = 0 . (25) 5 Mor e over, Q ′′ satisfies at le ast one of the fol lowing c onditions: a) the ro w sp ac e of B ′′ is n ot a subsp ac e of t he r ow sp ac e of D ′′ ; b) the c olumn sp ac e of C ′′ is not a subsp ac e of the c olumn sp ac e of D ′′ ; c) Q ′′ is not a symmet ric matrix. (2) T r( U T X QV X ) + k U X T QV X k ∗ > 0 . (26) Our pro of for Theo rems 3.1 and 3.5 dep ends on the characterization of the sub differential for the nu cle a r norm of rea l-num be r ed and complex-num b ered matrice s [ 2 , 18 , 2 0 ]. Next, we give Lemmas 3.2 , 3 .3 , a nd 3.4 which pla y an imp ortant role in establis hing Theor em 3.1 . Lemma 3.2 . L et us assume that X =  A B C D  is a squar e matrix in R n × n , wher e A ∈ R m × m and D ∈ R ( n − m ) × ( n − m ) . L et A = U A Λ A V ∗ A and D = U D Λ D V ∗ D b e singular value de c omp ositions of A and D r esp e ct ively. L et X = U X Λ X V ∗ X b e the singular value de c omp osition of X . Then it always holds that      A B C D      ∗ ≥      A 0 0 D      ∗ . (27) Mor e over, the e qu ality holds if and only if ther e exists a matrix Q such t hat Q ∗ Q = I , U X =  U A 0 0 U D  Q, (28) and V X =  V A 0 0 V D  Q. (29) W e r emark that in [ 14 , 15 ], the a utho r s established ( 27 ) witho ut sp ecifying the conditions under which ( 27 ) takes strict inequality or equality . Lemma 3.3. L et X =  A B C D  b e a squar e matrix in R n × n , wher e A ∈ R m × m and D ∈ R ( n − m ) × ( n − m ) . L et X = U X Λ X V ∗ X b e the singular value de c omp osition of X , and let A = U A Λ A V ∗ A and D = U D Λ D V ∗ D b e the singular value de c omp osition of A and D . L et [ U A U A ] and [ V A V A ] b e unitary matric es in R m × m , and let [ U D U D ] and [ V D V D ] b e unitary matric es in R ( n − m ) × ( n − m ) . L et us define E =  U A U A 0 0 0 0 U D U D  ∗ ×  A B C D  ×  V A V A 0 0 0 0 V D V D  . (30) Then      A B C D      ∗ =      A 0 0 D      ∗ (31) 6 if and only if the fol lowing c onditions hold simult ane ously: (1) t he r ow sp ac e of X is a subsp ac e of the r ow sp ac e of  V A 0 0 V D  ∗ ; (2) the c olumn sp ac e of X is a subsp ac e of the c olumn sp ac e of  U A 0 0 U D  ; (3) E is a symmetric p ositive semidefinite matrix. Lemma 3 .4. L et A b e an n × n matrix. Then k A k ∗ ≥ n X i =1 | A i,i | . (32) Mor e over, k A k ∗ = n X i =1 A i,i (33) if and only if A is a symmstric p ositive semidefinite matrix. W e note that Lemma 3.4 improves ov er Lemma 3 in [ 12 ]. With the lemmas abov e, w e now present the pro of for Theorem 3.1 . Pr o of for The or em 3.1 : W e define a function f ( X ) = k X k ∗ . W e lo ok at three cases for the sig n of T r( U T X QV X ) + k U X T QV X k ∗ : a ) negative, b) p ositive, and c) b eing equa l to 0 . Case a) : W e first assume that, for a certain nonzero Q from the null space of A , T r( U T X QV X ) + k U X T QV X k ∗ < 0 . (34) Because T r( U T X QV X ) + k U X T Q V X k ∗ = sup W ∈ ∂ k X k ∗ < W, Q > , (35) we ha ve sup W ∈ ∂ k X k ∗ < W, Q > < 0. F ro m Theorem 2 3.4 in [ 16 ], we know that f ′ ( X ; Q ) = sup W ∈ ∂ k X k ∗ < W, Q > < 0 , (36) where f ′ ( X ; Q ) = inf t> 0 k X + tQ k ∗ − k X k ∗ t (37) is the one- sided dir ectional deriv ative defined in [ 16 ]. Then there must e xist a p ositive num b er λ such that k X + λQ k ∗ − k X k ∗ < 0, namely Q is a direction alo ng which we can reduce the nuclear norm o f X . This shows that X canno t b e the s olution to the nuclear norm minimization problem, if T r( U T X QV X ) + k U X T QV X k ∗ < 0 for a certain nonzero Q from the null space of A . 7 Case b): Now w e instead assume that, a nonzero matrix Q from the null space of A sa tisfies T r( U T X QV X ) + k U X T Q V X k ∗ > 0. W e know that the sub differential ∂ f ( X ) o f f ( X ) is g iven by the set of matrice s in the for m of U X V T X + U X M V X T , wher e M is any matrix (o f appropr iate dimensio n) with spectr a l norm no more than 1. Then by the definition of sub differential, we hav e k X + Q k ∗ ≥ k X k ∗ + sup B ∈ ∂ f ( X ) h Q, B i (38) = k X k ∗ + T r( U T X QV X ) + k U X T QV X k ∗ (39) > k X k ∗ , (40) where we use the fact that the dual nor m o f s pec tr al norm is the nuclear nor m. This implies that X + tQ ca nnot be the solution to nuclear norm minimization for t 6 = 0 and any nonzero Q fro m the nu ll spa ce of A s atisfying T r( U T X QV X ) + k U X T QV X k ∗ > 0. Case c:) Now, we only need to consider the case T r( U ∗ X QV X ) + k U X ∗ QV X k ∗ = 0. W e know that a ny matrix Y satisfying b = A ( X ) must b e of the fo r m Y = X + tQ, (41) where t > 0, and Q is a nonzero element in the null space of A . Let us consider a matrix Q in the null space o f A such that T r( U T QV ) + k ¯ U T Q ¯ V k ∗ = 0 . (42) Let [ U X U X ] and [ V X V X ] b e unitary matrices in R n × n . T he n we can express Q a s Q = [ U X U X ] × Q ′ × [ V X V X ] ∗ , (4 3) where Q ′ ∈ R n × n . W e write Q ′ as a blo ck matrix Q ′ =  A ′ B ′ C ′ D ′  , (44) where A ′ ∈ R r × r and D ′ ∈ R ( n − r ) × ( n − r ) . W e let the singular v alue decomp osition of D ′ be D ′ = U D ′ Λ D ′ V ∗ D ′ . Then we further expres s Q ′ as Q ′ =  I r × r 0 0 U D ′ U D ′  × Q ′′ ×  I r × r 0 0 V D ′ V D ′  ∗ (45) where Q ′′ ∈ R n × n , and [ U D ′ U D ′ ] a nd [ V D ′ V D ′ ] a re unitar y matr ic e s in R ( n − r ) × ( n − r ) . W e can write Q ′′ as a blo ck matrix Q ′′ =  A ′′ B ′′ C ′′ D ′′  (46) 8 where A ′′ ∈ R r × r , D ′′ ∈ R ( n − r ) × ( n − r ) . Mo reov er , D ′′ =  Λ D ′ 0 0 0  ∈ R ( n − r ) × ( n − r ) . (47) F ro m the singular v alue decomp osition of X , we also have X =  U X U X  ×  I r × r 0 0 U D ′ U D ′  ×  Λ X 0 0 0  ×  I r × r 0 0 V D ′ V D ′  ∗ ×  V X V X  ∗ . (48) Then Y = X + tQ =  U X U X  ×  I r × r 0 0 U D ′ U D ′  × Y ′′ ×  I r × r 0 0 V D ′ V D ′  ∗ ×  V X V X  ∗ (49) where Y ′′ =  Λ X 0 0 0  + t  A ′′ B ′′ C ′′ D ′′  . (50) This means that Y ′′ and Y ha ve the same nuclear nor m. So in or der for us to see whe ther there exists a t > 0 such that k Y k ∗ = k X k ∗ , we only need to see w hether k Y ′′ k ∗ = k X k ∗ . W e obser ve that k Y ′′ k ∗ =      Λ X 0 0 0  + t  A ′′ B ′′ C ′′ D ′′      ∗ . (51) F urthermo re, b ecause T r( U T QV ) + k U T QV k ∗ = 0, we hav e r X i =1 A ′′ i,i + k D ′′ k ∗ = 0 . (52) According to Lemma 3.4 , we first no te tha t k Λ X + tA ′′ k ∗ ≥ P r i =1 | (Λ X ) i,i + tA ′′ i,i | ≥ P r i =1 ((Λ X ) i,i + tA ′′ i,i ). Mo reov er , by Lemma 3.2 , we ha ve      Λ X 0 0 0  + t  A ′′ B ′′ C ′′ D ′′      ∗ ≥ k Λ X + tA ′′ k ∗ + t k D ′′ k ∗ ≥ k Λ X k ∗ + t r X i =1 A i,i + t k D ′′ k ∗ = k Λ X k ∗ = k X k ∗ . 9 Thu s if for a certain t > 0, k Y ′′ k ∗ = k X k ∗ , we must hav e      Λ X 0 0 0  + t  A ′′ B ′′ C ′′ D ′′      ∗ =      Λ X + tA ′′ 0 0 tD ′′      ∗ =      Λ X + t diag( A ′′ ) 0 0 tD ′′      ∗ , (53) where diag( A ′′ ) is a diag onal ma trix ha ving sa me dia gonal as A ′′ , and Λ X + t dia g( A ′′ ) has nonneg- ative diagonal elements. By Lemma 3.4 , ( 53 ) holds and Λ X + t diag( A ′′ ) has nonnega tive diagonal elements, if a nd o nly if  Λ X 0 0 0  + t  A ′′ B ′′ C ′′ D ′′  (54) is a symmetr ic p ositive semidefinite matrix for s o me t > 0 . W e now prov e that, there exists t > 0 s uch tha t  Λ X 0 0 0  + t  A ′′ B ′′ C ′′ D ′′  (55) is a symmetric p ositive semidefinite ma tr ix if and only if 1) the r ow space of B ′′ is a subspace of the r ow space of D ′′ ; 2) the column spa ce o f C ′′ is a subspace of the c o lumn space of D ′′ ; 3)  A ′′ B ′′ C ′′ D ′′  is a symmetric matrix. T o see this, we first assume these thr ee conditions ar e sa tis fie d. Then when t > 0 is small enough,  Λ X 0 0 0  + t  A ′′ B ′′ C ′′ D ′′  m ust b e a symmetric p ositive semidefinite matr ix. This is b eca use tD ′′ = t  Λ D ′ 0 0 0  is a p o sitive semidefinite matrix, the row spa ce of tB ′′ is a subspa ce of the row s pa ce of tD ′′ , the c o lumn spa ce of tC ′′ is a subspace of column space of tD ′′ , and the Sch ur complement Λ X + tA ′′ − ( tB ′′ )( tD ′′ ) † ( tC ′′ ) must b e pos itive semidefinite when t is sufficien tly small (noting that Λ X is a full-rank p ositive semidefinite matr ix ), wher e ( tD ′′ ) † is the Moor e-Penrose inv erse of ( tD ′′ ). Now w e show that if either one of the three conditions fails, then for every t > 0 ,  Λ X 0 0 0  + t  A ′′ B ′′ C ′′ D ′′  (56) is not a sy mmetric p ositive semidefinite ma tr ix. This is b eca use, by Sch ur c omplement criterio n,  Λ X 0 0 0  + t  A ′′ B ′′ C ′′ D ′′  is a symmetric p os itive semidefinite ma tr ix, only if 1) the ro w space of tB ′′ is a subspace o f the row space of tD ′′ ; 2) the column space of tC ′′ is a subspace o f the column space of tD ′′ ; a nd 3 )  A ′′ B ′′ C ′′ D ′′  is a symmetric matrix. 10 If either of the three c onditions fails , we hav e      Λ X 0 0 0  + t  A ′′ B ′′ C ′′ D ′′      ∗ > k Λ X + tA ′′ k ∗ + t k D ′′ k ∗ ≥ k Λ X k ∗ + t r X i =1 A i,i + t k D ′′ k ∗ = k Λ X k ∗ = k X k ∗ . Thu s if either of these three conditions fails, the the ground truth X will be the unique so lution to the nuclear norm minimization. Combining Cases a), b) and c), we hav e prov ed Theor em 3.1 .  Now we extend our res ults to non-sq uare complex- n umber ed matrices, leading to Theorem 3.5 . W e remark that our pro of of Theorem 3.5 depends on the characterizatio n of the sub differential for the nuclear no rm of complex-num b ered matr ices [ 2 , 18 , 20 ]. Other than that, our pro of of Theore m 3.5 fo llows the same line o f r easoning as in the pr o of of Theor em 3.1 , and so we omit its pr o of. Theorem 3.5 . C onsider a r ank- r matrix X ∈ C m × n ( m ≤ n ) , and let its singular value de c omp o- sition b e X = U X Λ X V ∗ X . Su pp ose that we observe A ( X ) , and use the nucle ar norm minimization to r e c over the matrix. F or any matrix Q ∈ C m × n , we define a matrix Q ′ =  A ′ B ′ 0 C ′ D ′ 0  ∈ C m × n satisfying Q = [ U X U X ] × Q ′ × [ V X V X ] ∗ , (57) wher e [ U X U X ] and [ V X V X ] ar e u nitary matric es in C m × m and C n × n r esp e ctively, and A ′ ∈ C r × r and D ′ ∈ C ( m − r ) × ( m − r ) . We let the singular value de c omp osition of D ′ b e D ′ = U D ′ Λ D ′ V ∗ D ′ . We further define Q ′′ =  A ′′ B ′′ 0 C ′′ D ′′ 0  ∈ C m × n such t hat Q ′ =  I r × r 0 0 U D ′ U D ′  × Q ′′ ×   I r × r 0 0 V D ′ V D ′ 0 0   ∗ , (58) wher e [ U D ′ U D ′ ] and [ V D ′ V D ′ ] ar e unitary matric es in C ( m − r ) × ( m − r ) , A ′′ ∈ C r × r and D ′′ ∈ C ( m − r ) × ( m − r ) . Then X is t he u nique solution to the nu cle a norm minimization if and only if every nonzer o element Q ∈ C m × n fr om t he nu l l sp ac e of A satisfi es one of the fol lowing c onditions: 1) Re { T r( U ∗ X QV X ) } + k U X ∗ QV X k ∗ = 0 . (59) Mor e over, Q ′′ satisfies at le ast one of the fol lowing c onditions: a) the ro w sp ac e of B ′′ is n ot a subsp ac e of t he r ow sp ac e of D ′′ ; b) the c olumn sp ac e of C ′′ is not a subsp ac e of the c olumn sp ac e 11 of D ′′ ; c)  A ′′ B ′′ C ′′ D ′′  is not a H erm itian matrix. 2) Re { T r( U ∗ X QV X ) } + k U X ∗ Q V X k ∗ > 0 . (60) 4 Comparisons with the w eak n ull space condition for sparse reco ve ry W e would like to contrast the necessa ry a nd sufficient weak null space co ndition fo r nuclear norm minimization, with the necess ary a nd sufficient weak null space co ndition for recovery of sparse vectors using ℓ 1 minimization. F or the ℓ 1 minimization to successfully (uniquely) recover a s parse vector x ∈ R n whose supp ort is an index set K , the weak null spa c e c ondition for spa rse recovery [ 6 , 7 , 19 ] requir es that for e very no nzero element y in the null space of the linear op er a tor A , h sign( x K ) , y K i + k y K k 1 > 0 , (61) where K = { 1 , 2 , ..., n } \ K . W e rema r k that one do es r equire strict inequality in the conditio n ( 61 ), in contrast to o ur new null space condition for n uclea r no rm minimization where we hav e shown nuclear norm minimization c an still succeed even if e q uality holds in the null spa ce co nditio n equation in Theore m 3.1 . W e note that, one ca n a ls o deduce ( 6 1 ) directly from Theo rem 3.1 by sp ecializing X to b e a diagonal matrix with its diag onal elements cor resp onding to the elements o f the vector x . Because none of the three conditions for case “(1)” of Theo rem 3.1 holds, one must require strict ineq ua lity in case “2)” of T he o rem 3 .1 , which is equiv alent to ( 61 ). App endix 5 Pro of for Lemma 3.2 It is known that the dual norm o f nuclear norm is the sp ectral no rm. Using this fact, it ha s b een shown in [ 15 ] tha t ( 27 ) holds. No w we prov e the condition under which the equa lit y holds in ( 2 7 ). Suppo se that the equality in ( 27 ) ho lds . W e consider the ma trix P =  U A V ∗ A 0 0 U D V ∗ D  . (62) After s o me alg ebra, we know that      A 0 0 D      ∗ =  P,  A 0 0 D   . (6 3) Because we assume that      A B C D      ∗ =      A 0 0 D      ∗ , (64) 12 using the fact that the dual no rm of nuclear norm is sp ectral no rm, we hav e sup k Q k≤ 1  Q,  A B C D   =      A B C D      ∗ =  P,  A 0 0 D   =  P,  A B C D   . (65) Namely Q = P achiev es the supremum of  Q,  A B C D   ov er the set of Q ’s with sp ectral norm no mor e than 1. W e have      A B C D      ∗ = sup k Q k≤ 1  Q,  A B C D   = sup k Q k≤ 1 T r( Q ∗ U X Λ X V ∗ X ) = sup k Q k≤ 1 T r( V ∗ X Q ∗ U X Λ X ) = sup k Q k≤ 1 h U ∗ X QV X , Λ X i = sup k Q k≤ 1 n X i =1 σ i ( U ∗ X QV X ) ii = sup k Q k≤ 1 n X i =1 σ i U ∗ X,i : QV X,i : = n X i =1 σ i U ∗ X,i : P V X,i : ≤ n X i =1 σ i (66) where U X,i : and V X,i : are the i - th column of U X and V X resp ectively . By the Cauch y-Sch wartz inequality , the equality in ( 66 ) holds if and o nly if, for every i , the i - th column o f U X , and the i -th column of V X can b e written as U X,i : =  U A 0 0 U D  b, V X,i : =  V A 0 0 V D  b res pec tively , where b is a vector of unit energ y . Namely , if      A B C D      ∗ =      A 0 0 D      ∗ , then there exists a matrix Q such that Q ∗ Q = I , U X =  U A 0 0 U D  Q, (67) and V X =  V A 0 0 V D  Q. (68) On the other hand, if ( 67 ) and ( 6 8 ) hold, w e hav e Q = P =  U A V ∗ A 0 0 U D V ∗ D  achieving the supremum sup k Q k≤ 1  Q,  A B C D   through the sa me ar g uments in ( 66 ). This leads to      A B C D      ∗ =      A 0 0 D      ∗ . 13 6 Pro of for Lemma 3.3 Suppo se the r ank of A is r A (po ssibly sma lle r than m ), and the rank of D is r D (po ssibly sma lle r than ( n − m )). Then we hav e  A B C D  =  U A Λ A V ∗ A B C U D Λ D V ∗ D  =  U A U A 0 0 0 0 U D U D  ×     Λ A 0 0 0 B ′ C ′ Λ D 0 0 0     ×  V A V A 0 0 0 0 V D V D  ∗ (69) where [ U A U A ] and [ V A V A ] a re unitary matrices in R m × m , [ U D U D ] and [ V D V D ] are unitary matrices in R ( n − m ) × ( n − m ) , a nd B ′ and C ′ are such that [ U D U D ] C ′ [ V A V A ] ∗ = C, (70) [ U A U A ] B ′ [ V D V D ] ∗ = B . (71) Since we ar e pe r forming unitary tr ansformations , we ha ve      A B C D      ∗ =             Λ A 0 0 0 B ′ C ′ Λ D 0 0 0             ∗ = k E k ∗ . (72) Because      A 0 0 D      ∗ = k A k ∗ + k D k ∗ = k Λ A k ∗ + k Λ D k ∗ , to see whether      A 0 0 D      ∗ =      A B C D      ∗ , we only need to see w hether             Λ A 0 0 0 B ′ C ′ Λ D 0 0 0             ∗ =             Λ A 0 0 0 0 0 Λ D 0 0 0             ∗ = k Λ A k ∗ + k Λ D k ∗ . (73) Let the SVD of E be U E Λ E V ∗ E , then by Le mma 3.2 , the equality holds in ( 73 ), if and only if there exis ts a matrix Q such that Q ∗ Q = I , (74) U E =     I r A × r A 0 I r D × r D 0     Q, (75) 14 and V E =     I r A × r A 0 I r D × r D 0     Q, (76) where Q has only nonzero rows ov er the indices c o rresp onding to r A + r D nonzero co lumns of     I r A × r A 0 I r D × r D 0     . (77) Then we can write E =     I r A × r A 0 I r D × r D 0     Q Λ E Q ∗     I r A × r A 0 I r D × r D 0     ∗ . (78) W e c a n further see that s uch a matr ix Q ex is ts, if a nd only if the following three conditions hold simult a neously: 1) the row spa c e of E is a subs pa ce of the row s pace o f     I r A × r A 0 I r D × r D 0     ; (79) 2) the column space of E is a subspa ce of the c o lumn s pace o f     I r A × r A 0 I r D × r D 0     ; (80) 3) E is a symmetric p ositive semidefinite ma trix. This proves the claims in this lemma. 7 Pro of for Lemma 3.4 Consider the blo c k matrix A =  A 1:( n − 1) , 1:( n − 1) A 1:( n − 1) ,n A n, 1:( n − 1) A n,n  (81) where A I ,J corres p o nds to submatrix of A with row indices in I and column indices in J . 15 F ro m Lemma 3 .3 , we hav e k A k ∗ ≥ k A 1:( n − 1) , 1:( n − 1) k ∗ + | A n,n | . By applying inductions, we can obtain k A k ∗ ≥ n X i =1 | A i,i | . So k A k ∗ = P n i =1 A i,i only if the diago nal elemen ts of A ar e nonnegative. Moreov er , k A k ∗ = P n i =1 A i,i if a nd o nly if k A k ∗ =               A 1 , 1 A 2 , 2 . . . A n,n               ∗ . (82) This means that the nuclear norm o f A should b e equa l to nuclear norm of dia gonal matrix      A 1 , 1 A 2 , 2 . . . A n,n      . Through similar arguments as in the pro of of Theor em 3.2 (now extending to multiple blo c k s ), we kno w that k A k ∗ = P n i =1 A i,i if a nd only if A = U Λ U ∗ , wher e U is a unitary matrix, a nd Λ is a diagonal matrix with nonnega tive e lement s. Then A must b e s ymmetry (or Hermitian in the field of complex num b ers) p ositive semidefinite matrix. Moreov er , when A is a p o sitive semidefinite matrix , we must hav e k A k ∗ = P n i =1 A i,i . References [1] J. C a i, E. Cand` es, and Z. Shen. A Singular v alue thresho lding alg orithm for matrix completion. SIAM J. Optim. , 20(4):19 56–19 82, January 2010. [2] J. C a i, X. Q u, W. Xu, and G. Y e. Robus t recovery of complex exp onential signa ls from ra ndom Gaussian pro jections via low ra nk Hankel matr ix reconstructio n. Applie d and Computational Harmonic Analysis , 41(2):4 70–49 0, September 2 016. [3] E. Cand` es, Y. Eldar, T. Strohmer , and V. V o roninski. Phase retrie v al via matrix co mpletion. SIAM J. Imaging Sci. , 6 (1):199– 225, January 2013. [4] E. J. Cand` es and B. Rech t. Exact matrix completion via conv ex o ptimization. F oundations of Computational Mathematics , 9(6):7 17–77 2, December 2 009. [5] E. J . Cand` es and T. T ao. The p ow er of conv ex rela x ation: near- optimal matrix completion. IEEE T r ansactions on Information The ory , 56(5):20 53–2 080, May 201 0. 16 [6] D. Donoho. High- dimens ional cent r ally symmetric p olyto pes with neighbor liness prop ortional to dimension. Discr et e Comput Ge om , 35(4):61 7–652 , May 2006. [7] D. L. Dono ho and J. T anner . Neighbor line s s of r andomly pro jected simplices in high di- mensions. Pr o c e e dings of the National A c ademy of Scienc es of the Unite d States of Americ a , 102(27 ):9452– 9457, 2 005. [8] M. F az e l. Matrix r ank minimization with applic ations . PhD Thesis, Stanford Univ ersity , 2002 . [9] Y. Li, Y. Z ha ng, and Z . Huang. A reweighted nuclear norm minimiza tion a lgorithm for low rank matrix recovery . Journal of Computational and Applie d Mathematics , 2 63:338 –350 , June 2014. [10] Z. Liu and L. V andenberg he. In terior- p o int metho d for nu cle a r no rm appr oximation with a p- plication to system identification. SIAM. J. Matrix Anal. & Appl. , 31(3):12 35–12 56, Nov ember 2009. [11] S. Ma , D. Go ldfarb, and L. Chen. Fixed p oint and Bregman iterative metho ds for matrix rank minimization. Ma t h. Pr o gr am. , 128(1- 2):321– 353, June 201 1 . [12] Samet Oymak and Baba k Hassibi. N ew null space results a nd recovery thr e sholds for matrix rank minimization. arXiv pr eprint arXiv:1011. 6326 , 2010 . [13] B. Rec ht, M. F azel, and P . Parrilo. Guaranteed minim um-rank so lutio ns of linear matrix equations via nuclear norm minimization. SIAM R ev. , 5 2(3):471 – 501, January 2 0 10. [14] B. Rech t, W. Xu, and B. Has sibi. Nece ssary a nd s ufficient conditions for success of the nuclear norm heuristic for rank minimization. In Pr o c e e dings of 47th IEEE Confer enc e on De cision and Contr ol , pages 3065 –307 0, December 2 008. [15] B. Rech t, W. Xu, and B. Hassibi. Null space conditions and thresholds for r a nk minimization. Math. Pr o gr am. , 1 27(1):17 5–202 , March 2011. [16] R. Ro ck afellar. Convex analysis . Princeton university press, 201 5. [17] C. T omasi and T. Kana de. Shape a nd motion from image strea ms under orthography: a factorization metho d. Int J Comput Vision , 9(2):137 –154 , Nov ember 1 992. [18] K. Usevich and P . Como n. Hankel low-rank matrix completion: p erforma nce o f the nuclear norm relaxa tion. IEEE J ou r n al of S ele cte d T opics in Signal Pr o c essing , 1 0(4):637 –646, J une 2016. [19] W. Xu and B. Hassibi. Precise stability phase transitions for ℓ 1 minimization: a unified geometric fra mework. IEEE T r ansactions on Information The ory , 57 (10):6894 –691 9, Oc tob e r 2011. [20] W. Xu, J. Yi, S. Dasgupta, J. Cai, M. Jacob, and M. Cho. Separ ation-free s up er -resolutio n from compressed measur ements is p ossible: an arthonorma l a tomic norm minimization approa ch. arXiv:171 1.01396 , Nov ember 2017. 17