Some Software Packages for Partial SVD Computation

arXiv:1108.1548v3 [math.OC] 8 May 2013 MANUSCRIPT TO BE ENRICHED 1 Some Software Packages for Partial SVD Computation Zhouchen Lin Abstract—This technical report introduces some software packages for partial SVD computation, including optimized PROPACK, modified PROPACK for computing singular values above a threshold and the corresponding singular vectors, and block Lanczos with warm start (BLWS). The current version is preliminary. The details will be enriched soon. Index Terms—Partial SVD, PROPACK, Lanczos Method ✦ 1 INTRODUCTION This technical report introduces some software pack- ages for partial SVD computation, including optimized PROPACK, modified PROPACK for computing singular values above a threshold and the corresponding singular vectors, and block Lanczos with warm start (BLWS). The current version is preliminary. The details will be enriched soon. 2 OPTIMIZED PROPACK PROPACK [3] is nowadays widely used in solving nuclear norm minimization problems in order to compute the partial SVD. We optimized PROPACK and obtained 15% to 20% speed up. The optimization is mainly by rewriting the Gram-Schmidt orthogonalization in the Lanczos procedure. The code is downloadable at http://www.cis.pku.edu.cn/faculty/vision/zlin/optPROPACK.zip 3 MODIFIED PROPACK The current PROPACK [3] can only output given number of principal singular values and vectors. However, when solving nuclear norm minimization problems, we are often faced with singular value thresholding [1], which requires the principal singular values that are greater than a given threshold. So we modified PROPACK to provide this functionality. The pseudo code is as in Algorithm 1 [2]. The code is downloadable at http://www.cis.pku.edu.cn/faculty/vision/zlin/ThresholdLANSVD.zip 4 BLOCK LANCZOS WITH WARM START (BLWS) When solving nuclear norm minimization problems, we have to solve the partial SVD multiple times. • Z. Lin is with Key Lab. of Machine Perception (MOE), School of EECS, Peking University, Beijing, China, 100871. E-mail: zlin@pku.edu.cn Algorithm 1 Partial SVD by a Threshold 1: Input: Matrix A, threshold svthr, initial vector p0, size K of the bidiagonal matrix B, β1 = ∥p0∥, u1 = p0/β1, i = 1, minsv = svthr + 1, 2: while minsv ≥svthr do 3: while i ≤K do 4: ri = AT ui −βivi−1, ri = reorth(ri), 5: αi = ∥ri∥, vi = ri/αi, 6: pi = Avi −αiui, pi = reorth(pi), 7: i ←i + 1. 8: end while 9: Compute the SVD of the bidiagonal matrix B to obtain the singular values, stored in a vector s. 10: neig = the number of accurate singular values. 11: minsv = the minimum of those accurate singular values. 12: Update K by K =    min(K + 100, max(2 + K, length(s > svthr) ∗K/neig)), if neig > 0, 2 ∗K, otherwise. 13: end while However, the matrix to compute the partial SVD only changes slightly over iteration. To utilize this fact, we propose using the block Lanczos with warm start technique to cut the computation in each iteration [5]. The code is downloadable at http://www.cis.pku.edu.cn/faculty/vision/zlin/BLWS.zip The code includes the BLWS technique for eigenvalue decomposition (BL EVD.m) and that for singular value decomposition (BL SVD.m), and also an exemplary us- age of BL SVD in solving the RPCA problem [4]. REFERENCES [1] J. Cai, E. Cand`es, and Z. Shen. A singular value threshold- ing algorithm for matrix completion. preprint, code available at http://svt.caltech.edu/code.html, Sep 2008. MANUSCRIPT TO BE ENRICHED 2 [2] M. Chen. Algorithms and implementations of matrix reconstruction (in Chinese). Master Thesis, available at http://www.cis.pku.edu.cn/faculty/vision/zlin/2010- MatrixReconstruction.pdf, 2010. [3] R. M. Larsen. Lanczos bidiagonalization with partial re- orthogonalization. Department of Computer Science, Aarhus University, Technical report, DAIMI PB-357, code available at http://soi.stanford.edu/∼rmunk/PROPACK/, Sep 1998. [4] Z. Lin, M. Chen, and Y. Ma. The augmented lagrange multi- plier method for exact recovery of corrupted low-rank matrix. Technical Report UILU-ENG-09-2215, UIUC, October 2009 (arXiv: 1009.5055)., 2009. [5] Z. Lin and S. Wei. A block Lanczos with warm start technique for accelerating nuclear norm minimization algorithms. Optimization Letters, submitted, arxiv:1012.0365.

This content is AI-processed based on open access ArXiv data.