Robust PCA via Outlier Pursuit

Singular Value Decomposition (and Principal Component Analysis) is one of the most widely used techniques for dimensionality reduction: successful and efficiently computable, it is nevertheless plagued by a well-known, well-documented sensitivity to outliers. Recent work has considered the setting where each point has a few arbitrarily corrupted components. Yet, in applications of SVD or PCA such as robust collaborative filtering or bioinformatics, malicious agents, defective genes, or simply corrupted or contaminated experiments may effectively yield entire points that are completely corrupted. We present an efficient convex optimization-based algorithm we call Outlier Pursuit, that under some mild assumptions on the uncorrupted points (satisfied, e.g., by the standard generative assumption in PCA problems) recovers the exact optimal low-dimensional subspace, and identifies the corrupted points. Such identification of corrupted points that do not conform to the low-dimensional approximation, is of paramount interest in bioinformatics and financial applications, and beyond. Our techniques involve matrix decomposition using nuclear norm minimization, however, our results, setup, and approach, necessarily differ considerably from the existing line of work in matrix completion and matrix decomposition, since we develop an approach to recover the correct column space of the uncorrupted matrix, rather than the exact matrix itself. In any problem where one seeks to recover a structure rather than the exact initial matrices, techniques developed thus far relying on certificates of optimality, will fail. We present an important extension of these methods, that allows the treatment of such problems.

💡 Research Summary

Robust Principal Component Analysis (RPCA) has long been recognized as a powerful tool for extracting low‑dimensional structure from high‑dimensional data, yet its classical formulations are notoriously fragile when faced with outliers. Most existing robust PCA methods assume that each observation may have a few corrupted entries, while the majority of the vector remains reliable. In many real‑world scenarios—such as collaborative filtering with malicious users, bio‑informatics experiments contaminated by faulty reagents, or financial records altered by fraudulent trades—entire data points can be completely corrupted. The paper “Robust PCA via Outlier Pursuit” introduces a novel convex optimization framework, called Outlier Pursuit, specifically designed to handle this more severe form of contamination, which the authors refer to as “column outliers.”

The authors model the observed data matrix (M \in \mathbb{R}^{d \times n}) as the sum of two components: a low‑rank matrix (L) that captures the genuine low‑dimensional subspace, and a column‑sparse matrix (C) whose non‑zero columns correspond to fully corrupted observations. The recovery problem is cast as a convex program:

\