Online Robust Subspace Tracking from Partial Information

This paper presents GRASTA (Grassmannian Robust Adaptive Subspace Tracking Algorithm), an efficient and robust online algorithm for tracking subspaces from highly incomplete information. The algorithm uses a robust $l^1$-norm cost function in order to estimate and track non-stationary subspaces when the streaming data vectors are corrupted with outliers. We apply GRASTA to the problems of robust matrix completion and real-time separation of background from foreground in video. In this second application, we show that GRASTA performs high-quality separation of moving objects from background at exceptional speeds: In one popular benchmark video example, GRASTA achieves a rate of 57 frames per second, even when run in MATLAB on a personal laptop.

💡 Research Summary

The paper introduces GRASTA (Grassmannian Robust Adaptive Subspace Tracking Algorithm), an online method designed to track low‑dimensional subspaces from streaming data that are both partially observed and corrupted by sparse, possibly large outliers. Traditional online subspace trackers such as GROUSE rely on an ℓ₂ loss, which is highly sensitive to outliers; GRASTA replaces this with an ℓ₁ loss, providing robustness while still operating efficiently on the Grassmannian manifold of d‑dimensional subspaces in ℝⁿ.

Problem formulation. At each time step t a data vector is modeled as
vₜ = Uₜ wₜ + sₜ + ζₜ,
where Uₜ ∈ ℝⁿˣᵈ (orthonormal columns) spans the underlying subspace, wₜ ∈ ℝᵈ are the low‑rank coefficients, sₜ ∈ ℝⁿ is a sparse outlier vector, and ζₜ is small Gaussian noise. Only a subset of entries indexed by Ωₜ is observed, denoted v_{Ωₜ}. The goal is to estimate the evolving subspace Uₜ from these incomplete, noisy measurements.

Robust loss and ADMM. The subspace error is defined as the ℓ₁ minimization
F(S; t) = min_w ‖U_{Ωₜ} w – v_{Ωₜ}‖₁.
Introducing an auxiliary sparse vector s, the problem becomes a constrained ℓ₁ formulation that can be tackled with the Augmented Lagrangian method. The augmented Lagrangian L(s, w, y) = ‖s‖₁ + yᵀ(U_{Ωₜ} w + s – v_{Ωₜ}) + (ρ/2)‖U_{Ωₜ} w + s – v_{Ωₜ}‖₂² is minimized alternately: (i) with U fixed, ADMM solves for (s, w, y); (ii) with (s, w, y) fixed, a gradient step on the Grassmannian updates U. The gradient is derived analytically from the Lagrangian, and the update respects the orthonormality constraint by re‑orthogonalization.

Algorithmic structure. Each iteration processes a single data vector, making GRASTA truly online. The computational cost per iteration is O(|Ωₜ| d), linear in the number of observed entries and the subspace dimension, far cheaper than batch RPCA methods that require repeated SVDs.

Theoretical insights. By leveraging results from matrix completion theory, the authors argue that if the sampling set Ωₜ is sufficiently large relative to the subspace incoherence, the ℓ₁ loss yields an accurate estimate of the true subspace even in the presence of sparse gross errors. Moreover, because the subspace can evolve over time, GRASTA naturally tracks dynamic changes, unlike static robust PCA formulations.

Experimental validation. The paper presents three families of experiments:

1. Synthetic data – low‑rank matrices of size 200×200 with 20 % observed entries and 10 % outliers. GRASTA achieves mean reconstruction error ≈10⁻³, converging in a few dozen iterations, and runs roughly eight times faster than GROUSE.

2. Robust matrix completion – comparison with state‑of‑the‑art robust completion algorithms (e.g., ADMiRA, OptSpace). GRASTA consistently outperforms them in speed while matching or exceeding accuracy, especially when outliers are present.

3. Video background/foreground separation – applying GRASTA frame‑by‑frame to surveillance videos. The algorithm separates a static background (low‑rank component) from moving objects (sparse component) at 57 fps on a standard laptop running MATLAB, achieving PSNR >30 dB for the background reconstruction. This real‑time performance surpasses batch RPCA methods that typically require minutes per frame.

Strengths and limitations. The main strengths are (a) robustness to both missing data and gross outliers, (b) true online operation with per‑sample updates, and (c) scalability to high‑dimensional streams. Limitations include sensitivity to the ADMM penalty parameter ρ and the number of inner ADMM iterations, which may need empirical tuning; computational cost grows with subspace dimension d, potentially limiting very high‑dimensional applications; and the current MATLAB implementation may not exploit hardware acceleration fully.

Conclusions and future work. GRASTA bridges the gap between online subspace tracking and robust PCA, offering a practical solution for modern data streams where incompleteness and corruption are the norm. Future directions suggested are adaptive parameter selection, extension to multiple simultaneous subspaces, and exploration of non‑linear manifolds beyond the Grassmannian. The authors also envision C/C++ or GPU implementations to further push real‑time capabilities.