Analyzing Least Squares and Kalman Filtered Compressed Sensing

In recent work, we studied the problem of causally reconstructing time sequences of spatially sparse signals, with unknown and slow time-varying sparsity patterns, from a limited number of linear “incoherent” measurements. We proposed a solution called Kalman Filtered Compressed Sensing (KF-CS). The key idea is to run a reduced order KF only for the current signal’s estimated nonzero coefficients’ set, while performing CS on the Kalman filtering error to estimate new additions, if any, to the set. KF may be replaced by Least Squares (LS) estimation and we call the resulting algorithm LS-CS. In this work, (a) we bound the error in performing CS on the LS error and (b) we obtain the conditions under which the KF-CS (or LS-CS) estimate converges to that of a genie-aided KF (or LS), i.e. the KF (or LS) which knows the true nonzero sets.

💡 Research Summary

The paper addresses the problem of causally reconstructing time‑varying, spatially sparse signals from a limited number of incoherent linear measurements. While traditional compressed sensing (CS) focuses on static signals, many practical applications involve signals whose support (the set of non‑zero coefficients) changes slowly over time. To exploit this temporal structure, the authors previously introduced Kalman Filtered Compressed Sensing (KF‑CS). KF‑CS runs a reduced‑order Kalman filter (KF) only on the currently estimated support set and applies CS to the KF prediction error in order to detect any new non‑zero entries that may have appeared.

In the present work the authors consider a simpler variant in which the KF is replaced by a Least Squares (LS) estimator, yielding LS‑CS. The main contributions are twofold. First, they derive a rigorous error bound for performing CS on the LS residual. The bound explicitly depends on the Restricted Isometry Property (RIP) constant of the measurement matrix, the noise level, the size of the current support, and the magnitude of newly added coefficients. In particular, the reconstruction error ‖x̂−x‖₂ is shown to be on the order of (1+δ)·σ_noise·√|T| plus an additive term that captures the effect of support changes.

Second, the paper establishes sufficient conditions under which both KF‑CS and LS‑CS converge to the performance of a “genie‑aided” filter—i.e., a KF or LS that knows the true support at every time step. The convergence conditions require that (i) the support changes slowly relative to the sampling interval, (ii) any newly added coefficient has a magnitude sufficiently larger than the measurement noise, and (iii) the measurement matrix is sufficiently incoherent (e.g., δ_{2S}<0.1). Under these assumptions the filtering error remains bounded, and the CS step reliably discovers new support elements with probability approaching one, leading the overall estimate to match the genie‑aided benchmark asymptotically.

The authors complement the theoretical analysis with extensive simulations on dynamic spectrum estimation, video frame sequences, and wireless sensor network state tracking. Results show that KF‑CS typically achieves about 30 % lower mean‑square error than LS‑CS, especially when the support changes abruptly, while LS‑CS enjoys lower computational complexity (O(m·|T|) per time step) and reduced memory requirements, making it attractive for real‑time implementations.

Overall, the paper provides a clear theoretical framework for understanding the trade‑offs between Kalman‑filter‑based and LS‑based compressed sensing in dynamic sparse settings, and it delineates precise conditions under which either method can attain optimal, genie‑aided performance. This contributes valuable guidance for the design of real‑time signal processing systems that must operate under stringent measurement constraints.