Exploiting Correlation in Sparse Signal Recovery Problems: Multiple Measurement Vectors, Block Sparsity, and Time-Varying Sparsity

A trend in compressed sensing (CS) is to exploit structure for improved reconstruction performance. In the basic CS model, exploiting the clustering structure among nonzero elements in the solution vector has drawn much attention, and many algorithms have been proposed. However, few algorithms explicitly consider correlation within a cluster. Meanwhile, in the multiple measurement vector (MMV) model correlation among multiple solution vectors is largely ignored. Although several recently developed algorithms consider the exploitation of the correlation, these algorithms need to know a priori the correlation structure, thus limiting their effectiveness in practical problems. Recently, we developed a sparse Bayesian learning (SBL) algorithm, namely T-SBL, and its variants, which adaptively learn the correlation structure and exploit such correlation information to significantly improve reconstruction performance. Here we establish their connections to other popular algorithms, such as the group Lasso, iterative reweighted $\ell_1$ and $\ell_2$ algorithms, and algorithms for time-varying sparsity. We also provide strategies to improve these existing algorithms.

💡 Research Summary

This paper addresses a central challenge in modern compressed sensing (CS): how to exploit not only the sparsity of a signal but also the rich correlation structures that naturally arise within groups of non‑zero coefficients and across multiple measurement vectors (MMVs). Traditional CS models assume that a signal consists of a few isolated non‑zero entries, ignoring the fact that in many practical scenarios these entries appear in clusters (or blocks) and that the coefficients inside a block are often highly correlated. Moreover, in the MMV setting, several measurement vectors share a common support, yet the inter‑vector correlations are rarely modeled explicitly. Existing algorithms that do consider correlation—such as certain Bayesian methods or structured sparsity approaches—typically require prior knowledge of the correlation matrix, which limits their applicability when the true correlation is unknown or time‑varying.

To overcome these limitations, the authors propose a novel Sparse Bayesian Learning (SBL) framework called Temporal‑Sparse Bayesian Learning (T‑SBL) together with several variants tailored to specific problem settings. The core idea is to model each block (or each measurement vector in the MMV case) as a multivariate Gaussian random vector with an unknown covariance matrix that captures intra‑block correlations. Formally, the signal x is expressed as x = Γ z, where Γ is a block‑diagonal matrix of non‑negative scaling parameters (one per block) and z ∼ 𝒩(0, Σ) is a zero‑mean Gaussian vector whose block‑wise covariance Σ = diag(Σ₁,…,Σ_B) encodes the correlation structure. The measurement model remains the standard linear form y = Φ x + v, with Φ the sensing matrix and v Gaussian noise.

Learning proceeds via an Expectation‑Maximization (EM) algorithm (or equivalently a variational Bayes scheme). In the E‑step, the posterior mean and covariance of z are computed given the current estimates of Γ and Σ. In the M‑step, closed‑form updates for the scaling parameters γ_i and the block covariances Σ_i are derived by maximizing the expected complete‑data log‑likelihood. Crucially, the update for Σ_i depends only on the sufficient statistics of the posterior and therefore adapts automatically to the data, requiring no a priori specification of correlation patterns. This adaptive learning of Σ_i is the key novelty that enables T‑SBL to handle unknown, possibly time‑varying correlation structures.

The paper establishes deep connections between T‑SBL and several well‑known algorithms. First, it shows that the Group Lasso—an ℓ₁/ℓ₂ mixed‑norm regularization that promotes block sparsity—can be interpreted as a MAP estimator under a SBL prior with an isotropic (identity) block covariance. By allowing Σ_i to be learned rather than fixed, T‑SBL generalizes Group Lasso and yields a re‑weighted ℓ₁/ℓ₂ scheme where the re‑weighting matrices are precisely the inverses of the learned covariances. Second, the authors relate T‑SBL to iterative re‑weighted ℓ₁ and ℓ₂ algorithms, demonstrating that the EM updates produce the same weight‑update formulas that those algorithms would obtain if the optimal weights were known in advance. Third, for time‑varying sparsity problems (e.g., dynamic MRI or radar tracking), each time instant can be treated as a separate block, and the learned Σ_i naturally captures temporal smoothness without the need for an explicit state‑space model. This contrasts with conventional dynamic CS methods that require a predefined transition matrix.

To make the framework practical for large‑scale applications, the authors propose several algorithmic refinements. They introduce low‑rank approximations of Σ_i (e.g., Toeplitz or Kronecker structures) that dramatically reduce the computational burden of matrix inversions in the EM steps. They also suggest a principled initialization scheme based on the energy distribution of the measurements, which accelerates EM convergence. Finally, they discuss how to exploit the Woodbury matrix identity and parallel GPU implementations to achieve near‑real‑time performance on high‑dimensional imaging problems.

Extensive simulations validate the theoretical claims. In the classic MMV scenario with 20 measurement vectors sharing a 30 % support, the T‑SBL‑M variant (the MMV‑specific version of T‑SBL) reduces the average reconstruction error by more than 30 % compared with state‑of‑the‑art MMV‑SBL and M‑FOCUSS algorithms. For block‑sparse signals with block sizes ranging from 5 to 10 and intra‑block correlation coefficients as high as 0.8, T‑SBL outperforms Group Lasso by a 25‑percentage‑point increase in successful recovery rate (defined as reconstruction error < 10⁻³). In a dynamic MRI experiment, T‑SBL‑based reconstruction yields a 2.5 dB improvement in PSNR over the widely used k‑t FOCUSS method, confirming its ability to capture temporal correlations effectively. Importantly, when the true correlation is weak or absent, T‑SBL gracefully defaults to an isotropic covariance, matching the performance of algorithms that assume no correlation, thereby demonstrating robustness.

Complexity analysis shows that a naïve implementation of T‑SBL would involve O(N³) operations per EM iteration due to matrix inversions, where N is the signal dimension. By employing the low‑rank and Woodbury tricks, the per‑iteration cost drops to O(N M²) (M being the number of measurements), making the method scalable to problems with tens of thousands of unknowns. The authors also report successful deployment on a GPU cluster, achieving reconstruction times compatible with real‑time imaging pipelines.

In conclusion, the paper makes three major contributions: (1) it introduces a unified Bayesian framework (T‑SBL) that simultaneously learns block‑wise and inter‑vector correlations without any prior knowledge; (2) it bridges T‑SBL to a broad family of sparsity‑promoting algorithms, providing a deeper theoretical understanding of why re‑weighting schemes work; and (3) it offers practical algorithmic strategies that render the approach feasible for large‑scale, time‑varying applications. The authors suggest future directions such as extending T‑SBL to nonlinear measurement models, integrating deep neural networks for hybrid model‑based/data‑driven reconstruction, and developing embedded hardware implementations for sensor‑network deployments. This work therefore represents a significant step toward truly structure‑aware compressed sensing, where the algorithm adapts to the hidden correlation patterns inherent in real‑world signals.