Efficient Adaptive Compressive Sensing Using Sparse Hierarchical Learned Dictionaries

Recent breakthrough results in compressed sensing (CS) have established that many high dimensional objects can be accurately recovered from a relatively small number of non- adaptive linear projection observations, provided that the objects possess a sparse representation in some basis. Subsequent efforts have shown that the performance of CS can be improved by exploiting the structure in the location of the non-zero signal coefficients (structured sparsity) or using some form of online measurement focusing (adaptivity) in the sensing process. In this paper we examine a powerful hybrid of these two techniques. First, we describe a simple adaptive sensing procedure and show that it is a provably effective method for acquiring sparse signals that exhibit structured sparsity characterized by tree-based coefficient dependencies. Next, employing techniques from sparse hierarchical dictionary learning, we show that representations exhibiting the appropriate form of structured sparsity can be learned from collections of training data. The combination of these techniques results in an effective and efficient adaptive compressive acquisition procedure.

💡 Research Summary

The paper presents a hybrid approach that combines structured sparsity—specifically tree‑sparsity—with adaptive measurement strategies to improve the efficiency of compressive sensing (CS). Traditional CS relies on a fixed, often random, measurement matrix and guarantees accurate recovery of a k‑sparse signal from O(k log (n/k)) linear measurements, provided the non‑zero coefficients are sufficiently large (typically on the order of √(log n)). However, many natural signals, such as wavelet coefficients of images, exhibit hierarchical dependencies: non‑zero entries tend to form a connected subtree of a known tree (e.g., a binary wavelet tree). By exploiting this structure, the number of admissible support patterns shrinks dramatically, allowing for stronger theoretical guarantees with fewer measurements.

The authors first propose a simple adaptive sensing algorithm tailored to tree‑sparse signals. Starting from the root of the tree, the algorithm measures the corresponding dictionary atom scaled by a factor β, obtains a noisy observation y = β α_root + w (w ∼ N(0,1)), and compares |y| to a threshold τ. If the measurement is deemed significant (|y| ≥ τ), the algorithm enqueues (or pushes) the d children of that node for future measurement; otherwise it discards that branch. By using a stack the traversal becomes depth‑first, while a queue yields breadth‑first exploration. The process stops when the data structure empties. The total number of measurements is m = dk + 1, i.e., linear in the sparsity level k, independent of the ambient dimension n.

Theorem 1 formalizes the performance: assuming the signal’s coefficient vector α is k‑tree‑sparse, the measurement noise is i.i.d. Gaussian, and the measurement energy satisfies Σ‖φ_i‖² ≤ R, there exists a constant c₃ such that if the smallest non‑zero coefficient α_min satisfies

 α_min ≥ c₃ √(log k)/β

and τ = c₂ β α_min (with 0 < c₂ < 1), then with probability at least 1 − k^{−c₁} the algorithm recovers the exact support. The scaling β is chosen as β = √(R/((d+1)k)) to respect the energy budget. Consequently, the required signal amplitude scales as √((d+1)k log k / R), a substantial improvement over non‑adaptive CS (which needs √(n log n / R)) and even over adaptive CS without structure (which needs √(n log k / R)). Corollary 1 extends the result to a two‑stage procedure (support recovery followed by coefficient estimation), showing that the ℓ₂ error of the final estimate is O(k log k / R) under the same amplitude condition.

To make the method applicable when a suitable dictionary is not known a priori, the authors integrate hierarchical dictionary learning. Given a training matrix X ∈ ℝ^{n×q}, they solve a constrained optimization that factorizes X ≈ DA, where D ∈ ℝ^{n×p} has orthonormal columns and each column a_i of A is forced to be tree‑sparse. The sparsity constraint is encoded via a group‑lasso regularizer Ω(a_i) = ∑_{g∈G} ω_g‖a_i(g)‖_p, where G contains groups corresponding to each node together with all its descendants in the tree. Alternating minimization over D and A yields a dictionary whose atoms respect the hierarchical structure. This learned dictionary can then be used in the adaptive sensing algorithm; the combined framework is named LASeR (Learning Adaptive Sensing Representations).

Experimental validation uses the Psychological Image Collection (72 hand‑drawn and 91 natural images). Images are resized to 128 × 128, vectorized (dimension 16,384), and centered. A balanced binary tree dictionary with 7 levels (127 atoms) is learned from 163 training samples. Two test images are then sensed using LASeR under three different total sensing energy budgets (R = 128·128, (128·128)/8, (128·128)/32) and various thresholds τ (including τ = 0). Reconstruction quality is measured by signal‑to‑noise ratio (SNR) as a function of the number of measurements. LASeR consistently outperforms baseline methods: PCA‑based reconstruction, model‑based CS, ℓ₁‑Lasso, and direct wavelet sensing (both with τ = 0 and τ > 0). The gains are especially pronounced when the measurement budget is tight; appropriate choice of τ reduces unnecessary measurements while preserving accurate support detection.

In summary, the paper makes three principal contributions: (1) a rigorous analysis showing that adaptive sensing exploiting tree‑sparsity can recover support with O(k) measurements and modest signal amplitude, (2) a hierarchical dictionary learning scheme that automatically discovers tree‑structured sparse representations from data, and (3) an integrated LASeR system that combines learned dictionaries with adaptive measurements, delivering superior empirical performance over a range of competing CS techniques. The work highlights how structural priors and measurement adaptivity synergistically reduce both the number of required samples and the signal strength needed for reliable recovery. Future directions include extending the framework to more general graph‑based sparsity models, handling quantized or nonlinear measurements, and implementing the adaptive acquisition logic on low‑power hardware platforms.