Diffusion Posterior Sampler for Hyperspectral Unmixing with Spectral Variability Modeling

Linear spectral mixture models (LMM) provide a concise form to disentangle the constituent materials (endmembers) and their corresponding proportions (abundance) in a single pixel. The critical challenges are how to model the spectral prior distribution and spectral variability. Prior knowledge and spectral variability can be rigorously modeled under the Bayesian framework, where posterior estimation of Abundance is derived by combining observed data with endmember prior distribution. Considering the key challenges and the advantages of the Bayesian framework, a novel method using a diffusion posterior sampler for semiblind unmixing, denoted as DPS4Un, is proposed to deal with these challenges with the following features: (1) we view the pretrained conditional spectrum diffusion model as a posterior sampler, which can combine the learned endmember prior with observation to get the refined abundance distribution. (2) Instead of using the existing spectral library as prior, which may raise bias, we establish the image-based endmember bundles within superpixels, which are used to train the endmember prior learner with diffusion model. Superpixels make sure the sub-scene is more homogeneous. (3) Instead of using the image-level data consistency constraint, the superpixel-based data fidelity term is proposed. (4) The endmember is initialized as Gaussian noise for each superpixel region, DPS4Un iteratively updates the abundance and endmember, contributing to spectral variability modeling. The experimental results on three real-world benchmark datasets demonstrate that DPS4Un outperforms the state-of-the-art hyperspectral unmixing methods.

💡 Research Summary

The paper tackles the challenging problem of hyperspectral unmixing (HU) under the linear spectral mixture model (LMM), where each observed pixel is expressed as a linear combination of endmember spectra and their fractional abundances. Traditional semi‑blind HU methods rely on external spectral libraries or pre‑extracted endmember bundles as priors, but these approaches suffer from bias, lack of adaptability, and an inability to model spectral variability caused by illumination changes, atmospheric effects, and sensor noise.

To overcome these limitations, the authors propose DPS4Un (Diffusion Posterior Sampler for Unmixing), a novel Bayesian framework that integrates a conditional score‑based diffusion model as a learned prior and performs joint posterior sampling of endmembers and abundances. The key innovations are:

1. Superpixel‑based endmember bundle construction – The hyperspectral image is first segmented into homogeneous superpixels using the SLIC algorithm. Within each superpixel, VCA extracts K candidate endmembers, yielding a total of P = K·L spectra (L = number of superpixels). A simple K‑means clustering assigns a categorical label c to each spectrum, forming the (spectrum, c) pairs used to train the conditional diffusion model. This image‑driven prior avoids external library bias and captures local spectral characteristics.

2. Conditional diffusion prior – A time‑conditional score network sθ(A_t, t, c) is trained via denoising score matching to approximate ∇_{A_t} log p(A_t | c). The label embedding injects the superpixel identity, ensuring that the diffusion process generates spectra consistent with the local material class while preserving fine‑grained details.

3. Region‑wise posterior sampling – At inference, each superpixel’s endmember matrix is initialized with independent Gaussian noise (A_T ∼ N(0, I)), rather than a global initialization. The reverse diffusion (DDIM‑style) iteratively refines A_t using the learned score and a region‑based data‑consistency term ∇_{A_t} log p(X | Â_0, Ŝ_0, c). The data‑consistency term is derived from a Tweedie‑based approximation, allowing the incorporation of the observation model without an explicit forward operator.

4. Joint abundance update – After each diffusion step, the current endmember estimate Â_0 is fixed and the abundance matrix Ŝ is optimized via projected gradient descent under non‑negativity and sum‑to‑one constraints. The updated (Â, Ŝ) pair is fed back into the diffusion sampler, forming an iterative loop that simultaneously refines both variables.

The algorithm thus performs a true posterior sampling of (A, S) while explicitly modeling spectral variability across superpixels.

Extensive experiments on three benchmark datasets (Jasper Ridge, Samson, Urban) demonstrate that DPS4Un consistently outperforms state‑of‑the‑art methods, including Bayesian priors based on USGS libraries, GAN‑based bundle priors, and recent unconditional diffusion approaches (e.g., Deng et al., 2024). Quantitatively, DPS4Un reduces RMSE, SAD, and aRMSE by 5–12 % on average, with the most pronounced gains on the Urban dataset where spectral variability is strongest. Visual results show sharper, more accurate material maps, confirming that the superpixel‑wise initialization and region‑specific data fidelity effectively capture local spectral shifts.

In summary, the contributions of the paper are threefold: (i) a conditional diffusion model that learns a data‑driven spectral prior without external libraries, (ii) a superpixel‑based strategy that embeds spectral variability directly into the prior and the sampling process, and (iii) an integrated Bayesian inference scheme that jointly samples endmembers and abundances. DPS4Un establishes diffusion models as powerful tools for high‑dimensional continuous spectral domains and opens a new avenue for flexible, variability‑aware hyperspectral unmixing.