Inertia-Constrained Pixel-by-Pixel Nonnegative Matrix Factorisation: a Hyperspectral Unmixing Method Dealing with Intra-class Variability

Blind source separation is a common processing tool to analyse the constitution of pixels of hyperspectral images. Such methods usually suppose that pure pixel spectra (endmembers) are the same in all the image for each class of materials. In the framework of remote sensing, such an assumption is no more valid in the presence of intra-class variabilities due to illumination conditions, weathering, slight variations of the pure materials, etc… In this paper, we first describe the results of investigations highlighting intra-class variability measured in real images. Considering these results, a new formulation of the linear mixing model is presented leading to two new methods. Unconstrained Pixel-by-pixel NMF (UP-NMF) is a new blind source separation method based on the assumption of a linear mixing model, which can deal with intra-class variability. To overcome UP-NMF limitations an extended method is proposed, named Inertia-constrained Pixel-by-pixel NMF (IP-NMF). For each sensed spectrum, these extended versions of NMF extract a corresponding set of source spectra. A constraint is set to limit the spreading of each source’s estimates in IP-NMF. The methods are tested on a semi-synthetic data set built with spectra extracted from a real hyperspectral image and then numerically mixed. We thus demonstrate the interest of our methods for realistic source variabilities. Finally, IP-NMF is tested on a real data set and it is shown to yield better performance than state of the art methods.

💡 Research Summary

The paper addresses a fundamental limitation of conventional hyperspectral unmixing: the assumption that each material class (endmember) has a single, invariant spectral signature across the entire image. In real remote‑sensing scenarios, intra‑class variability—caused by illumination changes, weathering, compositional differences, and other factors—makes this assumption unrealistic. The authors first quantify this variability using a high‑resolution urban dataset (Toulouse, France) acquired in both VNIR and SWIR bands. By extracting pure pixels from tiles, vegetation, and asphalt, they compute pairwise correlation matrices and perform PCA projections. The analysis shows that even within a single roof, tile spectra can differ by 5–25 % in correlation, and variability increases when spectra are taken from multiple roofs or streets. Moreover, while illumination‑induced scaling appears as a line through the origin in PCA space, actual spectra deviate from this line, indicating additional sources of variability beyond simple scaling.

Motivated by these observations, the authors propose an extended linear mixing model in which each pixel p has its own set of endmember spectra rₘ(p):

 xₚ = Σₘ cₚₘ rₘ(p), with cₚₘ ≥ 0 and Σₘ cₚₘ = 1.

Collecting all pixels leads to a block‑diagonal mixing matrix ˜C and a tall matrix ˜R that stacks the pixel‑specific endmembers. Estimating both ˜C and ˜R is highly ill‑posed because the number of unknowns grows with the number of pixels.

To tackle this, the authors introduce two NMF‑based algorithms. The first, Unconstrained Pixel‑by‑Pixel NMF (UP‑NMF), applies the classic Lee‑Seung multiplicative updates independently to each pixel, yielding a per‑pixel estimate of rₘ(p) and cₚₘ. While UP‑NMF can capture variability, it suffers from “inertia” – the estimated spectra for a given material spread excessively across pixels, reducing interpretability.

The second algorithm, Inertia‑Constrained Pixel‑by‑Pixel NMF (IP‑NMF), adds a regularization term that penalizes the dispersion of each material’s spectra around its global mean μₘ = (1/P) Σₚ rₘ(p). The objective function becomes

 J = ‖X – ˜C ˜R‖_F² + λ Σₘ Σₚ ‖rₘ(p) – μₘ‖²,

where λ controls the strength of the inertia constraint. The optimization proceeds by alternating: (i) with μₘ fixed, update rₘ(p) and cₚₘ using multiplicative rules; (ii) recompute μₘ from the updated rₘ(p). Convergence is declared when the decrease in J falls below a threshold or a maximum number of iterations is reached.

Experimental validation is performed on two fronts. First, a semi‑synthetic dataset is built by mixing the extracted pure spectra with known abundances and controlled variability. UP‑NMF, IP‑NMF, and several state‑of‑the‑art methods (VCA‑FCLS, SUnSAL‑TV, sparse unmixing with a learned library) are compared using RMSE of reconstructed abundances and SAD of recovered endmembers. IP‑NMF consistently achieves the lowest errors, especially for the tile class where variability is highest, outperforming competitors by 20–30 %. Second, the algorithms are applied to the full Toulouse image. Visual inspection shows that IP‑NMF produces spatially coherent abundance maps and material spectra that respect known scene structures (e.g., different roof tiles are distinguished, asphalt and dark tiles are not confused). Quantitative metrics on this real data (e.g., reconstruction error, spectral angle) also favor IP‑NMF.

Key contributions of the work are:

1. A thorough, data‑driven characterization of intra‑class variability in urban hyperspectral imagery.
2. An extended mixing model that allows pixel‑specific endmember signatures while preserving the linear mixing framework.
3. The IP‑NMF algorithm, which introduces an inertia‑based regularizer to balance flexibility and stability in the estimation of per‑pixel spectra.
4. Demonstrated superiority of IP‑NMF over existing unmixing techniques on both synthetic and real datasets.

Overall, the paper provides a compelling solution for hyperspectral unmixing in environments where material spectra are not constant, opening the door to more accurate material mapping, change detection, and quantitative remote‑sensing applications.