Kernel Nonnegative Matrix Factorization Without the Curse of the Pre-image - Application to Unmixing Hyperspectral Images

The nonnegative matrix factorization (NMF) is widely used in signal and image processing, including bio-informatics, blind source separation and hyperspectral image analysis in remote sensing. A great challenge arises when dealing with a nonlinear formulation of the NMF. Within the framework of kernel machines, the models suggested in the literature do not allow the representation of the factorization matrices, which is a fallout of the curse of the pre-image. In this paper, we propose a novel kernel-based model for the NMF that does not suffer from the pre-image problem, by investigating the estimation of the factorization matrices directly in the input space. For different kernel functions, we describe two schemes for iterative algorithms: an additive update rule based on a gradient descent scheme and a multiplicative update rule in the same spirit as in the Lee and Seung algorithm. Within the proposed framework, we develop several extensions to incorporate constraints, including sparseness, smoothness, and spatial regularization with a total-variation-like penalty. The effectiveness of the proposed method is demonstrated with the problem of unmixing hyperspectral images, using well-known real images and results with state-of-the-art techniques.

💡 Research Summary

The paper tackles a fundamental limitation of kernel‑based non‑negative matrix factorization (kernel‑NMF): the curse of the pre‑image. Traditional kernel‑NMF methods map the data into a feature space, perform a linear NMF there, and then must map the learned basis vectors back to the original input space. This inverse mapping is ill‑posed, especially when non‑negativity constraints are required, making the resulting bases unusable for many applications such as hyperspectral unmixing.

To overcome this, the authors propose a novel framework that keeps the factor matrices in the input space while still exploiting kernel functions. By expressing the inner products Φ(xₜ)·eₙ as kernel evaluations κ(xₜ, eₙ), the basis vectors eₙ can be directly optimized in the original space under non‑negativity constraints. This eliminates the need for any pre‑image reconstruction.

Two iterative optimization schemes are derived. The first is an additive (gradient‑descent) update that explicitly computes the gradient of the reconstruction error with respect to both the basis matrix E and the coefficient matrix A. The second is a multiplicative update in the spirit of Lee and Seung’s algorithm, which naturally preserves non‑negativity and often converges faster. Closed‑form update rules are provided for polynomial, Gaussian, and linear kernels, together with a theoretical discussion of convexity and convergence.

Beyond the basic model, the paper incorporates three widely used regularizations: sparsity (ℓ₁ or hard zero‑forcing), smoothness (second‑order difference penalty on spectra), and spatial coherence via a total‑variation‑like term on the abundance maps. Each regularizer is seamlessly integrated into the update formulas by adding appropriate weighting matrices, allowing the method to adapt to the physical constraints typical of hyperspectral unmixing.

Experimental validation is performed on two benchmark hyperspectral datasets, Cuprite and Moffett. The proposed kernel‑NMF (especially with the Gaussian kernel) is compared against conventional linear NMF, earlier kernel‑NMF variants, and recent nonlinear unmixing approaches. Evaluation metrics include reconstruction error, spectral angle distance of the recovered endmembers, and visual quality of the abundance maps. Results show that the new method achieves significantly lower reconstruction errors and more accurate endmember spectra, particularly in regions where nonlinear mixing is strong. The added regularizations further improve the interpretability and smoothness of the abundance maps without sacrificing accuracy.

Complexity analysis indicates that the dominant cost lies in kernel matrix computation, which can be pre‑computed and reused, making the algorithm practical for moderately sized images. Convergence experiments demonstrate that the multiplicative scheme reaches a stationary point faster than the additive one while yielding comparable solutions.

In summary, the authors present a theoretically sound and practically effective kernel‑NMF that sidesteps the pre‑image problem, supports non‑negativity, and accommodates common physical constraints. This contribution advances nonlinear matrix factorization and opens new possibilities for hyperspectral unmixing and other signal‑processing tasks involving nonlinear mixtures.