On the clustering aspect of nonnegative matrix factorization

This paper provides a theoretical explanation on the clustering aspect of nonnegative matrix factorization (NMF). We prove that even without imposing orthogonality nor sparsity constraint on the basis and/or coefficient matrix, NMF still can give clustering results, thus providing a theoretical support for many works, e.g., Xu et al. [1] and Kim et al. [2], that show the superiority of the standard NMF as a clustering method.

💡 Research Summary

The paper addresses a fundamental question in the theory of nonnegative matrix factorization (NMF): why does the standard NMF, without any explicit orthogonality or sparsity constraints, often produce meaningful clustering results? The authors provide a rigorous mathematical justification based on the Karush‑Kuhn‑Tucker (KKT) optimality conditions and a Lagrangian formulation of the classic NMF objective.

First, the problem is set up as the approximation (X \approx WH) where (X\in\mathbb{R}^{m\times n}{+}) is the data matrix, (W\in\mathbb{R}^{m\times k}{+}) contains basis vectors (interpreted as cluster centroids), and (H\in\mathbb{R}^{k\times n}_{+}) holds the coefficients (soft assignments of data points to clusters). The only constraints are non‑negativity of (W) and (H). The loss function is the Frobenius‑norm reconstruction error (J(W,H)=|X-WH|_F^2). By introducing non‑negative Lagrange multipliers (\Lambda) and (\Gamma) for the inequality constraints, the Lagrangian is written as

\