Towards a Fairer Non-negative Matrix Factorization

There has been a recent critical need to study fairness and bias in machine learning (ML) algorithms. Since there is clearly no one-size-fits-all solution to fairness, ML methods should be developed alongside bias mitigation strategies that are practical and approachable to the practitioner. Motivated by recent work on ``fair" PCA, here we consider the more challenging method of non-negative matrix factorization (NMF) as both a showcasing example and a method that is important in its own right for both topic modeling tasks and feature extraction for other ML tasks. We demonstrate that a modification of the objective function, by using a min-max formulation, may \textit{sometimes} be able to offer an improvement in fairness for groups in the population. We derive two methods for the objective minimization, a multiplicative update rule as well as an alternating minimization scheme, and discuss implementation practicalities. We include a suite of synthetic and real experiments that show how the method may improve fairness while also highlighting the important fact that this may sometime increase error for some individuals and fairness is not a rigid definition and method choice should strongly depend on the application at hand.

💡 Research Summary

The paper addresses a critical gap in the fairness literature by focusing on non‑negative matrix factorization (NMF), a widely used unsupervised learning technique for topic modeling, feature extraction, and downstream predictive tasks. Standard NMF minimizes the overall Frobenius reconstruction error, which implicitly treats all data points equally. When the dataset contains protected groups of disparate sizes, this average‑based objective can hide large reconstruction errors for minority groups, leading to biased representations and downstream decisions.

Motivated by the “Fair PCA” framework (Samadi et al., 2018), the authors propose a min‑max formulation for NMF, called Fairer‑NMF. The data matrix X is partitioned into groups (e.g., based on race, gender, or other protected attributes). For each group g they define the average reconstruction loss ℓ_g = ‖X_g – W_g H‖_F² / |g|. The Fairer‑NMF objective is

  min_{W_g ≥ 0, H ≥ 0} max_g ℓ_g,

which seeks to reduce the worst‑case group loss rather than the overall average. This formulation directly encodes a notion of group‑level fairness: the algorithm is forced to balance reconstruction quality across groups, preventing a dominant majority from monopolizing the representation capacity.

To solve the resulting non‑convex min‑max problem, the authors develop two algorithms:

1. Multiplicative Updates (MU) with group weights – Extending the classic Lee & Seung (2000) updates, they introduce dynamic scaling factors α_g for each group. The update rules become

 W_g ← W_g ⊙ (X_g Hᵀ) / (W_g H Hᵀ) · α_g,

 H ← H ⊙ (∑_g W_gᵀ X_g) / (∑_g W_gᵀ W_g H).

The α_g are adjusted at each iteration to drive the group losses toward equality, preserving non‑negativity while approximating the min‑max objective.

2. Alternating Minimization (AM) with a Lagrange multiplier – Holding H fixed, each group’s representation matrix W_g is solved as a non‑negative least‑squares (NNLS) subproblem. A scalar λ is introduced to enforce the constraint ℓ_A = ℓ_B (or more generally, equalizing the maximal loss). The algorithm iterates: (i) solve NNLS for each W_g, (ii) update λ to satisfy the equality condition, (iii) solve an NNLS problem for H given the concatenated W =