Applying non-negative matrix factorization with covariates to structural equation modeling for blind input-output analysis

Structural equation modeling (SEM) describes directed dependence and feedback, whereas non-negative matrix factorization (NMF) provides interpretable, parts-based representations for non-negative data. We propose NMF-SEM, a unified non-negative framework that embeds NMF within a simultaneous-equation structure, enabling latent feedback loops and a reduced-form input-output mapping when intermediate flows are unobserved. The mapping separates direct effects from cumulative propagation effects and summarizes reinforcement using an amplification ratio. We develop regularized multiplicative-update estimation with orthogonality and sparsity penalties, and introduce structural evaluation metrics for input-output fidelity, second-moment (covariance-like) agreement, and feedback strength. Applications show that NMF-SEM recovers the classical three-factor structure in the Holzinger-Swineford data, identifies climate- and pollutant-driven mortality pathways with negligible feedback in the Los Angeles system, and separates deprivation, general morbidity, and deaths-of-despair components with weak feedback in Mississippi health outcomes.

💡 Research Summary

The paper introduces NMF‑SEM, a novel statistical framework that unifies non‑negative matrix factorization (NMF) with structural equation modeling (SEM) to enable blind input‑output analysis when intermediate flows are unobserved. Traditional NMF yields additive, parts‑based representations of non‑negative data but lacks an explicit causal or feedback structure. Conversely, SEM specifies directed dependencies among observed and latent variables but typically does not enforce non‑negativity or provide a parts‑based interpretation. NMF‑SEM bridges this gap by imposing a non‑negative simultaneous‑equation constraint on the NMF coefficient matrix:

 B = Θ₁Y₁ + Θ₂Y₂,

where Y₁ (P₁×N) are endogenous variables, Y₂ (P₂×N) are exogenous covariates, X (P₁×Q) is a non‑negative latent profile matrix, and Θ₁ (Q×P₁) and Θ₂ (Q×P₂) are non‑negative feedback and direct‑effect matrices, respectively. Substituting B into the NMF approximation Y₁ ≈ X B yields the structural form

 Y₁ ≈ X(Θ₁Y₁ + Θ₂Y₂).

Rearranging gives (I – XΘ₁)Y₁ ≈ XΘ₂Y₂. Under the stability condition ρ(XΘ₁) < 1 (spectral radius), (I – XΘ₁) is invertible and an equilibrium input‑output operator can be defined:

 M_model = (I – XΘ₁)⁻¹ XΘ₂, Y₁ ≈ M_model Y₂.

This operator is the non‑negative analogue of the classic Leontief inverse and can be expanded as a Neumann series

 M_model = XΘ₂ + XΘ₁XΘ₂ + (XΘ₁)²XΘ₂ + …,

making explicit the contribution of each feedback round. The authors introduce an amplification ratio

 AR = ‖M_model‖₁ / ‖XΘ₂‖₁,

which quantifies total amplification relative to direct effects; AR ≥ 1 with the bound 1 ≤ AR ≤ 1/(1 – ‖XΘ₁‖₁).

Estimation proceeds via a regularized multiplicative‑update algorithm that respects non‑negativity and the simultaneous‑equation constraint. The objective function combines reconstruction error, an orthogonality penalty λ_X‖XᵀX – diag(XᵀX)‖_F² (to encourage distinct latent profiles), and ℓ₁ penalties λ₁‖Θ₁‖₁ + λ₂‖Θ₂‖₁ (to induce sparsity in feedback and direct pathways). The updates generalize the Lee–Seung rules and guarantee monotonic decrease of the loss. Initialization uses a feed‑forward NMF‑with‑covariates solution (Θ₁ = 0) to obtain X₀ and Θ₀, which provide a stable starting point. Hyper‑parameters λ₁ and λ₂ are selected by K‑fold cross‑validation on equilibrium prediction error, while ensuring the stability condition ρ(XΘ₁) < 1.

Three structural evaluation metrics are proposed:

1. Input‑output fidelity (SC_map) – the Pearson correlation between vec(M_model) and vec(M_simple) where M_simple = X₀Θ₀ is the direct‑effect benchmark. High SC_map indicates that the feedback‑enhanced mapping remains consistent with the feed‑forward structure.

2. Second‑moment fidelity (SC_cov) – the correlation between vec(S_model) and vec(S_sample), where S_model = M_model S_Y₂ M_modelᵀ and S_sample = Y₁Y₁ᵀ. This assesses whether the equilibrium mapping reproduces the observed covariance‑like dependence among endogenous variables.

3. Feedback strength – measured by the spectral radius ρ(XΘ₁) and the amplification ratio AR, with bootstrap confidence intervals to gauge uncertainty.

Monte‑Carlo simulations varying noise level (σ) and true feedback strength (ρ_true) demonstrate that AR and estimated ρ increase systematically with feedback magnitude and sample size, confirming the theoretical bounds. Even with zero true feedback, small positive estimates of Θ₁ appear due to sampling variability and non‑negativity constraints, explaining a slight bias observed in the simulations.

Empirical applications illustrate the method’s versatility:

• Holzinger‑Swineford cognitive data – NMF‑SEM recovers the classic three‑factor structure, achieving SC_map ≈ 0.995 and SC_cov ≈ 0.998, confirming that the latent profiles align with established psychological constructs.

• Los Angeles climate‑pollutant mortality system – The estimated Θ₁ is near zero, indicating negligible latent feedback between climate/pollutant drivers and mortality outcomes. The model thus provides a parsimonious representation compared with traditional SEM, while preserving high input‑output fidelity.

• Mississippi health outcomes – The analysis separates deprivation, general morbidity, and “deaths‑of‑despair” components, with weak feedback (low ρ and AR). Again, SC_map and SC_cov exceed 0.99, demonstrating that the equilibrium mapping captures both direct and indirect health pathways accurately.

Across all case studies, the proposed evaluation metrics consistently report high fidelity, and the amplification ratio offers an intuitive scalar summary of feedback effects. The paper concludes that NMF‑SEM delivers a coherent, interpretable, and statistically sound framework for simultaneously learning parts‑based latent representations, directed causal structures, and equilibrium input‑output relationships, especially valuable when internal flows are unobserved (the “blind” IO problem).