FSCNMF: Fusing Structure and Content via Non-negative Matrix Factorization for Embedding Information Networks

Analysis and visualization of an information network can be facilitated better using an appropriate embedding of the network. Network embedding learns a compact low-dimensional vector representation for each node of the network, and uses this lower dimensional representation for different network analysis tasks. Only the structure of the network is considered by a majority of the current embedding algorithms. However, some content is associated with each node, in most of the practical applications, which can help to understand the underlying semantics of the network. It is not straightforward to integrate the content of each node in the current state-of-the-art network embedding methods. In this paper, we propose a nonnegative matrix factorization based optimization framework, namely FSCNMF which considers both the network structure and the content of the nodes while learning a lower dimensional representation of each node in the network. Our approach systematically regularizes structure based on content and vice versa to exploit the consistency between the structure and content to the best possible extent. We further extend the basic FSCNMF to an advanced method, namely FSCNMF++ to capture the higher order proximities in the network. We conduct experiments on real world information networks for different types of machine learning applications such as node clustering, visualization, and multi-class classification. The results show that our method can represent the network significantly better than the state-of-the-art algorithms and improve the performance across all the applications that we consider.

💡 Research Summary

The paper addresses a fundamental limitation of most network embedding methods: they rely solely on graph structure while ignoring the rich textual (or multimodal) content attached to nodes. To bridge this gap, the authors propose FSCNMF, a non‑negative matrix factorization (NMF) framework that jointly learns two low‑dimensional embeddings—one from the adjacency matrix (structure) and one from the content matrix (e.g., TF‑IDF vectors). Rather than forcing a single shared embedding (as in earlier joint factorization approaches such as TADW), FSCNMF treats the content embedding as a regularizer for the structural embedding and vice‑versa. Concretely, the structural factorization A ≈ B₁B₂ is optimized with an additional term α₁‖B₁ − U‖² that pulls B₁ toward the content‑derived matrix U; similarly, the content factorization C ≈ UV includes β₁‖U − B₁‖². The parameters α₁ and β₁ control the strength of this mutual regularization, while α₂,α₃,β₂,β₃ are standard L₂ regularizers to avoid over‑fitting.

Optimization proceeds via alternating minimization with multiplicative update rules that respect the non‑negativity constraints, guaranteeing convergence properties similar to classic NMF. The basic model (FSCNMF) already yields embeddings that capture both topological proximity and semantic similarity. To incorporate higher‑order network proximity, the authors extend the model to FSCNMF++. Here, powers of the adjacency matrix (A², A³, …) are linearly combined (with coefficients γₗ) to form a richer proximity matrix Ā, which is then factorized in the same way. This enables the embedding to reflect not only immediate neighbors but also multi‑step relationships, improving performance on networks where community structure extends beyond 1‑hop connections.

The framework is further generalized to handle multiple content modalities (text, images, video, etc.). For each modality m a separate content factorization (C^{(m)} ≈ U^{(m)}V^{(m)}) is performed, and modality‑specific regularization terms λ_m‖U^{(m)} − B₁‖² are added, allowing the model to fuse heterogeneous information while still preserving the mutual regularization principle.

Experimental evaluation is conducted on several real‑world datasets: citation networks (Cora, Citeseer, Pubmed), a scholarly collaboration network (DBLP), and a social media network with image metadata (Flickr). The authors assess three representative downstream tasks: (1) node clustering (using NMI, ARI, and clustering accuracy), (2) 2‑D visualization (t‑SNE plots), and (3) multi‑class node classification (logistic regression / SVM). Baselines include classic random‑walk based methods (DeepWalk, node2vec, LINE), content‑aware joint factorization (TADW), and recent attributed embedding approaches (AANE, GraphSAGE). Across all datasets, FSCNMF consistently outperforms baselines, achieving up to 65 % relative improvement in clustering accuracy on the most challenging datasets where structural and content signals are noisy or partially contradictory. FSCNMF++ further boosts performance by 5–8 % on networks where higher‑order proximities are informative. Visualizations show clearly separated clusters that align with ground‑truth labels, confirming that the learned embeddings preserve both structural and semantic relationships.

Key strengths of the proposed approach are: (i) explicit mutual regularization that makes the model robust to inconsistencies between graph and content, (ii) interpretability due to non‑negative embeddings, (iii) flexibility to incorporate higher‑order proximities and multiple content modalities without redesigning the core algorithm, and (iv) relatively simple implementation based on matrix operations, which scales to medium‑size graphs without requiring deep learning infrastructure.

Limitations include the need to manually tune regularization weights (α₁, β₁, γₗ, λ_m) and the computational cost of forming high‑order adjacency matrices for very large graphs, which may require sparse approximations or sampling techniques. The authors suggest future work on automatic hyper‑parameter learning, scalable sparse matrix computations, and hybrid models that combine NMF with neural architectures to further enhance expressive power.

In summary, FSCNMF and its extension FSCNMF++ present a principled, effective, and versatile solution for fusing structural and content information in network embedding, delivering state‑of‑the‑art performance across a range of network analysis tasks.