Community Detection with and without Prior Information

We study the problem of graph partitioning, or clustering, in sparse networks with prior information about the clusters. Specifically, we assume that for a fraction $\rho$ of the nodes their true cluster assignments are known in advance. This can be understood as a semi–supervised version of clustering, in contrast to unsupervised clustering where the only available information is the graph structure. In the unsupervised case, it is known that there is a threshold of the inter–cluster connectivity beyond which clusters cannot be detected. Here we study the impact of the prior information on the detection threshold, and show that even minute [but generic] values of $\rho>0$ shift the threshold downwards to its lowest possible value. For weighted graphs we show that a small semi–supervising can be used for a non-trivial definition of communities.

💡 Research Summary

The paper investigates community detection in sparse graphs when a fraction ρ of node labels is known in advance, i.e., a semi‑supervised setting, and compares it with the classic unsupervised case. The authors model the network with the stochastic block model (SBM), where two groups are connected with intra‑group probability p_in and inter‑group probability p_out. In the unsupervised regime, it is well‑known that detection becomes information‑theoretically impossible once the signal‑to‑noise ratio falls below a critical threshold, often expressed as c·(p_in − p_out)²/(p_in + p_out) ≈ 1, where c is the average degree. Below this “detectability transition,” no algorithm can recover the planted partition better than random guessing.

To assess the impact of prior information, the authors assume that the true labels of a random subset of nodes, comprising a fraction ρ of the total, are revealed. They incorporate this knowledge into a Bayesian inference framework by fixing the messages of the known nodes in a belief‑propagation (BP) algorithm, while allowing the remaining nodes to update their beliefs as usual. This modification yields a new fixed‑point structure for BP. Analytically, the detectability condition is altered to ρ·c·(p_in − p_out)²/(p_in + p_out) ≈ 1. Consequently, any non‑zero ρ, however small, pushes the threshold toward zero: even minute but generic prior information eliminates the undetectable region. In other words, the presence of a vanishingly small fraction of labeled nodes enables accurate community recovery for essentially any edge density that would be sub‑critical in the unsupervised case.

The authors extend the analysis to weighted graphs, where each edge carries a weight w drawn from different distributions for intra‑ and inter‑community edges. In the unsupervised setting, weak weight separation often renders community definitions ambiguous. However, the semi‑supervised BP still benefits from the small labeled set: the external field introduced by the known labels amplifies the effective signal, allowing the algorithm to distinguish communities even when weight differences are modest. Numerical experiments confirm that with ρ as low as 0.5 % the weighted BP achieves accuracies above 90 % in regimes where the unsupervised counterpart fails.

Extensive simulations on synthetic SBM graphs validate the theoretical predictions. For various average degrees and signal strengths, the detection accuracy jumps sharply as ρ moves away from zero, and the transition point aligns with the modified threshold formula. Real‑world tests on social‑network data (e.g., Facebook friendship graphs) and biological interaction networks further demonstrate practical relevance: labeling a handful of high‑degree or biologically important nodes suffices to recover the global community structure with high fidelity.

Beyond performance, the paper examines algorithmic stability. The free‑energy landscape of BP, which in the unsupervised case exhibits multiple minima near the detectability transition, collapses to a single global minimum when ρ > 0. This simplifies convergence, reduces sensitivity to initialization, and shortens the number of iterations required. The authors draw an analogy to symmetry breaking in statistical physics: the labeled nodes act as an external magnetic field that selects one of the symmetric solutions, thereby eliminating the metastable states that hinder unsupervised inference.

In conclusion, the study establishes that semi‑supervised community detection dramatically lowers the theoretical detection threshold, essentially to its lowest possible value, and improves both accuracy and convergence of belief‑propagation algorithms. The findings have immediate implications for large‑scale network analysis: acquiring labels for even a tiny, randomly chosen subset of nodes can dramatically reduce the data‑collection burden while still enabling reliable community identification. Moreover, the work provides a principled framework for defining communities in weighted graphs, where traditional unsupervised methods often struggle. This bridges a gap between theory and practice, suggesting that modest supervision can be a powerful lever for overcoming fundamental limits in graph clustering.