Using Model-based Overlapping Seed Expansion to detect highly overlapping community structure

As research into community finding in social networks progresses, there is a need for algorithms capable of detecting overlapping community structure. Many algorithms have been proposed in recent years that are capable of assigning each node to more than a single community. The performance of these algorithms tends to degrade when the ground-truth contains a more highly overlapping community structure, with nodes assigned to more than two communities. Such highly overlapping structure is likely to exist in many social networks, such as Facebook friendship networks. In this paper we present a scalable algorithm, MOSES, based on a statistical model of community structure, which is capable of detecting highly overlapping community structure, especially when there is variance in the number of communities each node is in. In evaluation on synthetic data MOSES is found to be superior to existing algorithms, especially at high levels of overlap. We demonstrate MOSES on real social network data by analyzing the networks of friendship links between students of five US universities.

💡 Research Summary

The paper addresses the increasingly important problem of detecting highly overlapping community structures in social networks, where a single node may belong to many groups simultaneously. Traditional community detection methods assume disjoint partitions, and even recent overlapping algorithms typically handle only low levels of overlap (most nodes belonging to two communities). In real‑world platforms such as Facebook, users often participate in multiple social circles—academic, extracurricular, familial—making robust detection of high‑order overlap essential for tasks like targeted marketing, influence maximization, and anomaly detection.

To meet this challenge, the authors propose MOSES (Model‑based Overlapping Seed Expansion), a scalable algorithm built on a probabilistic generative model of community formation. MOSES proceeds in three stages. First, it selects seed nodes based on local clustering coefficients, identifying promising “core” members of potential communities. Second, it expands each seed by evaluating the posterior probability that a neighboring node belongs to the seed’s community. This posterior is derived from a Bernoulli edge model where the probability of an edge between two nodes i and j is 1 − exp(−θ·|C_i ∩ C_j|), with θ controlling the overall density of intra‑community links. Third, an Expectation‑Maximization (EM) loop iteratively refines both the community membership matrix and the model parameters (θ and the baseline link probability), allowing nodes to acquire multiple memberships with calibrated probabilities. A hard threshold then converts these probabilities into final assignments, while low‑probability memberships are pruned automatically.

A key strength of MOSES is its explicit allowance for overlap: each node’s membership vector is updated independently, and the prior distribution over the number of communities per node captures real‑world variance. Computationally, the expansion step only touches the immediate neighborhood of each seed, yielding an overall time complexity of O(|E|·k), where |E| is the number of edges and k is the average number of communities per node. This makes the method applicable to graphs with tens of thousands of nodes and hundreds of thousands of edges without prohibitive runtime.

The authors evaluate MOSES on both synthetic and real data. Synthetic tests use a modified LFR benchmark that systematically varies the average number of community memberships per node from 2 up to 5, thereby creating increasingly overlapping ground truth. Across all overlap levels, MOSES outperforms state‑of‑the‑art overlapping detectors such as CPM, OSLOM, SLPA, and GCE, achieving higher Normalized Mutual Information, Adjusted Rand Index, and F1‑score—particularly when the average overlap reaches four or five communities per node, where competing methods suffer a sharp decline in accuracy.

Real‑world validation involves friendship networks from five U.S. universities, each comprising 2,000–5,000 students and tens of thousands of undirected edges. MOSES successfully recovers known social partitions (departments, clubs, dormitories) and, crucially, reveals nodes that belong simultaneously to several of these groups. Visualizations illustrate that MOSES captures fine‑grained, multi‑layered structures that other algorithms either merge into a single large community or split into fragmented pieces. The algorithm’s memory footprint remains modest, and runtime experiments confirm near‑linear scaling with edge count.

The paper also discusses limitations and future work. Sensitivity to the choice of the density parameter θ and to the initial seed set is not fully explored; systematic parameter sweeps could improve robustness. Extending MOSES to dynamic networks, where community memberships evolve over time, and incorporating node attribute information (e.g., profile data) are promising directions. Additionally, scaling to massive online networks with millions of nodes may require distributed implementations or approximation schemes.

In summary, MOSES offers a principled statistical framework combined with an efficient seed‑expansion heuristic that together enable accurate detection of highly overlapping community structures. Its superior performance on both synthetic benchmarks and real university friendship graphs demonstrates that it overcomes the accuracy degradation typical of existing methods under high overlap. This contribution is likely to impact a broad range of applications in social network analysis, recommendation systems, and information diffusion modeling.