Efficiently Detecting Overlapping Communities through Seeding and Semi-Supervised Learning

Seeding then expanding is a commonly used scheme to discover overlapping communities in a network. Most seeding methods are either too complex to scale to large networks or too simple to select high-quality seeds, and the non-principled functions used by most expanding methods lead to poor performance when applied to diverse networks. This paper proposes a new method that transforms a network into a corpus where each edge is treated as a document, and all nodes of the network are treated as terms of the corpus. An effective seeding method is also proposed that selects seeds as a training set, then a principled expanding method based on semi-supervised learning is applied to classify edges. We compare our new algorithm with four other community detection algorithms on a wide range of synthetic and empirical networks. Experimental results show that the new algorithm can significantly improve clustering performance in most cases. Furthermore, the time complexity of the new algorithm is linear to the number of edges, and this low complexity makes the new algorithm scalable to large networks.

💡 Research Summary

The paper introduces ITEM (Information‑theoretic seeding and EM‑based semi‑supervised learning for overlapping community detection), a novel framework that reconceptualizes a graph as a text corpus: each edge becomes a document and each node a term. By constructing a Jaccard matrix that records the presence of a node in an edge’s neighbor set, the authors obtain a sparse binary (or tf‑idf) document‑term matrix suitable for classic text‑classification techniques.

Seed selection proceeds in two stages. First, a local RSS (Reputation, Strength, Specificity) score is computed for every edge using only its immediate neighborhood. Reputation is derived from the average Hamming similarity of SimHash fingerprints of incident edges, strength measures the number of common neighbors normalized by the larger endpoint degree, and specificity captures how exclusive the shared neighborhood is. An edge survives the RSS filter only if it has the highest score among its incident edges, guaranteeing a set of locally optimal candidates. Second, the candidate set is refined globally by Maximizing Global Information Gain (MGIG), which selects a diverse, representative subset of seeds that maximizes the overall information gain across the whole graph.

These seeds constitute a labeled training set for a Naïve Bayes (NB) classifier. The NB model treats each edge’s Jaccard row as a bag‑of‑words representation of a “topic”, i.e., a community. An Expectation‑Maximization (EM) loop iteratively (E‑step) assigns provisional community labels to unlabeled edges based on current NB probabilities, and (M‑step) re‑estimates NB parameters using both the original seeds and newly labeled edges. Because the EM process leverages both labeled and unlabeled data, it substantially improves classification accuracy over purely heuristic fitness functions traditionally used in expansion phases.

Complexity analysis shows that building the Jaccard matrix, computing RSS scores, and running MGIG all scale linearly with the number of edges |E|. The NB training and each EM iteration also operate in O(|E|·K) time, where K is the number of communities, leading to an overall linear‑time algorithm that can handle graphs with hundreds of thousands of edges.

Experimental evaluation spans synthetic LFR benchmarks with varying mixing parameters and overlap levels, as well as real‑world networks such as Zachary’s Karate club, Dolphin social network, a literary co‑appearance network (LM), and political blogs. ITEM is compared against four state‑of‑the‑art overlapping community detectors: Greedy Clique Expansion (GCE), Local Fitness Maximization (LFM), OSLOM, and SLPA. Across all datasets, ITEM consistently achieves higher Normalized Mutual Information (NMI), F1‑score, and modularity—often improving by 10–20 %—while maintaining comparable or lower runtimes. The gains are especially pronounced on networks with high overlap, where traditional local fitness functions struggle.

The authors acknowledge limitations: edge‑centric community definitions may over‑assign nodes to multiple groups, and extremely sparse or weighted graphs could render the Jaccard matrix too sparse for reliable NB estimation. Future work is outlined to extend ITEM to weighted, directed, and dynamic graphs, to integrate deep topic models (e.g., neural variational document models) for richer representations, and to explore adaptive seed updating mechanisms in evolving networks.

In summary, ITEM offers a principled, scalable solution to overlapping community detection by unifying a text‑mining transformation, a hybrid local‑global seeding strategy, and semi‑supervised learning. Its linear complexity and strong empirical performance make it a compelling addition to the toolkit of network scientists and practitioners dealing with large, complex graphs.