Clustering Memes in Social Media

The increasing pervasiveness of social media creates new opportunities to study human social behavior, while challenging our capability to analyze their massive data streams. One of the emerging tasks is to distinguish between different kinds of activities, for example engineered misinformation campaigns versus spontaneous communication. Such detection problems require a formal definition of meme, or unit of information that can spread from person to person through the social network. Once a meme is identified, supervised learning methods can be applied to classify different types of communication. The appropriate granularity of a meme, however, is hardly captured from existing entities such as tags and keywords. Here we present a framework for the novel task of detecting memes by clustering messages from large streams of social data. We evaluate various similarity measures that leverage content, metadata, network features, and their combinations. We also explore the idea of pre-clustering on the basis of existing entities. A systematic evaluation is carried out using a manually curated dataset as ground truth. Our analysis shows that pre-clustering and a combination of heterogeneous features yield the best trade-off between number of clusters and their quality, demonstrating that a simple combination based on pairwise maximization of similarity is as effective as a non-trivial optimization of parameters. Our approach is fully automatic, unsupervised, and scalable for real-time detection of memes in streaming data.

💡 Research Summary

The paper tackles the problem of grouping short social‑media messages into coherent “memes,” defined as units of information that spread through a network. Recognizing that traditional lexical clustering fails on micro‑blog posts due to extreme sparsity, the authors introduce “protomemes” – elementary clusters built around easily extracted entities: hashtags, user mentions, URLs, and cleaned textual phrases. A single tweet can belong to multiple protomemes, providing an overlapping, fine‑grained pre‑clustering that can be updated in real time.

Four similarity measures are defined between protomemes: (1) user similarity (cosine of user‑frequency vectors), (2) tweet similarity (binary tweet overlap), (3) content similarity (TF‑IDF cosine), and (4) diffusion similarity (cosine of the set of users mentioned or retweeted, used as a proxy for the underlying follower network). Each measure lies in the interval