Generating a Diverse Set of High-Quality Clusterings

We provide a new framework for generating multiple good quality partitions (clusterings) of a single data set. Our approach decomposes this problem into two components, generating many high-quality partitions, and then grouping these partitions to obtain k representatives. The decomposition makes the approach extremely modular and allows us to optimize various criteria that control the choice of representative partitions.

💡 Research Summary

The paper tackles the problem of producing a diverse collection of high‑quality clusterings for a single data set. Rather than treating “alternate clustering” (finding one additional clustering far from a given one) or “k‑consensus clustering” (selecting k representatives from a pre‑existing pool) as separate tasks, the authors propose a unified two‑stage framework that decouples the generation of candidate partitions from the selection of representative partitions.

Stage 1 – Generation of many high‑quality partitions.
The authors formalize the space of all possible partitions P of a data set X, noting that its cardinality is given by Stirling numbers of the second kind. They introduce a quality function Q: P → ℝ⁺ that can be instantiated with classic compactness‑separation measures (e.g., k‑means inverse sum of squared distances, Dunn index) or with a kernel‑based similarity score Q_K that aggregates intra‑cluster kernel values. The key idea is to sample partitions with probability proportional to Q, thereby biasing the sample toward regions of the space that contain high‑quality solutions.

To achieve proportional sampling, the authors employ a Metropolis‑Hastings (MH) random walk combined with Gibbs‑style updates. Starting from an arbitrary partition, they repeatedly select a random ordering σ of the data points. For each point x in that order, they consider moving x to each of the s clusters, compute the resulting quality scores, and then assign x to a cluster with probability proportional to those scores. After processing all points, the new partition becomes the current state. After a burn‑in period (e.g., 1 000 iterations), the chain is assumed to have mixed, and every subsequent state is taken as an independent sample. By repeating this process, they obtain m ≥ k samples that densely cover high‑quality regions while still preserving diversity.

Stage 2 – Grouping and selection of k representatives.
With a large collection Z of sampled partitions, the second stage clusters these partitions into k groups and picks one representative from each group. The authors stress that any distance metric on P can be used; they discuss three families:

1. Membership‑based distances (Rand index, Variation of Information, Normalized Mutual Information) that count pairwise agreements but ignore spatial layout.
2. Spatially‑aware distances such as LiftEMD, which represent each cluster by a point set and compare clusters using the Earth Mover’s Distance, thereby capturing geometric information.
3. Density‑adjusted distances d_Z, defined as the number of sampled partitions that are closer to a given partition than another, which expands densely populated regions to avoid over‑clustering them.

For the actual clustering of partitions, the authors adopt Gonzalez’s greedy far‑thest‑point algorithm, which provides a 2‑approximation to the optimal k‑center problem (minimizing the maximum distance from any point to its assigned center). Using LiftEMD as the base distance, the algorithm iteratively selects the partition that maximizes its distance to the current set of centers until k centers are chosen. Each center becomes a representative clustering.

Experimental evaluation.
The methodology is tested on three types of data: (i) synthetic data with known multi‑modal cluster structures, (ii) several UCI benchmark data sets, and (iii) a subset of the Yale Face image database. Quality is measured both by the kernel‑based score Q_K and by relative quality against a reference partition (e.g., the best single‑run k‑means solution). The proposed approach consistently discovers partitions whose quality equals or exceeds that of the state‑of‑the‑art consensus method LiftSSD, sometimes reaching a relative quality close to 1.0. Moreover, the minimum pairwise distance among the selected representatives is substantially larger than the distances from non‑representative samples to their nearest representative, confirming that the final set is both high‑quality and diverse.

Key contributions and implications.

1. Decoupling generation and selection removes the bias that earlier methods suffered from when distance considerations interfered with quality‑driven exploration. This allows the sampler to fully explore dense high‑quality regions without being forced away by previously selected partitions.
2. Quality‑proportional sampling via MH‑Gibbs provides a principled way to obtain a representative “cloud” of partitions that reflects the underlying quality landscape of P.
3. Modular distance framework lets practitioners plug in domain‑specific metrics (e.g., spatially aware for image data, membership‑based for categorical data) and even density‑aware adjustments, tailoring the notion of diversity to the problem at hand.
4. Empirical superiority over existing meta‑clustering techniques demonstrates that the method can uncover useful alternative clusterings that were previously missed, which is valuable for exploratory data analysis where multiple plausible segmentations may exist.

In summary, the paper presents a flexible, theoretically grounded pipeline for generating a rich set of high‑quality clusterings and distilling them into a concise, diverse collection of k representatives. By separating sampling from clustering, and by allowing interchangeable quality and distance measures, the framework can be adapted to a wide range of domains and opens avenues for future work on more efficient sampling schemes, automatic metric selection, and integration with user‑guided relevance feedback.