Active Clustering: Robust and Efficient Hierarchical Clustering using Adaptively Selected Similarities

Hierarchical clustering based on pairwise similarities is a common tool used in a broad range of scientific applications. However, in many problems it may be expensive to obtain or compute similarities between the items to be clustered. This paper investigates the hierarchical clustering of N items based on a small subset of pairwise similarities, significantly less than the complete set of N(N-1)/2 similarities. First, we show that if the intracluster similarities exceed intercluster similarities, then it is possible to correctly determine the hierarchical clustering from as few as 3N log N similarities. We demonstrate this order of magnitude savings in the number of pairwise similarities necessitates sequentially selecting which similarities to obtain in an adaptive fashion, rather than picking them at random. We then propose an active clustering method that is robust to a limited fraction of anomalous similarities, and show how even in the presence of these noisy similarity values we can resolve the hierarchical clustering using only O(N log^2 N) pairwise similarities.

💡 Research Summary

The paper tackles the problem of hierarchical clustering when acquiring or computing all pairwise similarities is prohibitively expensive. It asks whether the true hierarchy can be recovered from a small subset of the N(N‑1)/2 possible similarities. The authors introduce the “Tight Clustering” (TC) condition: for any three items i, j, k, if i and j belong to the same cluster while k does not, then the similarity s_{i,j} must be larger than both s_{i,k} and s_{j,k}. Under this condition, standard bottom‑up agglomerative methods (single, average, complete linkage) would recover the correct tree if the full similarity matrix were available, but they still require O(N²) queries.

The first theoretical contribution shows that random sampling of similarities is insufficient. Proposition 1 proves that to reliably detect a cluster of size m one must observe at least Ω(N m) random pairs; for small clusters this essentially forces sampling almost the entire matrix. Hence an adaptive (active) strategy is necessary.

The core algorithm, named OUTLIER‑cluster, builds on a “leadership test” originally used for causal inference on binary variables. For any triple (i, j, k) the algorithm computes an “outlier” based on the ordering of the three pairwise similarities: the item whose two incident similarities are both smaller than the similarity between the other two is declared the outlier. Lemma 1 proves that, when the TC condition holds, this outlier coincides exactly with the leader of the triple in the underlying hierarchical tree (the leaf whose path to the root does not contain the lowest common ancestor of the other two leaves). Consequently, each leadership test can be performed with only three adaptively chosen similarity queries.

Using a reconstruction scheme from prior work on binary trees, the authors show that O(N log N) such leadership tests suffice to recover the entire hierarchy. Since each test needs three similarity values, the total number of pairwise queries is bounded by 3 N log N (Theorem 3.1). This matches the information‑theoretic lower bound up to constant factors and demonstrates that adaptive selection yields an order‑of‑magnitude reduction compared with naïve random sampling.

The paper also addresses robustness. In many realistic settings a fraction of similarities may be corrupted, violating the TC condition. The authors extend the method to tolerate a limited proportion of anomalous entries. By incorporating redundancy and majority‑vote style verification across multiple tests, they prove that the hierarchy can still be recovered with high probability using O(N log² N) adaptively chosen similarities, even when some of the queried values are erroneous.

Experimental validation is performed on synthetic balanced and unbalanced binary trees of sizes 128, 256, 512, and on a synthetic Internet‑topology tree with 768 nodes. The OUTLIER‑cluster algorithm requires only 3–11 % of the total pairwise similarities to achieve the same clustering accuracy as full agglomerative clustering. When a small number of outlier tests are deliberately corrupted, the algorithm’s performance degrades gracefully, confirming its robustness.

Key strengths of the proposed approach are:

1. Adaptive efficiency – the algorithm queries only those similarities that are informative for the current sub‑tree, achieving a near‑optimal O(N log N) query complexity.
2. Monotonic‑invariant – it relies solely on the ordering of similarity values, making it insensitive to monotonic transformations (scaling, shifting) and thus robust to calibration issues.
3. Noise tolerance – the robust extension handles a bounded fraction of erroneous similarities with only a modest increase to O(N log² N) queries.
4. Theoretical guarantees – rigorous proofs establish correctness under the TC condition and quantify the impact of noise.

Limitations include the reliance on the TC condition, which may not hold in datasets where intra‑cluster similarity is not uniformly higher than inter‑cluster similarity, and the focus on binary hierarchical trees (though any tree can be transformed into a binary representation). Future work suggested by the authors involves relaxing the TC assumption, extending the method to multi‑branch trees, and integrating multiple similarity modalities.

In summary, the paper demonstrates that hierarchical clustering can be performed both efficiently and robustly by actively selecting a small, logarithmic number of pairwise similarities. This result has practical implications for domains where similarity acquisition is costly—such as network probing, biological assays, or human‑annotated similarity judgments—offering a principled way to dramatically reduce measurement effort while preserving exact hierarchical structure recovery.