Scaling Analysis of Affinity Propagation

We analyze and exploit some scaling properties of the Affinity Propagation (AP) clustering algorithm proposed by Frey and Dueck (2007). First we observe that a divide and conquer strategy, used on a large data set hierarchically reduces the complexity ${\cal O}(N^2)$ to ${\cal O}(N^{(h+2)/(h+1)})$, for a data-set of size $N$ and a depth $h$ of the hierarchical strategy. For a data-set embedded in a $d$-dimensional space, we show that this is obtained without notably damaging the precision except in dimension $d=2$. In fact, for $d$ larger than 2 the relative loss in precision scales like $N^{(2-d)/(h+1)d}$. Finally, under some conditions we observe that there is a value $s^$ of the penalty coefficient, a free parameter used to fix the number of clusters, which separates a fragmentation phase (for $s<s^$) from a coalescent one (for $s>s^*$) of the underlying hidden cluster structure. At this precise point holds a self-similarity property which can be exploited by the hierarchical strategy to actually locate its position. From this observation, a strategy based on \AP can be defined to find out how many clusters are present in a given dataset.

💡 Research Summary

The paper presents a comprehensive scaling analysis of the Affinity Propagation (AP) clustering algorithm and introduces a hierarchical divide‑and‑conquer framework, called Hierarchical AP (Hi‑AP), that dramatically reduces computational cost while preserving clustering quality.
First, the authors recast AP as a min‑sum version of belief propagation (BP) and extend the model to a soft‑constraint variant (SCAP) that relaxes the hard requirement that each exemplar must point to itself. This formulation clarifies the role of the penalty parameter s (the self‑similarity) and sets the stage for a systematic analysis of algorithmic complexity.
Hi‑AP works by recursively partitioning the data set of size N into b sub‑sets at each level of a tree of depth h. AP (or its weighted version, W‑AP) is run independently on each sub‑set, producing a set of exemplars with associated weights equal to the number of original points they represent. The weighted exemplars are then clustered at the next level, and the process repeats until the root is reached.
The authors derive the total computational cost as
C(h) ≈ K²·(N/K)^{(h+2)/(h+1)} ,
where K is an upper bound on the number of exemplars kept at each intermediate step. For h = 0 this recovers the classic O(N²) cost of AP; for h = 1 the cost becomes O(N^{3/2}); and as h → ∞ the cost approaches linear O(N). Thus, by increasing the hierarchy depth, AP can be scaled to very large data sets without sacrificing the algorithmic simplicity that makes AP attractive.
To assess the impact of this reduction on clustering accuracy, the paper analyzes the loss of information when data points are drawn from a centered distribution in ℝ^d. Assuming that each sub‑set contains points that are close together (average mutual distance ε), the relative loss in precision scales as
Δ_rel ≈ N^{(2‑d)/