Beyond Kemeny Medians: Consensus Ranking Distributions Definition, Properties and Statistical Learning

In this article we develop a new method for summarizing a ranking distribution, \textit{i.e.} a probability distribution on the symmetric group $\mathfrak{S}_n$, beyond the classical theory of consensus and Kemeny medians. Based on the notion of \textit{local ranking median}, we introduce the concept of \textit{consensus ranking distribution} ($\crd$), a sparse mixture model of Dirac masses on $\mathfrak{S}_n$, in order to approximate a ranking distribution with small distortion from a mass transportation perspective. We prove that by choosing the popular Kendall $τ$ distance as the cost function, the optimal distortion can be expressed as a function of pairwise probabilities, paving the way for the development of efficient learning methods that do not suffer from the lack of vector space structure on $\mathfrak{S}_n$. In particular, we propose a top-down tree-structured statistical algorithm that allows for the progressive refinement of a CRD based on ranking data, from the Dirac mass at a Kemeny median at the root of the tree to the empirical ranking data distribution itself at the end of the tree’s exhaustive growth. In addition to the theoretical arguments developed, the relevance of the algorithm is empirically supported by various numerical experiments.

💡 Research Summary

The paper addresses the fundamental challenge of summarizing probability distributions over permutations, a task that becomes intractable as the number of items n grows because the space size n! explodes. Traditional consensus ranking, i.e., finding a Kemeny median that minimizes the sum of Kendall‑τ distances to observed rankings, provides only a single point estimate. While useful, a single median fails to capture multimodality or substantial heterogeneity that is common in modern preference data (e.g., recommendation systems, search logs).

To overcome this limitation, the authors introduce the notion of a Consensus Ranking Distribution (CRD). The idea is to partition the symmetric group Sₙ into a collection of cells 𝒫 = {C₁,…,C_K} such that each cell has non‑zero probability under the true distribution P. For each cell C, they consider the conditional distribution P_C = P(· | Σ ∈ C) and compute a local Kemeny median σ*_C, i.e., a permutation that minimizes the expected Kendall‑τ distance within that cell. The CRD is then defined as the sparse mixture

  P* = ∑{C∈𝒫} α_C δ{σ*_C},  α_C = P(C).

Thus, instead of a single Dirac mass, the model uses a weighted sum of Dirac masses located at locally optimal rankings.

A key theoretical contribution is the expression of the optimal transport distortion between P and its CRD approximation when the cost function is the Kendall‑τ distance. The authors prove that the Wasserstein‑τ distance W_{τ}(P, P*) can be written solely in terms of pairwise probabilities p_{i,j|C}=P{Σ(i)<Σ(j) | Σ∈C}. Specifically,

  W_{τ}²(P, P*) = ∑{C∈𝒫} α_C ∑{i<j} min{p_{i,j|C}, 1−p_{i,j|C}}.

Consequently, if each cell contains enough samples (so that α_C is large), the pairwise probabilities can be estimated accurately, and the distortion bound becomes tight. This result sidesteps the need for a vector‑space structure on Sₙ and shows that all relevant information for optimal approximation is captured by pairwise comparisons.

Building on this theory, the authors propose COAST (Consensus rAnking deciSion Tree), a top‑down, binary decision‑tree algorithm that constructs the partition 𝒫 adaptively from data. The algorithm proceeds as follows:

1. Root – Compute the global Kemeny median σ* (the minimizer of the empirical Kendall‑τ risk) and store it as the root’s representative.
2. Splitting rule – Choose a pair of items (i, j) that maximizes a criterion related to uncertainty (e.g., the empirical estimate of |p_{i,j}−½|). The rule “σ(i) < σ(j)” creates two child nodes: one containing all rankings where i precedes j, the other where j precedes i.
3. Recursive refinement – For each child node, recompute the conditional pairwise probabilities, estimate the local median σ*_C, and evaluate whether the node contains enough samples to justify further splitting. If so, repeat step 2; otherwise stop.
4. Leaves – When the recursion stops, the leaf’s Dirac mass is the local median of its cell, and the leaf weight α_C is the empirical proportion of rankings falling into that cell.

Because each split is defined by a simple pairwise comparison, the computational cost per node is O(n²), and the depth of the tree automatically adapts to the data’s heterogeneity. The algorithm yields a hierarchy: the root provides a coarse global summary, intermediate nodes give increasingly refined local summaries, and the leaves reproduce the empirical distribution exactly when the tree is fully grown.

The authors provide non‑asymptotic risk bounds for the empirical CRD obtained by COAST, showing that the excess distortion decays at a rate O(1/√N) under mild stochastic transitivity assumptions, and at O(1/N) when a low‑noise condition (minimum margin h on pairwise probabilities) holds. These rates match the optimal statistical rates for ranking aggregation under the same assumptions.

Empirical evaluation includes synthetic experiments with multimodal Mallows mixtures and real‑world datasets (e.g., movie rating rankings). Results demonstrate that CRD consistently achieves lower average Kendall‑τ distance to the true distribution than a single Kemeny median, with improvements ranging from 10 % to 20 %. Moreover, the tree‑based representation uncovers interpretable clusters of rankings, each associated with a distinct local median, which is valuable for downstream tasks such as personalized recommendation or anomaly detection.

In summary, the paper makes three major contributions:

1. Conceptual – Introduction of the Consensus Ranking Distribution, a parsimonious mixture of local Kemeny medians that captures multimodality while remaining statistically tractable.
2. Theoretical – Derivation of a closed‑form distortion expression based solely on pairwise probabilities, together with risk bounds for empirical CRDs.
3. Algorithmic – Development of COAST, a scalable, tree‑structured learning procedure that adaptively refines the partition of Sₙ and yields a hierarchy of increasingly accurate approximations.

The work opens several avenues for future research, such as extending the framework to partial rankings, incorporating alternative distance metrics (e.g., Spearman footrule), and designing distributed implementations for massive ranking datasets.