Distributed Convoluted Rank Regression for Non-Shareable Data under Non-Additive Losses

We study high-dimensional rank regression when data are distributed across multiple machines and the loss is a non-additive U-statistic, as in convoluted rank regression (CRR). Classical communication-efficient surrogate likelihood (CSL) methods crucially rely on the additivity of the empirical loss and therefore break down for CRR, whose global loss couples all sample pairs across machines. We propose a distributed convoluted rank regression (DCRR) framework that constructs a similar surrogate loss and demonstrate its validity under the non-additive losses. We show that this surrogate shares the same population minimizer as the full-data CRR loss and yields estimators that are statistically equivalent to centralized CRR. Building on this, we develop a two-stage sparse DCRR procedure – an iterative $\ell_1$-penalized stage followed by a folded-concave refinement – and establish non-asymptotic error bounds, a distributed strong oracle property, and a DHBIC-type criterion for consistent model selection. A scaling result shows that the number of machines may diverge as $M = o({N/(s^2\log p)})$ while achieving centralized oracle rates with only $O(\log N)$ communication rounds. Simulations and a large-scale real data example demonstrate substantial gains over naive divide-and-conquer, particularly under heavy-tailed errors.

💡 Research Summary

This paper tackles the challenging problem of high‑dimensional rank‑based regression when the data are stored across multiple machines and the loss function is a non‑additive U‑statistic, as in Convoluted Rank Regression (CRR). Classical communication‑efficient distributed methods, such as the Communication‑Efficient Surrogate Likelihood (CSL) framework, rely on the empirical loss being a simple average of local losses. In CRR the loss couples all observation pairs, so the global loss cannot be decomposed into a sum of local losses and the usual gradient aggregation fails, leading to bias if naïvely applied.

The authors’ key insight is that, despite this non‑additivity at the sample level, the global CRR loss and each local CRR loss share the same population risk function because they are built from the same kernel and i.i.d. sampling scheme. Consequently, they have the same population minimizer β*. Leveraging this, they construct a surrogate loss that combines the CRR loss computed on a single “master” machine with a gradient‑correction term aggregated from all machines: \