Who's Winning? Clarifying Estimands Based on Win Statistics in Cluster Randomized Trials

Treatment effect estimands based on win statistics, including the win ratio, win odds, and win difference are increasingly popular targets for summarizing endpoints in clinical trials. Such win estimands offer an intuitive approach for prioritizing outcomes by clinical importance. The implementation and interpretation of win estimands is complicated in cluster randomized trials (CRTs), where researchers can target fundamentally different estimands on the individual-level or cluster-level. We numerically demonstrate that individual-pair and cluster-pair win estimands can substantially differ when cluster size is informative: where outcomes and/or treatment effects depend on cluster size. With such informative cluster sizes, individual-pair and cluster-pair win estimands can even yield opposite conclusions regarding treatment benefit. We describe consistent estimators for individual-pair and cluster-pair win estimands and propose a leave-one-cluster-out jackknife variance estimator for inference. Despite being consistent, our simulations highlight that some caution is needed when implementing individual-pair win estimators due to finite-sample bias. In contrast, cluster-pair win estimators are unbiased for their respective targets. Altogether, careful specification of the target estimand is essential when applying win estimators in CRTs. Failure to clearly define whether individual-pair or cluster-pair win estimands are of primary interest may result in answering a dramatically different question than intended.

💡 Research Summary

This paper addresses the growing use of win‑statistics—win ratio, win odds, and win difference—as treatment‑effect estimands in clinical trials, focusing on the special challenges that arise in cluster randomized trials (CRTs). In CRTs, the unit of randomization is a cluster (e.g., hospital, school, community) rather than an individual, which creates two fundamentally different targets for win‑based estimands: an individual‑pair estimand that treats every participant equally, and a cluster‑pair estimand that gives each cluster equal weight. The authors first formalize both estimands within the potential‑outcomes framework, showing how the usual win‑probability, loss‑probability, and tie‑probability are constructed for each approach.

A central contribution is the demonstration that when cluster size is informative—meaning that baseline outcomes or treatment effects depend on the number of individuals in a cluster—these two estimands can diverge dramatically. In such “informative cluster size” (ICS) settings, the individual‑pair estimand over‑weights large clusters, potentially inflating or deflating the apparent treatment benefit. Conversely, the cluster‑pair estimand, by applying inverse‑cluster‑size weighting, remains unbiased for the cluster‑level average effect. The paper provides consistent estimators for both targets and proposes a leave‑one‑cluster‑out jackknife variance estimator, which naturally accounts for intra‑cluster correlation and is computationally straightforward.

Through extensive simulation studies, the authors show that the individual‑pair estimator suffers from finite‑sample bias, especially when the number of clusters is modest, whereas the cluster‑pair estimator is essentially unbiased for its target. Both estimators achieve consistency as the number of clusters grows, but their finite‑sample properties differ markedly. An illustrative data example with an ordinal outcome and both type‑I (baseline outcome varies with size) and type‑II (treatment effect varies with size) informative cluster sizes is presented. In this example, the individual‑pair win ratio is 2.24, suggesting a strong benefit, while the cluster‑pair win ratio is 0.88, indicating possible harm. This stark contrast underscores the importance of explicitly stating which estimand is of primary interest.

The authors conclude with practical guidance: (1) assess whether cluster size is likely to be informative before analysis; (2) decide whether the scientific question concerns the average effect among individuals or among clusters; (3) select the corresponding win‑based estimand and apply the appropriate weighting scheme; (4) use the proposed jackknife variance estimator for inference; and (5) report the chosen estimand, weighting, and assumptions transparently. By clarifying the distinction between individual‑pair and cluster‑pair win estimands and providing robust estimation tools, the paper offers essential methodological insight for researchers applying win statistics in increasingly complex CRT settings.