Benchmarking covariate-adjustment strategies for randomized clinical trials

Covariate adjustment is widely recommended to improve statistical efficiency in randomized clinical trials (RCTs), yet empirical evidence comparing available strategies remains limited. This lack of real-world evaluation leaves unresolved practical questions about which adjustment methods to use and which covariates to include. To address this gap, we conduct a large-scale empirical benchmarking using individual-level data from 50 publicly accessible RCTs comprising 29,094 participants and 574 treatment-outcome pairs. We evaluate 18 analytical strategies formed by combining six estimators-including classical regression, inverse probability weighting, and machine-learning methods-with three covariate-selection rules. Across diverse therapeutic areas, covariate adjustment consistently improves precision, yielding median variance reductions of 13.3% relative to unadjusted analyses for continuous outcomes and 4.6% for binary outcomes. However, machine-learning algorithms implemented with default hyperparameter settings do not yield efficiency gains beyond simple linear models. Parsimonious regression approaches, such as analysis of covariance, deliver stable, reproducible performance even in moderate sample sizes. Together, these findings provide the first large-scale empirical evidence that transparent and parsimonious covariate adjustment is sufficient and often preferable for routine RCT analysis. All curated datasets and analysis code are openly released as a reproducible benchmark resource to support future clinical research and methodological development.

💡 Research Summary

**
Background and Objectives
Covariate adjustment is widely endorsed by regulatory agencies (FDA, EMA) as a means to increase statistical efficiency in randomized clinical trials (RCTs) without compromising validity. However, empirical evidence comparing the performance of available adjustment methods in real‑world trials is scarce; most prior work relies on simulations or a handful of case studies. This study addresses this gap by conducting a large‑scale empirical benchmark using individual‑level data from 50 publicly available RCTs (29,094 participants, 574 treatment‑outcome pairs) spanning diverse therapeutic areas, sample sizes, and outcome types.

Methods
Six estimators were evaluated: (1) classical ANCOVA (linear regression with baseline covariates), (2) ANHECOVA (ANCOVA with treatment‑covariate interactions, implemented via g‑computation), (3) inverse‑probability weighting (IPW/IPTW), (4) g‑logistic (logistic regression g‑computation for binary outcomes), (5) double/debiased machine learning (DML), and (6) targeted minimum loss estimation (TMLE). Each estimator was combined with three covariate‑selection strategies:

• All – adjust for every available baseline variable,
• Top‑3 – adjust for the three covariates most strongly correlated with the outcome,
• Baseline+ – a prespecified set of commonly recommended variables (baseline outcome, stratification factors, age, sex, weight).

Performance was assessed using: (a) proportional variance reduction (PVR) relative to the unadjusted mean‑difference estimator, (b) scaled difference in point estimates (S‑Diff), (c) covariate‑adjustment gain (CAG) – probability that adjustment yields a statistically significant treatment effect when the unadjusted analysis does not, (d) covariate‑adjustment loss (CAL) – the reverse, and (e) the proportion of analyses in which R reported errors.

Results – Precision Gains
Across all methods, covariate adjustment improved precision. For continuous outcomes, the median PVR was 13.3%; for binary outcomes, 4.6%. The largest gains were observed with ANCOVA under the All strategy (median PVR ≈ 17%). When adjustment was limited to Top‑3 or Baseline+, median PVR fell to 6–8% for continuous outcomes but remained positive across all sample sizes. g‑logistic performed comparably to ANCOVA for binary outcomes, achieving a median variance reduction of 10.6% under Baseline+.

Sample‑Size Dependence
In small trials (≤ 100 participants), methods that incorporated many covariates (All‑ANHECOVA, All‑IPW, TMLE, DML) suffered from inflated variance due to loss of degrees of freedom and over‑fitting. Restricting adjustment to a few strong prognostic covariates (Top‑3, Baseline+) mitigated this instability, delivering modest yet reliable precision gains. In large trials (> 400 participants), TMLE and DML matched the performance of ANCOVA, but did not surpass it, indicating that default machine‑learning implementations add little practical benefit when sample size is ample.

Bias and Estimate Shifts
The S‑Diff distribution was centered near zero for all methods, indicating no systematic bias introduced by adjustment. The variability of S‑Diff was larger for the All strategy, reflecting greater susceptibility to random fluctuations when many covariates are included.

Real‑World Impact (CAG/CAL)
Covariate‑adjustment gain averaged 8% across all analyses, whereas loss averaged only 1%, demonstrating that adjustment more often creates new statistically significant findings than it eliminates them.

Conclusions and Recommendations

1. Covariate adjustment consistently improves efficiency in RCTs and should be standard practice.
2. Simple, transparent regression‑based adjustments (ANCOVA or ANHECOVA) with a modest, prespecified covariate set (Baseline+) provide the best trade‑off between precision and stability, especially in moderate‑size trials.
3. Machine‑learning‑based estimators (TMLE, DML) do not deliver additional efficiency when used with default hyper‑parameters; substantial tuning or larger sample sizes would be required to realize potential gains.
4. When the number of baseline variables is large relative to sample size, limiting adjustment to the top prognostic covariates (Top‑3) or a pre‑specified core set (Baseline+) avoids over‑fitting while preserving most of the precision benefit.

Data and Code Availability
All curated trial datasets, preprocessing scripts, and analysis pipelines are publicly released (GitHub repository). This resource enables future methodological research, including the evaluation of alternative machine‑learning algorithms, automated covariate‑selection procedures, and Bayesian adjustment frameworks.

Implications
The findings provide concrete, empirically validated guidance for statisticians, trialists, and regulators. By favoring parsimonious, transparent adjustment strategies, investigators can achieve meaningful sample‑size reductions and cost savings without sacrificing validity, thereby accelerating the development of effective therapies.