Evaluation of machine-learning models to measure individualized treatment effects from randomized clinical trial data with time-to-event outcomes

Objective: In randomized clinical trials, prediction models can be used to explore the relationships between patients’ variables (e.g., clinical, pathological, or lifestyle variables, and also biomarker or genomic data) and treatment effect magnitude. Our aim was to evaluate flexible machine learning models capable of incorporating interactions and nonlinear effects from high-dimensional data to estimate individualized treatment recommendations in trials with time-to-event outcomes. Methods: We compared survival models based on neural networks (CoxCC and CoxTime) and random survival forests (Interaction Forests) against a Cox proportional hazards model with an adaptive LASSO (ALASSO) penalty as a benchmark. For individualized treatment recommendations in the survival setting, we adapted metrics originally designed for binary outcomes to accommodate time-to-event data with censoring. These adapted metrics included the C-for-Benefit, the E50-for-Benefit, and the root mean squared error for treatment benefit. An extensive simulation study was conducted using two different data generation processes incorporating nonlinearity and interactions. The models were applied to gene expression and clinical data from three cancer clinical trial data sets. Results: In the first data generation process, neural networks outperformed ALASSO in terms of calibration while the Interaction Forests showed superior C-for-benefit performance. In the second data generation process, both machine learning methods outperformed the benchmark linear ALASSO method across discrimination, calibration, and RMSE metrics. In the cancer trial data sets, the machine learning methods often performed better than ALASSO, particularly IF in terms of C-for-benefit, and either a neural network or IF for calibration measures addressing treatment benefit.

💡 Research Summary

This paper investigates how to estimate individualized treatment effects (ITEs) from randomized controlled trials (RCTs) when the outcome is a time‑to‑event variable, such as overall survival. Traditional RCT analyses focus on the average treatment effect (ATE) and ignore heterogeneity across patients. The authors therefore compare three flexible machine‑learning (ML) survival models—two feed‑forward neural networks (CoxCC and CoxTime) and Interaction Forests (IF)—against a benchmark Cox proportional hazards model with an adaptive LASSO penalty (Cox‑ALASSO).

The neural‑network models are built on a case‑control loss (CoxCC) that samples a subset of the risk set at each iteration, improving computational efficiency, while CoxTime adds the actual event time as an input to relax the proportional‑hazards assumption. Interaction Forests extend random survival forests by explicitly searching for quantitative and qualitative biomarker‑by‑treatment interactions through bivariate splits. The benchmark Cox‑ALASSO imposes adaptive L1 penalties to achieve variable selection in high‑dimensional settings but retains linear main‑effects and proportional hazards.

To evaluate performance, the authors adapt three metrics originally designed for binary outcomes to censored survival data: C‑for‑Benefit (a discrimination measure analogous to the AUC for treatment benefit), E50‑for‑Benefit (a calibration metric focusing on the median predicted benefit), and the root‑mean‑square error for benefit (RMSE‑Benefit). These metrics quantify how well a model predicts the difference in survival between treatment and control for each individual.

Two simulation scenarios are constructed. The first generates data from a full biomarker‑by‑treatment interaction model with a simple nonlinear transformation f(x) = (2x − 1)², fixed coefficients, and Weibull baseline hazards. The second scenario introduces more complex, higher‑order nonlinearities and multiple interactions. In each case, 2,000 patients are randomly assigned 1:1 to treatment or control, and censoring is introduced independently of covariates. Results show that in the first scenario, the neural networks achieve better calibration (E50, RMSE) than Cox‑ALASSO, while IF attains the highest C‑for‑Benefit. In the second, more challenging scenario, both ML approaches outperform the benchmark across all three metrics, demonstrating robustness to strong nonlinearity and interaction structures.

The authors then apply the methods to three real cancer trial datasets that combine gene‑expression profiles (hundreds to thousands of features) with clinical covariates. Because the sample sizes are modest (≈100–300 patients), a double 5‑fold cross‑validation scheme (outer loop for test folds, inner loop for hyper‑parameter tuning) is used to mimic external validation. Across these datasets, Interaction Forests consistently achieve the best C‑for‑Benefit, while either CoxCC or CoxTime provide the best calibration scores, confirming the simulation findings. Notably, IF excels when qualitative interactions (i.e., the direction of treatment effect flips depending on biomarker levels) are present, highlighting its ability to capture complex decision boundaries.

The discussion emphasizes that ML‑based survival models can capture nonlinearities and high‑dimensional interactions that linear Cox models miss, leading to more accurate individualized treatment recommendations. However, the authors acknowledge limitations: reduced interpretability of neural networks and forests, potential over‑fitting in small samples, sensitivity to censoring rates, and the need for external validation on independent cohorts. Future work is suggested to integrate causal inference techniques (e.g., propensity‑score matching, instrumental variables) with ML models, develop ensemble approaches, and create user‑friendly visualization tools for clinicians.

In conclusion, the study provides empirical evidence that flexible ML survival models—particularly Interaction Forests and neural‑network‑based Cox extensions—offer superior discrimination and calibration for estimating conditional average treatment effects in RCTs with time‑to‑event outcomes, especially when the underlying data exhibit strong nonlinearity and interaction effects. This positions them as valuable alternatives to traditional regression‑based methods for developing personalized treatment rules in oncology and other fields where high‑dimensional biomarker data are available.