Learning from Literature: Integrating LLMs and Bayesian Hierarchical Modeling for Oncology Trial Design

Designing modern oncology trials requires synthesizing evidence from prior studies to inform hypothesis generation and sample size determination. Trial designs based on incomplete or imprecise summaries can lead to misspecified hypotheses and underpowered studies, resulting in false positive or negative conclusions. To address this challenge, we developed LEAD-ONC (Literature to Evidence for Analytics and Design in Oncology), an AI-assisted framework that transforms published clinical trial reports into quantitative, design-relevant evidence. Given expert-curated trial publications that meet prespecified eligibility criteria, LEAD-ONC uses large language models to extract baseline characteristics and reconstruct individual patient data from Kaplan-Meier curves, followed by Bayesian hierarchical modeling to generate predictive survival distributions for a prespecified target trial population. We demonstrate the framework using five phase III trials in first-line non-small-cell lung cancer evaluating PD-1 or PD-L1 inhibitors with or without CTLA-4 blockade. Clustering based on baseline characteristics identified three clinically interpretable populations defined by histology. For a prospective randomized trial in the mixed-histology population comparing mono versus dual immune checkpoint inhibition, LEAD-ONC projected a modest median overall survival difference of 2.8 months (95 percent credible interval -2.0 to 7.6) and an estimated probability of at least a 3-month benefit of approximately 0.45. As LEAD-ONC remains under active development, these results are intended as preliminary demonstrations of the frameworks potential to support evidence-driven oncology trial design rather than definitive clinical conclusions.

💡 Research Summary

The paper introduces LEAD-ONC (Literature to Evidence for Analytics & Design in Oncology), an AI‑assisted framework that converts unstructured information from published oncology trial reports into quantitative data suitable for evidence synthesis and prospective trial design. The workflow begins with investigator‑curated PDFs that meet predefined eligibility criteria. Large multimodal language models (specifically Google Gemini 2.0 Flash) are employed to extract baseline covariate tables (“Table 1”) and Kaplan‑Meier (KM) figures, including the associated risk tables. The LLM output is validated against a JSON schema; if confidence thresholds are not met or structural rules are violated, an optical‑character‑recognition (OCR) fallback pipeline (OpenCV region detection → preprocessing → PaddleOCR → Tesseract) is triggered. Results from both paths are reconciled, and a three‑point calibration step (clicking the origin, the maximum x‑tick, and the top y‑tick) allows rapid, robust mapping from pixel coordinates to data coordinates, accommodating skewed or partially labeled axes. Users then trace each KM curve on an HTML5 canvas; the system automatically enforces KM constraints (survival = 100 % at time 0, monotonic time, non‑increasing survival) and resamples curves to 500 evenly spaced time points.

To address heterogeneity across historical trials, the framework computes standardized differences for each baseline variable, aggregates them into a dissimilarity matrix Dₖⱼ (either by averaging or taking the maximum), and clusters trials using a K‑medoid algorithm. This clustering isolates subpopulations that are more comparable to the intended target trial.

Within the cluster most similar to the target population, LEAD-ONC fits a Bayesian hierarchical model (BHM) based on a beta‑Stacy process (BSP). The BSP comprises a precision function c(t) and a mean survival function G(t), which is modeled as a Weibull distribution (shape k, scale λ). The precision parameter c represents an equivalent “pseudo‑sample size” and is given a non‑informative prior, allowing the data to dominate inference. The hierarchical model treats each historical trial’s reconstructed individual‑patient data (IPD) as a random draw from the same BSP, yielding a predictive posterior distribution of survival curves for the future trial.

The authors demonstrate the approach on five phase‑III NSCLC first‑line trials evaluating PD‑1/PD‑L1 inhibitors with or without CTLA‑4 blockade (KEYNOTE‑189, KEYNOTE‑407, POSEIDON, CheckMate‑227, CheckMate‑9LA). Baseline characteristics extracted from the publications reveal three distinct clusters: all‑non‑squamous, all‑squamous, and mixed histology. For a hypothetical randomized trial in the mixed‑histology cluster comparing mono‑ versus dual‑immune checkpoint inhibition, the BHM predicts a median overall‑survival (OS) difference of 2.8 months (95 % credible interval −2.0 to 7.6). The probability that dual therapy improves median OS by at least three months is approximately 0.45, indicating a modest and uncertain benefit.

The paper highlights several practical advantages: (1) automated, batch‑scale extraction of KM curves and risk tables reduces manual effort by >30 % compared with traditional point‑and‑click tools; (2) the three‑point calibration mitigates axis‑detection errors across heterogeneous figure layouts; (3) clustering based on baseline covariates focuses evidence synthesis on the most relevant subpopulation, improving the interpretability of the Bayesian prior for the target trial. Limitations include the continued need for human verification of LLM/OCR outputs, potential inaccuracies in reconstructed IPD, and the reliance on Weibull‑based BSP assumptions which may not hold for all tumor types or treatment modalities. Future work will explore fine‑tuning multimodal LLMs for higher extraction fidelity, extending the hierarchical model to non‑parametric or mixture survival processes, and prospectively validating LEAD‑ONC in actual trial design settings.

In summary, LEAD‑ONC showcases how integrating large language models with Bayesian hierarchical evidence synthesis can transform narrative trial reports into actionable quantitative priors, thereby supporting more data‑driven, transparent, and potentially more efficient oncology trial designs.