Reframing Population-Adjusted Indirect Comparisons as a Transportability Problem: An Estimand-Based Perspective and Implications for Health Technology

📝 Abstract

Population-adjusted indirect comparisons (PAICs) are widely used to synthesize evidence when randomized controlled trials enroll different patient populations and head-to-head comparisons are unavailable. Although PAICs adjust for observed population differences across trials, adjustment alone does not ensure transportability of estimated effects to decision-relevant populations for health technology assessment (HTA). We examine and formalize transportability in PAICs from an estimand-based perspective. We distinguish conditional and marginal treatment effect estimands and show how transportability depends on effect modification, collapsibility, and alignment between the scale of effect modification and the effect measure. Using illustrative examples, we demonstrate that even when effect modifiers are shared across treatments, marginal effects are generally population-dependent for commonly used non-collapsible measures, including hazard ratios and odds ratios. Conversely, collapsible and conditional effects defined on the linear predictor scale exhibit more favorable transportability properties. We further show that pairwise PAIC approaches typically identify effects defined in the comparator population and that applying these estimates to other populations entails an additional, often implicit, transport step requiring further assumptions. This has direct implications for HTA, where PAIC-derived effects are routinely applied within cost-effectiveness and decision models defined for different target populations. Our results clarify when applying PAIC-derived treatment effects to desired target populations is justified, when doing so requires additional assumptions, and when results should instead be interpreted as population-specific rather than decision-relevant, supporting more transparent and principled use of indirect evidence in HTA and related decision-making contexts.

💡 Analysis

Population-adjusted indirect comparisons (PAICs) are widely used to synthesize evidence when randomized controlled trials enroll different patient populations and head-to-head comparisons are unavailable. Although PAICs adjust for observed population differences across trials, adjustment alone does not ensure transportability of estimated effects to decision-relevant populations for health technology assessment (HTA). We examine and formalize transportability in PAICs from an estimand-based perspective. We distinguish conditional and marginal treatment effect estimands and show how transportability depends on effect modification, collapsibility, and alignment between the scale of effect modification and the effect measure. Using illustrative examples, we demonstrate that even when effect modifiers are shared across treatments, marginal effects are generally population-dependent for commonly used non-collapsible measures, including hazard ratios and odds ratios. Conversely, collapsible and conditional effects defined on the linear predictor scale exhibit more favorable transportability properties. We further show that pairwise PAIC approaches typically identify effects defined in the comparator population and that applying these estimates to other populations entails an additional, often implicit, transport step requiring further assumptions. This has direct implications for HTA, where PAIC-derived effects are routinely applied within cost-effectiveness and decision models defined for different target populations. Our results clarify when applying PAIC-derived treatment effects to desired target populations is justified, when doing so requires additional assumptions, and when results should instead be interpreted as population-specific rather than decision-relevant, supporting more transparent and principled use of indirect evidence in HTA and related decision-making contexts.

📄 Content

Health technology assessment (HTA) agencies make decisions that determine access to new interventions across diverse patient populations and real-world healthcare settings. [1][2][3] Randomized controlled trials (RCTs) are pivotal in HTA, as they are regarded as the gold standard for establishing causal effects. 4 While RCTs typically achieve strong internal validity, their external validity-the degree to which findings can be meaningfully applied beyond the study sample-may be limited. [5][6][7] RCTs are conducted under controlled conditions and often enroll narrowly defined populations based on strict inclusion and exclusion criteria. 8 Consequently, treatment effects observed in RCTs may not directly reflect outcomes in broader, more heterogeneous patient populations encountered in routine clinical practice-the populations most relevant to HTA decision-making. 5,[9][10][11] External validity encompasses two interrelated but distinct concepts: generalizability and transportability. 12,13 Generalizability is determined by the extent to which the study sample is representative of the target population. A sample that closely reflects the target population allows for greater confidence in generalizing the study’s findings to that population. 12,13 Transportability refers to the ability to transfer a causal effect learned in one population (often a study sample) to a different external target population (e.g., another trial’s population or a real-world population) that may differ in key characteristics such as baseline risk factors or distributions of effect modifiers. 9,[12][13][14][15][16][17] These distinctions have been extensively discussed in the epidemiologic and causal inference literature, which has also produced a range of formal statistical frameworks for transporting causal effects across populations. 9,[12][13][14] In the HTA context, transportability is the key consideration, since evidence must often be extended from study populations to the real-world or jurisdictional populations for which coverage and reimbursement decisions are made. 11,18 Transportability issues are especially prominent in indirect treatment comparisons (ITCs), which are widely used in HTA when no head-to-head RCTs directly compare interventions of interest. 19,20 ITCs aim to integrate and contrast evidence from different trials or data sources to estimate relative treatment effects-a process sometimes referred to as data fusion. Conventional ITC methods, including Bucher adjustments and network meta-analyses (NMAs), assume that all included studies are exchangeable, meaning that they represent samples from a common underlying population. [21][22][23][24] Under this assumption, relative treatment effects are invariant across studies and can be applied without further adjustment. However, in practice, the populations enrolled in different RCTs often differ meaningfully in baseline characteristics and treatment effect modifiers, violating the exchangeability assumption and limiting the validity of unadjusted indirect comparisons. 22,23,25,26 To address these challenges, population-adjusted indirect comparisons (PAICs)-such as the matching-adjusted indirect comparison (MAIC) and simulated treatment comparison (STC)-have been developed. 19,[27][28][29][30][31][32] These methods aim to improve comparability across studies by adjusting for differences in observed effect modifiers between study populations, typically leveraging patient-level data (PLD) from at least one trial and aggregate data from comparator studies. When correctly specified, they can improve the comparability of treatment effect estimates across populations and enhance the relevance of evidence for HTA decision-making. 19,27,33 However, the use of PAICs introduces important methodological and practical challenges. These analyses rely on strong, often unverifiable assumptions-most notably, that all relevant effect modifiers have been measured and that the relationships between effect modifiers and treatment effects are consistent across treatments. 19,[27][28][29]34 Moreover, because PLD are rarely available for all relevant studies, it is typically not possible to fully test the underlying assumptions or assess model adequacy, which presents unique challenges to ensuring transportable and unbiased estimates of treatment effects. Importantly, even when standard PAIC assumptions appear plausible, uncertainty may remain about whether-and under what conditions-PAIC-derived treatment effects can be validly applied to the populations and settings relevant to HTA. 35 In particular, the relationship between adjustment for population differences and transportability to new target populations is not always explicit.

In this paper, we examine the methodological foundations of transportability in the context of HTA, with a particular focus on its implications for indirect comparisons. Adopting an estimand-based perspective, we first outline the conceptual underpinnings of

This content is AI-processed based on ArXiv data.