Measures for Assessing Causal Effect Heterogeneity Unexplained by Covariates

There has been considerable interest in estimating heterogeneous causal effects across individuals or subpopulations. Researchers often assess causal effect heterogeneity based on the subjects’ covariates using the conditional average causal effect (CACE). However, substantial heterogeneity may persist even after accounting for the covariates. Existing work on causal effect heterogeneity unexplained by covariates mainly focused on binary treatment and outcome. In this paper, we introduce novel heterogeneity measures, P-CACE and N-CACE, for binary treatment and continuous outcome that represent CACE over the positively and negatively affected subjects, respectively. We also introduce new heterogeneity measures, P-CPICE and N-CPICE, for continuous treatment and continuous outcome by leveraging stochastic interventions, expanding causal questions that researchers can answer. We establish identification and bounding theorems for these new measures. Finally, we show their application to a real-world dataset.

💡 Research Summary

The paper tackles a gap in the causal‑inference literature: existing measures of treatment‑effect heterogeneity (such as the treatment benefit rate (TBR) and treatment harm rate (THR)) are limited to binary outcomes and, when extended, rely on arbitrary thresholds. The authors propose four new metrics that capture heterogeneity unexplained by observed covariates for both binary‑treatment/continuous‑outcome and continuous‑treatment/continuous‑outcome settings.

1. Binary treatment, continuous outcome – P‑CACE and N‑CACE
Using the potential‑outcome framework, individuals are classified according to whether the treatment moves their outcome across a threshold y. For any covariate value w, the “positively affected” probability at point y is P(Y₀ < y ≤ Y₁ | W = w) and the “negatively affected” probability is P(Y₁ < y ≤ Y₀ | W = w). Integrating these probabilities over the whole outcome support yields
 P‑CACE(w) = ∫ P(Y₀ < y ≤ Y₁ | W = w) dy,
 N‑CACE(w) = ∫ P(Y₁ < y ≤ Y₀ | W = w) dy.
These integrals are equivalent to the expected positive (or absolute negative) individual causal effect (ICE) conditional on the sign of ICE and on W. Consequently, P‑CACE measures the average effect among subjects whose ICE > 0, while N‑CACE measures the average absolute effect among subjects whose ICE < 0. When the outcome is binary, the definitions collapse to the classic TBR and THR, showing a clean generalisation.

2. Continuous treatment, continuous outcome – P‑CPICE and N‑CPICE
The authors adopt stochastic interventions: two distributions π₀ and π₁ replace the original treatment with random draws X_{π₀} and X_{π₁}. The corresponding potential outcomes are Y_{π₀} and Y_{π₁}. Analogous to the binary‑treatment case, they define
 P‑CPICE(w) = ∫ P(Y_{π₀}<y ≤ Y_{π₁} | W = w) dy,
 N‑CPICE(w) = ∫ P(Y_{π₁}<y ≤ Y_{π₀} | W = w) dy.
These capture the average causal effect of shifting the treatment distribution from π₀ to π₁ among subjects who benefit (positive ICE) or are harmed (negative ICE) by that shift. The framework subsumes the Population Intervention Causal Effect (PICE) and provides a decomposition into beneficial and harmful components.

3. Identification and bounding
Under two key assumptions—conditional exogeneity (Yₓ ⟂ X | W) and conditional monotonicity of the outcome with respect to the treatment level—the authors prove that the above integrals are identified from observed data. Specifically, the probability terms can be expressed as differences of conditional cumulative distribution functions, e.g.,
 P(Y₀<y ≤ Y₁ | W) = max{F_{Y|X=0,W}(y) − F_{Y|X=1,W}(y), 0}.
When monotonicity fails or when the required densities are not fully observed, they derive sharp upper and lower bounds for each metric, ensuring that practitioners can still obtain informative intervals.

4. Decomposition of the conditional average causal effect (CACE)
A central result is that for any covariate level w, the traditional CACE decomposes as
 CACE(w) = P‑CACE(w) − N‑CACE(w).
Thus, the net average effect is the difference between the beneficial and harmful components. Figure 1 in the paper visualises this decomposition across a range of W values, illustrating how the overall CACE can mask substantial opposing sub‑effects.

5. Illustrative examples
Four synthetic examples clarify the behavior of the new measures: (i) homogeneous positive ICE (P‑A CE = 1, N‑A CE = 0); (ii) symmetric heterogeneous ICE with zero net ACE but non‑zero P‑A CE and N‑A CE; (iii) degenerate case with zero ICE for everyone; and (iv) a model with covariates where the distribution of ICE varies with W. These examples demonstrate that P‑A CE and N‑A CE capture heterogeneity invisible to the average effect.

6. Real‑world application
The authors apply the methodology to a medical dataset (e.g., a drug’s effect on blood pressure). They estimate the conditional outcome distributions using flexible machine‑learning models (e.g., Bayesian additive regression trees) and specify stochastic interventions such as a “single‑shift” (adding a fixed dose) and a “double‑shift” (increasing and decreasing dose). The estimated P‑CACE/N‑CACE reveal a sizable subgroup that benefits from the drug and another that is harmed, information that is lost when looking only at the overall ACE. Similarly, P‑CPICE/N‑CPICE identify patient groups for whom a dosage increase is advantageous versus detrimental, supporting personalized dosing decisions.

7. Comparison with existing work
Appendix A provides a concise table contrasting the settings covered by prior heterogeneity measures (binary‑treatment/binary‑outcome, binary‑treatment/continuous‑outcome with thresholds, etc.) with the four new metrics, highlighting that the proposed framework is the most general to date.

8. Limitations and future directions
The identification results rely on monotonicity, which may be violated in practice; the choice of stochastic intervention distributions introduces subjectivity; and high‑dimensional covariates can lead to instability in non‑parametric estimation. The authors suggest extending the theory to relax monotonicity (e.g., using partial identification) and developing data‑driven methods for selecting π₀, π₁.

Overall contribution
The paper makes three substantive contributions: (1) it defines principled, integrative measures of causal‑effect heterogeneity for continuous outcomes and treatments; (2) it provides rigorous identification and bounding results under realistic causal assumptions; and (3) it demonstrates practical utility through simulation and a real‑world medical example. By separating beneficial from harmful sub‑effects, the proposed metrics enable researchers and policymakers to uncover hidden heterogeneity, design targeted interventions, and move beyond average‑effect summaries toward more nuanced causal insight.