Combining BART and Principal Stratification to estimate the effect of intermediate variables on primary outcomes with application to estimating the effect of family planning on employment in Nigeria and Senegal

There is interest in learning about the causal effects of modern contraceptive use on empowerment outcomes. Data on this question often come from family planning (FP) programs that increase access to FP and facilitate contraceptive use among some women, rather than directly assigning use. Women whose contraceptive behavior changes because of these programs (“compliers”) may differ from target populations in ways that alter the consequences of contraceptive use for empowerment outcomes. We propose a two-step approach. First, we use principal stratification and Bayesian Additive Regression Trees (BART) to estimate the effect of modern contraceptive use among compliers in the study population, treating the FP program as an instrument rather than as the treatment of interest. Second, we generalize these complier-specific effects to a broader population by averaging conditional effects over the covariate distribution in the target population, with uncertainty in that distribution quantified via a Bayesian bootstrap applied to external complex survey data. We examine performance in simulation designs previously used to evaluate IV estimators. We then apply the approach to employment among urban women in Nigeria and Senegal, finding strong and heterogeneous effects of contraceptive use. Sensitivity analyses suggest robustness to violations of assumptions for internal and external validity.

💡 Research Summary

The paper addresses the challenge of estimating the causal impact of modern contraceptive use on women’s empowerment outcomes, specifically employment, when the exposure cannot be randomized but is influenced by family‑planning (FP) programs. The authors treat the FP program as an encouragement (instrument) and use principal stratification to focus on the sub‑population of “compliers” – women whose contraceptive behavior changes because of the program. To estimate the complier‑specific effect, they embed a Bayesian mixture model for principal strata within Bayesian Additive Regression Trees (BART). BART’s non‑parametric, tree‑based structure captures nonlinear relationships and heterogeneous effects between modern contraceptive use (binary treatment) and employment (binary outcome). The mixture model, implemented with data‑augmentation and MCMC, separates latent strata (compliers, never‑takers, defiers) and yields posterior draws of conditional average treatment effects (CATEs) for each covariate profile.

Because the complier group in the impact evaluation may not represent the broader target population of urban women, the second step generalizes the findings. The authors obtain the covariate distribution of the target population from Demographic and Health Surveys (DHS), which are complex, stratified two‑stage probability samples. Using a Bayesian bootstrap that respects survey weights and design, they generate posterior draws of the target covariate distribution. By averaging the CATEs over these draws (a plug‑in g‑formula), they produce a posterior distribution for the population average treatment effect (PATE). This “Prince‑BART Generalized” (PBG) approach thus combines internal causal identification with external transportability.

The paper carefully enumerates the assumptions required for internal validity (instrument relevance, conditional independence given covariates, exclusion restriction, monotonicity, and a single‑mode mixture) and for external validity (support overlap, conditional transportability). Sensitivity analyses probe violations: city‑level unobserved confounding, lack of covariate support in the target, and differences in the outcome model between source and target. Parametric and non‑parametric perturbations show that the main conclusions are robust.

Simulation studies, based on designs previously used to evaluate IV estimators, compare PBG with conventional linear IV, two‑stage least squares, and other non‑parametric IV methods. Results demonstrate that when treatment effects are heterogeneous and relationships are nonlinear, PBG yields substantially lower bias and mean‑squared error, while preserving credible interval coverage.

The empirical application uses longitudinal data from the Measurement, Learning & Evaluation (MLE) project on FP interventions in four initial and two delayed cities in Nigeria and Senegal. The sample includes women who had never used modern contraception at baseline (6,808 in Nigeria, 4,380 in Senegal) and follows them to endline, measuring employment. After fitting the principal‑stratification BART model, the authors find that modern contraceptive use raises the probability of employment by roughly 5–8 percentage points among compliers, with effect sizes varying by age, education, and city characteristics. Generalizing to all urban women using DHS covariates yields similar positive PATE estimates, confirming that the program’s impact extends beyond the trial cities. Sensitivity checks indicate that modest violations of the key assumptions do not overturn the substantive findings.

In discussion, the authors position PBG as an integrative, flexible tool for settings where an intermediate variable is manipulated by an encouragement design and where policymakers need to extrapolate results to broader populations. They acknowledge limitations such as reliance on observed covariates for transportability, computational intensity of Bayesian non‑parametrics, and the need for high‑quality external survey data. Nonetheless, the methodology offers a principled way to combine instrumental‑variable identification with modern machine‑learning estimation and Bayesian uncertainty propagation, advancing causal inference in public‑health and development research.