Forest Kernel Balancing Weights: Outcome-Guided Features for Causal Inference

While balancing covariates between groups is central for observational causal inference, selecting which features to balance remains a challenging problem. Kernel balancing is a promising approach that first estimates a kernel that captures similarity across units and then balances a (possibly low-dimensional) summary of that kernel, indirectly learning important features to balance. In this paper, we propose forest kernel balancing, which leverages the underappreciated fact that tree-based machine learning models, namely random forests and Bayesian additive regression trees (BART), implicitly estimate a kernel based on the co-occurrence of observations in the same terminal leaf node. Thus, even though the resulting kernel is solely a function of baseline features, the selected nonlinearities and other interactions are important for predicting the outcome – and therefore are important for addressing confounding. Through simulations and applied illustrations, we show that forest kernel balancing leads to meaningful computational and statistical improvement relative to standard kernel methods, which do not incorporate outcome information when learning features.

💡 Research Summary

The paper “Forest Kernel Balancing Weights: Outcome-Guided Features for Causal Inference” addresses a fundamental challenge in observational causal inference: selecting which features to balance when weighting control units to match the treated group distribution. While balancing covariates is central to removing confounding bias, traditional methods often struggle to capture the complex, non-linear relationships in the outcome model that are crucial for effective confounding adjustment.

The authors propose a novel method called Forest Kernel Balancing. This approach innovatively leverages the fact that tree-based ensemble models—specifically Random Forests (RF) and Bayesian Additive Regression Trees (BART)—implicitly estimate a similarity kernel between units based on how often they co-occur in the same terminal leaf nodes across many trees. Critically, because these models are trained to predict the outcome variable (using only control group data), the resulting “forest kernel” encodes non-linearities and interactions that are most relevant for predicting the potential outcome. Although the kernel is still a function of only the baseline covariates, its structure is now outcome-guided, ensuring it captures features pertinent to addressing confounding.

The methodological framework integrates this forest kernel into the established kernel balancing workflow. After training an RF or BART model on the control group, a kernel matrix K is constructed for all units. A low-dimensional spectral approximation of K (e.g., its top principal components) is then computed. Finally, weights for the control units are optimized to balance these kernel-derived features against the treated group mean, typically using a regularized balancing objective that trades off imbalance and weight dispersion.

Through comprehensive simulations, the paper demonstrates that forest kernel balancing yields meaningful improvements over standard design-based kernel methods (e.g., polynomial or Gaussian kernels). It consistently achieves lower bias and root-mean-square error (RMSE) in estimating the Average Treatment Effect on the Treated (ATT), especially in settings with complex, non-linear outcome surfaces. The BART-based kernel often shows slightly more robust performance due to its Bayesian averaging over MCMC samples. The method also offers computational benefits, as the forest kernel tends to have lower effective dimensionality.

The practical utility of the approach is illustrated with two applied examples: evaluating an educational training program and studying the effects of child soldiering. These cases show how forest kernel balancing can be seamlessly integrated into real-world observational study design. In conclusion, the paper successfully bridges machine learning and causal inference by using powerful predictive models to inform the design stage, creating a more targeted and efficient approach to covariate balancing that directly incorporates outcome information.