Bayesian Inference for Gaussian Mixed Graph Models

We introduce priors and algorithms to perform Bayesian inference in Gaussian models defined by acyclic directed mixed graphs. Such a class of graphs, composed of directed and bi-directed edges, is a representation of conditional independencies that is closed under marginalization and arises naturally from causal models which allow for unmeasured confounding. Monte Carlo methods and a variational approximation for such models are presented. Our algorithms for Bayesian inference allow the evaluation of posterior distributions for several quantities of interest, including causal effects that are not identifiable from data alone but could otherwise be inferred where informative prior knowledge about confounding is available.

💡 Research Summary

The paper introduces a comprehensive Bayesian framework for Gaussian models defined by acyclic directed mixed graphs (ADMGs), which combine directed edges with bidirected edges to encode conditional independencies that arise from latent confounding. The authors first construct appropriate prior distributions that respect the graphical structure: normal priors for regression coefficients, inverse‑Wishart priors for error variances, and additional inverse‑Wishart components for the covariance blocks induced by bidirected edges. By embedding the structural constraints directly into the priors, the resulting posterior distribution remains coherent under marginalization and reflects both parameter uncertainty and graph‑level uncertainty.

Two inference algorithms are presented. The first is a Gibbs‑sampling based Markov chain Monte Carlo (MCMC) scheme that exploits the conditional conjugacy of each parameter block (β, σ², Θ) to draw exact samples from their full conditional distributions. The ADMG constraints reduce the dimensionality of the covariance block Θ, improving mixing. The second algorithm is a variational Bayesian (VB) approach that adopts a mean‑field factorization q(β)q(σ²)q(Θ). Coordinate ascent updates are derived analytically, and the bidirected‑edge constraints are enforced through Lagrange multipliers within the variational objective. VB provides a fast deterministic approximation, scaling to hundreds of variables where MCMC becomes computationally burdensome.

A central contribution is the treatment of causal effects that are non‑identifiable from observational data alone. In traditional structural equation modeling, latent confounders render certain directed effects (e.g., X → Y) unidentifiable. Within the Bayesian ADMG framework, informative priors on the latent covariance structure encode expert knowledge about the strength and direction of confounding. The posterior then yields a distribution over the causal effect, allowing inference even when the effect is not point‑identified. Simulation studies demonstrate that, with reasonably informative priors, the posterior mean of a non‑identifiable effect closely matches the true effect, while the posterior variance correctly reflects remaining uncertainty. An empirical application to a socioeconomic dataset shows that incorporating expert priors on unmeasured confounding leads to causal estimates that differ substantially from those obtained by purely frequentist methods, highlighting the practical value of the approach.

The experimental evaluation compares MCMC and VB in terms of accuracy, convergence speed, and scalability. MCMC achieves high fidelity to the true posterior but requires thousands of iterations and exhibits slower mixing for high‑dimensional covariance blocks. VB converges within a few dozen iterations, delivering accurate approximations of marginal means and variances, and remains stable as the number of variables grows. Both methods are sensitive to prior specification; weaker priors allow the data to dominate, while stronger priors shift the posterior toward the prior belief, as expected.

Finally, the authors discuss extensions and future work. They propose generalizing the framework to non‑Gaussian exponential family outcomes (binary, count), integrating structure learning with prior elicitation to jointly infer graph topology and parameters, and developing sparse representations of the latent covariance to handle very high‑dimensional confounding structures.

In summary, this work provides the first unified Bayesian treatment of Gaussian ADMGs, offering both exact (MCMC) and approximate (variational) inference tools, and demonstrates how prior knowledge about latent confounding can be leveraged to estimate causal effects that are otherwise non‑identifiable. The methodology bridges causal inference, graphical modeling, and Bayesian computation, opening new avenues for rigorous analysis in fields where unmeasured confounding is a central concern.