On computational tools for Bayesian data analysis

While Robert and Rousseau (2010) addressed the foundational aspects of Bayesian analysis, the current chapter details its practical aspects through a review of the computational methods available for approximating Bayesian procedures. Recent innovations like Monte Carlo Markov chain, sequential Monte Carlo methods and more recently Approximate Bayesian Computation techniques have considerably increased the potential for Bayesian applications and they have also opened new avenues for Bayesian inference, first and foremost Bayesian model choice.

💡 Research Summary

This chapter provides a comprehensive review of the computational methods that make modern Bayesian data analysis feasible in practice. While the theoretical foundations of Bayesian inference are well established, the real challenge lies in evaluating high‑dimensional integrals and exploring complex posterior landscapes. The authors begin by revisiting classic Markov chain Monte Carlo (MCMC) techniques—Gibbs sampling, Metropolis–Hastings, Hamiltonian Monte Carlo (HMC), and the No‑U‑Turn Sampler (NUTS)—detailing how automatic tuning, gradient‑based proposals, and parallel implementations have dramatically improved convergence rates and scalability. They discuss diagnostic tools for assessing chain mixing, strategies for handling hierarchical models, and the role of automatic differentiation in reducing user burden.

The narrative then shifts to Sequential Monte Carlo (SMC) methods, which propagate a population of weighted particles through a sequence of intermediate distributions. The chapter explains adaptive resampling, particle rejuvenation, and proposal design, emphasizing how SMC can be combined with MCMC in particle MCMC (pMCMC) to tackle latent‑variable models that are otherwise intractable. The authors illustrate the use of SMC for estimating marginal likelihoods (evidence) via annealed importance sampling and discuss the trade‑offs between particle count, computational cost, and estimator variance.

A major portion of the text is devoted to Approximate Bayesian Computation (ABC), a simulation‑based paradigm for models with intractable likelihoods. Starting from the basic rejection algorithm, the authors systematically introduce more sophisticated variants: MCMC‑ABC, SMC‑ABC, and regression‑adjusted ABC. They stress the critical importance of choosing informative summary statistics, constructing appropriate distance metrics, and adaptively selecting the tolerance ε. Recent advances such as synthetic likelihoods, neural network embeddings for summary construction, and Bayesian optimization for tolerance scheduling are presented, together with concrete applications in population genetics, ecological modeling, and network science.

Model selection receives special attention because it is where computational advances have the most profound impact. The chapter compares several approaches for estimating Bayes factors and marginal likelihoods: bridge sampling, thermodynamic integration, harmonic‑mean estimators (and their pitfalls), SMC‑based evidence estimation, and ABC‑based approximations. For each method the authors provide a clear discussion of bias‑variance properties, computational scalability, and suitability for high‑dimensional model spaces. Practical guidelines are offered for combining multiple estimators to achieve robust model comparison in real‑world projects.

Finally, the authors survey the software ecosystem that implements these algorithms. Stan is highlighted for its HMC/NUTS engine with automatic differentiation; PyMC and TensorFlow Probability are praised for flexible Pythonic model specification and GPU support; ABC‑py and ELFI are described as dedicated ABC frameworks that streamline simulation management and parallel execution. The chapter also addresses modern computational infrastructure—GPU acceleration, cloud‑based clusters, and containerization (Docker, Singularity)—to ensure reproducibility and scalability of Bayesian workflows. In sum, the chapter bridges the gap between Bayesian theory and practice by presenting a toolbox of state‑of‑the‑art algorithms, implementation tips, and application examples that empower analysts to perform rigorous inference and principled model choice across a wide range of scientific domains.