Approximate Bayesian Computation: a nonparametric perspective

Approximate Bayesian Computation is a family of likelihood-free inference techniques that are well-suited to models defined in terms of a stochastic generating mechanism. In a nutshell, Approximate Bayesian Computation proceeds by computing summary statistics s_obs from the data and simulating summary statistics for different values of the parameter theta. The posterior distribution is then approximated by an estimator of the conditional density g(theta|s_obs). In this paper, we derive the asymptotic bias and variance of the standard estimators of the posterior distribution which are based on rejection sampling and linear adjustment. Additionally, we introduce an original estimator of the posterior distribution based on quadratic adjustment and we show that its bias contains a fewer number of terms than the estimator with linear adjustment. Although we find that the estimators with adjustment are not universally superior to the estimator based on rejection sampling, we find that they can achieve better performance when there is a nearly homoscedastic relationship between the summary statistics and the parameter of interest. To make this relationship as homoscedastic as possible, we propose to use transformations of the summary statistics. In different examples borrowed from the population genetics and epidemiological literature, we show the potential of the methods with adjustment and of the transformations of the summary statistics. Supplemental materials containing the details of the proofs are available online.

💡 Research Summary

This paper provides a rigorous non‑parametric analysis of Approximate Bayesian Computation (ABC), focusing on the asymptotic properties of three widely used estimators of the posterior distribution: simple rejection sampling, linear regression adjustment, and a newly proposed quadratic adjustment. The authors begin by formalising ABC as the problem of estimating the conditional density (g(\theta \mid s_{\text{obs}})) where (s_{\text{obs}}) are summary statistics computed from the observed data. In the basic rejection‑sampling scheme, parameter draws (\theta) are generated from the prior, simulated summary statistics (s) are produced, and those draws for which (|s-s_{\text{obs}}|\le\varepsilon) are retained. By expanding the joint density of ((\theta,s)) around ((\theta_0,s_{\text{obs}})), the authors derive that the bias of the resulting posterior estimator is of order (\varepsilon^{2}) while its variance scales as (O\bigl(1/(n h^{d})\bigr)) (with (n) the number of simulations, (h) the kernel bandwidth, and (d) the dimension of the summary statistics).

Linear adjustment, introduced by Beaumont et al. (2002), fits a local linear regression (\theta = a + b^{\top}(s-s_{\text{obs}}) + \varepsilon) to the retained draws and then corrects each (\theta) to the value it would have at (s_{\text{obs}}). The paper shows that this correction reduces the leading bias term to order (\varepsilon^{3}) while leaving the variance essentially unchanged. However, the analysis also reveals a crucial limitation: the linear correction assumes a homoscedastic relationship between (\theta) and (s). When the conditional variance (\operatorname{Var}(\theta\mid s)) varies with (s) (heteroscedasticity) or the mean relationship is markedly non‑linear, the linear adjustment can introduce additional bias that outweighs its theoretical advantage.

To address these shortcomings, the authors propose a quadratic adjustment. They fit a second‑order local regression (\theta = a + b^{\top}(s-s_{\text{obs}}) + (s-s_{\text{obs}})^{\top}C(s-s_{\text{obs}}) + \varepsilon) and use the fitted model to map each retained draw back to the observed summary statistics. The asymptotic bias expansion for this estimator contains fewer terms than the linear case: the first‑order bias component cancels, leaving a bias of order (\varepsilon^{4}). The variance again remains of the same order as in the rejection sampler, meaning that the quadratic adjustment can achieve a strictly smaller mean‑squared error whenever the underlying (\theta)–(s) relationship is close to homoscedastic and moderately smooth.

Recognising that homoscedasticity is rarely satisfied in raw summary statistics, the paper devotes a substantial part to transformations of the summaries. By applying monotone transformations such as logarithms, square‑roots, or Box‑Cox powers, the authors demonstrate empirically that the conditional variance of (\theta) can be stabilised across the range of (s). After transformation, both linear and quadratic adjustments exhibit markedly reduced bias and variance, often outperforming the plain rejection sampler even when the tolerance (\varepsilon) is relatively large.

The theoretical developments are illustrated with two applied examples. The first concerns population genetics: estimating the selection coefficient in a Wright–Fisher diffusion model using allele‑frequency summaries. Transforming the allele‑frequency variance and applying the quadratic adjustment yields posterior credible intervals that are both narrower and better centred on the true selection coefficient than those obtained by rejection sampling. The second example involves epidemiological modelling of an SIR (susceptible–infectious–recovered) process, where the goal is to infer transmission and recovery rates from time‑series counts of infected individuals. Here, a square‑root transformation of the infection counts makes the variance of the rates approximately constant, and a linear adjustment suffices to produce posterior estimates with substantially lower mean‑squared error than the baseline ABC.

Supplementary material provides full proofs of the bias and variance expansions, as well as code for reproducing the simulation studies. In conclusion, the paper clarifies that adjustment methods are not universally superior to simple rejection sampling; their advantage depends critically on the degree of homoscedasticity in the summary‑statistic–parameter relationship. By judiciously transforming summary statistics, practitioners can often bring this relationship close to the homoscedastic regime, thereby unlocking the full potential of linear or quadratic adjustments to achieve more accurate, efficient, and robust likelihood‑free Bayesian inference.