A note on target distribution ambiguity of likelihood-free samplers

Methods for Bayesian simulation in the presence of computationally intractable likelihood functions are of growing interest. Termed likelihood-free samplers, standard simulation algorithms such as Markov chain Monte Carlo have been adapted for this setting. In this article, by presenting generalisations of existing algorithms, we demonstrate that likelihood-free samplers can be ambiguous over the form of the target distribution. We also consider the theoretical justification of these samplers. Distinguishing between the forms of the target distribution may have implications for the future development of likelihood-free samplers.

💡 Research Summary

The paper addresses a subtle but important source of ambiguity in likelihood‑free Bayesian samplers, such as Approximate Bayesian Computation (ABC) and its many MCMC, SMC, and PMC extensions. In settings where the likelihood is intractable, these algorithms replace the exact likelihood with a simulation‑based discrepancy between observed data and data generated from proposed parameter values. While this approach has become standard, the authors point out that the literature often fails to specify which probability distribution the algorithm is actually targeting.

Two distinct targets are identified. The first is the joint (or augmented) distribution of the model parameters θ together with auxiliary variables u that represent simulated data, summary statistics, or any other latent quantities introduced by the algorithm: p(θ,u | y). The second is the marginal posterior of the parameters alone, obtained by integrating out the auxiliary variables: p(θ | y). Whether an algorithm samples from the joint distribution or only from the marginal posterior has profound implications for its theoretical justification, convergence properties, and practical performance.

To make the distinction concrete, the authors rewrite a broad class of likelihood‑free methods within a unified framework. Each iteration consists of (i) proposing a new parameter value θ′ from a proposal kernel q(θ′|θ), (ii) generating auxiliary data u′ from the model g(u′|θ′), (iii) computing a distance ρ(s(u′),s(y)) between summary statistics of the simulated and observed data, and (iv) deciding whether to accept the move. The acceptance probability can be constructed in two ways. If it includes the ratio of the auxiliary‑variable densities g(u′|θ′)/g(u|θ) (or an equivalent weight), the resulting Markov chain has p(θ,u | y) as its invariant distribution. If the acceptance rule depends only on the indicator 𝟙{ρ≤ε}, the chain is designed to approximate the marginal posterior p(θ | y) but does not, in general, satisfy detailed balance with respect to that distribution unless ε→0 and the summaries are sufficient.

The same ambiguity appears in sequential methods. In ABC‑SMC or PMC‑ABC, particle weights are typically proportional to a kernel Kε(ρ) multiplied by the prior density. Whether the weight also contains the auxiliary‑variable density term determines if the particle system targets the joint distribution or the marginal posterior. The authors provide rigorous proofs that, under the joint‑target formulation, standard convergence results for Markov chains and sequential Monte Carlo hold. By contrast, when only the marginal posterior is intended, the algorithm may suffer from bias that does not vanish unless the tolerance ε is driven to zero at an appropriate rate and the summary statistics are sufficient.

Empirical illustrations are presented on a simple Gaussian model and a more complex biological simulation. When the joint target is used, the auxiliary variables’ posterior is recovered alongside accurate parameter estimates, and credible intervals achieve nominal coverage. When the marginal‑only formulation is employed with a relatively large ε, the posterior estimates become biased, especially in multimodal settings where the chain can become trapped in a single mode. Computationally, the joint‑target approach incurs extra cost because the auxiliary variables must be stored and propagated, but the authors argue that this cost is justified when downstream inference (e.g., model checking, predictive simulation) requires knowledge of the full augmented posterior.

In the discussion, the authors stress that practitioners must explicitly decide which target is appropriate for their problem. If the goal is solely parameter inference and computational resources are limited, a marginal‑target algorithm may be acceptable, provided that ε is sufficiently small and the summaries are carefully chosen. If, however, one needs the full posterior over simulated data—for example, to assess model fit, perform posterior predictive checks, or to reuse simulated datasets—then the joint‑target formulation is indispensable. The unified framework proposed in the paper offers a clear recipe for constructing either type of algorithm while preserving theoretical guarantees.

Finally, the paper outlines directions for future work: adaptive schemes that automatically tune ε and the weighting of auxiliary‑variable densities, methods for selecting or learning sufficient summary statistics, and scalable strategies for sampling from high‑dimensional joint posteriors. By clarifying the target‑distribution ambiguity, the authors provide a foundation for more robust and transparent development of likelihood‑free inference methods.