Relevant statistics for Bayesian model choice

The choice of the summary statistics used in Bayesian inference and in particular in ABC algorithms has bearings on the validation of the resulting inference. Those statistics are nonetheless customarily used in ABC algorithms without consistency checks. We derive necessary and sufficient conditions on summary statistics for the corresponding Bayes factor to be convergent, namely to asymptotically select the true model. Those conditions, which amount to the expectations of the summary statistics to asymptotically differ under both models, are quite natural and can be exploited in ABC settings to infer whether or not a choice of summary statistics is appropriate, via a Monte Carlo validation.

💡 Research Summary

This paper addresses a fundamental issue in Bayesian model selection, especially within the framework of Approximate Bayesian Computation (ABC): the choice of summary statistics T(y) and its impact on the consistency of the resulting Bayes factor. The authors first recall that ABC replaces the full data y by a low‑dimensional statistic T(y) and accepts simulated parameter–data pairs when the simulated statistic is close to the observed one. While this approximation is attractive for intractable likelihoods, the validity of model comparison hinges on whether T(y) contains enough information to discriminate between competing models M₁ and M₂.

The core contribution is a set of necessary and sufficient conditions on T(y) under which the Bayes factor based on the summary, B_T12(y) = m₁(T)/m₂(T), converges to the correct model as the sample size n grows. The authors formalize four technical assumptions (A1–A4). (A1) requires that T_n obey a central‑limit‑type scaling: v_n(T_n−μ₀) converges in distribution to a non‑degenerate limit Q, with v_n→∞. (A2) introduces sieves F_{n,i} in the parameter spaces and imposes tail‑probability control (τ_i, α_i) on the prior mass outside these sieves, together with a uniform concentration bound for the distribution of T_n around its model‑specific mean μ_i(θ_i). (A3) assumes that if a model is “compatible” with the statistic—i.e., the set {μ_i(θ_i)} contains μ₀—then the prior mass of parameters whose means lie within a radius u of μ₀ behaves like u^{d_i} v_n^{-d_i}. (A4) guarantees the existence of a set E_n on which the model‑specific density g_i(t|θ_i) dominates the true density g_n(t) by a fixed factor δ with high probability.

Under these assumptions, Lemma 1 shows that the marginal density of the statistic under model i, m_i(T_n), satisfies
C_l v_n^{-d_i} g_n(T_n) ≤ m_i(T_n) ≤ C_u v_n^{-d_i} g_n(T_n)
when the model is compatible, and decays at a faster rate O(v_n^{-τ_i}+v_n^{-α_i}) when it is not. Consequently, the Bayes factor behaves as

- B_T12(y) → 0 or ∞ if exactly one model is compatible (i.e., the asymptotic mean of T under the true model cannot be reproduced by the wrong model), guaranteeing consistent model selection;

- B_T12(y) converges to a finite non‑degenerate constant when both models are compatible, in which case the statistic does not discriminate asymptotically.

Thus the key condition for consistency is that the asymptotic expectation of T(y) under the two models differ; equivalently, μ₀ must belong to the closure of {μ_1(θ)} but not to that of {μ_2(θ)} (or vice‑versa). If the expectations coincide, the Bayes factor based on T fails to select the true model, regardless of the tolerance ε or the number of simulations.

The paper translates this theoretical insight into a practical validation procedure for ABC practitioners. By simulating a large reference table under each model and estimating the empirical means of the candidate summary statistics, one can test whether the means are distinct. If they are, the statistic is deemed suitable for model choice; if not, it should be discarded or augmented.

Illustrations include a synthetic example contrasting a Normal N(θ,1) model with a Laplace L(θ,1/√2) model. Statistics such as the sample mean, median, and variance have identical expectations under both models and consequently yield Bayes factors that do not separate the models, as shown by overlapping posterior‑model‑probability histograms. In contrast, the median absolute deviation (MAD) has different expectations, leading to Bayes factors that concentrate near 0 or 1 depending on the true model, confirming the theory. Additional simulations with fourth and sixth moments further demonstrate how the rate of convergence improves when the statistic’s expectation differs.

A more realistic application to population genetics (not detailed in the excerpt) showcases the method on high‑dimensional summary vectors, reinforcing that the validation step scales to complex settings.

In conclusion, the authors provide a rigorous characterization of when summary statistics lead to consistent Bayesian model selection, a set of verifiable regularity conditions, and a Monte‑Carlo based diagnostic that can be incorporated into any ABC workflow. This work bridges the gap between theoretical guarantees and practical implementation, offering a clear guideline: before running ABC for model comparison, ensure that the chosen summaries have asymptotically distinct expectations under the competing models.