A copula based approach to adaptive sampling

Our article is concerned with adaptive sampling schemes for Bayesian inference that update the proposal densities using previous iterates. We introduce a copula based proposal density which is made more efficient by combining it with antithetic variable sampling. We compare the copula based proposal to an adaptive proposal density based on a multivariate mixture of normals and an adaptive random walk Metropolis proposal. We also introduce a refinement of the random walk proposal which performs better for multimodal target distributions. We compare the sampling schemes using challenging but realistic models and priors applied to real data examples. The results show that for the examples studied, the adaptive independent \MH{} proposals are much more efficient than the adaptive random walk proposals and that in general the copula based proposal has the best acceptance rates and lowest inefficiencies.

💡 Research Summary

The paper addresses the problem of improving the efficiency of Markov chain Monte Carlo (MCMC) sampling for Bayesian inference by adapting the proposal distribution as the chain progresses. Traditional adaptive schemes such as the adaptive random‑walk Metropolis (RWM) and adaptive mixtures of multivariate normal (MVN) proposals often struggle with high‑dimensional, non‑Gaussian, or multimodal target distributions, leading to low acceptance rates and slow mixing. To overcome these limitations, the authors propose a novel adaptive independent Metropolis–Hastings (MH) sampler that (i) constructs a proposal density using a copula framework, and (ii) augments it with antithetic variable sampling to reduce variance in the proposal step.

Copula‑based proposal. At each iteration the algorithm estimates the marginal posterior distribution of each parameter from the accumulated samples (using kernel density estimation or a parametric fit). A Gaussian copula is then fitted to capture the dependence structure among the marginals. The resulting joint proposal density is the product of the estimated marginals linked by the copula correlation matrix, which is updated online using the empirical covariance of the chain. This construction allows the proposal to mimic the shape, tail behaviour, and inter‑parameter correlations of the target distribution far more faithfully than a simple multivariate normal mixture.

Antithetic sampling. For every draw from the copula proposal a paired “antithetic” draw is generated by reflecting the underlying uniform random numbers (u → 1‑u) while preserving the same marginal CDFs and copula correlation. The two proposals are evaluated simultaneously, and the standard Metropolis–Hastings acceptance rule is applied to each. Because the antithetic pair tends to lie on opposite sides of the high‑density region, the combined estimator exhibits reduced Monte‑Carlo variance, leading to higher overall acceptance probabilities and more effective exploration.

Experimental design. The authors benchmark four samplers on three realistic Bayesian models: (1) a multimodal logistic regression with a high‑dimensional coefficient vector, (2) a nonlinear mixed‑effects model with hierarchical random effects, and (3) a structural time‑series model (Bayesian VAR) fitted to real economic data. For each model they compare (a) the proposed copula‑antithetic adaptive independent MH, (b) an adaptive MVN mixture proposal, (c) a standard adaptive RWM, and (d) a refined RWM that incorporates multiple scaling factors and local re‑tuning to aid multimodal exploration. Performance metrics include average acceptance rate, effective sample size (ESS), integrated autocorrelation time (IACT), and ESS per unit of CPU time.

Results. Across all experiments the copula‑antithetic sampler achieves the highest acceptance rates (≈ 0.45–0.62) and the largest ESS per second, often 1.8–3.2 times greater than the best competing method. In the multimodal logistic regression, the plain RWM becomes trapped in a single mode, while the MVN mixture occasionally jumps but still shows poor mixing. The copula‑based proposal, by accurately representing inter‑parameter dependence, moves freely between modes; the antithetic component further boosts acceptance by roughly 5–8 % relative to a non‑antithetic copula proposal. Similar gains are observed in the mixed‑effects and time‑series examples, where the adaptive independent schemes produce substantially lower IACT values and more stable posterior estimates even when the prior is weakly informative.

Discussion and limitations. The study demonstrates that a copula‑driven adaptive proposal can capture complex posterior geometry that normal‑based mixtures miss, and that antithetic sampling is an inexpensive variance‑reduction tool compatible with the independent‑proposal framework. However, fitting a high‑dimensional copula incurs additional computational cost, and the empirical correlation matrix may become unstable when the number of parameters exceeds a few hundred. The authors suggest extensions such as using t‑copulas for heavier tails, imposing sparsity on the correlation matrix, and developing online updating rules for streaming data.

Conclusion. By integrating copula modeling with antithetic sampling, the authors deliver an adaptive independent MH algorithm that outperforms conventional adaptive RWM and MVN mixture proposals on challenging Bayesian problems. The method offers a practical, statistically principled way to enhance MCMC efficiency for high‑dimensional, non‑Gaussian, and multimodal posterior distributions, making it a valuable addition to the toolbox of Bayesian practitioners.