Approximating the marginal likelihood using copula

Model selection is an important activity in modern data analysis and the conventional Bayesian approach to this problem involves calculation of marginal likelihoods for different models, together with diagnostics which examine specific aspects of model fit. Calculating the marginal likelihood is a difficult computational problem. Our article proposes some extensions of the Laplace approximation for this task that are related to copula models and which are easy to apply. Variations which can be used both with and without simulation from the posterior distribution are considered, as well as use of the approximations with bridge sampling and in random effects models with a large number of latent variables. The use of a t-copula to obtain higher accuracy when multivariate dependence is not well captured by a Gaussian copula is also discussed.

💡 Research Summary

Model selection in the Bayesian framework hinges on the accurate evaluation of marginal likelihoods, a task that quickly becomes computationally prohibitive as the dimensionality of the parameter space grows or when the posterior distribution exhibits strong non‑Gaussian features. The traditional Laplace approximation, which expands the log‑posterior around its mode and approximates the posterior by a multivariate normal distribution, works well only when the posterior is close to Gaussian and the curvature is well captured by the Hessian. In many realistic applications—hierarchical models with many random effects, mixture models, or models with heavy‑tailed errors—these conditions are violated, leading to substantial bias in the estimated marginal likelihood.

The authors propose a principled extension of the Laplace method that leverages copula theory to separate marginal behavior from dependence structure. The key idea is to model each parameter’s marginal distribution individually (using kernel density estimates, parametric families, or analytic approximations) and then to couple these marginals with a copula that captures the multivariate dependence. Two copula families are examined: the Gaussian copula, which is computationally convenient and captures linear correlation through a correlation matrix, and the Student‑t copula, which adds a degrees‑of‑freedom parameter to model tail dependence and therefore provides a more accurate representation when the posterior exhibits heavy tails or asymmetric dependence.

Two practical variants of the approach are developed. The first, a “sample‑based copula approximation,” assumes that posterior draws (e.g., from an MCMC run) are available. Empirical marginal distributions are constructed from the draws, and the pseudo‑observations (ranks transformed to the unit interval) are used to estimate the copula parameters via maximum likelihood or method‑of‑moments. This variant inherits the accuracy of the underlying Monte‑Carlo sample and can be made arbitrarily precise by increasing the number of draws. The second, a “Laplace‑based copula approximation,” requires only the posterior mode, the gradient, and the Hessian at the mode—exactly the information already computed in a standard Laplace approximation. The Hessian provides a Gaussian approximation to the dependence structure, which is then refined by replacing the normal marginals with more appropriate univariate approximations (e.g., skew‑normal, gamma). The resulting copula‑augmented density serves as a more faithful surrogate for the true posterior, especially in the presence of skewness or kurtosis.

A major contribution of the paper is the integration of these copula surrogates with bridge sampling, a technique that estimates the ratio of normalizing constants between two distributions. By using the copula‑based density as the proposal distribution in bridge sampling, the authors achieve a proposal that is much closer to the target posterior than a simple Gaussian, thereby reducing variance and improving effective sample size. This synergy is particularly valuable for hierarchical models with a large number of latent variables, where the dimensionality of the integration problem can be in the hundreds or thousands. In such settings, the authors demonstrate that the copula‑augmented bridge estimator dramatically outperforms both the plain Laplace estimator and bridge sampling with a naïve Gaussian proposal.

The empirical evaluation comprises three sets of experiments. First, synthetic data are generated from multivariate distributions with known marginal shapes (e.g., skew‑normal, beta) and known dependence structures (Gaussian and t‑copulas). The copula‑augmented Laplace estimator recovers the true marginal likelihood with relative errors typically below 5 %, whereas the standard Laplace method exhibits errors up to 30 % in the same scenarios. Second, the method is applied to a Bayesian logistic regression with random intercepts for a large number of groups, a classic example of a high‑dimensional random‑effects model. Here, the t‑copula version yields a log‑marginal‑likelihood estimate that is within 0.2 log units of a long‑run thermodynamic integration benchmark, while the Gaussian copula is within 0.5 log units. Third, a real‑world medical dataset involving hierarchical survival models is analyzed; the copula‑based bridge estimator provides stable estimates across different bridge tuning parameters, whereas the ordinary Laplace estimate varies dramatically with the choice of prior scale.

The paper also discusses limitations and future directions. Estimating high‑dimensional marginal densities remains challenging; the authors suggest possible remedies such as dimension‑reduction techniques, parametric marginal families fitted via variational Bayes, or the use of normalizing flows. Automatic selection of the t‑copula degrees of freedom could be incorporated via profile likelihood or Bayesian model averaging. Moreover, the framework is not restricted to Gaussian or t‑copulas; other Archimedean copulas (Clayton, Gumbel) could be explored to capture asymmetric tail dependence. Finally, the authors envision coupling the copula surrogate with modern deep‑learning based posterior approximations (e.g., normalizing‑flow variational inference) to create hybrid estimators that retain the interpretability of copulas while leveraging the expressive power of neural networks.

In summary, the article delivers a versatile, computationally efficient extension of the Laplace approximation that harnesses copula theory to better capture marginal non‑Gaussianity and complex dependence in posterior distributions. By providing both sample‑based and analytic variants, and by demonstrating seamless integration with bridge sampling, the authors equip practitioners with a practical toolkit for accurate marginal likelihood estimation in a wide array of Bayesian models, from modest regression problems to large‑scale hierarchical and random‑effects structures.