On a Class of Shrinkage Priors for Covariance Matrix Estimation

We propose a flexible class of models based on scale mixture of uniform distributions to construct shrinkage priors for covariance matrix estimation. This new class of priors enjoys a number of advantages over the traditional scale mixture of normal priors, including its simplicity and flexibility in characterizing the prior density. We also exhibit a simple, easy to implement Gibbs sampler for posterior simulation which leads to efficient estimation in high dimensional problems. We first discuss the theory and computational details of this new approach and then extend the basic model to a new class of multivariate conditional autoregressive models for analyzing multivariate areal data. The proposed spatial model flexibly characterizes both the spatial and the outcome correlation structures at an appealing computational cost. Examples consisting of both synthetic and real-world data show the utility of this new framework in terms of robust estimation as well as improved predictive performance.

💡 Research Summary

The paper introduces a novel Bayesian framework for estimating covariance (or precision) matrices in high‑dimensional settings by employing scale mixtures of uniform (SMU) distributions as shrinkage priors. Traditional Bayesian shrinkage priors for covariance estimation have largely relied on scale mixtures of normal distributions (e.g., normal‑Gamma, horseshoe) or on the conjugate G‑Wishart prior. While theoretically appealing, these approaches suffer from two major drawbacks: (1) the prior densities are often analytically intractable, making Gibbs sampling impossible and forcing reliance on Metropolis‑Hastings or other local proposal schemes; (2) the G‑Wishart prior requires computation of a normalizing constant that is unavailable in closed form for non‑decomposable graphs, leading to expensive Monte‑Carlo approximations.

The authors observe that any symmetric, unimodal density can be expressed as a scale mixture of uniform distributions (Theorem 1). Concretely, for a density π(θ) with mode at zero, one can write

π(θ)=∫₀^∞ ½ t⁻¹ 1{|θ|<t} h(t) dt,

where the mixing density h(t) is proportional to –2t π′(t). This representation yields a simple hierarchical model: conditional on a latent scale t, the parameter θ follows a uniform distribution on (–t, t). The latent t itself has a tractable conditional distribution that can be sampled directly via the inverse‑cumulative‑distribution method (Theorem 2).

Using this construction, the paper builds a family of shrinkage priors that includes:

• Exponential‑power priors (|θ|^q penalty) with mixing density h(t)∝t^q exp(–t^q/τ^q). Special cases q=2 (Gaussian) and q=1 (Laplace) recover the Bayesian lasso and graphical lasso priors.
• Student‑t priors with h(t)∝t^2 (1+t^2/τ^2)^{-(ν+3)/2}, linking t^2/τ^2 to an inverse‑Beta distribution.
• Generalized double‑Pareto priors where h(t)∝t (1+t/τ)^{-(2+α)}.
• A newly proposed logarithmic prior π(θ)∝log(1+τ^2/θ^2) whose mixing density is half‑Cauchy. This prior possesses an infinite spike at zero and heavy tails, mirroring the desirable properties of the horseshoe prior but with an explicit closed‑form density.

All these priors share the crucial computational advantage that posterior inference can be performed with a pure Gibbs sampler. The Gibbs steps are: (i) sample each element ω_ij of the precision matrix from its truncated normal (or more generally, truncated likelihood) distribution given the current latent scales; (ii) sample each latent scale t_ij from its conditional density proportional to –π′(t)·1{t>|ω_ij|}, which is straightforward because the inverse CDF is analytically available for the priors listed above. No Metropolis‑Hastings acceptance step is needed, and the proposals are global rather than local, leading to faster mixing especially when p (the dimension) is large.

Beyond the basic precision‑matrix model, the authors extend the SMU framework to multivariate conditional autoregressive (CAR) models for areal data. In this setting, both the spatial dependence (through an adjacency matrix) and the cross‑outcome dependence (through a covariance matrix) are shrunk simultaneously using SMU priors. The same data‑augmentation Gibbs scheme applies, preserving computational scalability while allowing flexible modeling of spatial and outcome correlations.

The empirical section conducts two sets of experiments. In synthetic simulations, the SMU priors (especially the logarithmic and exponential‑power families) achieve lower Frobenius norm errors and higher true‑positive rates for edge recovery compared with G‑Wishart and point‑mass priors. The Gibbs sampler exhibits rapid convergence and low autocorrelation even when p≈100 and n is modest. In a real‑world application to multivariate disease incidence across U.S. states, the multivariate CAR model with SMU priors yields higher predictive log‑likelihoods and better cross‑validated performance than competing Bayesian graphical lasso and G‑Wishart approaches. The logarithmic prior, in particular, demonstrates robustness to outliers due to its heavy tails while still aggressively shrinking near‑zero elements.

Overall, the paper makes three substantive contributions:

1. Theoretical unification – it shows that any symmetric unimodal prior can be recast as an SMU, providing a systematic way to design new shrinkage priors with desired tail and spike behavior.
2. Computational innovation – by exploiting the SMU representation, it delivers a fully Gibbs‑based posterior sampler that scales to high dimensions without the need for costly Metropolis steps or normalizing‑constant approximations.
3. Modeling extension – it demonstrates that the SMU framework naturally extends to complex hierarchical models such as multivariate CAR, enabling simultaneous spatial and multivariate shrinkage with modest computational overhead.

The work therefore offers a practical, flexible, and computationally efficient alternative to existing Bayesian shrinkage methods for covariance estimation, with clear advantages in both theory and application.