Computational aspects of Bayesian spectral density estimation

Gaussian time-series models are often specified through their spectral density. Such models present several computational challenges, in particular because of the non-sparse nature of the covariance matrix. We derive a fast approximation of the likelihood for such models. We propose to sample from the approximate posterior (that is, the prior times the approximate likelihood), and then to recover the exact posterior through importance sampling. We show that the variance of the importance sampling weights vanishes as the sample size goes to infinity. We explain why the approximate posterior may typically multi-modal, and we derive a Sequential Monte Carlo sampler based on an annealing sequence in order to sample from that target distribution. Performance of the overall approach is evaluated on simulated and real datasets. In addition, for one real world dataset, we provide some numerical evidence that a Bayesian approach to semi-parametric estimation of spectral density may provide more reasonable results than its Frequentist counter-parts.

💡 Research Summary

The paper tackles the long‑standing computational bottleneck in Gaussian time‑series models that are defined through their spectral density. Because the covariance matrix is generally dense, exact likelihood evaluation scales poorly with the series length. The authors derive a fast approximation to the likelihood by moving to the frequency domain: after applying the discrete Fourier transform to the observed series, each Fourier coefficient is treated as approximately independent Gaussian, yielding an O(n log n) likelihood approximation. This “approximate likelihood” is then combined with a prior to form an approximate posterior distribution.

Sampling directly from this approximate posterior is straightforward, but it does not equal the true posterior. To correct this, the authors employ importance sampling: each draw θ_i from the approximate posterior is re‑weighted by the ratio of the true posterior density to the approximate one. A key theoretical contribution is the proof that the variance of these importance weights vanishes as the number of samples N → ∞, guaranteeing that the importance‑sampling correction becomes exact in the large‑sample limit.

A practical difficulty arises because the approximate posterior is often multimodal. The multimodality stems from the semi‑parametric nature of the spectral density and the non‑convexity of the likelihood in the parameter space. Simple Markov chain Monte Carlo (MCMC) would tend to get trapped in a single mode. To overcome this, the authors design a Sequential Monte Carlo (SMC) sampler that uses an annealing sequence β∈