Reconstructing signals from noisy data with unknown signal and noise covariance

We derive a method to reconstruct Gaussian signals from linear measurements with Gaussian noise. This new algorithm is intended for applications in astrophysics and other sciences. The starting point of our considerations is the principle of minimum Gibbs free energy which was previously used to derive a signal reconstruction algorithm handling uncertainties in the signal covariance. We extend this algorithm to simultaneously uncertain noise and signal covariances using the same principles in the derivation. The resulting equations are general enough to be applied in many different contexts. We demonstrate the performance of the algorithm by applying it to specific example situations and compare it to algorithms not allowing for uncertainties in the noise covariance. The results show that the method we suggest performs very well under a variety of circumstances and is indeed qualitatively superior to the other methods in cases where uncertainty in the noise covariance is present.

💡 Research Summary

The paper addresses the problem of reconstructing a Gaussian signal from linear measurements contaminated by Gaussian noise when both the signal covariance matrix (S) and the noise covariance matrix (N) are unknown. Starting from the standard linear model d = R s + n, where R is a known response operator, the authors assume zero‑mean Gaussian priors for the signal and the noise, each characterized solely by its covariance. They parameterize S and N as sums over known projection operators onto eigenspaces, with unknown scalar eigenvalues p_k and η_j respectively. Independent inverse‑Gamma priors (a generalized Jeffreys prior) are placed on each eigenvalue, allowing the algorithm to learn both the scale and the structure of the covariances from the data.

To perform Bayesian inference, the joint probability P(s,d) is obtained by marginalising over the hyper‑parameters p_k and η_j. The resulting expression defines an information Hamiltonian H = –log P(s,d). The posterior P(s|d) is approximated by a Gaussian G(s‑m,D) whose mean m and covariance D are to be determined. By identifying the posterior with a canonical distribution at temperature T and constructing the Gibbs free energy G = U – T S_B (where U is the internal energy and S_B the Boltzmann entropy), the authors show that minimising G for T=1 is equivalent to minimising the Kullback‑Leibler divergence between the true posterior and the Gaussian approximation.

Carrying out functional derivatives of the approximate Gibbs energy with respect to m and D yields a set of coupled fixed‑point equations:

• m = D j,
• j = Σ_j δ_j \tilde{r}_j R† N_j⁻¹ d,
• D⁻¹ = Σ_k γ_k \tilde{q}_k S_k⁻¹ + Σ_j δ_j \tilde{r}_j R† N_j⁻¹ R, where the auxiliary quantities \tilde{q}_k and \tilde{r}_j are expectations of the quadratic forms involving the current estimates of m and D, and γ_k, δ_j incorporate the shape parameters of the inverse‑Gamma priors. The eigenvalues themselves are updated via
• p_k = q_k + ½ tr