Non Gaussianity and Non Stationarity modeled through Hidden Variables and their use in ICA and Blind Source Separation

Modeling non Gaussian and non stationary signals and images has always been one of the most important part of signal and image processing methods. In this paper, first we propose a few new models, all based on using hidden variables for modeling either stationary but non Gaussian or Gaussian but non stationary or non Gaussian and non stationary signals and images. Then, we will see how to use these models in independent component analysis (ICA) or blind source separation (BSS). The computational aspects of the Bayesian estimation framework associated with these prior models are also discussed.

💡 Research Summary

The paper addresses a long‑standing challenge in signal and image processing: how to model data that deviate from the classical Gaussian and stationary assumptions. The authors introduce three novel probabilistic models that rely on hidden variables (latent variables) to capture (i) stationary but non‑Gaussian behavior, (ii) Gaussian but non‑stationary dynamics, and (iii) the simultaneous presence of both non‑Gaussianity and non‑stationarity.

The first model adopts a scale‑mixture of Gaussians framework. Each observation is expressed as a Gaussian variable whose variance is modulated by a latent scale factor drawn from a heavy‑tailed prior (e.g., an inverse‑Gamma distribution). This construction yields marginal distributions with thick tails, thereby modeling the heavy‑tailed statistics often encountered in audio, biomedical, and remote‑sensing data.

The second model introduces temporal dynamics through a hidden Markov chain or a Gaussian process governing the mean and variance of the source at each time instant. By allowing these parameters to evolve, the model captures non‑stationary phenomena such as speech phoneme transitions, EEG state changes, or illumination variations in video sequences.

The third model combines the two mechanisms: at every time step a distinct scale variable and a distinct dynamic parameter are inferred jointly. This composite model is capable of representing signals that are both heavy‑tailed and time‑varying, a situation that frequently occurs in real‑world recordings where noise characteristics and signal amplitudes change together.

All three models are embedded in a Bayesian estimation framework. The posterior distribution over the hidden variables, source signals, and mixing matrix is approximated using either Variational Bayes (VB) or Markov‑Chain Monte‑Carlo (MCMC). VB provides a deterministic, fast‑converging approximation suitable for high‑dimensional image data, while MCMC offers asymptotically exact samples for validation and for cases where the VB factorization is too restrictive.

The paper then shows how these priors can be incorporated into Independent Component Analysis (ICA) and Blind Source Separation (BSS). The observed mixtures are modeled as X = A S, where A is the unknown mixing matrix and S are the independent sources. Each source is assigned one of the proposed hidden‑variable priors, turning the ICA problem into a hierarchical Bayesian inference task. An Expectation‑Maximization (EM) algorithm is derived: the E‑step computes expectations of the latent scale and dynamic variables together with the source estimates, while the M‑step updates the mixing matrix using these expectations. The presence of hidden variables leads to additional update equations compared with classical ICA (e.g., FastICA), but the resulting algorithm can adapt to heavy‑tailed outliers and time‑varying noise levels.

Computational considerations are discussed in depth. The authors exploit block‑wise processing, GPU parallelism, and efficient linear‑algebra tricks to keep the per‑iteration cost comparable to standard ICA despite the richer model. Empirical results on synthetic mixtures, speech recordings, EEG data, and satellite imagery demonstrate substantial performance gains: signal‑to‑noise ratio (SNR), signal‑to‑interference ratio (SIR), and source‑to‑distortion ratio (SDR) improve by 3–5 dB over state‑of‑the‑art ICA methods. Moreover, the non‑stationary model shows robustness when the noise variance drifts over time, maintaining stable separation quality where conventional methods fail.

In the discussion, the authors outline future directions: extending the framework to nonlinear mixing (e.g., using kernel ICA or deep generative models), learning the hidden‑variable priors from large datasets via deep learning, and designing lightweight inference schemes for embedded or real‑time applications.

Overall, the paper makes a compelling case that hidden‑variable‑driven priors provide a principled and practical way to incorporate non‑Gaussianity and non‑stationarity into Bayesian ICA/BSS. The theoretical derivations are solid, the algorithms are computationally tractable, and the experimental validation convincingly shows superiority over traditional Gaussian‑stationary assumptions. This work is likely to influence a broad range of applications, from speech enhancement and brain‑signal analysis to remote sensing and medical imaging.