Noise Level Estimation for Overcomplete Dictionary Learning Based on Tight Asymptotic Bounds

1  Abstract—In this letter, we address the problem of estimating Gaussian noise level from the trained dictionaries in update stage. We first provide rigorous statistical analysis on the eigenvalue distributions of a sample covariance matrix. Then we propose an interval-bounded estimator for noise variance in high dimensional setting. To this end, an effective estimation method for noise level is devised based on the boundness and asymptotic behavior of noise eigenvalue spectrum. The estimation performance of our method has been guaranteed both theoretically and empirically. The analysis and experiment results have demonstrated that the proposed algorithm can reliably infer true noise levels, and outperforms the relevant existing methods.

Index Terms—Dictionary learning, sample covariance matrix, random matrix theory, noise level estimation. I. INTRODUCTION HE dictionary learning is a matrix factorization problem that amounts to finding the linear combination of a given signal N M   Y  with only a few atoms selected from columns of the dictionary N K    D  In an overcomplete setting, the dictionary matrix D has more columns than rows , K N  and the corresponding coefficient matrix K M   X  is assumed to be sparse. For most practical tasks in the presence of noise, we consider a contamination form of the measurement signal ,   Y DX w where the elements of noise w are independent realizations from the Gaussian distribution 2 (0, ) n  N . The basic dictionary learning problem is formulated as:

2 0 , min . . F i s t L i    D X Y DX x (1) Therein, L is the maximal number of non-zero elements in the coefficient vector ix . Starting with an initial dictionary, this minimization task can be solved by the popular alternating approaches such as the method of optimal directions (MOD) [1] and K-SVD [2]. The dictionary training on noisy samples can incorporate the denoising together into one iterative process. In general, the residual errors of learning process are determined

Manuscript received December XX, 2017; revised XX, 2017; accepted XX, 2017. Date of publication XX, 2017; date of current version XX, 2017. This work was supported by Beijing major science and technology projects under Grant Z171100000117008. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. XXXX. R. chen is with the School of Microelectronics, Tianjin University, Tianjin 300072, China (e-mail: rchen@jdl.ac.cn). C. Yang* (Corresponding author), H. Jia and X. Xie are with National Engineering Laboratory for Video Technology, Peking University, Beijing 100871, China (e-mail: {csyang, hzjia, donxie}@pku.edu.cn). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org . Digital Object Identifier 10.1109/LSP.2015.2448732 by noise levels [3]. Noise incursion in a trained dictionary can affect the stability and accuracy of sparse representation [4]. So the performance of dictionary learning highly depends on the estimation accuracy of unknown noise level 2n  when the noise characteristics of trained dictionaries are unavailable.
The main challenge of estimating the noise level lies in effectively distinguishing the signal from noise by exploiting sufficient prior information. The most existing methods have been developed to estimate the noise level from image signals based on specific image characteristics [5]-[8]. Generally, these works assume that a sufficient amount of homogeneous areas or self-similarity patches are contained in natural images. Thus empirical observations, singular value decomposition (SVD) or statistical properties can be applied on carefully selected patches. However, it is not suitable for estimating the noise level in dictionary update stage because only few atoms for sparse representation cannot guarantee the usual assumptions. To enable wider applications and less assumptions, more recent methods estimate the noise level based on principal component analysis (PCA) [9], [10]. These methods underestimate the noise level since they only take the smallest eigenvalue of block covariance matrix. Although later work [11] has made efforts to tackle these problems by spanning low dimensional subspace, the optimal estimation for true noise variance is still not achieved due to the inaccuracy of subspace segmentation. As for estimating the noise variance techniques, the scaled median absolute deviation of wavelet coefficients has been widely adopted [12]. Leveraging the results from random matrix theory (RMT), the median of sample eigenvalues is also used as an estimator of noise variance [13]. However, these estimators are no longer consistent and unbiased when the dictionary matrix ha

…(Full text truncated)…