Application of dual-tree complex wavelet transform for spectra background reduction

Application of dual-tree complex w a v elet transform for sp ectra bac kground reduction Kazimierz Skrobas*, Kamila Stefa ´ nsk a-Skrobas, Cyprian Mieszczy´ nski, Renata Rata jczak National Cen tre for Nuclear Research, A. Soltana 5, 05-400 Ot wock-Swierk, P oland Keyw ords: exp erimen tal data pro cessing, bac kground cutting metho d, w a velet transform, sim ulations *coresp onding author: Kazimierz.Skrobas@ncb j.go v.pl Abstract This pap er presents a method for unique background remo v al in exp erimen tal data pro cessing using the Dual-T ree Complex W a velet T ransform (DTCWT) algorithm. The prop osed technique is based on discrete w av elet theory (DWT) and o vercomes numerous obstacles encountered in information retriev al that are presen t in commonly used numerical techniques, particularly fitting or filtering metho ds. The DTCWT also outp erforms metho ds based on the F ourier T ransform. The metho d is universal, allowing the analysis of an arbitrarily selected data range and its position in time. F urthermore, it meets the specific requirements of signal analysis and optimization, including signal preserv ation and bias reduction. This paper discusses the implementation of an algorithm for bac kground reduction, enabling the extraction and enhancemen t of v aluable information from spectra. The capabilities of w av elets for spectral data processing are demonstrated using tw o significantly different t yp es of sp ectra: X-ra y powder diffraction and photoluminescence measured for the Ga 2 O 3 crystal. Issues typical of DWT applications that are imp ortan t for robust and reliable data pro cessing—suc h as the c hoice of w av elet family and the n um b er of decomposition levels—are also discussed. The metho d is av ailable as a softw are pac k age for bac kground reduction. 1 Motiv ations Exp erimen tal data alw ays consists of informative and non-informative parts. The last one is a compilation of differen t types of distortions and often mak es extracting the v aluable part a challenging job. T ypically , the F ourier T ransform (FT) is employ ed for distortion separation [22, 15, 14, 1]. The FT decomp oses the signal in the frequency domain into slow, medium, and fast-changing comp onen ts, where: (i) the low est frequencies are resp onsible for background, (ii) the medium band is usually an interesting region for further analysis, and (iii) the highest frequencies’ region originates from bursts of short-duration impulses (high-frequency noise). By suppressing the extreme frequencies and leaving the medium sub-band untouc hed, it is p ossible to synthesize a signal free of distortion. The disadv antage of the FT tec hniques is the sensitivity to the range of data taken for analysis, which means that regions that are to o short generate spurious ripples near the correct p eaks in the frequency domain. Another dra wback is ov erlapping frequency ranges and aliasing artifacts. All of the ab o ve are sources of errors in FT analysis of signals consisting of comp onen ts with frequencies dep enden t on time, e.g., human sp eec h, trends, c hirp-like sounds, or an y non-stationary deviations. W a velet-based tec hniques can address the ab o v e issues. They originate from the theory of orthogonal function systems [9], which has b een transformed to the W av elet Theory (WT) by [7, 13, 2, 3]. Contrary to FT, wa velets are b oth well lo calized in time and frequency domains, which allows for a significan tly b etter in tro duction of algorithms in signal processing. The high effectiv eness of the WT has led to its acceptance in n umerous fields. It outp erforms commonly used techniques for signal pro cessing, like the Windo wed F ourier T ransform, and other domains for image pro cessing or data compression. W a velet-based approaches ha ve recen tly been used as tec hniques for bac kground remov al. This is particularly imp ortan t when the level of slowly v arying comp onen ts is comparable to, or even higher than, the level of useful information contain ed in the signal. F or example, this issue w as addressed b y [10] et al., who used a WT- based algorithm for the decon tamination of fluorescence microscopy images from b oth low- and high-frequency 1 noise. F urthermore, they demonstrated that this approach helps mitigate hardware-related issues that ma y significan tly affect the final results. The first bac kground remov al algorithm was prop osed b y Gallow ay et al. [8] and was applied to data pro- cessing in surface-enhanced Raman spectroscopy (SERS). The Gallow ay algorithm was later further developed b y Zhao [23] and Cotret [6] through the in tro duction of the Dual-T ree Complex W av elet T ransform (DTCWT) [19], whic h originated from efforts to improv e the Discrete W av elet T ransform (DWT). The DTCWT enabled a significant reduction of oscillations in the vicinity of sharp p eaks and the elimination of aliasing artifacts. The use of DTCWT for background subtraction from energy-disp ersiv e X-ra y fluorescence (EDXRF) sp ectra w as demonstrated in [23], for ultrafast electron scattering (UES) 1D sp ectra in [6], and for 2D scattering pat- terns in [5]. In the latter case, the remov al of image hotsp ots using the Discrete W av elet T ransform was also demonstrated. 2 W a v elet T ransformations Similarly to the F ourier T ransform, t wo techniques can b e distinguished, corresp onding to signals that are con tinuous or discrete in time. The Contin uous W a velet T ransform (CWT) of a f ( t ) function given b y [13] is the follo wing: W f ( b, a ) = Z + ∞ −∞ f ( t ) ψ ⋆ ab ( t ) dt (1) where ψ ab ( t ) is a w av elet function (WF): ψ ab ( t ) = 1 √ a ψ  t − b a  a ∈ R + , b ∈ R (2) and the In verse Con tinuous W av elet T ransform (ICWT): f ( t ) = 1 C Z + ∞ 0 Z + ∞ −∞ W f ( b, a ) 1 a ψ ab ( t ) dadb (3) The a con trols WF dilation and is called a sc ale p ar ameter ; the b is called a shift parameter, giving the w av elet lo cation in time; C is a wa v elet-dep enden t constant. In general, WF are complex functions, non- orthogonal to themselves, but orthogonal to the dual basis. The Equ.1 can b e interpreted as a con volution of the input f ( t ) function with a WF. The CWT produces a very detailed image of a signal with a significan t amount of redundant information. F or numerous purp oses, such as m ultilevel decomp osition, it is feasible to substitute CWT with a more concise view. In this case, orthogonal, real-v alue functions are selected, and the contin uous a,b parameters are replaced b y v alues from a discrete set. T ypically , in such a dy adic decomposition, the scale and shift parameters are p o w ers of 2. This approach, called a Discrete W av elet T ransform (DWT), transforms a discrete signal f [ n ] as follo ws: D W T ( j, k ) = X n f [ n ] ψ j k [ n ] (4) ψ j k [ n ] = 1 √ 2 j ψ  k − n 2 j 2 j  (5) and the In verse Discrete W a velet T ransform (IDWT): f [ n ] = X j k D W T ( j, k ) ψ j k [ n ] (6) for in tegers j, k , n ∈ Z . Here, the j parameter controls the scale and selects the subband of frequencies co vered b y a wa v elet. The low est range of frequencies is represented b y the highest j v alues. The j parameter is the sub ject of adjusting to control the n umber of decomposition lev els. The maxim um v alue of lev els is giv en b y L max = log 2 N , where N stands for the n umber of data p oin ts. There are numerous collections, called families, from which one can select mother wa velet (MW), whic h is next a sub ject of mo dification b y adjusting the i,j parameters. F or example, the orthogonal ψ ( n ) functions for D WT are sho wn in Fig.2 from Daub ec hies db , Symlet sym , and Coiflet c oif families. The db family first w as giv en b y Daubechies [2]; the Symlet and Coiflet w av elets are mo dified Daub ec hies wa velets as to obtain greater symmetry and n umber of v anishing moments (smo othness) [4] of an MW. The common feature of all wa velets, in con trast to trigonometric functions used in the F ourier transform, is that they are finite and well lo calized both in time, see Fig.1a) and frequency , Fig.1b) (solid lines). They can b e treated as a band-pass filter, with a central frequency prop ortional to 2 − j and gain and bandwidth prop ortional 2 Figure 1: a) The scaling (dashed lines) and w av elet (solid lines) functions from the Coiflet family (coif8); the n umber stands for a decomp osition lev el. b) Corresp onding frequency characteristics Figure 2: Example wa velets used in DWT selected from 3 families, i.e. Daub ec hies (db6), Symlet (sym6), Coiflet (coif6) of 8th level (the num b er follow ed by the name of the w av elet stands for the corresp onding filter’s order) to 2 j . The lo west frequencies range with a constant comp onen t are supplemen ted b y sc aling function (SF), see in see Fig.1a,b) (dashed line), which acts as a lo w-pass filter. The collection of WF and the corresponding SF functions is called a bank. The presen ted method of a signal decomp osition based on filter banks is called m ultiresolution analysis (MA) and was prop osed b y Mallat[12]. The original MA uses real w av elets only and has b een extended to complex w av elets, what b ecome a foundation of the DTCWT [19] algorithm. The DTCWT addresses issues such as oscillations, shift v ariance, and aliasing. It also found an application in the bac kground remov al softw are and in the results presen ted here. 3 Results β − Ga 2 O 3 is one of the most promising wide-bandgap semiconductors, meeting the demands of mo dern ap- plications in high-pow er electronics, opto electronics, and solar-blind detectors [16]. Moreo ver, it is a radiation- resistan t material, meaning that devices made from β − Ga 2 O 3 can op erate in radiation-in tensive en vironments, suc h as space. As a result, studying radiation-induced defects and other phenomena in this material is crucial. In addition, β − Ga 2 O 3 is considered to be an excellen t host material for rare earth (RE) ions, which can pro vide their more efficien t ligh t emission compared to other wide-bandgap semiconductors. Hence, such systems are in tensively studied [16, 18, 17]. How ev er, the structural and optical resp onse of the RE dopan t is very subtle due to its low concentration in the matrix. Therefore, it is essential to eliminate an y electronic disturbances con tributing to the exp erimen tal sp ectra or extract the information coming from the very weak signals of the RE on a strong background. In the present work, the WT-based algorithms hav e b een used for bac kground analyses of X-ray and photoluminescence (PL) sp ectra processing obtained for β − Ga 2 O 3 crystals implanted at ro om temp erature with 150 k eV rare earth ions like Yb, and Eu with fluency 1 × 10 15 1 /cm 2 . The obtained spectra significantly differ in terms of peak prop erties (width, heigh t, relative distances), bac k- ground lev els, and high-frequency noise lev els, whic h allow ed testing the application of the DTCWT algorithm on differen t conditions. The bac kground reduction and CWT analysis results of raw and pro cessed data are shown for the X-ray p o wder diffraction sp ectrum of the β − Ga 2 O 3 : Y b system in Fig. 3. In the first step, the intensities of the 3 ra w sp ectrum hav e been reduced b y calculating their logarithm log 10 , to decrease the heigh t ratio and enhance readabilit y . Next, the sp ectrum p oin ts hav e been pro cessed b y application of DTCWT algorithm based on db5 w av elet with decomp osition up to the 6th lev el. The presented diffraction sp ectrum (represen ted by a solid blue line) exhibits a high degree of natural bac kground, gradually diminishing tow ards the highest-v alued angles. The calculated background, depicted as a dashed red line, accurately repro duces the natural course, exhibiting a high level of compliance and b eing free of medium and high-frequency comp onen ts. It has b een subtracted from the ra w data, resulting in a final, free of slo w-changing comp onen ts spectrum (green line). The processed sp ectrum kept the general shap e of the p eaks. In the case of the 020 p eak, the algorithm detected and subsequently reduced the lo cal excess of bac kground, whic h also allo wed the extraction of the initially w eak p eak (marked as a ) around 63 ◦ . The neighbor p eak b has remained untouc hed. The sp ectrum also consists of slightly lifted regions, indicated by the arro ws, that ha ve no ph ysical meaning but are the remains of DWT synthesis only . F or the ab o ve cases, CWT sp ectra hav e b een calculated (see Fig.4) to show the impact of the DTCWT algorithm on the data. T o enhance visibility , only p ositiv e CWT co efficien ts are shown, and high-frequency noise ( f > 1 a.u. ) has b een remov ed; high-intensit y regions are marked in dark-orange colors. The sampling frequency is equal to: f s = number of data points ang le r ange ≈ 3808 100 ≈ 38 . 08 (7) The CWT sp ectrum of the initial XRD pattern (Fig.4a), sho ws that all angle ranges consist of a high num b er and significant intensit y of low-frequencies ( f < 0 . 25 a.u. ) comp onen ts. The diffraction p eaks are w ell lo calized on the angle axis and co ver mainly the narro w range of medium frequencies, i.e., 0 . 25 < f < 1. After background reduction (Fig.4b), the regions of lo w frequencies ha ve been significan tly reduced. How ever, in the case of the medium frequencies sub-band, the comp onen ts corresp onding to diffraction p eaks are preserv ed and also partially preserve the region of very low frequencies, i.e., f < 2 − 6 = 0 . 015625 corresp onding to the bac kground under the 020 p eak. One can also notice that the background reduction resulted in a decrease in o verall in tensities. In summary , the CWT clearly shows that the DTCWT algorithm efficiently recognizes and separates lo w frequencies from other frequency groups. As previously men tioned, the exp erimen tal data comprises numerous components with distinct characteris- tics. Therefore, input is pro cessed in many and sometimes non-trivial w ays to meet requirements. In the case of the D WT applications, one m ust only consider the selection of the w av elet family and the n umber of decom- p osition lev els. The results of such analysis are demonstrated in Fig.5, where the PL sp ectra of Ga 2 O 3 : E u system w ere used. T o compare the effect of bac kground reduction, the χ -metric has b een calculated as follows: χ = s P N i ( y i − b i ) 2 N (8) where: y i stands for input data; b i - background; N - num b er of data p oin ts. The low v alues of χ -metric demonstrate the bac kground ov erfitting to input data and underfitting in the opp osite. Studies of PL sp ectra sho w a significan t reduction in bac kground, regardless of p eak characteristics, ov erall bac kground shap e, and the presence of high-frequency noise. The common feature for all cases, indep enden t of the selected family or the n umber of decomposition levels, is enhancing the high-frequency noise for energies ab o v e 3 eV . It is the most pronounced if decomp osition is ab o ve the 4th lev el. There is a visible dependence on wa v elet family and decomp osition level selection, whic h can interfere with p eak magnitude and the presence of artifacts. The selection of the wa velet family has a minor impact on the final results. There are only negligible differences b et ween families seen for the highest lev el of decomp ositions. The essen tial factor for background reduction is the num b er of decomp osition levels. Here, increasing the decomp osition level clearly go es with increasing the v alue of χ -metric. This dep endence also indicates that for low χ v alues, i.e., ab out 0 . 18, see in Fig.5 the first column, there is an unfav orable effect of bac kground ov erfitting to data, esp ecially for narrow p eaks placed at energies near 2 eV , and gives a disruption in their heigh t prop ortions. In contrast, the unfav orable underfitting effect is presen t at the highest lev el of decomp osition, see in Fig.5 the last column, where the χ is ab out 0 . 32. It induces the creation of spurious p eaks s (indicated by arro ws) with significant height. F or the db5 and sym5 families, one can notice a po orly fitted lo cal background for real p eaks at E = 2 eV . The presence of s illustrates the capability of discontin uity detection b y w av elets, which is the source of error here. An exception one can observe for c oif5 , see in Fig.5, the first row. In this case, both the 5th and 6th levels giv e exactly the same χ , equal to 0 . 238, well-separated p eaks and spurious peaks with very low heigh ts. This example demonstrates that decomp osition based on Coiflet w av elets detects irrelev an t differences for DTCWT without a significan t negative impact on bac kground reduction. 4 Figure 3: The logarithm of X-ra y measured sp ectrum intensities from (blue solid line) of Ga 2 O 3 crystal, with w ell visible, high-level bac kground; the sp ectrum with remov ed bac kground is given as a green line. Arro ws indicate artifacts remained after D WT synthesis ( db5 ) Figure 4: CWT sp ectra of a) initial X-ra y in tensities and b) after slow changing comp onen ts ( f < 0 . 25) reduction. T o enhance readabilit y , only CWT co efficien ts ab o ve 0 are shown; high-frequency ( f > 1) noise has b een remov ed by means of SG filtering 5 Figure 5: Comparison of w av elet families and levels of decomposition for PL sp ectra. T o enhance the visibilit y , the l og 10 ( I ) of input data (blue line) has b een pro cessed; bac kground and outcomes are given resp ectiv ely as red and green lines 6 4 Discussion and conclusions The abov e results demonstrate that DTCWT is a very effective to ol for background remov al from data with minimal impact on p eak prop erties and user in terven tions. How ev er, it requires more atten tion if the sub ject of analysis is a neighborho o d of real p eaks. In this case, w av elets generate spurious ripples around the main p eaks, even if the selection of the family and num b er of decomp osition levels is done correctly . The same effect is giv en if the num b er of decomp osition levels is to o high. In the case of co existence of a wide, slo wly changing bac kground with a relatively short region of high-frequency noise, see Fig.5, the presence of noise makes it difficult to find a correct level of background. This issue must b e addressed by the high-frequency filtering, e.g., with the Sa vitzky-Golay method. It is shown that the most imp ortan t for DTCWT is the proper selection of the num b er of decomposition lev els. The n umber of lev els should be close to the maximal v alue L max for a given set of data p oin ts. Here, the b est trade-off has b een found if the n umber of levels is equal to L max − 1 = 5. The choice of the wa velet family (WF) is of lesser imp ortance. F or X-ray diffraction, the most suitable family was db5 , while for PL spectra, both db5 and sym5 wa velet families. Slightly worse results were obtained for c oif5 wa v elets. T o conclude, the DTCWT algorithm requires fewer initial parameters for data pro cessing than typical fitting metho ds. Moreo ver, it is resistan t to high-frequency noise, enables the extraction of weak features, and separates regions of interest v ery well. The final results are reliable due to the small risk of numerical errors’ in tro duction. The softw are for background remov al, namely tlor em.py , based on DTCWT, written in Python language, is a v ailable from[20]. It also provides CWT analysis supp orted by pywt pack ages[11] and data denoising based on SciPy pack age [21]. Other Python scripts and data used for the pap er preparation are av ailable up on request. Ac knowledgemen ts The researc h was co-founded b y the NCN pro ject UMO-2022/45/B/ST5/02810. The exp erimen tal w ork was supp orted by the Helmholtz-Zen trum Dresden-Rossendorf (20002208-ST, 21002661-ST, 21002663- ST) References [1] S. Cole, K. B. Fisher, and D. H. W einberg. F ourier analysis of redshift-space distortions and the determi- nation of Ω. Monthly Notic es of the R oyal Astr onomic al So ciety , 267(3):785–799, 1994. [2] I. Daubechies. Orthonormal bases of compactly supported wa v elets. Comm. Pur e and Appl. Math. , 41:909– 996, 1988. [3] I. Daub ec hies. The wa velet transform, time-frequency lo calization and signal analysis. IEEE T r ansaction on Information The ory , 36:961–1005, 1990. [4] I. Daub ec hies. T en L e ctur es on Wavelets, . CBMS-NSF Conference on W av elets and Applications, 1992. [5] L. P . R. de Cotret, M. R. Otto, M. J. Stern, and B. J. Siwic k. An op en-source soft ware ecosystem for the in teractive exploration of ultrafast electron scattering data. A dv. Struct. Chem. Imaging , 4:1–5, 2018. [6] L. P . R. de Cotret and B. J. Siwick. A general metho d for baseline-remov al in ultrafast electron p o wder diffraction data using the dual-tree complex w av elet transform. Structur al Dynamics , 4:044004, 2017. [7] D. Gab or. Theory of comm unication. J. Inst. Ele ctr. Eng. , 93, 1946. [8] C. M. Gallow a y , E. C. L. Ru, and P . G. E. and. An iterativ e algorithm for background remo v al in sp ectroscop y b y wa velet transforms. Appl. Sp e ctr osc. , 63:1370–1376, 2009. [9] A. Haar. Zur theorie der orthogonalen funktionensysteme. Mathematische A nnalen , 69:331–371, 1910. [10] M. H¨ upfel, A. Y. Kobitski, W. Zhang, and G. Nienhaus. W av elet-based bac kground and noise subtraction for fluorescence microscop y images. Biome d Opt Expr ess , 12(2):969–980, 2021. [11] G. R. Lee, R. Gommers, F. W aselewski, K. W ohlfahrt, and A. O’Leary . Pyw av elets: A p ython pack age for w av elet analysis. Journal of Op en Sour c e Softwar e , 4(36):1237, 2019. [12] S. G. Mallat. Multiresolution approximations and wa velet orthonormal bases of l2(r). T r ansactions of the A meric an Mathematic al So ciety , 315:69–87, 1989. 7 [13] J. Morlet, G. Arens, E. F ourgeau, and D.Giard. W a ve propagation and sampling theory—part i and ii: Complex signal and scattering in m ultilay ered media. Ge ophysics , 1982. [14] F. A. S. Neves, H. E. P . de Souza, M. C. Ca v alcanti, F. Bradasc hia, and E. J. Bueno. Digital filters for fast harmonic sequence comp onen t separation of unbalanced and distorted three-phase signals. IEEE T r ansactions on Industrial Ele ctr onics , 59(10):3847–3859, 2012. [15] M. W. Peterson, S. E. Gorrell, and M. G. List. F ourier descriptors for improv ed analysis of distor- tion transfer and generation. V olume 1: Aircraft Engine; F ans and Blo wers; Marine; Honors and Aw ards:V001T01A029, 2017. [16] R. Rata jczak, M. Mahwish, D. Kalita, P . Jozwik, C. Mieszczynski, J. Matulewicz, M. Wilczop olsk a, W. W ozniak, U. Kentsc h, R. Heller, and E. Guziewicz. Anisotropy of radiation-induced defects in yb- implan ted β -ga2o3. Scientific R ep orts , 14:24800, 2024. [17] M. Sarwar, R. Rata jczak, V. Iv anov, S. Gieralto wsk a, A. Wierzbic k a, W. W ozniak, R. Heller, S. Eisen- winder, and E. Guziewicz. Crystal lattice reco very and optical activ ation of yb implan ted into β -ga2o3. Materials , 17(16), 2024. [18] M. Sarw ar, R. Rata jczak, V. Iv anov, M. T urek, R. Heller, L. W achnic ki, W. W ozniak, and E. Guziewicz. Structural defects and luminescence in sm-implan ted β -ga2o3. physic a status solidi (RRL) – R apid R ese ar ch L etters , n/a(n/a):2400415, 2025. [19] I. Selesnick, R. Baraniuk, and N. Kingsbury . The dual-tree complex wa velet transform. IEEE Signal Pr o c essing Magazine , 22(6):123–151, 2005. [20] K. Skrobas. A program for exp erimen tal data background reduction. ” https://github.com/kskrobas/ tlorem ”. [21] P . Virtanen, R. Gommers, T. E. Oliphant, M. Hab erland, T. Reddy , D. Cournap eau, E. Buro vski, P . Pe- terson, W. W eck esser, J. Bright, S. J. v an der W alt, M. Brett, J. Wilson, K. J. Millman, N. May orov, A. R. J. Nelson, E. Jones, R. Kern, E. Larson, C. J. Carey , ˙ I. Polat, Y. F eng, E. W. Moore, J. V anderPlas, D. Laxalde, J. Perktold, R. Cimrman, I. Henriksen, E. A. Quintero, C. R. Harris, A. M. Arc hibald, A. H. Rib eiro, F. P edregosa, P . v an Mulbregt, and SciPy 1.0 Contributors. SciPy 1.0: F undamental Algorithms for Scien tific Computing in Python. Natur e Metho ds , 17:261–272, 2020. [22] M. F. W ahab, F. Gritti, and T. C. O’Hav er. Discrete fourier transform techniques for noise reduction and digital enhancemen t of analytical signals. T rAC T r ends in Analytic al Chemistry , 143:116354, 2021. [23] F. Zhao and A. W ang. A bac kground subtraction approach based on complex w av elet transforms in edxrf. X-R ay Sp e ctometry , 44:41–47, 2014. 8