Random Euler Complex-Valued Nonlinear Filters

1 Random Euler Comple x-V alued Nonlinear Filte rs Jiashu Zhang, She n g Zhang, and Defang Li Abstract —Over the last decade, both the neu ral network and kernel adaptive filter ha v e successfully been used for nonlinear signal processing . Ho wev er , they suffer from high compu tational cost caused by their complex/gr owing network structures. In th is paper , we p ropose two random Eu ler filters for complex-va lued nonlinear filtering problem, i.e., lin ear random Euler complex- valued filter (LRECF ) and its wid ely-linear versio n (WLRECF), which possess a simple and fixed network structure. The transient and steady-state perf ormances ar e studied in a non-stationary en vironment. Th e analytical minimum mean squ are error (MSE) and optimum step-size ar e deriv ed. Finally , numerical simulations on complex-valued nonl inear system identifi cation and nonlinear channel equalization are presented to show the effectiveness of the proposed methods. Index T erms —Nonlinear Filter , random Euler , transient anal- ysis, steady-state analysis. I . I N T R O D U C T I O N W ith the d evelopment of adaptive filter ing, comp lex-valued adaptive filter has fo u nd app lications in di verse fields of radar imaging, fourier analysis, mobile commun ications, seismics, estimation of direction of ar riv al and beamform ing [1]–[3]. In modeling and ide ntification of complex-valued non linear systems, traditional linear adaptive filtering techn iq ues suf fer from poor per forman ce. Ex amples for such situations inclu de nonlinear system identification, nonlin ear chann el equa liza - tion. In orde r to mode l nonline ar systems, serval method s have been proposed in the last h alf cen tury [3]–[ 9], which in clude the neural networks, p olynom ia l, sp line and Fourier filters, to just mentio n a few . In order to directly process complex values by neural n et- works, the comp lex-valued neural ne twork ( CVNN) have been developed [10]–[13], where the splitting-c o mplex and fully - complex activ ation f u nctions are used. The major drawback of the CVNNs is the h eavy co mputation a l complexity . Se veral different types o f CVNNs have b een pr esented in [12], such as multiplayer perc etron (MLP) n etworks, radial basis func tion (RBF) network s, an d recurren t neu r al n etworks (RNN). In [14], the echo state network fo r complex noncirc u lar signals was proposed , which separates the RNN arch itecture in to two con stituent comp o nents: a recurr ent architecture and a memory less o u tput layer . W ith a comp lex-c h ebyshev exp a n- sion for the input signal, the comp lex-ch ebyshev fun ctional- link network (CCFLN) was design ed [15], which is a linear filtering of the expa n ded signal in the higher dimensional space. Based on the rep roducin g kernel Hilbert space (RKHS) theory , the kernel adap ti ve filters (KAFs) were developed Manuscript rece i ved **, 2017. This work was supporte d by the Natio nal Science Foundati on of China (Grants 61671392, U1562218). The authors are with the School of Information Scienc e and T ech- nology , Southwest Jiaotong Uni ve rsity , Chengdu 611756, China (e-mail : jszhang@swj tu.edu.cn, dr .s. zhang@i eee.or g, Ldf125@swjtu.edu.cn). in [5], wh ich maps the o riginal input space into an infinite dimensiona l RKHS w ith a specific kernel. When the kernel is chosen as Gaussian kernel, the KAF is the gr owing RBF network. Over th e real kern e l filter, ser val ad aptive algor ith ms were proposed in [16]–[19], such as the kernel least mean square (KL MS), kern el affine projectio n, kernel recur si ve least-squares, kern el p r ojected sub gradient meth ods. Using the complexification of real RKHSs, or complex repro ducing ker- nels, th e co mplex kernel a d aptive filtering has been introd uced in [2 0]. W ith the wide-linear mod el, further enh ancements to the complex-valued/qu aternion-valued kernel appro ach can be found in [21]–[23]. Howe ver , the order of th e se kerne l filters grows linearly with the num ber of inp ut data. T o overcome this se vere dr awback , se vera l low-complexity techniques h ave been developed in [24]–[28], such as the sparse KLMS, quantized KLMS, KLMS with l 1 -norm regulariza tio n. Recently , acco rding to Bochner’ s th eorem, Rah imi and Recht suggested a p opular app roach, i.e. , rando m four ier features, to approx imate the re a l kern e l ev a lu ation in KAFs [29]. Based o n the rand om fourie r f eatures, the ran dom fou rier filtering ( RFF) has been prop osed in [30], where the origina l input data is ma pped to a finite d imensional space. Thus, compare d with the KAF , it enab les learn ing o f nonlinea r function s in an efficient fashio n . In [ 30], the m e a n squar e (LMS) and r ecursive least squares (RLS) wer e developed into the RFF . Fu r thermor e, a distributed RFF was p resented f or networks in [31]. Unf ortunately , these RFFs only deal with real-valued no nlinear systems. In this pape r , we pr opose two random Euler co m plex-valued filters to dea l with th e co m plex-valued n o nlinear filtering problem . Firstly , based on the complexification of real RKHSs and Boch ner’ s theorem , a de tailed deriv ation of the linear random Euler complex-valued filter ( L RECF) is pr esented. Then, employing the widely-linear model and the W irtinger’ s deriv ati ve, the widely-linea r random Euler complex-valued filter ( WLRECF) is designe d. Due to th e fixed n etwork structure, the pro posed two schem es enjoy low co m putational complexity compared to the kern el filter . Theoretical analysis on the mean stability and mean-squa re con vergence of the propo sed method s is perfor med in a no n-stationary en viron- ment mo d eled by a r andom- walk model. From these results, the closed -form expression for th e steady-state mean squar e error (M SE ) is obtained , wh ich ind icates that there is an optimum step-size in the non-stationar y environment. Finally , experiments are conducted to ev aluate the perform ance of the propo sed filters, in c luding comp lex-valued non linear sy stem identification and n onlinear chan n el e qualization. The rest of th is p aper is o rganized as fo llows . I n Section II, a brief review of the RFF is presented. Section I II p rovides the deriv ation of the LRECF and WLRECF . T h e mean an d mean-squ are b ehaviors are analyzed in Section IV. Section V 2 Fig. 1. The real random Fourier filter . presents Monte Carlo simulations. Finally , conclusio ns are drawn in Section VI. In this paper, matrices are rep resented by boldface capital letters, and all vectors are colum n vectors denoted by bold face lowercase letters. The o ther symbols are listed as f ollows: ( · ) T transpose operato r; ( · ) ∗ conjuga te operato r ; ( · ) H Hermitian transpose o perator; λ max ( · ) largest eigen value of a matrix; T r( · ) trace of a matrix; I identity matrix with a ppropr iate dimen sion; 0 zero vector with ap propria te dimension ; A ⊗ B Kron ecker pro duct of two matrice s A and B ; vec ( · ) column vector formed by stacking the column s of a matrix ; | · | absolute value of a co mplex number; real ( · ) real part of complex num ber; imag ( · ) imaginary part of complex nu mber; E {·} e xpectation operator . I I . R E V I E W O F R E A L R A N D O M F O U R I E R F I LT E R Consider a continuous nonlinear input- output ma p ping, y = f ( x ) (1) where x ∈ F m is the m -dim ensional vector 1 , and y ∈ F is the o utput sign a l. Based on the trainin g sequence s of th e fo r m { ( x n , y n ) , n = 1 , 2 , · · · } , the goal o f the learn ing tasks is to learn the non-linear input-ou tput depen dence. In th e case o f real Hilbe rt spaces (i.e., F = R ) , the real random Fourier non linear filter algorithm is w n = w n − 1 + µ z ( x n ) e n (2) where µ is the step-size and e n = y n − z T ( x n ) w n − 1 with w n − 1 being a weigh t vector for the rando m F ourier features 1 F is a general field, which can be either R or C . vector z ( x n ) . In [29], it gives two such embed dings about the random Fourier featur es z ( x ) : z ( x ) = r 1 D        sin( r T 1 x ) cos( r T 1 x ) . . . sin( r T 2 D x ) cos( r T 2 D x )        , r 2 D      cos( r T 1 x + a 1 ) cos( r T 2 x + a 2 ) . . . cos( r T D x + a D )      , (3) where r i is drawn fr om a Gaussian distribution with zero mean and covariance m atrix σ 2 I , a i is th e unifo r m distrib ution on [0 , 2 π ] . Fig. 1 shows the RFF with later embeddin g in ( 3). As can be seen, in the RFF , the origina l data x n ∈ R m is transform ed to a h igh dime n sional feature space, via a map, z ( x n ) ∈ R D . I I I . P RO P O S E D R A N D O M E U L E R C O M P L E X - V A L U E D N O N L I N E A R F I LT E R In this section , we are inter e sted on comp lex Hilber t spaces, i.e., F = C , and will design the LRECF and WLRECF . L et z ∈ C m and z = z 1 + i z 2 , z 1 , z 2 ∈ R m . W e adopt the complexification methodolog y of real RKHSs [20], Φ c ( z ) =Φ( z 1 ) + i Φ( z 2 ) = κ R ( · , [ z 1 , z 2 ]) + iκ R ( · , [ z 1 , z 2 ]) (4) where κ R ( · , · ) is chosen as a real Gaussian kernel. A. Linea r Random Euler Comple x-V alued F ilter By the u se of the cost fu n ction, 1 2 | y n − w H Φ c ( x n ) | 2 , and the gradient d escent meth od, at time n , the weight-up date equation giv es w n = P n i =1 µe ∗ i Φ c ( x i ) 2 , where the initial e stima te is assumed to be zero a n d e n = y n − w H n − 1 Φ c ( x n ) . Th e system’ s output ˆ y n , at time n , can be estima te d as ˆ y n = w H n − 1 Φ c ( x n ) (5) = 2 n − 1 X i =1 α i κ R ([ real ( x n ) , imag ( x n )] , [ real ( x i ) , imag ( x i )]) where α i = µe i . Acco rding to Bochner’ s th eorem, we can have κ R ([ real ( x n ) , imag ( x n )] , [ real ( x i ) , imag ( x i )]) = E c { ζ c ([ real ( x n ) , imag ( x n )]) ζ ∗ c ([ real ( x i ) , imag ( x i )]) } (6) where ζ c ([ real ( x n ) , imag ( x n )]) = e j c T [ real ( x n ); imag ( x n )] , and the random vector c is drawn from a pr obability distribution 3 that is the F ourier transfor m of the Gaussian kernel. W e choo se a sample average to appro ximate (6) using D random vectors { c 1 , c 2 , · · · , c D } , E c { ζ c ([ real ( x n ) , imag ( x n )]) ζ ∗ c ([ real ( x i ) , imag ( x i )]) } ≈ 1 D D X i ζ c i ([ real ( x n ) , imag ( x n )]) ζ ∗ c i ([ real ( x i ) , imag ( x i )]) (7) 2 This is well-kno wn comple x-value d KL MS (CKLMS) via complex ificati on of real kernel s in [20]. 3 It is actually the multi v ariat e Gaussian distributi on wit h zero mean and cov ariance matrix σ 2 I [31]. 3 Fig. 2. The linear random Euler complex-v alued filter . Substituting (6) and (7) into ( 5), the estimate o f the filtering output can be approxim ated as ˆ y n = u H n − 1 z c ( x n ) (8) where u n = P n i =1 α ∗ i z c ( x i ) with z c ( x i ) being z c ( x i ) = r 2 D       e j c T 1 [ real ( x i ); imag ( x i )] e j c T 2 [ real ( x i ); imag ( x i )] . . . e j c T D [ real ( x i ); imag ( x i )]       (9) In (8), u n − 1 can be seen as a weight vector for th e rand om features vector z c ( x n ) , which c an be r ewritten as u n = u n − 1 + µe ∗ n z c ( x n ) (10) where the initial weight vector is assumed to be zero. Because of the Euler representatio n in (9), we coin this approa c h as LRECF . Fig. 2 illustrates its architecture . As can be seen, the LRECF h as fixed network structure , which is obviously different from th e growing structure of the CKLMS. B. W idely- Lin ear Rando m E uler Complex-V alued F ilter Inspired by (8) and using th e widely-lin ear mo del 4 , th e estimator for m of the nonlinea r filter is written as ˆ y n = u H z c ( x n ) + v H z ∗ c ( x n ) (11) T o design a filter { u , v } , we establish th e following cost function L ( e n ) = 1 2 | y n − u H z c ( x n ) − v H z ∗ c ( x n ) | 2 (12) where the er r or signal is e n = y n − ˆ y n . Using the stochastic gr a d ient adaptation and the W irtinger’ s deriv ati ve with r espect to { u , v } , as follows: ∇ u L ( e n ) = 2 ∂ L ( e n ) ∂ u ∗ = ∂ L ( e n ) ∂ u r + i ∂ L ( e n ) ∂ u i (13) and ∇ v L ( e n ) = 2 ∂ L ( e n ) ∂ v ∗ = ∂ L ( e n ) ∂ v r + i ∂ L ( e n ) ∂ v i (14) 4 The widely-li near model enables the processing of the noncircul ar comple x-v alued signals, which provide improve d performance than the con- vent ional linear model [32]–[35]. Fig. 3. The widely-l inear random Euler complex-v alued filter . we get th e update equation for the w e ight vector u n = u n − 1 − µ ∇ u L ( e n ) = u n − 1 + e ∗ n z c ( x n ) (15) and v n = v n − 1 − µ ∇ v L ( e n ) = v n − 1 + µe ∗ n z ∗ c ( x n ) (16) where e n = y n − u H n − 1 z c ( x n ) − v H n − 1 z ∗ c ( x n ) . T he step-size µ controls the conv ergence rate of the propo sed algo rithm. Fig. 3 illustrates the ar c hitecture of the WLRECF . T o simplify the notatio n, using an aug mented weight vector w n = [ u T n v T n ] T and a complex a ugmented vector z c,c ( x n ) = [ z T c ( x n ) z ∗ T c ( x n )] T , we can r ewrite the p roposed WLRECF algorithm (15)-(16) as w n = w n − 1 + µe ∗ n z c,c ( x n ) (17) I V . P E R F O R M A N C E A N A L Y S I S In this section, the perfo rmances of the propo sed schem es in terms of mean stability and mea n-square conv ergence are investigated. Instead of analyzin g each prop osed method separately , we main ly foc us on the WLRECF scheme that includes the LRECF method as a specia l case. In order to make the perfor mance analysis trac ta b le, some assumptions are introd uced. Assumption 1: Inspired by (1 1), we consider an alternative observation mo del y n = u H opt ,n z c ( x n ) + v H opt ,n z ∗ c ( x n ) + υ n , w H opt ,n z c,c ( x n ) + υ n (18) where w opt ,n = [ u T opt ,n v T opt ,n ] T represents the optimal aug- mented weight vector o f an unk nown system. 4 Assumption 2: In ( 18), the noise υ n is an in depend ent and identically distributed ( i.i.d. ) complex-valued ra n dom se- quence with zero-mean and E { | υ n | 2 } = σ 2 υ , and is indepen- dent of th e inpu t x j for all j . Assumption 3: In (18), the time- varying unk nown we ig ht vector w opt ,n is defined as a rando m walk m odel, i.e., w opt ,n = w opt ,n − 1 + q n , where the ra n dom perturbatio n q n is a station ary white noise vector with zero mean a n d covariance matrix E { q n q H n } = σ 2 q I , which is mutu a lly indepen dent o f the inpu t { z c ( x n ) } an d noise { υ n } . A. Mean Con verg ence Analysis Let the weight error vector ˜ w n = w opt ,n − w n . The output error e n becomes e n = ˜ w H n − 1 z c,c ( x n ) + υ n (19) while its c o njugate is e ∗ n = ˜ w T n − 1 z ∗ c,c ( x n ) + υ ∗ n (20) Inserting ( 2 0) into (1 7), the recursion of the weight error vector ˜ w n is ˜ w n = ˜ w n − 1 − µ ( ˜ w T n − 1 z ∗ c,c ( x n ) + υ ∗ n ) z c,c ( x n ) + q n =  I − µ z c,c ( x n ) z H c,c ( x n )  ˜ w n − 1 − µυ ∗ n z c,c ( x n ) + q n (21) Using assumptions 2-3 an d well-k nown indepen dence as- sumption [38]–[4 1], we have E { υ ∗ n z c,c ( x n ) } = 0 , E { q n } = 0 , an d hence E { ˜ w n } =  I − µE  z c,c ( x n ) z H c,c ( x n )  E { ˜ w n − 1 } (22) Thus, when the step -size satisfies 0 < µ < 2 λ max  E  z c,c ( x n ) z H c,c ( x n )  (23) the pr oposed scheme is stab le in th e mean sense, and is unbiased, i.e., E { ˜ w n } → 0 . B. Mean -Squa r e Con ver gence Analy sis The MSE performance is defined as E {| e n | 2 } = E { ˜ w H n − 1 z c,c ( x n ) z H c,c ( x n ) ˜ w n − 1 } + σ 2 υ =T r { E { z c,c ( x n ) z H c,c ( x n ) } E { ˜ w n − 1 ˜ w H n − 1 }} + σ 2 υ , T r { R z E { ˜ w n − 1 ˜ w H n − 1 }} + σ 2 υ (24) where R z = { z c,c ( x n ) z H c,c ( x n ) } . Upon multiplying both sides of (21) by ˜ w H n yields the following r elation ˜ w n ˜ w H n = ˜ w n − 1 ˜ w H n − 1 − µ z c,c ( x n ) z H c,c ( x n ) ˜ w n − 1 ˜ w H n − 1 − µ ˜ w n − 1 ˜ w H n − 1 z c,c ( x n ) z H c,c ( x n ) + µ 2 z c,c ( x n ) z H c,c ( x n ) ˜ w n − 1 ˜ w H n − 1 z c,c ( x n ) z H c,c ( x n ) + µ 2 | υ n | 2 z c,c ( x n ) z H c,c ( x n ) + q n q H n − µυ n  I − µ z c,c ( x n ) z H c,c ( x n )  ˜ w n − 1 z H c,c ( x n ) − µυ ∗ n z c,c ( x n ) ˜ w H n − 1  I − µ z c,c ( x n ) z H c,c ( x n )  +  I − µ z c,c ( x n ) z H c,c ( x n )  ˜ w n − 1 q H n + q n ˜ w H n − 1  I − µ z c,c ( x n ) z H c,c ( x n )  − µυ ∗ n z c,c ( x n ) q H n − µυ n q n z H c,c ( x n ) . (25) Proceeding in a man n er similar to [ 36], [37] and using assumptions 2-3 5 vec ( E { ˜ w n ˜ w H n } ) =( I − µ A + µ 2 B )vec ( E { ˜ w n − 1 ˜ w H n − 1 } ) + µ 2 σ 2 υ vec ( R z ) + σ 2 q vec( I ) (26) where A = I ⊗ E { z c,c x n ) z H c,c ( x n ) } + E { z ∗ c,c ( x n ) z T c,c ( x n ) } ⊗ I , (27) B = E { ( z ∗ c,c ( x n ) z T c,c ( x n )) ⊗ ( z c,c ( x n ) z H c,c ( x n )) } , (28) Thus, when the step - size is such that th e matrix I − µ A + µ 2 B is stable, the pro posed algor ithm is conver gent in the mean-squ are sense. Mo reover , we can ob tain the steady-state mean-squ are deviation (MSD) lim i →∞ E { ˜ w n ˜ w H n } = µσ 2 υ vec − 1 { ( A − µ B ) − 1 vec ( R z ) } + σ 2 q vec − 1 { ( A − µ B ) − 1 vec ( I ) } µ (29) Inserting (29) into ( 24), we finally get th e steady-state MSE MSE nonsta = σ 2 υ + µσ 2 υ vec T ( R z )( A − µ B ) − 1 vec ( R z ) + σ 2 q vec T ( R z )( A − µ B ) − 1 vec ( I ) µ (30) Remark: (i) A sufficiently small step -size can guarantee the propo sed alg o rithm to be stable. This is becau se wh en µ is sufficiently small, the terms µ B compared with A can be neglected, i.e., I − µ A + µ 2 B ≈ I − µ A . In this case, the step-size should satisfy 0 < µ < 2 λ max ( A ) . (ii) For a stationar y system ( q ( n ) = 0 ), (30) simplifies to MSE sta = σ 2 υ + µσ 2 υ vec T ( R z )( A − µ B ) − 1 vec( R z ) . When the used step-size µ → 0 , MSE sta tends to the m inimum σ 2 υ . (iii) For the non- stationary system ( q ( n ) 6 = 0 ) , we k now that th ere is an optimu m step-size given by µ opt = σ q σ υ q φ ϕ , and the c orrespon ding minimum MSE is MSE nonsta , min = σ 2 υ + 2 σ υ σ q √ ϕφ , with φ = v e c T ( R z ) A − 1 vec ( I ) , ϕ = vec T ( R z ) A − 1 vec ( R z ) . 5 Under the assumptions 2-3, we kno w that the last six terms in (25) are equal to zero. 5 0 0.5 1 1.5 2 x 10 5 −30 −25 −20 −15 −10 −5 0 Iterations MSE (dB) CLMS proposed LRECF proposed WLRECF (a) 0 0.5 1 1.5 2 x 10 5 −30 −25 −20 −15 −10 −5 0 Iterations MSE (dB) CLMS proposed LRECF proposed WLRECF (b) Fig. 4. Performance comparison between the CLMS, L RE CF , an d WLRECF for th e nonlinea r system I. (a) the ci rcular input, (b) the noncircular input ρ = 0 . 1 . 0 1 2 3 4 5 6 7 8 x 10 5 −20 −15 −10 −5 0 5 Iterations MSE (dB) CLMS proposed LRECF proposed WLRECF (a) 0 1 2 3 4 5 6 7 8 x 10 5 −30 −25 −20 −15 −10 −5 0 5 Iterations EMSE (dB) CLMS proposed LRECF proposed WLRECF (b) Fig. 5. Performance comparison between the CLMS, LRECF , and WLRE CF for the nonlinear system II. (a) the MSE curve s, (b) the EMS E curves. V . M O N T E C A R L O S I M U L A T I O N S In this section, Mo nte Carlo simulations are presented. First, to examine th e co n vergence p erforma nce of the pro posed two filters, the nonlinear system identification task is c a rried out. Then, a n onlinear chann el e q ualization task is co n sidered. T o ev alua te the filterin g per forman ce, th e MSE in dB is used and defined as MSE = 10log 10 ( E {| e n | 2 } ) where the expectation is obtained by a veraging the results of 200 ind ependen t runs. A. Nonlin ear system identifi cation In the com plex-valued n onlinear system identification, we consider two different n o nlinear systems u sed in [20], [21]. Experime nts are conducte d on a set o f the input-ou tput signal { x n , x n − 1 , · · · , x n − m +1 , y n } with m > 0 . 1) Non linear System I: The first non linear sy stem is chosen as y n = t n + (0 . 15 − 0 . 1 i ) t 2 n (31) T ABLE I C O M PA R I S O N O F A V E R A G E D C O N S U M E D T I M E O V E R 1 × 10 5 L E A R N I N G S A M P L E S Adapti ve filters A veraged consumed time ( s ) CKLMS 7 . 83 × 10 3 LRECF 21 . 61 WLRECF 52 . 3 where t n is an output signal of a linear filter t n = 5 X k =1 h k x n − k +1 (32) with h k being h k = 0 . 43 2  1 + cos  2 π ( k − 3) 5  − i  1 + cos  2 π ( k − 3) 10  (33) In the inpu t-output relatio nship (3 1)-(3 3), y n and x n are complex-valued outp ut and input signals. At the rece i ver end of the system, the outpu t sign al is corr u pted by white Gaussian 6 0 2 4 6 8 10 x 10 4 −25 −20 −15 −10 −5 0 5 Iterations EMSE (dB) CKLMS proposed LRECF proposed WLRECF Fig. 6. The EMSE compari son between the CKLMS, LRE CF , and WLRECF . noise an d the level of th e n oise is set to 30 d B. Th e inpu t signal is obtaine d by the u sing of the form x n = p 1 − ρ 2 s 1 ,n + iρs 2 ,n (34) where s 1 ,n , s 2 ,n are zero-mean Gaussian random v ariables, ρ ∈ [0 , 1] d etermines th e per f ormance of x n . If ρ approaches 0 or 1, the input is h ighly noncircular and for ρ = √ 2 2 it is circular . Th e step-size of the complex-valued least mean square (CLMS) is µ = 0 . 05 . For a fair comp arison, the step- sizes of p roposed algor ithms are also set to µ = 0 . 05 . The other parameter s are m = 5 , D = 500 , σ 2 = 0 . 2 . In Fig. 4, the MSE learning cur ves f or c ircular and n o ncircular inp ut signals are d epicted. 2) Non linear System II: The seco n d nonlin ear system con- sists of a linear filter: t n = ( − 0 . 9 + 0 . 8 i ) x n + (0 . 6 − 0 . 7 i ) x n − 1 (35) and a memoryless nonlinearity y n = t n + (0 . 1 + 0 . 15 i ) t 2 n + (0 . 06 + 0 . 0 5 i ) t 3 n (36) The input signal has th e form x n = x 1 ,n + i x 2 ,n , where x 1 ,n , x 2 ,n are u n iform randomly distributed signals an d their ranges are [ − 1 , 1] . The o bservation noise corrup ts the output signal with the variance 16 dB. The step-sizes of the CLMS, LRECF , an d WLRECF ar e 0 . 005 . Th e other par ameters are set to be m = 2 , D = 500 , σ 2 = 1 . The MSE and excess mean- square err o r (EMSE) learning curves are shown in Fig. 5, where the EM SE is defined as EMSE = 10log 10 ( E {| y n − ˆ y n − υ n | 2 } ) with υ n being the noise added to the desired signal. Both Fig. 4 and Fig. 5 show that the pr o posed sche mes can achieve an im proved per f ormance compar e d with the CLMS. It is also shown that in proposed two metho ds the LRECF is inf erior to the WLRECF . Fro m the Section III, we know the LRECF is an approxim a tion fo r the CKLMS. Due to the growing network , the CKLMS poses both co m putational as well as memo ry issues f o r large lear ning samples, such as 8 × 10 5 samples. Thus, we only co mpare th e CKLMS with the propo sed method s for relatively small samples drawn in Fig. 6, and the av eraged consu med time is listed in T able I. It is mea- sured on a 3.2 GHz Intel Core i5 processor with 8 Gb of RAM, runnin g Matlab R20 17a o n Windo ws 10 . The experiment set- tings are the same as tho se used in Fig. 5. It demon strates that the LRECF effectively app roximates the CKLMS with lower complexity . Th e p roposed WLRECF me th od c o uld o utperfo rm the CKLMS an d LRECF schem e s. But the WLRECF requir es more compu tations than the LRECF . 3) Effect of the step -size, D , and σ 2 : T o examine the effect of the step- size, D , and σ 2 on the perfo rmance o f the pro p osed schemes, th e E MSE c u rves of the pro posed WLRECF with different µ , D , and σ 2 are d isplayed in Fig. 7. In Fig. 7(a), D = 500 , σ 2 = 0 . 2 . In Fig. 7 (b), µ = 0 . 05 , σ 2 = 0 . 2 . In Fig. 7(c) , D = 5 00 , µ = 0 . 05 . The o th er exper im ent settings are th e same as tho se u sed in Fig. 5. As can be seen, the choice o f the step-size d etermines a comp romise between fast conv ergence rate and small steady -state EMSE. W ith fixed µ and σ 2 , small D (i. e., D = 100 ) will suf fer from slow conver gence rate. Large D can lead to improved conv ergence perf o rmance but with high computational cost, shown in Fig. 7(b ). Wit h fixed µ and D , too large and two small σ 2 will suffer from poo r conv ergence perform ance illustrated by Fig. 7(c). Hen ce, to achieve fast co n vergence rate and low steady-state err or , µ, D and σ 2 should be chosen approp riately acc ording to the ap plication. B. Nonlin ear channel equalization In th e no n linear channel equalization scen arios, we consid- ered the equalization mod el whic h consists o f a linear filter [42], [43] t n =(0 . 34 − 0 . 27 i ) s n + (0 . 87 + 0 . 4 3 i ) s n − 1 + (0 . 34 − 0 . 2 1 i ) s n − 2 (37) and a nonlinear d istortion x n = t n + 0 . 1 t 2 n + 0 . 05 t 3 n (38) The nonlinear channel equalization stru c tu re is shown in Fig. 8(a) . The 4 QPSK symbols, s 1 = 1 + j, s 2 = 1 − j, s 3 = − 1 + j, s 4 = − 1 − j , are tested: The case 1) th e 4 symbols are equip robable; The case 2 ) the oc currence probability are p 1 = 0 . 4 , p 2 = 0 . 1 , p 3 = 0 . 4 , p 4 = 0 . 1 . The lear ning cur ves using the set o f the training samp les { x n , x n − 1 , · · · , x n − m +1 , s n − d } are drawn in Fig. 8 (b)-(c) , where m > 0 an d d is the equalizatio n time delay . The additive observation noise is a zer o-mean Gaussian sign a l with variance 15 dB. T h e values o f the parameter s are m = 5 , D = 500 , σ 2 = 0 . 05 , d = 2 , µ = 0 . 08 . By the use of 30 0000 testing samples, Fig. 9 gives the symbol classification pe r forman ce of the equalizer s with the pr oposed WLRECF . C. Theoretical Curves T o verify th e analyses in the section IV , the th eoretical transient MSE and MSD cur ves of the proposed WLRECF for the no n -stationary en vironme nt ( σ 2 q = 10 − 8 ) are plotted in Fig. 10 an d Fig. 11. Accord ing to th e mode l (18), th e unknown channe l is r andomly g e n erated an d its leng th is 128 . The in itial weig h t vector of the adaptive filter is an all one vector . Th e input sign al x ( n ) ∈ C 5 is generated by mea ns o f 7 0 1 2 3 4 5 x 10 6 −35 −30 −25 −20 −15 −10 −5 0 5 Iterations EMSE (dB) µ =0.1 µ =0.05 µ =0.01 µ =0.005 (a) 0 1 2 3 4 5 x 10 5 −30 −25 −20 −15 −10 −5 0 Iterations EMSE (dB) D=100 D=300 D=500 D=800 (b) 0 0.5 1 1.5 2 x 10 6 −30 −25 −20 −15 −10 −5 0 5 Iterations EMSE (dB) σ 2 =0.1 σ 2 =0.2 σ 2 =1.2 σ 2 =15 (c) Fig. 7. EMSE curve s of the proposed WLRECF with dif ferent step-sizes, D , and σ 2 . (a) Effect of µ , (b) Effect of D , (c) Effect of σ 2 . (a) 0 1 2 3 4 5 x 10 5 −14 −12 −10 −8 −6 −4 −2 0 2 4 Iterations MSE (dB) CLMS proposed LRECF proposed WLRECF (b) 0 1 2 3 4 5 x 10 5 −14 −12 −10 −8 −6 −4 −2 0 2 4 Iterations MSE (dB) CLMS proposed LRECF proposed WLRECF (c) Fig. 8. The MSE comparison between the CLMS, CKLMS, LRECF , and WLRE CF for the nonlinear channel equalizat ion problem. (a) the nonlinear channel equali zatio n, (b) the case 1, (c) the case 2. (a) (b) (c) (d) Fig. 9. Eye diagram of symbol classifica tion performance using the proposed WLRECF . (a) the input signal x n for the case 1, (b) the output signal ˆ y n for the case 1, (c) the output signal x n for the case 2, (d) the output signal ˆ y n for the case 2. (34). The vector c i in ( 9) is drawn from a white Gaussian distribution with D = 64 , σ 2 = 0 . 2 . Three different step- sizes µ = 0 . 01 , 0 . 005 and 0 . 0 01 are app lied. The variances of the measuremen t noise υ n are set to be 0 . 1 , 0 . 0 1 in Fig . 10 and Fig. 11, respe c ti vely . The th eoretical curves are calculated using (24) an d (29). It can be observed that the theoretical 8 analysis can predict th e perf ormance of the WLRECF well. V I . C O N C L U S I O N In this paper, we prop osed two random E uler filters, i.e., LRECF and WLRECF , for comp lex-valued nonline ar filter . On the basis of the comp lexification of real RKHSs an d Boc h ner’ s theorem, the LRECF filter was fir stly d erived. Then, further using the widely-linear model and th e Wirtinger’ s d e riv ative, the WLRECF was also ob tained. A s compared to the well- known complex-valued KLMS, the pr oposed rand om Euler filters enjoy low compu tational complexity bec a u se of the fixed network structur e s. In add ition, theor etical expressions to char acterize th e transient and steady-state behaviors o f propo sed schemes were p resented in the ran dom-walk non- stationary environment. Th r ough a series o f simulation s, we finally demon strated the effectiveness of the proposed metho ds and theo r etical r esults. R E F E R E N C E S [1] C. F . N. Co wan and P . M. Grant, Adaptiv e Filt ers . E ngle wood Cliffs, NJ: Prentice-Ha ll, 1985. [2] S. Haykin, Adaptive Fil ter Theory , 4th ed. Upper Saddle Rive r , NJ : Prentic e-Hall, 2002. [3] T . Adali and S. Haykin, Adaptive Signal Proc essing: Next Generation Solutio ns . Piscata way , NJ: Wile y-IEE E Press, 2010. [4] D. P . Mandic and V . S. L . Goh, Complex V alued Nonlinear Adaptive F ilte rs: Noncir cularity , W idely Linear and Neural Models. Ne w Y ork: W ile y , 2009 [5] W . Liu, J. C. Principe , and S. Haykin, Ke rnel Adaptive Fil tering . New Y ork: Wi le y , 2010. [6] H. Zhao and J . Zhang, “ Adapti vely combined FIR and functiona l link artificial neural network equalizer for nonline ar communicatio n channe l, ” IEEE T rans. Neural Netw . , vol . 20, no. 4, pp. 665-674, Apr . 2009. [7] M. Scarpiniti , D. Comminiello, R. Pa risi, and A. Uncini, “Nonlinea r spline adapti ve filtering, ” Signal Pr ocess. , vol. 93, no. 4, pp. 772-783, Apr . 2013. [8] A. Carini and G. L. Sicuranza , “Recursi ve ev en mirror Fourier nonlinea r filters and s implified s tructu res, ” IEE E T rans. Signal Pro cess. , vol. 62, no. 24, pp. 6534-6544, Dec. 2014. [9] S. Zhang, and W . X. Zheng, “Recursi ve adapti ve sparse expon ential functio nal link neura l network for nonline ar AEC in impulsiv e noise en vironment, ” IEEE T rans. Neural Netw . Learn. Syst. , vol. PP , no. 99, 2017, Doi: 10.1109/TNNLS.2017.2761259. [10] A. Hirose, Complex-v alued neural networks , S pringer-V erlag, Berli n, 2006. [11] H. Leung, S. H aykin, “The complex backpropagat ion algorit hm, ” IEEE T rans. Signal Proce ss. , vol. 39, no. 9, pp. 2101-2104, 1991. [12] D. P . Mandic and J. A. Chambers, Recurrent Neural Networks for Pred iction: Learning Algorith ms, A rchit ectur es and Stability . New Y ork: W ile y , 2001. [13] H. Li and T . Adali, “Comple x-v alued adapti ve signal processing using nonline ar functions, ” J . Adv . Signal Pr ocess. Specia l Issue on Emerg ing Mach . Learn. T ech. Signal P r ocess. , 2008, 765 615. [14] Y . Xia, B. Jelfs, M. M. V . Hulle, J. C. Prłncipe, and D. P . Mandic, “ An augmente d echo state network for nonlinea r adapti ve filte ring of comple x noncircula r signals, ” IEEE T rans. neural networks , vol. 22, no. 1, pp. 74-83, Jan. 2011. [15] M. Li, J. Liu, Y . Jiang, and W . Feng, “Complex -chebyshe v functi onal link neural network beha vioral model for broadband wireless power amplifiers, ” IEEE T rans. Micr ow . Theory T ech. , vol. 60, no. 6, pp. 1979- 1989, Jun. 2012. [16] K. Sla va kis, S. Theodori dis, and I. Y amada, “ Adapti ve constra ined learni ng in reproducing kernel Hilbert spaces: The robust beamforming case, ” IEEE Tr ans. Signal Proce ss. , vol. 57, no. 12, pp. 4744-4764, 2009. [17] W . Parreira , J. Bermudez, C. Richard, and J . T ourneret, “Stochastic beha vior analysi s of the Gaussian kernel-l east-mea n-square algorith m, ” IEEE T rans. Signal Pr ocess. , vol. 60, no. 5, p. 2208-2222, 2012. [18] W . Liu and J. C. Principe, “Kernel af fine projection algori thms, ” EURASIP J. Adv . Signal Pr ocess. , vol. 2008, no. 1, pp. 1-13, Mar . 2008. [19] Y . E ngel, S. Mannor , and R. Meir , “The kernel recursiv e least-squares algorit hm, ” IEEE T rans. Signal P r ocess. , vol. 52, no. 8, 2004. [20] P . Bouboulis, K. Slav akis, and S. T heodori dis, “ Adapti ve learning in complex reproducing kerne l Hilbert spaces employi ng W irtinge r’ s subgradie nts, ” IEEE T rans. Neural Netw . Learn . Syst. , v ol. 23, no. 3, pp. 425-438, 2012. [21] F . A. T obar , A. Kuh, and D. P . Mandic, “ A novel augmente d complex v alued kernel LMS, ” in Proc. IEEE 7th Sensor Array Multi channe l Signal Pro cess. W orkshop (SAM) , Jun. 2012, pp. 473-476. [22] P . Bouboul is, S. Theodori dis, and M. Mavro forakis, “The augmented comple x kernel LMS, ” IEEE T rans. Signal Pr ocess. , vol. 60, no. 9, pp. 4962-4967, Sept. 2012. [23] T . K. Paul, and T . Ogunfunmi, “ A kernel adapti ve algori thm for quatern ion-v alued inputs, ” IEEE T rans. Neural Netw . Learn. Syst. , vol. 26, no. 10, Oct. 2015. [24] W . Liu, I. M. Park, and J. C. Principe , “ An information theore tic approac h of designing sparse ke rnel adapti ve filters, ” IE EE T rans. Signal Pr ocess. , vol. 20, no. 12, pp. 1950-1961, Dec. 2009. [25] B. Chen, S. Zhao, P . Zhu, and J. Principe , “Quantized kernel least mean square algorithm, ” IEEE T rans. Neural Netw . Learn. Syst., vol. 23, no. 1, pp. 22-32, Jan. 2012. [26] B. Chen, S. Z hao, S. Seth, and J. C. Principe , “Online ef ficient learnin g with quantiz ed KLMS and L 1 regul ariza tion, ” in Proc. Int. Joint Conf . Neural Netw . (IJCNN) , 2012, pp. 1-6. [27] A. Singh, N. Ahuja, and P . Moulin, “Online learning with kernel s: Overc oming the gro wing sum problem, ” MLSP , Sept. 2012. [28] W . Gao, J. Chen, C. Richard, and J. Huang, “Online dictionary learning for ke rnel LMS, ” IEEE T rans. Signal Pr ocess. , vol. 62, no. 11, pp. 2765- 2777, 2014. [29] A. Ra himi and B. Rec ht, “Ra ndom feat ures for large scale ke rnel machines, ” in NIPS , vol. 20, 2007. [30] P . Bouboulis, S. Pougkakiotis, and S. T heodori dis, “Ef ficient KLMS and KRLS algorithms: A random fourier feature perspecti ve, ” in Proc. IEEE Statist . Signal Proce ss. W orkshop (SSP), pp. 1-5, 2016. [31] P . Bouboulis, S. Pougkakioti s, and S. Theodoridi s, “Onlin e distrib uted learni ng over networks in RKH spaces using random fourier features, ” arXi v preprint arXiv: 1703.08131 v2, 2017. [32] B. Picinbono and P . Che valie r , “Wi dely linear estimati on with complex data, ” IEEE T rans. Signal Proc ess. , vol. 43, no. 8, pp. 2030-2033, Aug. 1995. [33] Y . Xia, C. C. T ook, and D. P . Mandic, “ An augmented af fine projection algorit hm for the filtering of comple x noncircu lar signals, ” Signal Pr ocess. , vol. 9, no. 6, pp. 1788-1799, 2010. [34] A. Kuh, C. Manlol oyo, N. Corpuz, and R. K o wahl , “Wi nd prediction using comple x augmente d adapt i ve filters, ” in Proc . IEEE Int. Conf. Gre en Circ uits Syst. (ICGCS) , 2010, pp. 46-50. [35] Y . -M. Shi, L. Huang, C. Qian, and H. C. So, “Shrinkage linear and widely linear comple x-v alued least mean squares algorithms for adapti ve beamforming, ” IEEE T rans. Signal Proce ss. , vol. 63, no. 1, pp. 119-131, Jan. 2015. [36] S. Zhang and J. Zhang, “Transi ent anal ysis of zero at tracti ng NLMS algorit hms without Gaussian input signal, ” Signal P r ocess. , vol. 97, pp. 100-109, Apr . 2014. [37] S. Zhang and J. Zhang, “Ne w steady-state analysis results of var iable step-size LMS algorithm with differe nt noise distri buti ons, ” IEEE Signal Pr ocess. L ett. , vol. 21, no. 6, pp. 653-657, Jun. 2014. [38] M. Scarpinit i, D. Comminiello, G. Scarano, R. Parisi, and A. Uncini, “Steady -state performanc e of spline adapti ve filters, ” IEE E T rans. Signal Pr ocess. , vol. 64, no. 4, pp. 816-828, 2016. [39] J. Shi and J. Ni, “Diff usion sign subband ada pti ve filte ring al gorithm with enlarg ed cooperatio n and its v ariant, ” Cir cuits, Systems, and Signal Pr ocessing , vol. 36, no. 4, pp 1714-1724, 2016. [40] S. Zhang, J. Z hang, and H. C. So, “Mean square de viati on analysis of LMS and NL MS algorit hms with white reference inputs, ” Signal Pr ocess. , vol. 131, pp. 20-26, Feb . 2017. [41] A. H. Sayed, Fundamentals of A daptive Filt ering , New Y ork: Wile y- Intersci ence, 2003. [42] I. Cha, and S. A. Kassam, “Chann el equali zation using adapti ve complex radial basis function networks, ” IEEE Journal on Select ed Area s in Communicat ions , vol. 13, no. 1, pp. 122-131, 1995. [43] M. B. L i, G. B. Huang, P . Saratchan dran, and N. Sundararajan , “Fully comple x extre me learning machines, ” Neur ocomputi ng , vol. 68, pp. 306- 314, 2005. 9 0 0.5 1 1.5 2 x 10 5 −15 −10 −5 0 5 10 15 20 Iterations MSE (dB) µ =0.01 µ =0.005 µ =0.001 Simulation: dash line Theory: solid line (a) 0 0.5 1 1.5 2 x 10 5 −25 −20 −15 −10 −5 0 5 10 15 20 Iterations MSE (dB) µ =0.01 µ =0.005 µ =0.001 Simulation: dash line Theory: solid line (b) Fig. 10. T he MSE learnin g curves for differe nt step-size s in the non-stationary en vironment. (a) the noise varia nce 0.1, (b) the noise v ariance 0.01. 0 0.5 1 1.5 2 x 10 5 −25 −20 −15 −10 −5 0 5 10 15 20 25 Iterations MSD (dB) µ =0.01 µ =0.005 µ =0.001 Simulation: dash line Theory: solid line (a) 0 0.5 1 1.5 2 x 10 5 −25 −20 −15 −10 −5 0 5 10 15 20 25 Iterations MSD (dB) µ =0.01 µ =0.005 µ =0.001 Simulation: dash line Theory: solid line (b) Fig. 11. T he MSD learning curves for differe nt step-size s in the non-stationary en vironment. (a) the noise v arianc e 0.1, (b) the noise var iance 0.01.