Sparse System Identification in Pairs of FIR and TM Bases

Sparse System Iden tification in P airs of FIR and TM Bases Dan Xiong 1 , Li Chai 2 , Jingxin Zhang 3 1. Sc h o ol of Information Science and Engineering 2. Engineering Researc h Cen ter of Metal lurgical Au tomation and Measuremen t T ec hnology W uhan Univ ersit y of Science and T ec hn ology , Hub ei, W uhan, 430 081, C hina 3. Sc h o ol of Soft wa re and Electrical En gineering Swinburne Univ ers ity of T ec h nology , Melb ourne, VIC 3122, Austr alia Abstract : This pap er considers the reconstruction of a sparse co efficien t v ector θ fo r a rational transfer function, under a pa ir of FIR a nd T ak enak a-Malmquist (TM) bases and from a limited num b er of linear frequency-domain measureme n ts. W e pro- p ose to concatenate a limited n umber of F IR and TM basis functions in the represen- tation of the transfer function, and prov e the uniqueness of the sparse represen tation defined in the infinite dimensional function space with pair s of FIR and TM bases. The sufficien t condition is give n for replacing the ℓ 0 optimal solution by the ℓ 1 optimal solution using FIR and TM bases with random s amples on the upp er unit circle, as the foundation of reconstruction. T he sim ulations v erify that ℓ 1 minimization can recon- struct the co efficien t v ector θ with hig h probabilit y . It is shown t ha t the concatenated FIR and TM bases giv e a m uch sparser represen tation, with muc h low er reconstruction order than using only FIR basis functions and less dependency on the know ledge of the true system p o les than using only T M basis functions. Key words : sparse system iden tification; FIR basis; TM basis; ℓ 1 optimization. 1 In tro ductio n System iden tification has a long history in con trol theory . System iden tification using finite impulse resp onse (FIR) mo del ha s b een studied for man y years. FIR mo d- eling corresp onds to estimating the expansion co efficien ts of a partial expansion in the standard orthonormal function basis { z − k } . The main adv an ta ge of F IR mo del is that the parameters (the impulse resp onse co efficien ts) app ear linearly in the model, leading to a simple estimation problem. Man y excellen t w orks on the parameter estimation ha ve b een done [1], [2] and some metho ds and algorit hms ha ve b een w ell dev elop ed, for example, the least mean square [3], [4 ] and subspace iden tification [5] metho ds. Although all these metho ds can ac hiev e effectiv ely system identification, the main dis- adv an tage of the FIR mo del is t ha t in general o ne needs to estimate a large amoun t of expansion co efficie n ts if the p o le of the transfer function is close to the unit circle and hence the impulse res p onse deca ys slo wly , whic h will lead to a v ery high order recon- struction. T o ov ercome this problem, system iden tificatio n using rational orthonormal basis functions with structures w as intro duced. Bet w een the 1920s a nd 1 9 90s, most o f the r esearch o n using ratio nal orthonormal basis functions w as fo cused o n the construction using Laguerre functions with a single (rep eating for k > 1) real p ole a ∈ ( − 1 , 1) [6], [7], [8], and Kautz functions ha ving t wo complex conjugate p oles [9], [10]. Ninness et al. used an ar bitr ary sequence of p oles, whic h led t o the T ak enak a-Malmquist (TM) basis functions [11]. Since the w ork of [11], the Generalized Orthogona l Basis function (GOBF) based construction was in tro duced in to the arena of syste ms. Over the last t wen t y y ears, iden tification and con trol of linear stable dynamic system s using Orthonormal Ratio na l F unctions (ORFs) ha ve b een widely used, see for instance [12, 13, 14, 1 5, 16, 17, 1 8 , 19, 20, 21]. An L TI system can b e w ell a ppro ximated with a small num b er of O RFs if the p oles in the TM ba sis a re close to the true system p oles [22]. If su fficien t samples of the tra nsfer function on the unit circle are acquired, the co efficien t estimation can b e solv ed b y least squares (LS) metho d. Ho w ev er, it is g enerally diffi cult to iden tify the order and the poles of the transfer function in adv ance. This limits the usefulnes s of ORF based sys tem iden tification me tho ds and has inspired compresse d sensing based FIR system ide n tification in recen t years. Compressed sensing (CS) [23 ], [2 4], [25], [26] is a new framew ork fo r sim ulta neous sampling and compression of signals. It has drawn m uc h atten tio n since its adv ent sev eral ye ars ago, and has b een applied to the iden tification o f sparse systems. Sparse system iden tification using the least mean sq uare (LMS) alg o rithm w as discussed in [27], [28], [29], and the alg o rithm based on the pro jections on to w eigh ted ℓ 1 balls was prop osed in [30 ], [31]. The essence of the se metho ds is to find a sparse repre sen tation of the system confined to a single basis and may not yield the sparsest solution. It is w ell know n fro m the CS literature that a signal may hav e a muc h sparser rep- resen tation in a n ov ercomplete basis (r edundant dictionar y) consisting o f concatenated orthogonal bases [32 ], [33], [34 ], [3 5]. In the contex t of finite dimensional v ector spaces, [32] and [33 ] hav e presen ted and ana lyzed the sparse represen tatio n o f v ector signals under a pair of orthono r ma l bases. In the con text of finite dimensional function spaces, [36] has discusse d the ra ndom sampling in the b ounded orthonormal systems with one orthonormal basis from the p ersp ectiv e of structured random matrix. Inspired b y these w orks, this pap er inv estigates the sparse system iden tification under a pair of FIR and TM ba ses and from a limited n um b er of linear measuremen ts. Here the iden tification is to reconstruct a sparse co efficien t v ector θ fo r a rational trans- fer function under such pairs. In other word, the co efficien t v ector θ is the ob ject o f the iden tification. The aim is to obtain a sparse represen tation with muc h few er significan t co efficien ts than using only FIR basis functions and with w eaker dependence on the true system p oles than using only the O R Fs, and hence to ov ercome the dra wbac ks of these t wo ty p es of bases. Based on the analysis of [37], w e s ho w the uniq ueness of sparse represen tation for rational transfer functions in the infinite dimensional function space with pairs o f FIR and TM bases, using the uniform b ound of maximal absolute inner pro duct of suc h pairs as an index. W e then deriv e a compresse d sensing formulation for finding the sparse represen tation of the rationa l transfer function in the concatenated FIR and TM bases. W e further sho w that the replacemen t of ℓ 0 optimization b y ℓ 1 optimization with randomly sampled frequency domain measuremen ts and under a pair o f FIR and T M bases is guarante ed with high probability . Numerical ex p erimen ts v erify the effectiv eness of the prop o sed iden tification fra mew ork. The con tributio ns of this pap er are: • Analysis on the uniqueness prop erty of sparse re presen tation of ratio nal transfer function in the pa irs o f FIR and TM bases. • A no v el iden tification method for rational transfer function with the finite-order com bination of FIR basis and TM basis. • Sufficien t conditions on the n umber of measuremen ts needed to recov er the sparse co efficien t from the randomly sampled measuremen ts by solving the ℓ 1 -minimization problem in the pairs of FIR and TM bases and demonstration of the reconstruction p erformance of the pro p osed metho d. The rest of this pap er is organized a s follo ws. In Sec tion 2, the uniquenes s of the sparse represen tation of rational transfer functions in the pair s of FIR and TM bases is giv en. The sparse system iden tification using concatenated FIR and TM ba ses is give n in Section 3. Section 4 discusses compu tation issues of the prop osed metho d. Section 5 presen t s the sim ulation results, follow ed by conclusions in Section 6. 2 Sparse representation of transfer functions in p airs of FIR and TM b ases and Uniquenes s Prop ert y Let H ( z ) b e a prop er, stable, real-rational transfer function with at least o ne nonzero p ole. Assume that H ( z ) has a “sparse” represen t a tion under a pair of ORF bases, { φ k ( z ) , k = 1 , 2 , · · · , } and { ψ l ( z ) , l = 1 , 2 , · · · } , that is H ( z ) = ∞ X k =1 α k φ k ( z ) + ∞ X l =1 β l ψ l ( z ) . In this section, w e will show the uniquenes s prop ert y of the sparse r epresen t ation with real v alued sparse co efficien ts α = [ α 1 , α 2 , · · · ] T and β = [ β 1 , β 2 , · · · ] T . Here the sparsit y of α and β is in the sense that k α k 0( ε ) ≤ s 1 , k β k 0( ε ) ≤ s 2 , whic h means the rational transfer function H ( z ) is ( ε , s 1 + s 2 )-sparse in the pairs o f orthonormal ra tional functions, where k · k 0( ε ) is the ε -0 nor m defined as f o llo ws. Definition 1. F o r a fixe d thr eshold ε > 0 and an in finite se quenc e α = [ α 1 , α 2 , · · · ] T satisfying ∞ P k =1 | α k | < ∞ , let N ε ( α ) = min { K : ∞ X k = K | α k | ≤ ε } and defi n e the ε -supp ort of α as Γ ε ( α ) = { k : | α k | 6 = 0 , 1 ≤ k < N ε ( α ) } , and the c ar dinality of Γ ε ( α ) as the ε -0 norm o f α , denote d by k α k 0( ε ) . In this pap er, we fo cus on the concatenation of FIR basis φ k ( z ) = z − ( k − 1) , k = 1 , 2 , · · · and T ak enak a-Malmquist basis (TM basis) [38], [39] ψ l ( z ) := p 1 − | ξ l | 2 z − ξ l l − 1 Y j =1 1 − ¯ ξ j z z − ξ j , l = 1 , 2 , · · · , (2.1) where the p oles { ξ l } ⊂ D = { z | | z | < 1 } are giv en and ¯ ξ j is the complex conjugate of ξ j . The construction of TM basis holds for m ultiple p oles and complex p oles as w ell. If a n y of the p oles { ξ l } are chose n as complex, then the impulse resp o nses of TM basis are complex-v alued, whic h is ina ppropriate. Ho wev er, the construction of new basis functions whic h ha ve the same complex po les but hav e real v alued impulse responses can solv e this problem, see [16] for details. In addition, the necessary and suffic ien t condition for the completeness of TM basis functions is ∞ P l =1 (1 − | ξ l | ) = ∞ [11]. Both bases are or thonormal in t erms of the inner pro duct h φ k ( z ) , φ k ′ ( z ) i = 1 2 π i I T φ k ( z ) φ k ′ ( z ) dz z = 1 2 π Z 2 π 0 φ k ( e iω ) φ k ′ ( e iω ) dω , (2.2) where T = { z | | z | = 1 } . W e ha ve presen ted in [37] the uniqueness of sparse represen tation of transfer func- tion in pairs of general ORF bases with the represen tation co efficien ts satisfying ( q k α k 0( ε ) + ε ) 2 + ( q k β k 0( ε ) + ε ) 2 < 1 µ , where µ = sup k ,l |h φ k ( z ) , ψ l ( z ) i | is the m utual cohe rence of suc h t w o ORF bases. The concept of m utual coherence f o r matrices w as in tro duced by Da vid Donoho and Mic hael Elad [3 3]. The m utual coherence has b een used extensiv ely in the field of sparse repre- sen tations of signals since it is a k ey measure of the b ound for the unique represen tatio n of a sparse signal and the abilit y of sub o ptimal a lgorithms suc h as matchin g pursuit and basis purs uit to correctly iden tify the sparse signal. F ollo wing the terminology of compressed sensing, we denote µ the mutual coherence of tw o ORF bases { φ k ( z ) } and { ψ l ( z ) } . Notice that FIR and TM bases are t wo sp ecial cases of ORF bases. Express t he TM basis in impulse resp onse ψ l ( z ) := ∞ X d =0 a dl z − d , l = 1 , 2 , · · · . (2.3) Then the inner pro duct o f φ k ( z ) and ψ l ( z ) is give n by h φ k ( z ) , ψ l ( z ) i = h z − ( k − 1) , ∞ X d =0 a dl z − d i = ∞ X d =0 h z − ( k − 1) , z − d i a dl = a ( k − 1) ,l . The last equation follo ws f r om the orthonormality of FIR bases { z − ( k − 1) } ∞ k =1 . Hence the m utual coherence o f FIR and TM bases is µ = sup k ,l |h φ k ( z ) , ψ l ( z ) i | = sup k ,l | a ( k − 1) ,l | = sup d,l | a dl | . Denote the uniform b ound of the maximal absolute impulse resp onse of TM basis as ˜ µ = sup d,l | a dl | . (2.4) Using a similar proo f of Theorem 2 in [37], w e can establish the unique ness pro p ert y of the sparse represen ta tion of transfer function in pair of FIR and TM bases. Theorem 1. F or a tr ansfer function H ( z ) with a r epr esentation in the c onc atenate d FIR a n d TM b ases H ( z ) = ∞ X k =1 α k z − ( k − 1) + ∞ X l =1 β l ψ l ( z ) , wher e ψ l ( z ) is given in (2 . 1 ). F or a fixe d thr esholds ε > 0 , if the r epr esentation is sp arse in the sen se of ε -0 norm and ( q k α k 0( ε ) + ε ) 2 + ( q k β k 0( ε ) + ε ) 2 < 1 ˜ µ with ˜ µ as define d in (2.4), then this sp arse r epr esen tation is unique. Remark 1. F or the p air of g e ner al ORF b ases , if the numb ers of such two b ases ar e given as n 1 and n 2 , r esp e ctively, then the c omplexity of the mutual c oher enc e µ o f such two b ases is O ( n 1 n 2 ) . Howe ver fr om (2. 4), for the p air o f FIR and T M b ase s , the analytic formula of t he mutual c oher enc e ˜ µ is given, which shows t hat ˜ µ only dep e n ds on the maximal absolute impulse r esp onse of TM b asis functions, thus the c ompl e xity of ˜ µ is the numb er of TM b asis functions O ( n 2 ) . Tha t is, the mutual c oher enc e of F IR and TM b ases is e asier to c o m pute than the gener al ORF b ases. When ε = 0, the ε - 0 norm reduces to the s tandard definition of 0-norm. Then we ha ve the following Corollary . Corollary 1. If a tr ansfer function H ( z ) has a sp arse r epr esentation in the c onc ate- nate d F IR and TM b ases H ( z ) = ∞ X k =1 α k z − ( k − 1) + ∞ X l =1 β l ψ l ( z ) and k α k 0 + k β k 0 < 1 ˜ µ with ˜ µ as define d in (2.4), then this sp arse r epr esen tation is unique. The impulse resp o nses { a dl } in (2.3), whic h determine the v alue of ˜ µ , can be ob- tained b y t he fo llowing theorem for ψ l ( z ) with dis tinct p oles. Theorem 2. The L aur ent exp ansion o f the TM b asis function ψ l ( z ) with distinct p oles ξ j ( j = 1 , 2 , · · · , l ) in the an nulus { z | max 1 ≤ j ≤ l | ξ j | < | z | < 2 } is ψ l ( z ) := ∞ X d =0 a dl z − d , l = 1 , 2 , · · · , wher e a dl = p 1 − | ξ l | 2 l X j ′ =1 ξ d − 1 j ′ Q l − 1 j =1 (1 − ¯ ξ j ξ j ′ ) Q l j =1 ,j 6 = j ′ ( ξ j ′ − ξ j ) , for d = 1 , 2 , · · · , and a 0 l = 0 for al l l . Pr o of. In the ann ulus { z | max 1 ≤ j ≤ l | ξ j | < | z | < 2 } , from the Lauren t expansion, for d = 1 , 2 , · · · 1) l = 1 a d 1 = 1 2 π i I C ψ 1 ( z ) z − d +1 dz = R es  ψ 1 ( z ) z − d +1 , ξ 1  = lim z → ξ 1 ( z − ξ 1 ) z d − 1 p 1 − | ξ 1 | 2 z − ξ 1 = p 1 − | ξ 1 | 2 ξ 1 d − 1 , where Res[ · , · ] denotes the Residue of a complex function. 2) l ≥ 2 a dl = 1 2 π i I C ψ l ( z ) z − d +1 dz = l X j ′ =1 Res  ψ l ( z ) z − d +1 , ξ j ′  = l X j ′ =1 Res " z d − 1 p 1 − | ξ l | 2 z − ξ l l − 1 Y j =1 1 − ¯ ξ j z z − ξ j , ξ j ′ # = l X j ′ =1 lim z → ξ j ′ ( z − ξ j ′ ) z d − 1 p 1 − | ξ l | 2 1 − ¯ ξ l z l Y j =1 1 − ¯ ξ j z z − ξ j = l X j ′ =1 ξ d − 1 j ′ p 1 − | ξ l | 2 1 − ¯ ξ l ξ j ′ Q l j =1 (1 − ¯ ξ j ξ j ′ ) Q l j =1 ,j 6 = j ′ ( ξ j ′ − ξ j ) = p 1 − | ξ l | 2 l X j ′ =1 ξ d − 1 j ′ Q l − 1 j =1 (1 − ¯ ξ j ξ j ′ ) Q l j =1 ,j 6 = j ′ ( ξ j ′ − ξ j ) . Denote Q l − 1 j =1 (1 − ¯ ξ j ξ j ′ ) Q l j =1 ,j 6 = j ′ ( ξ j ′ − ξ j ) = 1 fo r l = 1. Then the ab o v e deriv ations can b e unified a s a dl = p 1 − | ξ l | 2 l X j ′ =1 ξ d − 1 j ′ Q l − 1 j =1 (1 − ¯ ξ j ξ j ′ ) Q l j =1 ,j 6 = j ′ ( ξ j ′ − ξ j ) , for l , d = 1 , 2 , · · · . F urther, it is ob vious that a 0 l = 0 for all l . 3 Sparse System Ide n tific ati o n using co ncatenated FIR and TM bases Theorem 1 sho ws that if the represen tation co efficien t v ector θ = [ α T β T ] T is sparse enough, the r epresen tation is unique. In this section, w e will pro p ose a reconstruction algorithm based on compressed sensing to reconstruct H ( z ) with a small f raction of the measuremen ts of H ( z ) on the unit circle. Precisely , define T N := { z r = e 2 π i ( r − 1) / N , r = 1 , 2 , · · · , N } . W e fo cus on the underdetermine d case with only a few of the comp onen ts of { H ( z r ) , r = 1 , 2 , · · · , N } sampled or observ ed. That is, only a small fraction of T N is kno wn. Giv en a subset Ω ⊂ { 1 , 2 , · · · , N } of size | Ω | = m (far less than the n umber of the basis functions), the goal is t o reconstruct the represen tatio n co efficien ts and hence t he transfer function H ( z ) from the muc h shorter m -dimensional measureme n ts { H ( z r ) , r ∈ Ω } . No w w e will res tate this problem in a matrix form. Com bining with Definition 1, H ( z ) can b e rewritten as H ( z ) = n 1 X k =1 α k z − ( k − 1) + n 2 X l =1 β l ψ l ( z ) + ∆ 1 + ∆ 2 , where n 1 = N ε ( α ) − 1 and n 2 = N ε ( β ) − 1, ∆ 1 = ∞ P k = n 1 +1 α k z − ( k − 1) and ∆ 2 = ∞ P k = n 2 +1 β l ψ l ( z ). By simple calculatio n, w e ha ve k ∆ 1 k 2 = h ∞ X k = n 1 +1 α k z − ( k − 1) , ∞ X k = n 1 +1 α k z − ( k − 1) i = ∞ X k = n 1 +1 | α k | 2 ≤ ε 2 . Similarly , w e ha ve k ∆ 2 k 2 ≤ ε 2 . Denote ∆ = ∆ 1 + ∆ 2 , then we ha v e k ∆ k ≤ p 2( k ∆ 1 k 2 + k ∆ 2 k 2 ) ≤ 2 ε. No w the transfer function H ( z ) can b e simplified as H ( z ) = n 1 X k =1 α k z − ( k − 1) + n 2 X l =1 β l ψ l ( z ) + ∆ , (3.1) with k ∆ k ≤ 2 ε . With a little bit abuse o f the notation, the unkno wn co efficien ts t o b e determined here are denoted as α = [ α 1 , α 2 , · · · , α n 1 ] T and β = [ β 1 , β 2 , · · · , β n 2 ] T . Due to the arbitra riness of ε , the no r m of the term ∆ can b e a rbitrarily small, and the term ∆ = 0 when ε is exactly zero. In the sequel, w e first discuss the equation (3.1) with the term ∆ omitted. Define [Φ Ψ] to b e a comp osite sample matrix with its r -th ( r = 1 , 2 , · · · , N ) ro w satisfying [Φ Ψ] r := [1 , z − 1 r , · · · , z − n 1 +1 r , ψ 1 ( z r ) , · · · , ψ n 2 ( z r )] (3.2) and H := [ H ( z 1 ) , H ( z 2 ) , · · · , H ( z N )] T . Then H = [Φ Ψ]  α β  , where Φ =     1 z − 1 1 · · · z − n 1 +1 1 1 z − 1 2 · · · z − n 1 +1 2 · · · · · · · · · · · · 1 z − 1 N · · · z − n 1 +1 N     . W e ra ndomly select the subset Ω of size m ( << n 1 + n 2 ) dra wn from the uniform distribution ov er the index set { 1 , 2 , · · · , N } , and denote the measuremen t by H Ω = [Φ Ψ] Ω  α β  , where H Ω is the m × 1 v ector consisting of { H ( z r ) , r ∈ Ω } , and [Φ Ψ] Ω is the m × ( n 1 + n 2 ) matrix with the r -th row [Φ Ψ] r , r ∈ Ω. As [Φ Ψ] Ω is the concatenation of t wo bases, the represen tation is not unique in general. Ho we v er, as sho wn in Theorem 1, if the represen tation is sufficien tly sparse, the uniqueness of the repres en tatio n is guaranteed . The goal is to find the sparsest represen tation f rom the ℓ 0 minimization ( P 0 ) : min α,β      α β      0 sub j ect to H Ω = [Φ Ψ] Ω  α β  , whic h is an infeasible se arc h problem [40]. An alternativ e approach is to solv e the ℓ 1 minimization problem (Basis Pursuit) [23], [24], [41] ( P 1 ) : min α,β      α β      1 sub j ect to H Ω = [Φ Ψ] Ω  α β  , (3.3) whic h can b e solv ed b y linear pro gramming or second order cone program [3 6], [42 ]. F urther, t a king the data with small p erturbatio ns in t o consideration, the measure- men t is giv en by H Ω = [Φ Ψ] Ω  α β  + η . Here the small p erturbations η can b e either the transfer functions that are not exactly sparse but nearly sparse (compressible), or the noise in the sampling pro cess, and is b ounded b y k η k 2 ≤ ǫ . The corresponding appro ac h is called Basis Pursu it Denoising (BPDN) min α,β      α β      1 sub j ect to     H Ω − [Φ Ψ] Ω  α β      2 ≤ ǫ. (3.4) F or the sparse represen ta tion using only o ne basis, compressed sensing theory has pre- sen ted the equiv a lence o f ℓ 0 optimization and ℓ 1 minimization when the represen tation is sufficien tly sparse [40], [41], and has pro vided the sufficien t conditions on the num b er of measuremen ts needed to reco ve r the sparse co efficien t from the r a ndomly sampled measuremen ts by solving the ℓ 1 -minimization problem [36], [40]. F or the setting of (3 .3) concerning t wo bases, [37] has presen ted the low er b ound of the num b er of measuremen ts whic h guaran tees the replacemen t of ℓ 0 optimization ( P 0 ) b y ℓ 1 optimization ( P 1 ) under a pair of general ORF bases for a fixed (but arbitrary) supp ort. As FIR and TM bases are sp ecial cases of ORF ba ses, the replacemen t holds as w ell. T o presen t the sufficien t conditio n on t he nu m b er of measuremen ts required f or the sparse reconstruction by ℓ 1 optimization in pairs of FIR and TM ba ses with random samples, w e first presen t t he orthonormality prop erty of [Φ Ψ], whic h directly deter- mines t he m utual coherence µ ( Φ , Ψ) of matrices Φ a nd Ψ. µ (Φ , Ψ) , as a k ey index in the reconstruction, is defined a s µ (Φ , Ψ) = max k ,l |h Φ k , Ψ l i| k Φ k k k Ψ l k , (3.5) where Φ k , Ψ l denote t he k -th, l -th column of Φ and Ψ, resp ectiv ely . Theorem 3. When N is sufficiently lar ge, the c omp osite sampling matrix [Φ Ψ] satisfies: (i) Φ ∗ Φ ≈ N I n 1 , wh e r e ∗ is the c onjugate tr ansp ose, I n 1 is the identity matrix of dimension n 1 . (ii) Ψ ∗ Ψ ≈ N I n 2 . (iii) Φ ∗ Ψ = ( P N r =1 ψ k ( z r ) z − l r ) ≈ N ( a k − 1 ,l ) , ( k = 1 , · · · , n 1 , l = 1 , · · · , n 2 ) , wher e { a k − 1 ,l } ar e the im pulse r esp onses de fi ne d in e quation (2.3). (iv) µ (Φ , Ψ) ≈ ˜ µ . Pr o of. The in tegral definition of inner pro duct in (2 .2) shows that when N is sufficien tly large, 1 2 π N X r =1 z − ( k − 1) r z − ( l − 1) r 2 π N → h z − ( k − 1) , z − ( l − 1) i = δ k l , where the kro neck er sym b ol δ k l equals 1 if k = l and 0 if k 6 = l . Then the ( k , l ) elemen t of Φ ∗ Φ is N X r =1 z − ( k − 1) r z − ( l − 1) r = N X r =1 z − ( k − 1) r z − ( l − 1) r → N δ k l , whic h implies (i). The ( k , l ) elemen t of Ψ ∗ Ψ is N X r =1 ψ k ( z r ) ψ l ( z r ) = N X r =1 ∞ X d ′ =0 a d ′ k z − d ′ r ∞ X d =0 a dl z − d r = ∞ X d ′ =0 ∞ X d =0 a d ′ k a dl N X r =1 z − d ′ r z − d r → N ∞ X d =0 a dk a dl = N δ k l , the last equation is based o n the orthonormality of { ψ l ( z ) } , h ψ k ( z ) , ψ l ( z ) i = h ∞ X d ′ =0 a d ′ k z − d ′ , ∞ X d =0 a dl z − d i = ∞ X d ′ = d h a d ′ k z − d ′ , a d ′ l z − d i + X d ′ 6 = d h a d ′ k z − d ′ , a dl z − d i = ∞ X d =0 a dk a dl h z − d , z − d i + X d ′ 6 = d a d ′ k a dl h z − d ′ , z − d i = ∞ X d =0 a dk a dl = δ k l , whic h implies (ii). As fo r (iii), the ( k , l ) elemen t of Φ ∗ Ψ is N X r =1 z − ( k − 1) r ψ l ( z r ) = N X r =1 z − ( k − 1) r ∞ X d =0 a dl z − d r = ∞ X d =0 a dl N X r =1 z − ( k − 1) r z − d r → N a ( k − 1) ,l . F rom the results (i)-(iii), we sho w that when N is sufficien t ly large, k Φ k k ≈ √ N ( k = 1 , 2 , · · · , n 1 ) and k Ψ l k ≈ √ N ( l = 1 , 2 , · · · , n 2 ). And h Φ k , Ψ l i , corresponding to the ( k , l )-elemen t of Φ ∗ Ψ, is approximately N a ( k − 1) ,l . Hence from the definition in (3 .5), the m utual coherence of [Φ Ψ] constructed b y FIR and TM bases appro ximately equals max k ,l {| a ( k − 1) ,l |} , i.e., ˜ µ in (2.4 ) when N is sufficien tly large. Denote T 1 = { k : | α k | 6 = 0 } and T 2 = { l : | β l | 6 = 0 } as the supp orts of co efficien ts α and β , resp ectiv ely . Let Φ T 1 b e the N × | T 1 | matrix corresp onding to the columns of Φ indexed by T 1 , and define Ψ T 2 similarly . Theorem 4. Fix a subset T = T 1 S T 2 of the c o efficient domain, with T 1 and T 2 b eing the supp orts of the c o efficients α and β , r esp e ctively. Ch o os e a s ubse t Ω of the me asur ement domain of size | Ω | = m , and a sign se quenc e τ on T uniformly at r andom. Supp ose m satisfies m ≥ C 1 + max 1 ≤ l ≤ n 2 | ξ l | 1 − max 1 ≤ l ≤ n 2 | ξ l | max 2 {| T | , log ( n 1 + n 2 δ ) , C ˜ µ,T ,δ } , wher e C ˜ µ,T ,δ = 4  ( 1 2 + k Φ ∗ T 1 Ψ T 2 / N k )[2 log ( 2( n 1 + n 2 ) δ )] − 1 2 − ˜ µ p | T |  2 . Then with the pr ob ability exc e e ding 1 − 6 δ and sufficiently la r ge N , every c o efficient ve ctor θ =  α β  supp orte d on T wi th sign matching τ c an b e r e c over e d fr om solving the ℓ 1 optimization p r oblem ( P 1 ) : min ˆ α, ˆ β      ˆ α ˆ β      1 subje ct to H Ω = [Φ Ψ] Ω  ˆ α ˆ β  for t he c o e ffi c ient ve ctor  ˆ α ˆ β  , wher e H Ω = [Φ Ψ] Ω  α β  . Pr o of. F rom [37], the ℓ 1 optimization can reconstruct the sparse coefficien t v ector for general ORF pairs using ra ndo m samples with high probability when n umber of mea- suremen t m satisfies m ≥ C µ 2 M max 2 {| T | , log ( n 1 + n 2 δ ) , C µ (Φ , Ψ) ,T ,δ } , where µ M = max { µ Φ , µ Ψ } w ith µ Φ = max | Φ ij | and µ Ψ = max | Ψ ij | being the largest magnitude among the en tries in Φ and Ψ, resp ectiv ely . And C µ (Φ , Ψ) ,T ,δ = 4  ( 1 2 + k Φ ∗ T 1 Ψ T 2 / N k )[2 log ( 2( n 1 + n 2 ) δ )] − 1 2 − µ (Φ , Ψ) p | T |  2 . F or the pair of orthonormal bases consisting of FIR and TM basis functions, w e ha ve µ Φ = 1 since all elemen ts in Φ ha s mo dulus equal to 1, while f o r the second basis Ψ, the elemen t in Ψ is Ψ l ( z r ) with mo dulus equal to √ 1 −| ξ l | 2 | z r − ξ l | . By using triangle inequalit y , we hav e 1 − | ξ l | = || z r | − | ξ l || ≤ | z r − ξ l | ≤ | z r | + | ξ l | = 1 + | ξ l | , whic h implies s 1 − | ξ l | 1 + | ξ l | ≤ | Ψ l ( z r ) | ≤ s 1 + | ξ l | 1 − | ξ l | . Then v u u u t 1 − min 1 ≤ l ≤ n 2 | ξ l | 1 + min 1 ≤ l ≤ n 2 | ξ l | ≤ µ Ψ ≤ v u u u t 1 + max 1 ≤ l ≤ n 2 | ξ l | 1 − max 1 ≤ l ≤ n 2 | ξ l | . Th us µ M ≤ s 1+ max 1 ≤ l ≤ n 2 | ξ l | 1 − max 1 ≤ l ≤ n 2 | ξ l | . According to Theorem 3, µ (Φ , Ψ) ≈ ˜ µ . The claim t hen follo ws. Theorem 4 sho ws that for most sparse co efficien t v ectors θ supp orted on a fixed (but arbitrary) set T , the co efficien t ve ctors can b e r ecov ered with ov erwhelming probability if the sign of θ on T and the o bserv ations H Ω = [Φ Ψ ] Ω θ are drawn at random. 4 Computation Issues Notice that H Ω and [Φ Ψ] Ω in (3.3) are all complex-v alued, and compressed sensing with complex-v alued da ta has been discussed in [43] and [44]. A metho d adopted is to conv ert the ℓ 1 -norm minimization of complex signals to the second-order cone programming (SOCP) [35]. In t his pap er, we rewrite the complex-v alued minimization in the real-v alued form b y se parating the real and imaginary parts of (3.3) and (3.4), resp ectiv ely . Denoting H Ω = H R Ω + iH I Ω , [Φ Ψ] Ω = [Φ Ψ] R Ω + i [Φ Ψ] I Ω , η = η R + iη I , w e g et the followin g equiv alen t optimization pro blems. Lemma 1. The optimiza tion (3.3) is e quivalent to min α,β      α β      1 subje ct to  H R Ω H I Ω  =  [Φ Ψ] R Ω [Φ Ψ] I Ω   α β  , (4.1) and the optimization (3.4) is e quivalen t to min α,β      α β      1 subje ct to      H R Ω H I Ω  −  [Φ Ψ] R Ω [Φ Ψ] I Ω   α β      2 ≤ ǫ. (4.2) Pr o of. H Ω = [Φ Ψ] Ω  α β  is equiv alen t to H R Ω + iH I Ω = ([Φ Ψ] R Ω + i [Φ Ψ] I Ω )  α β  , whic h is equiv alen t to            H R Ω = [Φ Ψ ] R Ω " α β # H I Ω = [Φ Ψ] I Ω " α β # . This pro ves (4.1). Similar pro of is also a pplicable to (4.2). In (4.1) and (4 .2 ), the dimens ion of the problem is doubled. F or brevit y , w e refer to (4 .1) and (4 .2) a s tw o-ortho mo del hereafter. On the other hand, the separation of the real and imaginary part o f sensing matrix do es not change the mutual coherence of matrix [Φ Ψ]. In fact, the following holds. Lemma 2. F or p airs of c omplex matric es Φ and Ψ , µ (Φ , Ψ) = µ  Φ R Φ I  ,  Ψ R Ψ I  , wher e µ (Φ , Ψ) as define d in (3.5). Pr o of. Based on the fact that Φ k = Φ R k + i Φ I k and Ψ l = Ψ R l + i Ψ I l , w e ha v e k Φ k k =      Φ R k Φ I k      and k Ψ l k =      Ψ R l Ψ I l      . The inner products h Φ k , Ψ l i can be calculated b y the off-diagonal elemen ts of the Gram matrix [Φ Ψ] ∗ [Φ Ψ]. And the G ram matrix of  Φ R Ψ R Φ I Ψ I  is  Φ R Ψ R Φ I Ψ I  T  Φ R Ψ R Φ I Ψ I  = [Φ R Ψ R ] T [Φ R Ψ R ] + [Φ I Ψ I ] T [Φ I Ψ I ] = [Φ Ψ] ∗ [Φ Ψ] , the last equalit y is based on [Φ Ψ] = [Φ Ψ] R + i [Φ Ψ] I . It follo ws that µ (Φ , Ψ) = µ  Φ R Φ I  ,  Ψ R Ψ I  . Note t ha t sampling on the upp er unit circle instead of t he en tire unit circle is sufficien t to reconstruct the co efficien ts due to the separation of the real and imag inary part of the mo del. In f a ct, for the structure o f matrix [Φ Ψ] satisfying ( 3 .2), w e ha v e z − ( k − 1) r = ( ¯ z r ) − ( k − 1) and ψ l ( z r ) = ψ l ( ¯ z r ) for t he real p ole in the TM basis. This means that the ( N − r + 2)-th row of Φ is the conjugate of the r -th ro w ( r = 2 , 3 , · · · N 2 when N is ev en or r = 2 , 3 , · · · N +1 2 when N is o dd). Hence, if w e sample the conjugate rows to formulate (4.1) , the optimization will fail for the redundance in the equations. Similarly , if the z r on the real line is sampled, (4.1) will include a equation with all zero co efficien ts, whic h will lead to the failure of (4.1) as w ell. The strategy in this pap er is to sample on the upp er unit circle and exclude the endp oin t s, inste ad of the en tire unit circle. 5 Sim ulation This section illustrates the use of the t w o- o rtho mo del in the sparse system identifi- cation with a sim ulation study . Supp ose an underlying transfer function is sparse under the pair of FIR and TM bases, we implemen t the following experimen ts to demonstrate the reconstruction p erformance of the pro p osed a lgorithm. The exp eriments include three a sp ects: (a) the r econstruction of the sparse co efficien ts in tw o-ortho mo del; (b) the reconstruction of underlying SISO transfer function with poles kno wn; (c) the re- construction of underlying SISO transfer function with p oles (of multiplicit y greater than 1) partially kno wn o r unknow n. 5.1 Reconstruction of the sparse co efficien ts in the t w o-ortho basis represen tation The following pro cedure is implemen ted t o reconstruct the sparse co efficien ts of the transfer function in the t wo-ortho basis represen tat io n. Step 1 : Randomly generate a n ( n 1 + n 2 )-dimensional co efficien t v ector θ = [ α T , β T ] T consisting of ( s 1 + s 2 ) spik es with amplitude +1, and uniformly sample from the in- terv al (0 , 1) the p oles { ξ 1 , ξ 2 , · · · , ξ n 2 } in the T M basis { ψ l ( z ) , l = 1 , 2 , · · · , n 2 } . Then construct the transfer function H ( z ) = n 1 X k =1 α k z − ( k − 1) + n 2 X l =1 β l ψ l ( z ) = [1 , z − 1 , · · · , z − n 1 +1 , ψ 1 ( z ) , · · · , ψ n 2 ( z )] θ . Step 2: Uniformly sample z 1 , z 2 , · · · , z N on the upper unit circle, and then randomly select m samples to form the measuremen t mo del H Ω = [Φ Ψ] Ω θ . Step 3: Use ℓ 1 minimization (4.1) to solv e the co efficien t v ector θ . The soft w are ℓ 1 -magic is emplo y ed to reco ver the sparse co efficien t θ f r om t he tw o- ortho mo del. The num b ers of FIR basis and TM basis functions used are b oth 50, with sparsit y 3 and 2 under resp ectiv e ba sis, the n umber of measuremen ts is 30 = 6( s 1 + s 2 ) ( N =40 0 0), and the exp erimen t is rep eated 100 times. The relativ e reconstruction error of the sparse co efficien ts v ector θ of the transfer function, measured b y || ˆ θ − θ || 2 || θ || 2 , is sho wn in T able 1, with the maximal, minimal, a v erage error o f 10 0 trials in detail. T able 1: The relative reconstruction error of sparse co efficien t s o f the tr a nsfer function Mo del Reconstruction error Reco ve r Max Min Av erage rate t wo-ortho 0 .7 004 6.3729 e-006 0.0 203 91 % The reco v er rate, whic h is the p ercen tage of reconstruction error b elow the give n threshold out of the 100 trials, is also g iv en in T able 1. The threshold is set as 0.0005, and the ra te 91% s ho ws that the optimization model (4.1) can accurately rec onstruct the sparse co efficien ts of t he tra nsfer function with high proba bility . 5.2 Reconstruction of stable SISO system with p oles kno wn This subsection giv es examples to illustrate that TM mo del [45 ] a nd F IR mo del are sp ecial cases of t he t wo-ortho mo del. It show s that the redundan t basis can handle the sp ecial case with o nly o ne basis. Example 1: T M model Consider an underlying SISO system with transfer function H ( z ) = 1 . 5 z − 0 . 871 z 2 − 0 . 87 6 z + 0 . 0 0866 = 1 z − 0 . 01 + 1 2 z − √ 3 . (5.1) W e f ollo w the aforemen tioned Steps 2 and 3 a nd rep eat the exp erimen t 100 times. The n umbers of FIR basis a nd TM bas is functions used are both 100, with s parsit y 0 and 2 under respectiv e bases. The n um b er of measuremen ts is 28 = 14( s 1 + s 2 ) ( N =10 00). The first tw o p oles in the TM basis are the true p oles and the others are all set to a constan t, sa y 0. The reconstruction error is measured b y the H 2 norm of the difference b et wee n the original and reconstructed transfer functions. The reconstruction p erformances, including reconstruction error and reco v er ra te (for the threshold 0.0005), are sho wn in T able 2 and compared with the TM mo del (i.e. no FIR basis t erms in tw o-ortho mo del) discusse d in [45], whose sparsity a nd n um b er of measuremen ts are t he same a s those of tw o-ortho mo del. T able 2: Comparison b etw een tw o-ortho and TM mo dels Mo del Reconstruction error Reco ve r Max Min Av erage rate t wo-ortho 0 .9 274 3.8716 e-006 0.0 945 88% TM 0.9415 1.72 61e-006 0.0735 89% As seen from T able 2, the tw o-ortho mo del can accurately re construct the orig inal transfer function with a high probabilit y , whic h is o nly sligh tly lo w er than that of the TM mo del for the concatenation of FIR bases. This show s that TM mo del is indeed a sp ecial case of tw o-ortho mo del. Example 2: FIR mo del Next, w e will give another example to sho w that the sparse FIR mo del (i.e. no TM basis terms in tw o-ortho mo del) is also a sp ecial case of t wo-ortho mo del. Consider the transfer function H ( z ) = 1 /z 3 + 1 / z 5 + 3 / z 8 . The n umbers of FIR basis a nd TM basis f unctions used ar e b oth 10 0 , with sparsit y 3 and 0 under resp ectiv e bases. T o a void the TM basis degenerating to FIR basis, the first three p oles in TM basis are randomly c hosen in the interv a l (0, 1) and the others are all zero. The n umber of measuremen ts is 3 0 = 10 ( s 1 + s 2 ) ( N =1000). The reconstruction p erformance is sho wn in T able 3, including the H 2 -norm of reconstruction error and comparison to the sparse FIR mo del with the same sparsit y and measuremen ts as those of the tw o-ortho mo del. The threshold used is 0.0 0 05. T able 3: Comparison b etw een t wo-ortho and FIR mo dels Mo del Reconstruction error Reco ve r Max Min Av erage rate t wo-ortho 0 .5 512e-003 0 .0 063e-003 0 .0521e-003 99% FIR 0.1838e-00 3 0.0049e-00 3 0.0286e-0 0 3 100% As seen fro m T able 3, t wo-ortho mo del and sparse F IR mo del can b oth accurately reco v er the transfer function with ve ry high probability . This sho ws that sparse FIR mo del is also a sp ecial case of tw o-ortho mo del. 5.3 Reconstruction of stable SISO system with p oles not ex- actly kno wn The ab ov e subsections ar e based o n the a ssumption that all the p oles giv en in the TM basis are the true p o les of the transfer function, whic h is often unrealistic. In this subsection, we consider the situations when the p oles are pa r tially know n or fully unkno wn, whic h is difficult to handle with either TM mo del or FIR mo del alone. Example 3: R econstruction with part of the p oles a v ailable Con tinue to consider the underlying SISO system (5.1). The n umbers of FIR basis and TM basis functions used are b oth 100, with sparsity 3 a nd 2 under r esp ective bases. The num b er of measuremen t s is 40 = 8( s 1 + s 2 ) ( N =100 0). Here w e assume one of the p oles √ 3 / 2 is kno wn. W e rep eat ed the exp erimen t for 100 times and compared with t he sparse F IR mo del with 500 FIR basis functions in T able 4. T able 4: Comparison b etw een t wo-ortho mo del and FIR m o dels Mo del Sparsit y Measuremen t Reconstruction error Recon. Reco ve r ♯ Max Min Av erage Order rate t wo-ortho 5 30 0.9203 0.0001 0.0453 3 94% FIR 30 180 0.2534 0.0002 0.0207 60 45% As sho wn in T able 4, the reco ver rate (for the threshold 0.0005) is 94% for tw o-ortho mo del. This means that if one of the p oles in the TM basis is the true po le ( √ 3 / 2), the optimization mo del (4.1) can rec onstruct the o riginal tra nsfer function a ccurately with very high probability . The reconstruction order in T able 4 is with resp ect to the minim um error fro m the 1 00 trials. Although the maximal a nd a verage errors of sparse FIR mo del are less than those of tw o-ortho mo del, it needs mor e than 6 times measuremen ts to get the recons truction ev en with m uc h higher order. Hence, when the p oles are partially kno wn, the t w o- o rtho mo del has muc h b etter p erformance than sparse FIR m o del, in terms of reco v er rate, num b er of measuremen ts and reconstruction order. Example 4: R econstruction with all the poles unkno wn Consider t he underlying SISO system (5 .1). Here the p o les are all unkno wn, w e set differen t p oles with differen t multiplicit y to exemplify the p erformance of the t wo-ortho mo del. The num b ers of FIR basis and TM basis functions used are b oth 100, with sparsit y 3 and 6 under resp ectiv e bases. The n umber of measuremen ts is 54 = 6 ( s 1 + s 2 ) ( N = 1 000). W e rep eated t he exp erimen t for 100 times. The reconstruction error of tw o- ortho mo del, with resp ect to differen t p oles (0.9, 0.85, 0.8) and multiplicit y (2 to 6) in TM basis, is show n in Fig. 1 . 2 3 4 5 6 0.46 0.48 0.5 0.52 0.54 0.56 0.58 0.6 0.62 0.64 0.66 maximal error 2 3 4 5 6 0 0.5 1 1.5 2 2.5 3 x 10 −3 minimal error multiplicity of pole 2 3 4 5 6 0.02 0.025 0.03 0.035 0.04 0.045 0.05 0.055 average error ξ =0.9 ξ =0.85 ξ =0.8 Figure 1: Reconstruction error of t w o-o rtho mo del. As seen from Fig . 1, the maximal error increase s with the m ultiplicit y , while the minimal error decreases with the m ultiplicit y . The av erage reconstruction error of the t wo-ortho mo del decreases as the m ultiplicit y increases when p ole is 0.9, while for p ole 0.85 and 0.8, the a v erage error increases . This sho ws that higher m ultiplicit y is necessary when the p o le in TM basis is far a w ay from the true po le. The succes sful reco v er rate (fo r the threshold 0.0005) is giv en in T able 5. T able 5: Recov er rate with resp ect to differen t poles and m ultiplicit y P ole Multiplicit y 2 3 4 5 6 ξ = 0 .9 0 0.06 0 .3 3 0.38 0.31 ξ = 0.85 0.08 0.66 0 .5 9 0.43 0.27 ξ = 0 .8 0 0 0.06 0.32 0.39 F rom Fig. 1 and T able 5, if the p ole in TM basis is not the true p ole of the transfer function, we can use tw o-ortho mo del with a nearb y m ultiple p ole to reconstruct. The farther the distance b etw een the p ole in TM basis and true p ole, the higher m ultiplicity is nec essary . And based on the reco v er rate, there is an optimal m ultiplicit y for each p ole in TM basis. The optimal m ultiplicity for the p ole 0.8 5 , 0.9, 0 .8 is 3, 5, 6, resp ectiv ely , and the corresp onding reconstruction order with resp ect to the minimal error is 8 , 5, 8, respectiv ely . While the order of reconstruction b y sparse FIR mo del (see previous section) with the minimal error is 6 0, and the reco ve r rate is 45%. Hence, t wo-ortho mo del has b etter r econstruction p erformance (reconstruction order, n umber of measuremen ts and reco v er rate) than sparse FIR mo del for the nearest p ole 0.85, and the farther p ole will lead to low er reco v er rate. Example 5: R econstruction with multiple poles W e consider the situation when the transfer function has multiple poles, whic h is difficult to handle with FIR mo del. The transfer function of in terest is H ( z ) = 1 ( z − 0 . 1) 8 + (2 − √ 3 z ) 4 (2 z − √ 3) 5 . The n um b ers o f FIR basis a nd TM basis functions use d are 100 and 50, respectiv ely , with sparsit y of 3 and 2 under respective bases. The num b er of measureme n ts is 50 = 1 0( s 1 + s 2 ) ( N =1000). 1) case 1: the dominan t pole √ 3 / 2 is known. W e set the first 5 p oles in the TM basis as the true dominant p oles, and the others are all set to zero. W e repeat the e xp erimen t for 100 times. The histog r a m of reconstruction error o f t w o-o r t ho mo del, compared with the FIR mo del with 500 FIR basis functions, is shown in Fig. 2. As show n in Fig. 2, the errors from tw o-ortho mo del are mostly within (0, 0.1 ), while the errors from FIR mo del scatter ov er (0, 1.2) . The parameters for furt her comparison a r e given in T able 6. 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0 20 40 60 80 100 reconstruction error number of trials reconstruction error of transfer function two−ortho model 0.7 0.8 0.9 1 1.1 1.2 1.3 0 10 20 30 reconstruction error number of trials FIR model Figure 2: Histogram of reconstruction error of tw o-ortho mo del and sparse FIR mo del. T able 6: Comparison b etw een t wo-ortho and FIR mo dels Mo del Sparsit y Measuremen ts ♯ Reconstruction error Recon. Order Max Min Av erage t wo-ortho 5 50 0.8044 0.0002 0.0245 20 FIR 30 120 1.1622 0.6199 0.8952 4 99 The p ercen ta g es of reconstruction error b elo w 0.001 are 53 % and 0% for t wo-ortho mo del and FIR mo del, resp ectiv ely . Hence tw o-ortho mo del is far b etter t ha n FIR mo del for system with m ultiple p oles. 2) case 2: the p oles of the or iginal transfer function are unkno wn b e- forehand. The n umbers of FIR and TM basis used are both 100, with the sparsit y 3 and 2, resp ectiv ely and the n umber o f measuremen ts is 60. The p ole ξ 1 in TM basis is set as 0.9 with m ultiplicity 7 (higher tha n the true m ultiplicity), and the others are all set t o 0. The histogr am of reconstruction error with comparison to FIR mo del is sho wn in Fig. 3 , and the reconstruc tion p erformance is giv en in T able 7. W e also consider the situations when ξ 1 is set to 0.85 and 0.8, resp ectiv ely , the results a re also giv en in Fig. 3 and T able 7. 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 0 10 20 30 reconstruction error number of trials reconstruction error of transfer function FIR model 0 0.5 1 1.5 0 10 20 30 reconstruction error number of trials two−ortho model( ξ =0.9) 0 0.2 0.4 0.6 0.8 1 1.2 1.4 0 20 40 60 reconstruction error number of trials two−ortho model( ξ =0.85) 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 0 10 20 30 reconstruction error number of trials two−ortho model( ξ =0.8) Figure 3: Histogram of reconstruction error of tw o-ortho mo del a nd sparse FIR mo del (with m ultiple p oles). T able 7: Reco ver rate of tw o-ortho and FIR mo dels for threshold 0.00 5 with resp ect to differen t poles Mo del Reconstruction error Reco ve r Recon. Max Min Av erage rate order FIR 1 .0906 0 .3247 0.725 05 0% 499 ξ = 0.9 1.4336 0.0019 0.5756 4 % 26 ξ = 0 .85 1.3 5 34 0.0 0 17 0.23 5 0 13% 19 ξ = 0.8 1.5505 0.0621 0.6077 0 % 10 6 It is obvious from Fig. 3 and T able 7, that tw o-ortho mo del and sparse FIR mo del b oth cannot handle we ll the system without prior infor mation ab out t he multiple p oles. Ev en so, the a ve rage reconstruction erro r and the reconstruction order o f t w o-o rtho mo del is less than those of sparse F IR mo del. The closer the p oles in the TM basis are to the t r ue p o le, the less the av erage reconstruction error is, and the order is m uch lo wer with a higher probability . In applicatio n, the true transfer function is unkno wn, so the reconstruction error cannot b e calculated. T o c ho ose a reconstruction with b etter p erformance, the clus- tering criterion is based. Here w e cluster the reconstruction coefficien ts from the 100 exp eriments when ξ = 0 . 85. The num b er o f clusters is 10, a nd the histogram of clusters is shown in Fig . 4. 1 2 3 4 5 6 7 8 9 10 0 5 10 15 20 25 30 35 cluster number of samples Figure 4: Clusters of reconstruction co efficien ts. F rom Fig. 4, we can see that the cluster 7 has most of the samples, and the reconstruction order of these sample is a lmo st 20. Therefore, we can c ho ose the recon- struction co efficien t v ector with the minim um error from this cluster. 6 Conclus ion W e hav e established the uniqueness of sparse represen tation f o r a rational SISO system under FIR and TM bases. With the uniqueness prop ert y and combinin g the principle of compressed sensing, w e ha ve prop o sed an iden tification metho d using suc h t wo bases . The iden tification metho d is linear in the t w o orthonormal bases, and w e ha ve pr esen ted the lo we r b ound of the nu m b er of measuremen t whic h guaran tees the efficien t reconstruction of sparse represen tation co efficien ts in pairs of FIR and TM bases b y the ℓ 1 minimization from a limited n umber of observ ations on the upp er unit circle with high probability . It is sho wn that the prop osed metho d can reconstruct the system with m uc h few er measuremen ts, m uc h less error and hig her probability than those of FIR mo del, ev en without the prior information of poles. Since the t w o-o rtho mo del needs m uc h few er significan t coefficien ts than tho se of FIR mo del, it leads to a m uch low er order reconstruction than FIR mo del. References [1] M. J. Levin, “Optim u m estimation of impu lse resp onse in the presence of noise,” IRE T ransactions on Circu it Theory , vo l. 7, no. 1, pp . 50-56, 1960. [2] L. Rabiner, R. Cro c hiere, and J. Allen, “FIR system mo deling and identi fication in the presence o f noise and with band -limited inpu ts,” IEEE T ransactio ns on Acoustics, Sp eec h and Signal Pro cessing, v ol. 26, no. 4, p p. 319-3 33, 1978. [3] B. Widro w, J. M. Mcco ol, M. G. L arimore, and C. R. Joh n son, “Stationary and nonsta- tionary learnin g c h aracteristics of the LMS adaptiv e filter,” in Pro ceedings of the IEEE, 1976, v ol. 64, p p. 1151-1162 . [4] H ¨ u and H. Kan , “Th e least-squares iden tification of FIR sy s tems sub ject to worst-ca se noise,” Systems & Con trol Letters, v ol. 23, n o. 5, p p. 329-338, 1994. [5] T. Kata yama , Su bspace metho ds for system identi fication, S pringer London, 2005. [6] B W ahlb erg, “System identi fication u sing Laguerre mo d els,” IEEE T ransactions on Au- tomatic Cont rol, v ol. 36, no. 5, pp. 551-562, 1991. [7] R. Malti, S. B. Ekongo lo, and J. Ragot, “Dynamic SISO and MIS O system approx- imations based on optimal Laguerre mo dels,” I E EE T ransactions on Automatic Cont rol, v ol. 43, n o. 9, pp. 1318-1323 , 1998. [8] C. T. C hou and M. V erhaegen, “Con tin u ous-time identificatio n of SISO systems us in g Laguerre functions,” IEE E T ransactio ns on S ignal P ro cessing, v ol. 47, no. 2, pp. 349-362, 1999. [9] B W ahlb erg, “System ident ification using Kautz mo d els,” IEEE T ransactions on Auto- matic Con trol, v ol. 39, n o. 6, pp. 1276-1282 , 1994. [10] B. W ahlb erg and P . M. M¨ akil¨ a, “ On appro ximation of stable linear dyn amical systems using Laguerre and K autz functions,” Automatica, v ol. 32, no. 5, p p. 693-7 08, 1996. [11] B. Ninness and F. Gustafsson, “A un ifying construction of orthonorm al bases f or system iden tification,” IEEE T r an s actions on Automatic Cont rol, v ol. 42, no. 4, pp. 51 5-521, 1997. [12] H. Ak ca y and B. Ninness, “Ratio n al basis functions for robust iden tification from fre- quency and time-domain measuremen ts,” in American Control Conferen ce, 1998, pp. 1101- 1117. [13] B. Ninness, “F requency domain estimation usin g orthonormal b ases,” in Pr o ceedings of IF A C W orld Congress, 1996, pp . 381-386 . [14] N. F. D. W ard and J. R. P artington, “Robust iden tification in the disc algebra us in g rational w av elets and orthonormal basis functions,” In ternational Journal of C on trol, v ol. 64, n o. 3, pp. 409-423, 1996. [15] P . V. Guc ht and A. Bultheel, “Orthogonal rational functions for system id en ti- fication: n umerical asp ects,” IEEE T ransactions on Au tomatic Con trol, v ol. 48, no. 4, pp. 705- 709, 2003. [16] B. Ninn ess, J. C. Gomez, and S. W eller, “MIMO sys tem iden tifi cation using orthonormal basis functions,” in Pro ceedings of the IEEE C onference on Decision and C ontrol, 2002, pp. 703-70 8. [17] M. W en and Q. T ao, “F requency-domain ident ification: an algorithm based on an adap- tiv e rational orthogonal system,” Automatica, v ol. 48, no. 6, p p . 1154-1162, 2012. [18] Q. Chen, W. Mai, L . Zh ang, and W. Mi, “System ident ification by d iscrete rational atoms,” Automatica, v ol. 56, no. Supp lemen t C, p p. 53-59 , 2015. [19] Y. Oh ta, “Sto c hastic system trans f ormation us ing generalized orthonormal basis func- tions with a pplications to co ntin uous -time system identificat ion,” Automat ica, v ol. 47, no. 5, pp. 1001-1006, 2011. [20] E. Hida ya t and A. Medv edev, “Laguerre domain iden tification of con tin uous linear time- dela y systems from imp ulse resp ons e data,” Au tomatica, v ol. 48, no. 11, pp. 2902-2907 , 2012. [21] K. Tiels and J. Schouk ens, “Wiener sys tem identificatio n with generalized orthonormal basis functions,” Automatica, v ol. 50, no. 12, pp. 3147-3 154, 2014. [22] P eter S . C. Heub erger, Mo delling and id en tification with rational orthogonal basis fu nc- tions, Springer London, 2005. [23] D. L. Donoho, “Compressed sensing,” I EEE T ransactio n s on In formation T h eory , v ol. 52, no. 4, p p. 1289- 1306, 2006. [24] E. J. Cand` es, J. Romberg, and T. T ao, “Robust uncertain t y principles: exact signal reconstruction from highly incomplete frequency information,” I E EE T ransactions on Information Theory , vo l. 52, no. 2, pp. 489-5 09, 2006. [25] E. J. Can d` es and T. T ao, “Near-optimal signal reco very from rand om p ro jections: uni- v ersal encoding strategies,” IEEE T rans actions on In f ormation Theory , v ol. 52, no. 12, pp. 5406-5 425, 2006. [26] E. J. Cand ` es, J . Rom b erg, and T. T ao, “Stable signal reco very from incomplete and inaccurate measurements,” Comm unications on Pure & Applied M athematics, v ol. 59, no. 8, pp. 410-412, 2006. [27] Y. Gu, J . Jin, and S . Mei, “ l 0 norm constraint LMS algorithm for spars e system iden ti- fication,” IEEE Signal Pro cessing Letters, v ol. 16, no. 9, p p. 774-7 77, 2009. [28] Y. Chen, Y. Gu , and A. O. Hero, “Sparse LMS for sy s tem identificat ion,” in IEEE In ternational Conference on Acoustics, Sp eec h and Signal Processing, 2009, pp. 3125 - 3128. [29] N. Kalouptsidis, G. Mileounis, B. Babadi, and et al, “Adaptiv e algorithms for spars e system ident ification,” Signal Pro cessing, vo l. 91, no. 8, pp. 1910- 1919, 2011. [30] Y. Kopsin is, K. Sla v akis, and S . Theo doridis, “Onlin e sp arse system iden tifica- tion and signal reconstruction using pro jections on to w eight ed ℓ 1 balls,” IEEE T ransactions on Signal Pro cessing, vo l. 59, no. 3, p p. 936-9 52, 2010. [31] K. Sla v akis, Y. Kopsinis, and S. Th eo d oridis, “Adaptiv e algorithm for sparse sy s tem iden tification u s ing p ro jections ont o weig h ted ℓ 1 balls,” in I E EE In ternational Conference on Acoustics, Sp eec h and S ignal Pro cessing, 2010, pp. 3742-3745. [32] M. Ela d and A. M. Bru c kstein, “A generalized uncertain t y principle and sparse repre- sen tation in pairs of bases,” IEEE T ransactions o n In formation Theory , vo l. 48, no. 9, pp. 2558-2 567, 2002. [33] D. L. Donoho and M. Elad, “Optimally sparse r epresen tation in general (nonorthogonal) dictionaries via ℓ 1 minimization,” in Pro ceedings of the National Academ y o f Science s of the United States of America, 2003, v ol. 100, pp . 2197- 2202. [34] R. Grib onv al and M. Nielsen, “Sp arse representati ons in unions of bases,” IEEE T rans- actions on Information Theory , vo l. 49, no. 12, pp. 3320-3 325, 2003. [35] D. M. Maliouto v, M. Cetin, and A. S. Willsky , “Optimal spars e r epresen tations in general o ve rcomplete b ases,” in IEE E Int ernational Conf erence on Acoustics, Sp eec h and Signal Pro cessing, 2004, v ol. 2, pp. ii-793- 6. [36] H. Rauh ut, “Compressive sensing and structured random matrices,” Radon Series on Computational and Applied Mathematics, v ol. 9, pp. 1-94, 2010. [37] D. Xiong, L. Chai, and J. Zhang, “Uncertain t y principle and sparse reco n struction in pairs of orthonormal rational function bases,” arXiv:1712 .02981 , 2017. [38] S. T ak enak a, “On th e orthogonal fu nctions and a new formula of int erp olation,” Japanese Journal of Mathematics, v ol. 2, pp. 129-1 45, 1925. [39] F. Malmquist, “Sur la d etermination dune class d e fonctions analytiques par leur v aleurs dans un ensemble donn e d e p oin ts,” in Pro c. C . R. 6ime Cong. Math. S cand., 1925, pp. 253-2 59. [40] B. K. Natara jan, “Sp arse appro ximate solutions to linea r systems,” SIAM Journal on Computing, v ol. 24, n o. 2, pp. 227-234, 2006. [41] S. C hen, D. L. Donoho, and M. A. S aunders, “A tomic decomp osition by b asis pursu it,” SIAM Review, v ol. 43, no. 1, p p . 129- 159, 2001. [42] S. Bo yd and L. V anden b erghe, Con vex O ptimization, Cam b ridge Univ ersity Press, 2004. [43] S. Y u , A. S . Khw a ja, and J. Ma, “Compressed sens in g of complex-v alued d ata,” Signal Pro cessing, v ol. 92, no. 2, pp. 357-362, 2012. [44] F. H. C. Tivive and A. Bouzerdoum, “A compressed sensing metho d for complex-v alued signals with application to through-the-w all radar imaging,” in I EEE In tern ational C on- ference on Acoustics, Sp eec h and Signal Pro cessing, 2013, pp. 2144 -2148. [45] D. Xiong, L. Ch ai, and J. Zhang, “Sparse system iden tification u sing orthogonal ratio- nal functions,” in Pro ceeding of th e 11th W orld Congress on In telligen t Con trol and Automation, 2014, pp. 2340- 2345.