Strongly Consistent Model Order Selection for Estimating 2-D Sinusoids in Colored Noise

Strongly Consisten t Mo del Order Selection for Estimating 2-D Sin usoids in Colored Noise Mark Kliger and Joseph M. F rancos ∗ Abstract W e consider the problem of join tly estima t ing the num b er as w ell as the pa- rameters of t wo-dimensional sin usoidal signals, observ ed in the presence of a n additiv e colored noise field. W e b egin by elaborat ing on the least squares estima- tion of 2-D sin usoidal s ignals, when t he a ssumed n umber of sin usoids is incorrect. In the case where the num b er of sinuso idal signals is under-estimated w e sho w the almost sure con v e rgence of the least squares estimates to the pa rameters of the dominan t sinusoids . In the case where this n um b er is o ver-estimated, the estimated pa ra meter v ector obtained by t he least squares estimator con tains a sub-v ector that con v erges almost surely to the correct parameters of the sin u- soids. Based on these results, w e pro v e t he strong consistency of a new mo de l order selection rule. Keyw ords: Tw o-dimensional random fields; mo del order selec tion; least squares estimation; strong c onsistency . ∗ M. Kliger is with the Department o f E lectrical Engineering a nd Computer Scie nce , Universit y of Mic higan, Ann A rb or, MI, 4 8109- 2122, USA. T el: (734 ) 6 47-83 89 F AX: (734) 763- 8041, email: mkliger@ umic h.edu. J. M. F rancos is with the Department of Electrical and Computer Engineering, Ben-Gurion Universit y , Beer-Shev a 84105 , Israel. T el: + 972 8 64618 42, F AX: +972 8 6472 949, email: francos@ ee.bgu.ac.il 1 1 In tro duc t ion W e consider the problem of jointly estimating the n um b er a s w ell a s the parameters of t wo- dimensional sin usoidal signals, observ ed in the presence of an additive noise field. This problem is, in fact, a sp ecial case of a m uch more general problem, [5]: F rom the 2-D W old-like decomp osition w e hav e tha t a ny 2- D regular and homogeneous discrete random field can b e represen ted as a sum of t w o m utually ortho g onal comp onen ts: a purely-indeterministic field and a deterministic one. In this pap er w e consider the sp ecial case where the deterministic comp onen t consists of a finite (unknown) num b er of sin usoidal comp onents , while the purely-indeterministic comp onen t is an infinite order non- symmetrical half plane, (or a quarter-plane), moving av erage field. This mo deling and estimation problem has fundamen t al theoretical imp or t a nce, as w ell as v arious applications in texture estimation of images (see, e.g., [4] and the references therein) and in w a v e propagation problems (see, e.g., [14] and the references therein). Man y algo rithms hav e b een devised to estimate the parameters of sin usoids observ ed in white noise and only a s mall fraction of the deriv ed metho ds has b een extended to the case where the noise field is colored (see, e.g. , F ra ncos et. al. [3], He [8], K undu and Nandi [11], Li and Stoica [12], Zhang and Mandrek ar [13 ], and the references therein). Most of these assume the n umber of sin usoids is a-prio ri kno wn. How eve r this assumption do es no t alw a ys hold in practice. In the past three decades the problem of mo del order selection for 1-D signals has receiv ed considerable att en tion. In general, mo del order selection rules are based (directly or indirectly) on three p o pular criteria: Ak aike informatio n criterion (AIC), the minim um description length (MDL ) , and the maxim um a-p osteriori probabilit y criterion (MAP). All these criteria ha ve a common fo rm composed of tw o terms: a data term and a p enalt y term, where the data term is the log-likelihoo d function ev aluated for the assumed mo del. The problem of mo delling m ultidimensional fields has receiv ed m uc h less atten tion. In [9], a MAP mo del order selection criterion for join tly estimating the n um b er and the parameters of t w o- dimensional sin usoids observ ed in the presence of an additive white G aussian noise field, is deriv ed. In [10], w e pro ved t he strong consistency of a large family o f mo del order selection rules, whic h includes the MAP based rule in [9] as a sp ecial case. In this pap er w e deriv e a strongly consisten t mo del order selection rule, for join tly estimating the n umber of sin usoidal comp onen ts and their parameters in the presence of colored noise. This deriv atio n extends the results o f [10] to the case where the additiv e noise is colored, mo deled b y an infinite order non-symmetrical half - plane or quarter-plane moving a v erage represen tat ion, suc h that the noise field is not necessarily Gaussian. T o the b est of our kno wledge this is the most general result av ailable in the area of mo del-o r der sele ction rules of 2 - D random fields with mixed sp ectrum. 1 The prop osed criterion has the usual form of a data t erm and a p enalt y term, where the first is the le ast squar es estimator ev aluated fo r the assumed mo del order and the latter is prop o rtional to the logarithm of the data size. Since w e ev aluate the data term for any assumed mo del order, including incorrect ones, w e should consider the problem of least squares estimation of the parameters of 2- D sin usoidal signals when the assumed num b er of sinus oids is incorrect. Let P denote the n um b er of sinu soidal signals in the o bserv ed field and let k denote their assumed num b er. In the case where the n um b er of sin usoidal signals is under-estimated, i.e. , k < P , we pro v e the almost sure con v ergence of the least squares estimates to the parameters of the k dominan t sin usoids. In the case where the n umber of sinus oidal signals is ov er-estimated, i.e. , k > P , w e pro v e the almost sure con v ergence of the estimates obt a ined b y the least squares estimator to the par a meters of the P sin usoids in the observ ed field. The additional k − P comp onen ts a ssumed to exist, are a ssigned b y t he least squares estimator to the dominan t comp onen ts of the p erio dogra m of the noise field. Finally , using this result, w e pro ve the strong consistency of a new mo del order selection criterion and show ho w differen t assumptions regarding a noise field para meters affect the p enalty term of the criterion. The prop osed criterion completely generalized the previous results [9], [10], and pro vides a strong ly consisten t estimator of t he num b er as w ell as of the parameters of the sin usoidal comp onen ts. 2 Notations, De finition s and Assumptio ns Let { y ( n, m ) } b e a real v alued field, y ( n, m ) = P X i =1 ρ 0 i cos( ω 0 i n + υ 0 i m + ϕ 0 i ) + w ( n, m ) , (1) where 0 ≤ n ≤ N − 1, 0 ≤ m ≤ M − 1 and for eac h i , ρ 0 i is non-zero. Due to phy sical considerations it is further assumed that for eac h i , | ρ 0 i | is bo unded . Recall that the non-symmetric al hal f - plan total-or der is defined b y ( i, j )  ( s, t ) iff ( i, j ) ∈ { ( k , l ) | k = s, l ≥ t } ∪ { ( k , l ) | k > s, −∞ ≤ l ≤ ∞} . (2) Let D b e an infinite or der non- symmetrical half-plane suppor t, defined b y D =  ( i, j ) ∈ Z 2 : i = 0 , 0 ≤ j ≤ ∞  ∪  ( i, j ) ∈ Z 2 : 0 < i ≤ ∞ , −∞ ≤ j ≤ ∞  . (3) 2 Hence the notations ( r , s ) ∈ D and ( r , s )  (0 , 0 ) are equiv alen t. W e assume that { w ( n, m ) } is an infinite order non-symmetrical half-pla ne MA noise field, i.e. , w ( n, m ) = X ( r ,s ) ∈ D a ( r , s ) u ( n − r , m − s ) , (4) suc h t ha t the follo wing assumptions are satisfied: Assumption 1: The field { u ( n, m ) } is a n i.i.d. real v alued zero-mean rando m field with finite v aria nce σ 2 , suc h that E [ | u ( n, m ) | α ] < ∞ for some α > 3 . Assumption 2: The sequence a ( i, j ) is an absolutely summable determinis tic sequence , i.e. , X ( r ,s ) ∈ D | a ( r , s ) | < ∞ . (5) Let f w ( ω , υ ) denote the sp ectral densit y function of the noise field { w ( n, m ) } . Hence, f w ( ω , υ ) = σ 2     X ( r ,s ) ∈ D a ( r , s ) e j ( ωr + υ s )     2 . (6) Assumption 3: The spatial frequencies ( ω 0 i , υ 0 i ) ∈ (0 , 2 π ) × (0 , 2 π ), 1 ≤ i ≤ P are pairwise differen t. In other words, ω 0 i 6 = ω 0 j or υ 0 i 6 = υ 0 j , when i 6 = j . Let { Ψ i } b e a sequence of rectangles suc h tha t Ψ i = { ( n, m ) ∈ Z 2 | 0 ≤ n ≤ N i − 1 , 0 ≤ m ≤ M i − 1 } . Definition 1 : The sequence of subsets { Ψ i } is said to tend to infinit y (w e adopt the notation Ψ i → ∞ ) as i → ∞ if lim i →∞ min( N i , M i ) = ∞ , and 0 < lim i →∞ ( N i / M i ) < ∞ . T o simplify notations, w e shall omit in the follo wing the subs cript i . Thu s, the notation Ψ( N , M ) → ∞ implies that b oth N and M tend to infinit y as functions of i , and at roughly the same rate. Definition 2 : Let Θ k b e a bo unded and closed subset of the 4 k dimensional space R k × ((0 , 2 π ) × (0 , 2 π )) k × [0 , 2 π ) k where for an y v ector θ k = ( ρ 1 , ω 1 , υ 1 , ϕ 1 , . . . , ρ k , ω k , υ k , ϕ k ) ∈ Θ k the co ordinate ρ i is non- zero and b ounded for ev ery 1 ≤ i ≤ k while the pairs ( ω i , υ i ) are pairwise differen t, so that no t w o regressors coincide. W e shall refer to Θ k as the p a r a m eter sp ac e . F r o m the mo del definition (1) and the ab ov e assumptions it is clear that θ 0 k = ( ρ 0 1 , ω 0 1 , υ 0 1 , ϕ 0 1 , . . . , ρ 0 k , ω 0 k , υ 0 k , ϕ 0 k ) ∈ Θ k . 3 Define the loss function due to the erro r of the k -th order regression mo del L k ( θ k ) = 1 N M N − 1 X n =0 M − 1 X m =0  y ( n, m ) − k X i =1 ρ 0 i cos( ω 0 i n + υ 0 i m + ϕ 0 i )  2 . (7) A v ector ˆ θ k ∈ Θ k that minimizes L k ( θ k ) is called the L e ast Squar e Estimate ( LSE). In the case where k = P , the LSE is a str ongl y c onsistent estimator of θ 0 P (see, e.g. , [11] and the references therein). 3 Strong C o nsiste ncy of the Ov er- and Under-D etermined LSE In the following subsections w e establish the strong consistency of this LSE when the num b er of sin usoids is under-estimated, or ov er-estimated. The first theorem establishes the strong consis- tency of the least squares estimator in the case where t he n um b er of the regressors is low er tha n the actual n umber of sin usoids. The second theorem establishes the strong consistency of the least squares estimator in the case where the num b er of the regressors is higher than the actual num b er of sin usoids. 3.1 Consistency of the LSE for an Under-Estimate d Mo del Order Let k denote the assumed n um b er of observ ed 2-D sin usoids, where k < P . F or an y δ > 0, define the set ∆ δ to b e a subset of the parameter space Θ k suc h that each v ector θ k ∈ ∆ δ is differen t from the v ector θ 0 k b y at least δ , at least in o ne of its coo r dinates, i.e. , ∆ δ = " k [ i =1 R iδ # ∪ " k [ i =1 Φ iδ # ∪ " k [ i =1 W iδ # ∪ " k [ i =1 V iδ # , (8) where R iδ =  θ k ∈ Θ k : | ρ i − ρ 0 i | ≥ δ ; δ > 0  , Φ iδ =  θ k ∈ Θ k : | ϕ i − ϕ 0 i | ≥ δ ; δ > 0  , W iδ =  θ k ∈ Θ k : | ω i − ω 0 i | ≥ δ ; δ > 0  , V iδ =  θ k ∈ Θ k : | υ i − υ 0 i | ≥ δ ; δ > 0  . (9) 4 T o prov e the main result of this section w e shall need an additional assumption a nd the follo wing lemmas: Assumption 4: F or conv enience, and without loss o f g eneralit y , we assume that the sin usoids are indexed according to a des cending o rder of their amplitudes, i.e. , ρ 0 1 ≥ ρ 0 2 ≥ . . . ρ 0 k > ρ 0 k +1 . . . ≥ ρ 0 P > 0 , (10) where w e assume that for a given k , ρ 0 k > ρ 0 k +1 to av o id trivial am biguities resulting fro m the case where the k -th dominan t comp onen t is not unique. Lemma 1. lim inf Ψ( N ,M ) →∞ inf θ k ∈ ∆ δ  L k ( θ k ) − L k ( θ 0 k )  > 0 a.s. (11) Pr o of: See Appendix A f or the pro of. Lemma 2. L et { x n , n ≥ 1 } b e a se quenc e of r ando m variabl e s. Then Pr { x n ≤ 0 i.o. } ≤ Pr { lim inf n →∞ x n ≤ 0 } , (12) where the abbreviation i.o. stands for infinitely often . Pr o of: See Appendix B f or the pro of. The next theorem establishes the strong consistency of the least squares estimator in the case where the n umber o f the regressors is low er than the actual n um b er of sin usoids. Theorem 1. L et Assumptions 1-4 b e satisfie d. Then, the k -r e gr ess o r p ar ameter ve ctor ˆ θ k = ( ˆ ρ 1 , ˆ ω 1 , ˆ υ 1 , ˆ ϕ 1 , . . . , ˆ ρ k , ˆ ω k , ˆ υ k , ˆ ϕ k ) that minimizes (7) i s a str ongly c onsistent estimator of θ 0 k = ( ρ 0 1 , ω 0 1 , υ 0 1 , ϕ 0 1 , . . . , ρ 0 k , ω 0 k , υ 0 k , ϕ 0 k ) as Ψ( N , M ) → ∞ . That is, ˆ θ k → θ 0 k a.s. as Ψ( N , M ) → ∞ . (13) Pr o of: The pro of follo ws an argumen t pro p osed b y W u [15], Lemma 1. Let ˆ θ k = ( ˆ ρ 1 , ˆ ω 1 , ˆ υ 1 , ˆ ϕ 1 , . . . , ˆ ρ k , ˆ ω k , ˆ υ k , ˆ ϕ k ) b e a pa rameter v ector that minimizes (7). Ass ume tha t the prop osition ˆ θ k → θ 0 k a.s. as Ψ( N , M ) → ∞ is not tr ue. Then, there exis ts some δ > 0, suc h that ([1], Theorem 4.2.2, p. 69), Pr( ˆ θ k ∈ ∆ δ i.o. ) > 0 . (14) This inequalit y together with the definition of ˆ θ k as a v ector that minimizes L k implies Pr( inf θ k ∈ ∆ δ  L k ( θ k ) − L k ( θ 0 k )  ≤ 0 i.o. ) > 0 . (15) 5 Using Lemma 2 w e o btain Pr( lim inf Ψ( N ,M ) →∞ inf θ k ∈ ∆ δ  L k ( θ k ) − L k ( θ 0 k )  ≤ 0) ≥ Pr( inf θ k ∈ ∆ δ  L k ( θ k ) − L k ( θ 0 k )  ≤ 0 i.o. ) > 0 , (16) whic h contradicts (11). Hence, ˆ θ k → θ 0 k a.s. as Ψ ( N , M ) → ∞ . (17) Remark : Lemm a 1 and Theorem 1 remain v a lid ev en under less restrictiv e assumptions regarding the noise field { w ( n, m ) } . If the field { u ( n, m ) } is an i.i.d. real v alued zero-mean random field with finite v ar ia nce σ 2 , and the sequence a ( i, j ) is a square summable deterministic sequence , i.e. , P ( r ,s ) ∈ D a 2 ( r , s ) < ∞ , then Lemma 1 and Theorem 1 hold. 3.2 Consistency of the LSE for an Ov er-Estimated Mo del Order Let k denote the assumed n um b er of observ ed 2-D sin usoids, where k > P . Without loss of generalit y , w e can assume that k = P + 1, (as the pro of fo r k ≥ P + 1 follows immediately b y r ep eating the same argumen ts). Let the p erio dogram (scaled b y a fa cto r of 2) of the field { w ( n, m ) } b e giv en b y I w ( ω , υ ) = 2 N M      N − 1 X n =0 M − 1 X m =0 w ( n, m ) e − j ( nω + mυ )      2 . (18) The parameter spaces Θ P , Θ P +1 are defined as in Definition 2. Theorem 2. L et Assumptions 1-4 b e satisfie d. The n , the p ar ameter ve ctor ˆ θ P +1 = ( ˆ ρ 1 , ˆ ω 1 , ˆ υ 1 , ˆ ϕ 1 , . . . , ˆ ρ P , ˆ ω P , ˆ υ P , ˆ ϕ P , ˆ ρ P +1 , ˆ ω P +1 , ˆ υ P +1 , ˆ ϕ P +1 ) ∈ Θ P +1 that minimizes (7) with k = P +1 r e gr essors as Ψ( N , M ) → ∞ is c omp ose d of the ve ctor ˆ θ P = ( ˆ ρ 1 , ˆ ω 1 , ˆ υ 1 , ˆ ϕ 1 , . . . , ˆ ρ P , ˆ ω P , ˆ υ P , ˆ ϕ P ) which is a str ongly c onsistent estimator of θ 0 P = ( ρ 0 1 , ω 0 1 , υ 0 1 , ϕ 0 1 , . . . , ρ 0 P , ω 0 P , υ 0 P , ϕ 0 P ) as Ψ( N , M ) → ∞ ; of the p air of sp atial fr e quencies ( ˆ ω P +1 , ˆ υ P +1 ) that maximize s the p erio do gr am of the observe d r e alization o f the field { w ( n, m ) } , i.e., ( ˆ ω P +1 , ˆ υ P +1 ) = arg max ( ω, υ ) ∈ (0 , 2 π ) 2 I w ( ω , υ ) , (19) and of the ele ment ˆ ρ P +1 that satisfi e s ˆ ρ 2 P +1 = 2 N M I w ( ˆ ω P +1 , ˆ υ P +1 ) . (20) 6 Pr o of: Let θ P +1 = ( ρ 1 , ω 1 , υ 1 , ϕ 1 , . . . , ρ P , ω P , υ P , ϕ P , ρ P +1 , ω P +1 , υ P +1 , ϕ P +1 ), b e some v ector in the parameter space Θ P +1 . W e hav e, L P +1 ( θ P +1 ) = 1 N M N − 1 P n =0 M − 1 P m =0  y ( n, m ) − P +1 P i =1 ρ i cos( ω i n + υ i m + ϕ i )  2 = 1 N M N − 1 P n =0 M − 1 P m =0  y ( n, m ) − P P i =1 ρ i cos( ω i n + υ i m + ϕ i )  2 + 1 N M N − 1 P n =0 M − 1 P m =0  ρ P +1 cos( ω P +1 n + υ P +1 m + ϕ P +1 )  2 − 2 N M N − 1 P n =0 M − 1 P m =0  y ( n, m ) − P P i =1 ρ i cos( ω i n + υ i m + ϕ i )  ρ P +1 cos( ω P +1 n + υ P +1 m + ϕ P +1 )  = L P ( θ P ) + ρ 2 P +1 2 + 1 2 N M N − 1 P n =0 M − 1 P m =0 ρ 2 P +1 cos(2 ω P +1 n + 2 υ P +1 m + 2 ϕ P +1 ) − 2 N M N − 1 P n =0 M − 1 P m =0 w ( n, m ) ρ P +1 cos( ω P +1 n + υ P +1 m + ϕ P +1 ) − 2 N M N − 1 P n =0 M − 1 P m =0  P P i =1 ρ 0 i cos( ω 0 i n + υ 0 i m + ϕ 0 i ) − P P i =1 ρ i cos( ω i n + υ i m + ϕ i )   ρ P +1 cos( ω P +1 n + υ P +1 m + ϕ P +1 )  = H 1 ( θ P +1 ) + H 2 ( θ P +1 ) + H 3 ( θ P +1 ) (21) where, θ P = ( ρ 1 , ω 1 , υ 1 , ϕ 1 , . . . , ρ P , ω P , υ P , ϕ P ) ∈ Θ P and, H 1 ( θ P +1 ) = L P ( ρ 1 , ω 1 , υ 1 , ϕ 1 , . . . , ρ P , ω P , υ P , ϕ P ) = L P ( θ P ) , (22) H 2 ( θ P +1 ) = ρ 2 P +1 2 − 2 N M N − 1 X n =0 M − 1 X m =0 w ( n, m ) ρ P +1 cos( ω P +1 n + υ P +1 m + ϕ P +1 ) , (23) H 3 ( θ P +1 ) = 1 2 N M N − 1 X n =0 M − 1 X m =0 ρ 2 P +1 cos(2 ω P +1 n + 2 υ P +1 m + 2 ϕ P +1 ) − 2 N M N − 1 X n =0 M − 1 X m =0  P X i =1 ρ 0 i cos( ω 0 i n + υ 0 i m + ϕ 0 i ) − P X i =1 ρ i cos( ω i n + υ i m + ϕ i )   ρ P +1 cos( ω P +1 n + υ P +1 m + ϕ P +1 )  . (24) Let ˆ θ P = ( ˆ ρ 1 , ˆ ω 1 , ˆ υ 1 , ˆ ϕ 1 , . . . , ˆ ρ P , ˆ ω P , ˆ υ P , ˆ ϕ P ) be a v ector in Θ P that minimizes H 1 ( θ P +1 ) = L P ( θ P ). F rom [11] (or using Theorem 1 in the previous section), ˆ θ P → θ 0 P a.s. as Ψ( N , M ) → ∞ . (25) The function H 2 is a function of ρ P +1 , ω P +1 , υ P +1 , ϕ P +1 only . Ev aluating the pa rtial deriv ativ es of H 2 with resp ect to these v ariables, it is easy to verify tha t the extrem um p oin ts of H 2 are also the 7 extrem um p oints of the p erio dogra m of the realization of the noise field. Moreov er, let ρ e , ω e , υ e , ϕ e denote an extrem um p oin t o f H 2 . Then at this p oin t H 2 ( ρ e , ω e , υ e , ϕ e ) = − I w ( ω e , υ e ) N M . (26) Hence, the minimal v alue of H 2 is obtained a t the co or dina t es ρ P +1 , ω P +1 , υ P +1 , ϕ P +1 where the p erio dogr a m of { w ( n, m ) } is maximal. Let ˆ ρ P +1 , ˆ ω P +1 , ˆ υ P +1 , ˆ ϕ P +1 denote the co o r dina t es tha t minimize H 2 . Then w e hav e ( ˆ ω P +1 , ˆ υ P +1 ) = arg min ( ω, υ ) ∈ (0 , 2 π ) 2 H 2 ( ρ P +1 , ω P +1 , υ P +1 , ϕ P +1 ) = arg max ( ω, υ ) ∈ (0 , 2 π ) 2 I w ( ω , υ ) , (27) and ˆ ρ 2 P +1 = 2 N M I w ( ˆ ω P +1 , ˆ υ P +1 ) . (28) By Assumption 1, 2 and Theorem 1, [1 3], w e hav e sup ω ,υ I w ( ω , υ ) = O (log N M ) . (29) Therefore, H 2 ( ˆ ρ P +1 , ˆ ω P +1 , ˆ υ P +1 , ˆ ϕ P +1 ) = O  log N M N M  . (30) Let ˆ θ P +1 ∈ Θ P +1 b e the v ector comp osed of the elemen ts of the v ector ˆ θ P ∈ Θ P and of ˆ ρ P +1 , ˆ ω P +1 , ˆ υ P +1 , ˆ ϕ P +1 , defined ab ov e, i.e. , ˆ θ P +1 = ( ˆ ρ 1 , ˆ ω 1 , ˆ υ 1 , ˆ ϕ 1 , . . . , ˆ ρ P , ˆ ω P , ˆ υ P , ˆ ϕ P , ˆ ρ P +1 , ˆ ω P +1 , ˆ υ P +1 , ˆ ϕ P +1 ) . W e need to v erify that this v ector minimizes L P +1 ( θ P +1 ) on Θ P +1 as Ψ( N , M ) → ∞ . Recall that for ω ∈ (0 , 2 π ) and ϕ ∈ [0 , 2 π ) N − 1 X n =0 cos( ω n + ϕ ) = sin  [ N − 1 2 ] ω + ϕ  + sin  ω 2 − ϕ  2 sin  ω 2  = O (1) . (31) Hence, as N → ∞ 1 log N N − 1 X n =0 cos( ω n + ϕ ) = o (1) , (32) and consequen tly 1 N N − 1 X n =0 cos( ω n + ϕ ) = o  log N N  . (33) 8 Next, w e ev aluate H 3 . Consider the first term in (24). By (33) w e hav e 1 2 N M N − 1 X n =0 M − 1 X m =0 ρ 2 P +1 cos(2 ω P +1 n + 2 υ P +1 m + 2 ϕ P +1 ) = o  log N M N M  , (34) for any set of v alues ρ P +1 , ω P +1 , υ P +1 , ϕ P +1 ma y assume. Consider the second term in (24). By (33) and unless there exists some i , 1 ≤ i ≤ P , suc h that ( ω P +1 , υ P +1 ) = ( ω 0 i , υ 0 i ), w e hav e as Ψ( N , M ) → ∞ , 1 N M N − 1 X n =0 M − 1 X m =0 P X i =1 ρ 0 i ρ P +1 cos( ω 0 i n + υ 0 i m + ϕ 0 i ) cos( ω P +1 n + υ P +1 m + ϕ P +1 ) = o  log N M N M  , (35) for any set of v alues ρ P +1 , ω P +1 , υ P +1 , ϕ P +1 ma y assume. Assume now that there exists some i , 1 ≤ i ≤ P , suc h that ( ω P +1 , υ P +1 ) = ( ω 0 i , υ 0 i ). Since b y assumption there a re no t w o differen t regressors with iden t ical spatial frequencies, it follows that one of the estimated frequencies ( ω i , υ i ) is due to noise con tribution. Hence, b y interc hanging the roles of ( ω P +1 , υ P +1 ) and ( ω i , υ i ), a nd rep eating the ab o ve a rgumen t w e conclude that this term has the same order as in (35) . Similarly , f or the third term in (24): By (33) and unless there exists some i , 1 ≤ i ≤ P , suc h that ( ω P +1 , υ P +1 ) = ( ω i , υ i ), w e hav e as Ψ( N , M ) → ∞ , 1 N M N − 1 X n =0 M − 1 X m =0 P X i =1 ρ i ρ P +1 cos( ω i n + υ i m + ϕ i ) cos( ω P +1 n + υ P +1 m + ϕ P +1 ) = o  log N M N M  . (36 ) Ho w ev er such i for whic h ( ω P +1 , υ P +1 ) = ( ω i , υ i ) cannot exist, as this amounts to reducing the n umber of regr essors from P + 1 to P , as t w o o f them coincide. Hence, for any θ P +1 ∈ Θ P +1 as Ψ( N , M ) → ∞ H 3 ( θ P +1 ) = o  log N M N M  . (37) On the other hand, the strong consistency (25 ) of the LSE under the correct mo del order a ssump- tion implies that as Ψ( N , M ) → ∞ the minimal v a lue of L P ( θ P ) = σ 2 P ( r ,s ) ∈ D a 2 ( r , s ) a.s., while from (30) we hav e for the minimal v alue of H 2 that H 2 ( θ P +1 ) = O  log N M N M  . Hence, t he v a lue of H 3 ( θ P +1 ) at any p oin t in Θ p +1 is negligible ev en relativ e to the v alues L P ( θ P ) and H 2 ( θ P +1 ) assume at their resp ectiv e minimum p oints . Therefore, ev a luating ( 2 1) as Ψ( N , M ) → ∞ we ha v e L P +1 ( θ P +1 ) = L P ( θ P ) + H 2 ( ρ P +1 , ω P +1 , υ P +1 , ϕ P +1 ) + H 3 ( θ P +1 ) = L P ( θ P ) + H 2 ( ρ P +1 , ω P +1 , υ P +1 , ϕ P +1 ) + o  log N M N M  . (38) Since L P ( θ P ) is a f unction of the parameter v ector θ P and is indep enden t of ρ P +1 , ω P +1 , υ P +1 , ϕ P +1 , while H 2 is a function of ρ P +1 , ω P +1 , υ P +1 , ϕ P +1 and is indep enden t of θ P , the problem of min- imizing L P +1 ( θ P +1 ) becomes sep ar able as Ψ( N , M ) → ∞ . Th us minimizing (38) is equiv alen t 9 to separately minimizing L P ( θ P ) and H 2 ( ρ P +1 , ω P +1 , υ P +1 , ϕ P +1 ) a s Ψ( N , M ) → ∞ . Using the foregoing conclusions, the theorem follow s. 3.3 Discussion In the ab ov e theorems, w e hav e considered the problem of least squares estimation of the param- eters of 2- D sin usoidal signals observ ed in the presence o f an additive colored noise field, when the assumed num b er of sinus oids is incorrect. In the case where the num b er of sin usoidal signals is under-estimated w e ha ve established t he almost sure conv ergence of the least squares estimates to the parameters of the dominan t sinus oids. This result can b e in tuitiv ely explained using the basic principles of least squares estimation: Since the least squares estimate is the set of mo del parameters that minimizes the ℓ 2 norm of the error b etw een the observ ations a nd the assumed mo del, it follow s that in the case where the mo del order is under-estimated the minim um error norm is ac hiev ed when the k most dominan t sin usoids are correctly estimated. Similarly , in the case where the num b er of sinus oidal signals is ov er- estimated, the estimated parameter vec tor ob- tained by the least squares estimator con tains a 4 P - dimensional sub-ve ctor that conv erges almost surely to t he correct parameters of the sin usoids, while the remaining k − P comp onen t s assumed to exist, are assigned to the k − P most dominan t sp ectral p eaks of the noise p o w er to further minimize the norm of the estimation error. 4 Strong Consiste ncy of a F amily of Mo del Orde r S elec- tion Rule s In this section we emplo y the results deriv ed in the previous section in or der to establish t he strong consistency of a new mo del order selection rule. It is assumed that there are Q comp eting mo dels, where Q is finite, Q > P , and that eac h com- p eting mo del k ∈ Z Q = { 0 , 1 , 2 , . . . , Q − 1 } is equiprobable. F ollo wing the MDL-MAP template, define the statistic χ ξ ( k ) = N M lo g L k ( ˆ θ k ) + ξ k log N M , (39) where ξ is some finite constan t to b e sp ecified later, and L k ( ˆ θ k ) is the minimal v a lue of the error v ar ia nce of the least squares estimator. 10 The n umber of 2-D sin usoids is estimated by minimizing χ ξ ( k ) ov er k ∈ Z Q , i.e. , ˆ P = arg min k ∈ Z Q  χ ξ ( k )  . (40) Let A := P ( r ,s ) ∈ D P ( q , t ) ∈ D | a ( r , s ) a ( q , t ) | P ( r ,s ) ∈ D a 2 ( r , s ) . (41) The ob jectiv e of the next theorem is to prov e the asymptotic consistency o f the mo del order selection pro cedure in (40). Theorem 3. L et Assumptions 1-4 b e satisfie d. L et ˆ P b e given by (40) w i th ξ > 14 A . Then as Ψ( N , M ) → ∞ ˆ P → P a.s . (42) Pr o of: F o r k ≤ P , χ ξ ( k − 1) − χ ξ ( k ) = N M lo g L k − 1 ( ˆ θ k − 1 ) + ξ ( k − 1) log N M − N M log L k ( ˆ θ k ) − ξ k log N M = N M lo g  L k − 1 ( ˆ θ k − 1 ) L k ( ˆ θ k )  − ξ lo g N M . (43) F r o m Theorem 1 as Ψ( N , M ) → ∞ ˆ θ k → θ 0 k a.s. , (44) and ˆ θ k − 1 → θ 0 k − 1 a.s. (45) F r o m the definition of L k ( ˆ θ k ), and (44) L k ( ˆ θ k ) = 1 N M N − 1 P n =0 M − 1 P m =0  y ( n, m ) − k P i =1 ˆ ρ i cos( ˆ ω i n + ˆ υ i m + ˆ ϕ i )  2 = 1 N M N − 1 P n =0 M − 1 P m =0  P P i =1 ρ 0 i cos( ω 0 i n + υ 0 i m + ϕ 0 i ) + w ( n, m ) − k P i =1 ˆ ρ i cos( ˆ ω i n + ˆ υ i m + ˆ ϕ i )  2 − → Ψ( N ,M ) →∞ 1 N M N − 1 P n =0 M − 1 P m =0  P P i = k +1 ρ 0 i cos( ω 0 i n + υ 0 i m + ϕ 0 i ) + w ( n, m )  2 . (46) 11 F r o m Lemma 3 in Appendix C w e ha v e that as Ψ( N , M ) → ∞ sup ω ,υ      1 N M N − 1 X n =0 M − 1 X m =0 w ( n, m ) cos( ω n + υ m )      → 0 a.s. (47) Hence, from the Assumption 3, (31), (47) and the Strong Law of Large Numbers, w e conclude that as Ψ( N , M ) → ∞ L k ( ˆ θ k ) → σ 2 X ( r ,s ) ∈ D a 2 ( r , s ) + P X i = k +1 ( ρ 0 i ) 2 2 a.s. (48) and similarly L k − 1 ( ˆ θ k − 1 ) → σ 2 X ( r ,s ) ∈ D a 2 ( r , s ) + P X i = k ( ρ 0 i ) 2 2 a.s. (49) Since log N M N M tends to zero, as Ψ( N , M ) → ∞ , then as Ψ( N , M ) → ∞ ( N M ) − 1 ( χ ξ ( k − 1) − χ ξ ( k )) → log  1 + ( ρ 0 k ) 2 2 σ 2 P ( r ,s ) ∈ D a 2 ( r , s ) + P P i = k +1 ( ρ 0 i ) 2  a.s. (50) Since log  1 + ( ρ 0 k ) 2 2 σ 2 P ( r,s ) ∈ D a 2 ( r ,s )+ P P i = k +1 ( ρ 0 i ) 2  is strictly p o sitiv e, then χ ξ ( k − 1) > χ ξ ( k ). Hence, for k ≤ P , the function χ ξ ( k ) is monotonically decreasing with k . W e next consider the case where k = P + l fo r any integer l ≥ 1 . Based on [13], Theorem 1 and Ass umptions 1 , 2 w e ha v e that lim sup Ψ( N ,M ) →∞ sup ω ,υ I w ( ω , υ ) sup ω ,υ f w ( ω , υ ) log ( N M ) ≤ 14 a.s. (51) Based on an extension of Theorem 2 we ha v e that a.s. as Ψ( N , M ) → ∞ L P + l ( ˆ θ P + l ) = L P ( ˆ θ P ) − U l N M + o  log N M N M  , (52) where U l = l X i =1 I w ( ω i , υ i ) , (53) is the sum of the l largest elemen ts of the p erio dogram of the noise field { w ( s, t ) } . Clearly U l ≤ l sup ω ,υ I u ( ω , υ ) . (5 4) 12 Similarly to (43), a.s. as Ψ( N , M ) → ∞ , χ ξ ( P + l ) − χ ξ ( P ) = N M lo g L P + l ( ˆ θ P + l ) + ξ ( P + l ) log N M − N M lo g L P ( ˆ θ P ) − ξ P lo g N M = ξ l log N M + N M log 1 − U l N M L P ( ˆ θ P ) + o  log N M N M  ! = ξ l log N M −  U l L P ( ˆ θ P ) + o (log N M )  (1 + o (1)) = log N M  ξ l − U l L P ( ˆ θ P ) log N M + o (1)  ≥ log N M  ξ l − l sup ω ,υ I w ( ω , υ ) L P ( ˆ θ P ) log N M + o (1)  = l log N M  ξ − sup ω ,υ I w ( ω , υ ) sup ω ,υ f w ( ω , υ ) log N M sup ω ,υ f w ( ω , υ ) L P ( ˆ θ P ) + o (1)  , (55) where the second equality is obtained by substituting L P + l ( ˆ θ P + l ) using the equalit y (52). The third equalit y is due to the prop ert y that for x → 0, log (1 + x ) = x (1 + o (1)), where the observ a t io n that the term U l N M L P ( ˆ θ P ) tends to zero a.s. as Ψ( N , M ) → ∞ is due to (51). F r o m [11] (or using Theorem 1 in the previous section), ˆ θ P → θ 0 P a.s. as Ψ( N , M ) → ∞ . (56) Hence, the strong consistency (56) o f the LSE under the correct mo del order assumption implies that as Ψ( N , M ) → ∞ L P ( ˆ θ P ) → σ 2 X ( r ,s ) ∈ D a 2 ( r , s ) a.s. (57) On the other hand using the tr ia ngle inequalit y sup ω ,υ f w ( ω , υ ) ≤ σ 2 X ( r ,s ) ∈ D X ( q , t ) ∈ D | a ( r , s ) a ( q , t ) | . (58) Substituting (51),(5 7) and (58) in to (55) w e conclude that χ ξ ( P + l ) − χ ξ ( P ) > 0 (59) for an y in teger l ≥ 1. Therefore, a.s. as Ψ ( N , M ) → ∞ , the function χ ξ ( k ) has a global minim um for k = P . 13 5 Sp ecial Case In tro ducing some additional restrictions on the structure of the noise field, we can establish a tigh ter (in terms of ξ ) mo del order selection rule. W e thus mo dify our earlier Assumption 1, 2 regarding the noise field as follo ws: Assumption 1’ The no ise field { w ( n, m ) } is an infinite order quarter-plane MA field, i.e. , w ( n, m ) = ∞ X r,s =0 a ( r , s ) u ( n − r , m − s ) (60) where the field { u ( n, m ) } is an i.i.d. real v alued zero-mean random field with finite v ariance σ 2 , suc h t ha t E [ u ( n, m ) 2 log | u ( n, m ) | ] < ∞ . Assumption 2’ The sequence a ( i, j ) is a deterministic sequence whic h satisfied the condition ∞ X r,s =0 ( r + s ) | a ( r , s ) | < ∞ . (61) In this case, based on [7], Theorem 3 .2 and Assumption 1’, 2’ w e hav e that lim sup Ψ( N ,M ) →∞ sup ω ,υ I w ( ω , υ ) sup ω ,υ f w ( ω , υ ) log( N M ) ≤ 8 a.s. (62) The results of Theorem 1 and 2 are not a ffected b y this assumption. The only change is in Theorem 3. Therefore w e can formulate the next theorem: Theorem 4. L et Assumptions 1’, 2’ , 3 and 4 b e satisfie d. L et ˆ P b e given by (40) with ξ > 8 A . Then as Ψ( N , M ) → ∞ ˆ P → P a.s . (63) The pro o f of this theorem is iden tical to the pro of of Theorem 3, where instead of (51 ) w e emplo y the inequalit y in (62). 6 Conclus ions W e hav e considered the problem of j oin t ly estimating the num b er as w ell as the par a meters of t w o- dimensional sinus oidal signals, observ ed in the presence of an additiv e colored noise field. W e ha v e established the strong consistency of the LSE when the nu m b er of sin usoidal signals is 14 under-estimated, or ov er-estimated. Based on these results, we hav e prov ed the strong consistency of a new mo del order selection rule for the n um b er of sin usoidal comp onents . App endix A Lemma 1. lim inf Ψ( N ,M ) →∞ inf θ k ∈ ∆ δ  L k ( θ k ) − L k ( θ 0 k )  > 0 a.s. (64) Pr o of: In the following w e first sho w that o n ∆ δ the sequence L k ( θ k ) − L k ( θ 0 k ) ( indexed in N , M ) is uniformly low er b ounded b y a strictly p ositiv e constan t as Ψ( N , M ) → ∞ . Since the se- quence elemen ts are uniformly low er b ounded b y a strictly p ositiv e constan t the sequence of infi- m ums, inf θ k ∈ ∆ δ ( L k ( θ k ) − L k ( θ 0 k )), is uniformly low er b ounded b y the same strictly positive constant as Ψ( N , M ) → ∞ , and hence, lim inf Ψ( N ,M ) →∞ inf θ k ∈ ∆ δ ( L k ( θ k ) − L k ( θ 0 k )). Th us, we first pro v e that the sequenc e L k ( θ k ) − L k ( θ 0 k ) is uniformly low er b ounded aw a y from zero on ∆ δ as Ψ( N , M ) → ∞ . L k ( θ k ) − L k ( θ 0 k ) = 1 N M N − 1 P n =0 M − 1 P m =0  y ( n, m ) − k P i =1 ρ i cos( ω i n + υ i m + ϕ i )  2 − 1 N M N − 1 P n =0 M − 1 P m =0  y ( n, m ) − k P i =1 ρ 0 i cos( ω 0 i n + υ 0 i m + ϕ 0 i )  2 = 1 N M N − 1 P n =0 M − 1 P m =0  P P i =1 ρ 0 i cos( ω 0 i n + υ 0 i m + ϕ 0 i ) + w ( n, m ) − k P i =1 ρ i cos( ω i n + υ i m + ϕ i )  2 − 1 N M N − 1 P n =0 M − 1 P m =0  P P i = k +1 ρ 0 i cos( ω 0 i n + υ 0 i m + ϕ 0 i ) + w ( n, m )  2 = 1 N M N − 1 P n =0 M − 1 P m =0  k P i =1 ρ 0 i cos( ω 0 i n + υ 0 i m + ϕ 0 i ) − k P i =1 ρ i cos( ω i n + υ i m + ϕ i )  2 + 2 N M N − 1 P n =0 M − 1 P m =0  P P i = k +1 ρ 0 i cos( ω 0 i n + υ 0 i m + ϕ 0 i )   k P i =1 ρ 0 i cos( ω 0 i n + υ 0 i m + ϕ 0 i ) − k P i =1 ρ i cos( ω i n + υ i m + ϕ i )  + 2 N M N − 1 P n =0 M − 1 P m =0 w ( n, m )  k P i =1 ρ 0 i cos( ω 0 i n + υ 0 i m + ϕ 0 i ) − k P i =1 ρ i cos( ω i n + υ i m + ϕ i )  = I 1 + I 2 + I 3 . (65) Th us, to c heck the asymptotic b eha vior of L.H.S. of (65) w e hav e to ev aluate lim Ψ( N ,M ) →∞ ( I 1 + I 2 + I 3 ) 15 for all v ectors θ k ∈ ∆ δ : lim Ψ( N ,M ) →∞ I 1 = lim Ψ( N ,M ) →∞ 1 N M N − 1 P n =0 M − 1 P m =0  k P i =1 ρ 0 i cos( ω 0 i n + υ 0 i m + ϕ 0 i )  2 − lim Ψ( N ,M ) →∞ " 2 1 N M N − 1 P n =0 M − 1 P m =0 k P i =1 k P j =1 ρ i ρ 0 j cos( ω i n + υ i m + ϕ i ) cos( ω 0 j n + υ 0 j m + ϕ 0 j ) # + lim Ψ( N ,M ) →∞ 1 N M N − 1 P n =0 M − 1 P m =0  k P i =1 ρ i cos( ω i n + υ i m + ϕ i )  2 = T 1 + T 2 + T 3 . (66) Recall that for | ρ | < ∞ and ϕ ∈ [0 , 2 π ) lim N → ∞ 1 N N − 1 X n =0 ρ cos( ω n + ϕ ) = 0 , (67) uniformly in ω on any closed in terv al in (0 , 2 π ). The same equalit y is hold for the sine function. Hence, due to Ass umption 3 and (67), w e hav e T 1 = lim Ψ( N ,M ) →∞ 1 N M N − 1 X n =0 M − 1 X m =0  k X i =1 ρ 0 i cos( ω 0 i n + υ 0 i m + ϕ 0 i )  2 = k X i =1 ( ρ 0 i ) 2 2 , (68) indep enden tly of θ k . Also, T 3 = lim Ψ( N ,M ) →∞ 1 N M N − 1 P n =0 M − 1 P m =0  k P i =1 ρ i cos( ω i n + υ i m + ϕ i )  2 = k P i =1 ( ρ i ) 2 2 + lim Ψ( N ,M ) →∞ 1 N M N − 1 P n =0 M − 1 P m =0 k P i =1 i 6 = j k P j =1 ρ i ρ j cos( ω i n + υ i m + ϕ i ) cos( ω j n + υ j m + ϕ j ) . (69) Since the pairs ( ω i , υ i ) are pairwise differen t, t hen o n any closed interv al in (0 , 2 π ) the sequence of partial sums 1 N M N − 1 P n =0 M − 1 P m =0 k P i =1 i 6 = j k P j =1 ρ i ρ j cos( ω i n + υ i m + ϕ i ) cos( ω j n + υ j m + ϕ j ) conv erges uniformly to zero as Ψ( N , M ) → ∞ . Hence, T 3 = k P i =1 ( ρ i ) 2 2 , (70) as Ψ( N , M ) → ∞ uniformly on ∆ δ . Lea ving T 2 unc hanged w e obtain lim Ψ( N ,M ) →∞ I 1 = k P i =1  ( ρ 0 i ) 2 2 + ( ρ i ) 2 2  − lim Ψ( N ,M ) →∞ 2 N M N − 1 P n =0 M − 1 P m =0 k P i =1 k P j =1 ρ i ρ 0 j cos( ω i n + υ i m + ϕ i ) cos( ω 0 j n + υ 0 j m + ϕ 0 j ) , (71) 16 uniformly on ∆ δ . Using the similar considerations to those emplo y ed in the ev aluation o f (68) w e obtain lim Ψ( N ,M ) →∞ I 2 = lim Ψ( N ,M ) →∞  2 N M N − 1 P n =0 M − 1 P m =0  P P i = k +1 ρ 0 i cos( ω 0 i n + υ 0 i m + ϕ 0 i )   k P i =1 ρ 0 i cos( ω 0 i n + υ 0 i m + ϕ 0 i ) − k P i =1 ρ i cos( ω i n + υ i m + ϕ i )  = − lim Ψ( N ,M ) →∞ " 2 N M N − 1 P n =0 M − 1 P m =0 k P i =1 P P j = k + 1 ρ i ρ 0 j cos( ω i n + υ i m + ϕ i ) cos( ω 0 j n + υ 0 j m + ϕ 0 j ) # . (72) By Lemma 3 in App endix C, w e hav e that a.s. as Ψ( N , M ) → ∞ : sup θ k ∈ ∆ δ      2 N M N − 1 X n =0 M − 1 X m =0 w ( n, m )  k X i =1 ρ 0 i cos( ω 0 i n + υ 0 i m + ϕ 0 i ) − k X i =1 ρ i cos( ω i n + υ i m + ϕ i )       → 0 . (73) Hence I 3 → 0 a.s. as Ψ( N , M ) → ∞ unifo rmly on ∆ δ . Using (71), (72) and (73) w e conclude that a.s. lim Ψ( N ,M ) →∞ ( L k ( θ k ) − L k ( θ 0 k )) = k P i =1  ( ρ 0 i ) 2 2 + ( ρ i ) 2 2  − lim Ψ( N ,M ) →∞ 2 N M N − 1 P n =0 M − 1 P m =0 k P i =1 P P j =1 ρ i ρ 0 j cos( ω i n + υ i m + ϕ i ) cos( ω 0 j n + υ 0 j m + ϕ 0 j ) . (74) T o complete t he ev aluation of (74) w e consider the v ectors θ k ∈ ∆ δ . Let us first assume that ∆ δ ≡ R q δ for some q , 1 ≤ q ≤ k . Th us, the coo r dinate ρ q of eac h v ector in this subs et is differen t from the corresp onding co ordinate ρ 0 q b y a t least δ > 0. Conside r first the case where all the o ther elemen ts of the vec tor θ k ∈ R q δ are identical to the corresp onding elemen ts of θ 0 k . Since b y this assumption ω j = ω 0 j , υ j = υ 0 j , ϕ j = ϕ 0 j for 1 ≤ j ≤ k , and ρ j = ρ 0 j for 1 ≤ j ≤ k , j 6 = q , on this set w e ha v e lim Ψ( N ,M ) →∞  L k ( θ k ) − L k ( θ 0 k )  =  ρ 0 q √ 2 − ρ q √ 2  2 − lim Ψ( N ,M ) →∞ 2 N M N − 1 X n =0 M − 1 X m =0 k X i =1 i 6 = j P X j =1 ρ i ρ 0 j cos( ω i n + υ i m + ϕ i ) cos( ω 0 j n + υ 0 j m + ϕ 0 j ) =  ρ 0 q √ 2 − ρ q √ 2  2 ≥ δ 2 2 > 0 , (75) uniformly in ρ q , where t he second equalit y is due to Assumption 3 and following the argumen ts emplo ye d to obtain (70). Assume next that θ k ∈ R q δ ( i.e. , the co ordinate ρ q is differen t from the corresponding co or- dinate ρ 0 q b y a t least δ > 0) and that in addition, there exists an elemen t ρ t of θ k , suc h that 17 1 ≤ t ≤ k , t 6 = q and | ρ t − ρ 0 t | ≥ λ, λ > 0 while all the other eleme n ts of the vec tor θ k are iden tical to the corresp onding elemen t s of θ 0 k . F ollowing a similar deriv ation to the one in (75) we conclude that lim Ψ( N ,M ) →∞  L k ( θ k ) − L k ( θ 0 k )  =  ρ 0 q √ 2 − ρ q √ 2  2 +  ρ 0 t √ 2 − ρ t √ 2  2 ≥ δ 2 2 + λ 2 2 > δ 2 2 , (76) uniformly in ρ q and ρ t . Consider the case where θ k ∈ R q δ while t here exists an elemen t ϕ l of θ k ∈ R q δ , suc h t hat | ϕ l − ϕ 0 l | ≥ η , η > 0 and all the other elemen t s of t he v ector θ k are iden tical to the corresp o nding elemen ts of θ 0 k . F ollo wing a similar deriv a tion to the one in (75) we conclude that lim Ψ( N ,M ) →∞  L k ( θ k ) − L k ( θ 0 k )  =   ρ 0 q √ 2 − ρ q √ 2  2 + ( ρ 0 l ) 2 − ( ρ 0 l ) 2 cos( ϕ l − ϕ 0 l ) , l 6 = q ( ρ 0 q ) 2 2 + ( ρ q ) 2 2 − ρ 0 q ρ q cos( ϕ q − ϕ 0 q ) , l = q > δ 2 2 , (77) uniformly in ρ q and ϕ l . Finally , consider the case where θ k ∈ R q δ while there exists an elemen t ω l of θ k ∈ R q δ , suc h that | ω l − ω 0 l | ≥ η , η > 0 and all the other elemen ts of the v ector θ k are iden tical to the corresp o nding elemen ts of θ 0 k . F ollo wing a similar deriv a tion to the one in (75) we conclude that lim inf Ψ( N ,M ) →∞  L k ( θ k ) − L k ( θ 0 k )  =   ρ 0 q √ 2 − ρ q √ 2  2 + ( ρ 0 l ) 2 , l 6 = q ( ρ 0 q ) 2 2 + ( ρ q ) 2 2 , l = q > δ 2 2 , ( 7 8) uniformly in ρ q and ω l . F r o m the ab ov e analysis it is clear t ha t lim Ψ( N ,M ) →∞ ( L k ( θ k ) − L k ( θ 0 k )) is low er b ounded by δ 2 2 uniformly in R q δ . F o llo wing similar reasoning, the next subset we consider is W q δ ∪ V q δ . W e first consider a subset of this set: Λ =  θ k ∈ W q δ ∪ V q δ : ∃ p, k + 1 ≤ p ≤ P , ( ω q , υ q ) = ( ω 0 p , υ 0 p )  ⊂ W q δ ∪ V q δ (79) This subse t includes v ectors in Θ k , suc h that their co ordinat e pairs ( ω q , υ q ) are differen t from the corresp onding pairs of θ 0 k and equal to some pair ( ω 0 p , υ 0 p ) where p ≥ k + 1. As ab ov e, the minim um is obta ined when a ll the other elemen ts of θ k are iden tical to the corresp onding elemen ts of θ 0 k . Hence, uniformly on Λ, w e hav e lim Ψ( N ,M ) →∞ ( L k ( θ k ) − L k ( θ 0 k )) ≥ ( ρ 0 q ) 2 2 + ( ρ q ) 2 2 − ρ 0 p ρ q = ( ρ 0 q ) 2 2 − ( ρ 0 p ) 2 2 +  ρ 0 p √ 2 − ρ q √ 2  2 ≥ ( ρ 0 q ) 2 2 − ( ρ 0 p ) 2 2 = ǫ Λ > 0 , (80) 18 where the last inequalit y is due to Assumption 4. On the complemen tary set: Λ c = ( W q δ ∪ V q δ ) \ Λ =  θ k ∈ W q δ ∪ V q δ : ( ω q , υ q ) 6 = ( ω 0 p , υ 0 p ) , ∀ p, k + 1 ≤ p ≤ P  (81) w e ha v e lim Ψ( N ,M ) →∞ ( L k ( θ k ) − L k ( θ 0 k )) ≥ ( ρ 0 q ) 2 2 + ( ρ q ) 2 2 ≥ ( ρ 0 q ) 2 2 = ǫ Λ c > 0 . (82) Finally , o n the set Φ q δ the co ordinate ϕ q of the each ve ctor in this subset is differen t fro m the corresp onding co ordinate ϕ 0 q b y at least δ > 0. As in previous cases , the minim um is o btained when all the other elemen t s of θ k ∈ Φ q δ are iden tical to the corresp onding elemen ts of θ 0 k . Hence, uniformly on Φ q δ , w e hav e lim Ψ( N ,M ) →∞ ( L k ( θ k ) − L k ( θ 0 k )) ≥ ( ρ 0 q ) 2 − ( ρ 0 q ) 2 cos( ϕ q − ϕ 0 q ) ≥ ( ρ 0 q ) 2 (1 − cos δ ) = ǫ Φ qδ > 0 . (83) Let ǫ q = min( δ 2 2 , ǫ Λ , ǫ Λ c , ǫ Φ qδ ). Collecting (75) ,( 8 0), (82) and (83) together w e conclude t hat the sequence L k ( θ k ) − L k ( θ 0 k ) is low er b ounded b y ǫ q > 0 uniformly on R q δ ∪ Φ q δ ∪ W q δ ∪ V q δ as Ψ( N , M ) → ∞ . By rep eating the same arguments for ev ery q , 1 ≤ q ≤ k , and b y letting ǫ = min( ǫ 1 , . . . , ǫ k ), we conclude that the sequence L k ( θ k ) − L k ( θ 0 k ) (indexed in N , M ) is lo w er b ounded b y ǫ > 0 uniformly on ∆ δ as Ψ( N , M ) → ∞ . Hence, it follows that sequence inf θ k ∈ ∆ δ ( L k ( θ k ) − L k ( θ 0 k )) ( indexed in N , M ) is also asymptotically lo w er b ounded b y ǫ > 0, i.e. , inf θ k ∈ ∆ δ  L k ( θ k ) − L k ( θ 0 k )  ≥ ǫ, (84) as Ψ( N , M ) → ∞ . Hence, b y the definition of lim inf lim inf Ψ( N ,M ) →∞ inf θ k ∈ ∆ δ  L k ( θ k ) − L k ( θ 0 k )  ≥ ǫ > 0 . (85) App endix B Lemma 2. L et { x n , n ≥ 1 } b e a se quenc e of r ando m variabl e s. Then Pr { x n ≤ 0 i.o. } ≤ Pr { lim inf n →∞ x n ≤ 0 } (86) 19 Pr o of: Let (Ω , Σ , p ) b e some probabilit y space. Let { x n ( ω ) , n ≥ 1 } b e a seque nce of random v ar ia bles. Let { A n ∈ Σ , n ≥ 1 } b e a sequence of subsets of Ω, suc h that A n = { ω ∈ Ω : x n ( ω ) ≤ 0 } . Define A m n = ∞ [ n = m { ω : x n ≤ 0 } . (87) Then A m n ⊆ { ω : inf n ≥ m x n ≤ 0 } . (88) Hence ∞ \ m 1 A m n ⊆ ∞ \ m 1 { ω : inf n ≥ m x n ≤ 0 } . (89) Consider the R.H.S. of (89), and let y m ( ω ) = inf n ≥ m x n . Since for all ω ∈ ∞ T m 1 { ω : inf n ≥ m x n ≤ 0 } , y m ( ω ) ≤ 0 for all m , then by definition sup m y m ( ω ) ≤ 0 as w ell. On the other hand if sup m y m ( ω ) ≤ 0, then for all m , y m ( ω ) ≤ 0. Hence we hav e the follow ing set equalit y ∞ \ m 1 { ω : inf n ≥ m x n ≤ 0 } = { ω : sup m inf n ≥ m x n ≤ 0 } . (90) Rewriting (89) w e ha v e ∞ \ m =1 ∞ [ n = m A n ⊆ { ω : sup m inf n ≥ m x n ≤ 0 } = { ω : lim inf n →∞ x n ( ω ) ≤ 0 } , (9 1) where the equality on the R.H.S. of (91 ) follo ws from the definition of lim inf n →∞ ( · ) of a sequence x n . Also b y definition, ∞ T m 1 ∞ S n = m A n = lim sup n →∞ A n . Hence, (see, e.g. , [1], p. 67) lim sup n →∞ A n = { ω : x n ( ω ) ≤ 0 i.o. } ⊆ { ω : lim inf n →∞ x n ( ω ) ≤ 0 } . (92) Due to the monotonicit y of the probabilit y measure, the lemma follo ws. App endix C Let D b e an in fi nite order non- symmetrical half-pla ne supp ort defined as in (3 ) and let D ( k , l ) b e a finite order non-symmetrical half-plane supp ort, defined b y D ( k , l ) =  ( i, j ) ∈ Z 2 : i = 0 , 0 ≤ j ≤ l  ∪  ( i, j ) ∈ Z 2 : 0 < i ≤ k , − l ≤ j ≤ l  (93) Let the field { w ( n, m ) } b e defined as in (4), and the field { u ( n, m ) } is a n i.i.d. real v alued zero-mean ra ndo m field with finite second or der momen t, σ 2 . The sequenc e a ( i, j ) is a square summable deterministic sequence, 20 X ( r ,s ) ∈ D a 2 ( r , s ) < ∞ . (94) The next lemma is an extension of a lemma originally prop osed by Hannan, [6] for the case of 1-D signals. Similar result can b e found in [11 ], Lemma 2, but with o nly a par tial pro of. Since this lemma is crucial for our work w e will prov e it here. Lemma 3. sup ω ,υ      1 N M N − 1 X n =0 M − 1 X m =0 w ( n, m ) cos ( ω n + ν m )      → 0 a.s. as Ψ ( N , M ) → ∞ (95) Pr o of: First, it is easy to see tha t , sup ω ,υ      1 N M N − 1 X n =0 M − 1 X m =0 w ( n, m ) cos ( ω n + ν m )      ≤ sup ω ,υ      1 2 N M N − 1 X n =0 M − 1 X m =0 w ( n, m ) e j ( ωn + ν m )      + sup ω ,υ      1 2 N M N − 1 X n =0 M − 1 X m =0 w ( n, m ) e − j ( ωn + ν m )      . (96) Hence it is sufficien t to pro ve the lemma fo r exponentials, i.e. , w e wish to pro v e that sup ω ,υ      1 N M N − 1 X n =0 M − 1 X m =0 w ( n, m ) e j ( ωn + ν m )      → 0 a.s. as Ψ( N , M ) → ∞ (97) Define the set D ( k , l ) C = D \ D ( k , l ). Then, w ( n, m ) = X D ( k, l ) a ( r , s ) u ( n − r , m − s ) + X D ( k, l ) C a ( r , s ) u ( n − r , m − s ) = v ( n, m ) + z ( n, m ) . (98) Then, sup ω ,υ      1 N M N − 1 X n =0 M − 1 X m =0 z ( n, m ) e j ( ωn + ν m )      ≤ 1 N M N − 1 X n =0 M − 1 X m =0 | z ( n, m ) | ≤ ( 1 N M N − 1 X n =0 M − 1 X m =0 z 2 ( n, m ) ) 1 2 . (99) By the SLLN, the R.H.S. of the la st inequalit y con v ergence, almost surely , to E [ z (0 , 0) 2 ] 1 2 =   σ 2 X D ( k, l ) C a ( r , s ) 2   1 2 , (100) 21 whic h due to ( 9 4) ma y b e made arbitra r y small b y taking k and l sufficien tly large. Hence it is sufficien t to pro ve the lemma with w ( n, m ) replaced b y v ( n, m ). sup ω ,υ      1 N M N − 1 X n =0 M − 1 X m =0 v ( n, m ) e j ( ωn + ν m )      ≤ X D ( k, l ) | a ( r , s ) | sup ω ,υ      1 N M N − 1 X n =0 M − 1 X m =0 u ( n − r , m − s ) e j ( ωn + ν m )      . (101) Since t he summation is finite and { u ( n, m ) } is i.i.d., it is sufficien t to pro v e the lemma with w ( n, m ) replaced b y u ( n, m ). Th us, w e consider the mean square of the discu ssed suprem um E   sup ω ,υ      1 N M N − 1 X n =0 M − 1 X m =0 u ( n, m ) e j ( ωn + ν m )      2   = E " sup ω ,υ 1 ( N M ) 2 N − 1 X n =0 M − 1 X m =0 N − 1 X k =0 M − 1 X l =0 u ( n, m ) u ( k , l ) e j ( ω ( n − k )+ ν ( m − l )) # . (102) By letting, n − k = p, m − l = r , (103) substitute, N − 1 P n =0 N − 1 P k =0 = P | p | R δ , and M > S δ , for δ > 2. Hence, for any suc h c hoice of N and M , f r om (109), E   sup ω ,υ      1 N M N − 1 X n =0 M − 1 X m =0 u ( n, m ) e j ( ωn + ν m )      2   ≤ K ( RS ) δ 2 . (110) Hence, if w e t a k e N = N ( R ) and M = M ( S ) to b e the smallest integers not smaller then R δ and S δ , resp ectiv ely , then (110) still holds. Hence, b y Cheb yshev inequalit y for ev ery ǫ > 0 P   sup ω ,υ       1 N ( R ) M ( S ) N ( R ) − 1 X n =0 M ( S ) − 1 X m =0 u ( n, m ) e j ( ωn + ν m )       ≥ ǫ   ≤ E   sup ω ,υ      1 N ( R ) M ( S ) N ( R ) − 1 P n =0 M ( S ) − 1 P m =0 u ( n, m ) e j ( ωn + ν m )      2   ǫ 2 ≤ K ǫ 2 ( RS ) δ 2 (111) and then since δ > 2 ∞ X R =1 ∞ X S =1 P   sup ω ,υ       1 N ( R ) M ( S ) N ( R ) − 1 X n =0 M ( S ) − 1 X m =0 u ( n, m ) e j ( ωn + ν m )       > ǫ   ≤ ∞ X R =1 ∞ X S =1 K ǫ 2 ( RS ) δ 2 < ∞ . (112) Hence, b y the Borel-Cantelly lemma, sup ω ,υ       1 N ( R ) M ( S ) N ( R ) − 1 X n =0 M ( S ) − 1 X m =0 u ( n, m ) e j ( ωn + ν m )       → 0 a.s. as Ψ( R, S ) → ∞ . (113) 24 No w, sup N ( R ) ≤ N ≤ N ( R +1) M ( S ) ≤ M ≤ M ( S +1) sup ω ,υ       1 N M N − 1 X n =0 M − 1 X m =0 u ( n, m ) e j ( ωn + ν m ) − 1 N M N ( R ) − 1 X n =0 M ( S ) − 1 X m =0 u ( n, m ) e j ( ωn + ν m )       ≤ sup N ( R ) ≤ N ≤ N ( R +1) M ( S ) ≤ M ≤ M ( S +1) sup ω ,υ 1 N M       N ( R ) − 1 X n =0 M − 1 X m = M ( S ) u ( n, m ) e j ( ωn + ν m )       + s up N ( R ) ≤ N ≤ N ( R +1) M ( S ) ≤ M ≤ M ( S +1) sup ω ,υ 1 N M       N − 1 X n = N ( R ) M ( S ) − 1 X m =0 u ( n, m ) e j ( ωn + ν m )       + s up N ( R ) ≤ N ≤ N ( R +1) M ( S ) ≤ M ≤ M ( S +1) sup ω ,υ 1 N M       N − 1 X n = N ( R ) M − 1 X m = M ( S ) u ( n, m ) e j ( ωn + ν m )       = I 1 + I 2 + I 3 . (114) Consider the first term in the previous equation. Using the triangular inequalit y I 1 ≤ 1 M ( S ) M ( S +1) − 1 X m = M ( S )   sup ω 1 N ( R )       N ( R ) − 1 X n =0 u ( n, m ) e j ωn         . (115) Let ˜ u ( m ) = sup ω 1 N ( R )       N ( R ) − 1 X n =0 u ( n, m ) e j ωn       . (116) Since { u ( n, m ) } is i.i.d., it is clear that { ˜ u ( m ) } is an i.i.d. seque nce of rando m v a riables. Moreo v er, from [6] (o r by rep eating the deriv ation in (98)-(1 10) for the pro cess u ( n, m ) with a fixed m ) w e ha v e E  ˜ u ( m ) 2  = E   sup ω 1 N ( R )       N ( R ) − 1 X n =0 u ( n, m ) e j ωn       2   ≤ K 1 R δ 2 . (117) T aking the mean of the square of the I 1 w e ha v e E  | I 1 | 2  ≤ 1 M ( S ) 2 M ( S +1) − 1 X m = M ( S ) M ( S +1) − 1 X m ′ = M ( S ) E [ ˜ u ( m ) ˜ u ( m ′ )] ≤ 1 M ( S ) 2 M ( S +1) − 1 X m = M ( S ) M ( S +1) − 1 X m ′ = M ( S ) E  ˜ u ( m ) 2  1 2 E  ˜ u ( m ′ ) 2  1 2 ≤ K 1 ( M ( S + 1) − 1 − M ( S )) 2 R δ 2 M ( S ) 2 ≤ K R δ 2 S 2 . (118) 25 Using once again the Cheb yshev inequalit y and the Bo rel-Can telli lemma we hav e that I 1 → 0 a.s. as Ψ( R, S ) → ∞ . Rep eating the same consideration fo r I 2 w e hav e that I 2 → 0 a.s. as Ψ( R, S ) → ∞ . Finally , for I 3 w e ha v e E [ | I 3 | 2 ] ≤ E         1 N ( R ) M ( S ) N ( R +1) − 1 X n = N ( R ) M ( R +1) − 1 X m = M ( S ) | u ( n, m ) |       2   = 1 ( N ( R ) M ( S )) 2 N ( R +1) − 1 X n = N ( R ) M ( S +1) − 1 X m = M ( S ) N ( R +1) − 1 X n ′ = N ( R ) M ( S +1) − 1 X m ′ = M ( S ) E [ | u ( n, m ) u ( n ′ , m ′ ) | ] ≤ σ 2 ( N ( R + 1) − 1 − N ( R )) 2 ( M ( S + 1) − 1 − M ( S )) 2 ( N ( R ) M ( S )) 2 ≤ K ( RS ) 2 . (119) Using again the Cheb yshev inequalit y and the Borel-Can telli lemma we ha ve that I 3 → 0 a.s. as Ψ( R, S ) → ∞ . Finally , we ha v e that sup ω ,υ       1 N M N − 1 X n =0 M − 1 X m =0 u ( n, m ) e j ( ωn + ν m ) − 1 N M N ( R ) − 1 X n =0 M ( S ) − 1 X m =0 u ( n, m ) e j ( ωn + ν m )       → 0 a.s. (120) for all N ( R ) ≤ N < N ( R + 1) and M ( S ) ≤ M < M ( S + 1), as Ψ ( R , S ) → ∞ , a nd hence as Ψ( N , M ) → ∞ , Since N ( R ) N ( R +1) → 1 and M ( S ) M ( S +1) → 1 as Ψ( R , S ) → ∞ w e can replace 1 N M in the second term b y 1 N ( R ) M ( S ) . Therefore, w e hav e sup ω ,υ       1 N M N − 1 X n =0 M − 1 X m =0 u ( n, m ) e j ( ωn + ν m ) − 1 N ( R ) M ( S ) N ( R ) − 1 X n =0 M ( S ) − 1 X m =0 u ( n, m ) e j ( ωn + ν m )       → 0 a.s. (121) F r o m (121) and (11 3) the lemma f ollo ws. References [1] K. L . Ch ung, A Course in Pr ob ability The ory , Harcout, Brace & W orld, 1968. [2] J. L. D o ob, Sto cha s tic Pr o c e s ses , John Wiley , 1953 . 26 [3] J. M. F rancos, A. Na rashimhan, a nd J. W. W o o ds, “Maxim um-Likelih o o d Parameter Esti- mation of Discrete Homogeneous Random Fields with Mixed Sp ectral D istribution,” IEEE T r ans. Signal Pr o c ess. , 44 (1996), 1242- 1255. [4] J. M. F rancos, A. Narashimhan, a nd J. W. W o o ds, “Maxim um-Likelihoo d Estimation of T extures Using a W old D ecomp osition Mo del,” IEEE T r a ns. Image Pr o c ess. , 4 (1995), 1655 - 1666. [5] J. M. F rancos, “Bounds on the Accuracy of Estimating t he Parameters of Discrete Homo- geneous Random Fields with Mixed Sp ectral Distributions,” IEEE T r ans . Info. The ory , 43 (1997), 908-922. [6] E. J. Hannan, “The Estimation o f F requency ,” J. Appl. Pr ob. ,10 (1973),51 0 -519. [7] S. He, “Uniform Conv ergency for W eigh ted Periodogra m of Sta tionary Linear Random Fields,” Chin. Ann. of Math. , 16B (1995), 331-340 . [8] S. He, “Parameter Estimation of Hidden P erio dic Mo del in Random Fields,” Scie n c e in China (Ser. A) , 42 (199 8 ), 238 -245. [9] M. Kliger and J. M. F rancos, “MAP Mo del Order Selection Rule for 2-D Sin usoids in White Noise,” IEEE T r ans. Signal Pr o c ess. , 53 (2005), 2563- 2575. [10] M. Kliger and J. M. F rancos, “Stro ng Consistency of a F amily of Mo del Order Selection Rules for Estimating the P a r a meters of 2- D Sin usoids in White Noise,” Pr o c. Int. Conf. A c oust., Sp e e ch, Signal Pr o c essing , Philadelphia, 20 05. [11] D. Kundu and S. Nandi, “D etermination o f Discrete Sp ectrum in a Random Field,” Statistic a Ne e rl a ndic a , 5 7 (2003), 258-283. [12] J. Li and P . Stoica, “Efficien t Mixed-Sp ectrum Estimation with Applications to T arget F ea- ture Extraction, IEEE T r ansactions on Signal Pr o c essing ,” 44 (1996), 281-29 5. [13] H. Zhang and V. Mandrek ar, “Estimation of Hidden F r equencies for 2D Stationary Pro cesses,” J. Time Ser. Anal. , 2 2 (200 1), 613-629. [14] J. W ard, Sp ac e - Time A daptive Pr o c essin g for Airb orne R adar , T ec hnical Rep ort 1015, Lincoln Lab orato r y , Massac h usetts Institute of T ec hno lo gy , 1994. [15] C.F.J. W u, “ Asymptotic Theory of Non-Linear Least Squares Estimation,” Ann. Stat. , 9 (1981), 510-513. 27