How Much Information can One Get from a Wireless Ad Hoc Sensor Network over a Correlated Random Field?

TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 1 Ho w Muc h Information can One G et from a Wireless A d Ho c Sensor Net w ork o v er a Correlated Random Field? Y oungc h ul Sung † , H. V incen t P o or and Hee jung Y u Abstract New large deviations results that c haracter ize the asymptotic information r ates for g eneral d -dimensional ( d -D) stationary Gaussian ﬁelds are obtained. By applying the gener a l re s ults to senso r no des on a tw o- dimensional (2 -D) lattice, the as y mptotic be havior o f ad h o c sensor netw o rks deployed o ver correla ted random ﬁelds for statistica l inference is inv estigated. Under a 2-D hidden Gauss -Markov random ﬁeld mo del with symmetric ﬁrst order co nditional autoregr ession a nd the assumption of no in-netw ork data fusion, the behavior of the total obtainable informatio n [nats ] and energ y eﬃciency [nats/J] deﬁned a s the ratio of total gathered infor mation to the required energy is o bta ined as the coverage area, no de densit y and ener gy v ary . When the sensor no de densit y is ﬁxed, the energy eﬃciency decreases to zero with rate Θ  area − 1 / 2  and the p er-no de infor mation under ﬁxed p er- no de energy also diminishes to zero with ra te O ( N − 1 / 3 t ) as the num b er N t of netw ork no des increa ses by incre a sing the coverage area. As the senso r spacing d n increases, the p er -no de infor mation conv erge s to its limit D with rate D − √ d n e − αd n for a given diﬀusion rate α . When the cov erage ar ea is ﬁxed a nd the no de densit y increases, the per -no de information is in versely pr op ortional to the no de density . As the total energy E t consumed in the netw ork increases, the total info r mation obtainable fro m the net work is given by O (log E t ) for the ﬁxed no de density and ﬁx e d cov erag e case and b y Θ  E 2 / 3 t  for the ﬁxed per- no de sensing energ y and ﬁxed density and incre asing co verage case. † Corresponding author Y oungch u l Sung and H eejung Y u are wi th the Dept. of Electrical Engineering, KAIST, Daejeon 305-701, South Korea. Email: ysung@ee.k aist.ac.kr and hjyu@stein.k aist.ac.kr. H. V. P o or is with the Dept. of Electrical Engineering, Princeton Universi ty , Princeton, NJ 0854 4. Email: p o or@princeton.edu. The work of Y. Sung was supp orted by the IT R&D program of MKE/I IT A. [20 08-F-004-01 “5G mobile comm unication systems based on b eam-division multiple access and rela ys with group coop eration”.] The work of H. V. P o or was supp orted in part by th e U. S . National Science F oun dation under Grants A NI-03-38807 and CNS-06-25637. October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 2 Index T erms — Ad ho c sensor netw o rks, lar ge deviations principle, a s ymptotic Kullba ck-Leibler in- formation ra te, asymptotic mut ual informa tion rate, statio nary Gaus s ian ﬁelds, Ga uss-Markov random ﬁelds, conditional autor egress ive model. I. Introduction Sensor n etw orks h a v e dra wn muc h atten tion in recent y ears b ecause of their p romising ap p li- cations such as scien tiﬁc r esearc h , environmen tal monitoring, and surveilla nce [1]. I n the design of sensor net wo rks, there are seve ral distinctiv e features. First, sensor net wo rks are d esigned to sense and monitor v arious physical phenomena su c h as temp er atur e, h umidity , densit y of a certain gas or stress lev el of diﬀerent lo cations in a structure. Man y of these physical pro cesses can b e mo delled as t w o-dimensional (2-D) random ﬁelds o v er a certain area, where the uncer- tain t y of the u nderlying signal is captured as the randomness of samp les and the pr oximit y of samples close in location is mo d elled by the correlation among the samples. Second, sensors in diﬀerent lo cations should b e able to deliv er the measured d ata to a con trol cen ter (or fus ion cen ter) wh ere the decision is made, and thus th e comm unication capabilit y is required as in ad ho c comm u nication net works. Such comm unication functionalit y can b e pro vided by netw orking sensor no des, for example, us in g multi-hop routing. Third , energy is one of the critical issues in sensor net work design since b oth sensing and comm un ication require energy and it is diﬃcult to rec harge batteries in already deploy ed s ensor no des. Hence, it is of interest to design energy eﬃcien t s en sor net wo rks. 000000000000000000000000000000000000000000000000 000000000000000000000000000000000000000000000000 000000000000000000000000000000000000000000000000 000000000000000000000000000000000000000000000000 000000000000000000000000000000000000000000000000 000000000000000000000000000000000000000000000000 000000000000000000000000000000000000000000000000 000000000000000000000000000000000000000000000000 000000000000000000000000000000000000000000000000 000000000000000000000000000000000000000000000000 000000000000000000000000000000000000000000000000 111111111111111111111111111111111111111111111111 111111111111111111111111111111111111111111111111 111111111111111111111111111111111111111111111111 111111111111111111111111111111111111111111111111 111111111111111111111111111111111111111111111111 111111111111111111111111111111111111111111111111 111111111111111111111111111111111111111111111111 111111111111111111111111111111111111111111111111 111111111111111111111111111111111111111111111111 111111111111111111111111111111111111111111111111 111111111111111111111111111111111111111111111111 P S f r a g r e p l a c e m e n t s Info rmation Senso r netw o rk Physical process (Uncertaint y) Fig. 1 A d ho c sensor netw ork over physical process October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 3 In this p ap er, we consider the design of suc h sensor net wo rks, and in vestig ate th e b eha vior and eﬃciency of these net w orks from an information-theoretic p ersp ectiv e. F rom th e information- theoretic viewp oin t, the p ro cess of sensing and communicati on mentio ned ab o v e can b e view ed as extracting in f ormation (ab out the underlying 2-D physical p ro cess) using imp erfect sensor no des by exp end ing en er gy f or statisti cal inference suc h as detection or reconstruction of th e sensed signal ﬁ eld [2, 3], as sho wn in Fig. 1. Relev an t questions regarding th e n et work design are as f ollo ws. Ho w m uc h information can one obtain from the net work for giv en co v erage and no d e density? Ho w do es the amount of gathered inf orm ation c hange as we increase the co verage area or no de d ensit y? How do th e ﬁeld correlation and measurement s ignal-to-noise (SNR) aﬀect the amount of information obtainable from the net w ork? What is the optimal n o de densit y? What are th e information and energy tr ad e-oﬀs in su c h a sensor netw ork with ad ho c routing? Answering these questions is diﬃcult, esp ecially , b ecause of the 2-D spatial correlation structure of the signal pr o cess inherent to th e t w o dimensionalit y of net w ork deplo ymen t. T o circum v en t this pr oblem, several stud ies b ased on on e-dimen sional (1-D) sp atial signal mo dels ha v e b een conducted (see, e.g., [2], [4], [5]). Ho we v er, there is an imp ortant diﬀerence b et w een 1-D signal mo dels and actual spatial signals. S u pp ose that we tak e observ ations from sens ors lo cated equidistan tly along a line transect laid o ve r an area. The observ ations m a y then b e view ed as samples generated b y a 1-D pro cess along the line transect and results from time series analysis could b e applied to examine their statistica l prop erties. In the 2-D case, h o w ev er, ther e is no natural notion of signal ﬂo w or dep en d ence d irection along the transect as there is in a more traditionally obtained time series. F or s amples from sensors placed o v er a 2-D area, it is necessary to consider the signal dep endence in all direction in the p lane. A. The Appr o ach and Summary of R esu lts In this p ap er, we consider ad ho c sensor netw orks d eplo y ed for making statistical inferences ab out u nderlying 2-D random ﬁelds, and address the ab o v e questions in a general 2-D setting. In p articular, we inv estigate the amoun t of inform ation obtainable from the n et work and related trade-oﬀs among information, co v erage, densit y and energy in v arious asymptotic settings, and rev eal the fund amen tal b eha vior of large scale p lanar ad ho c sensor net w orks. W e mod el the signal ﬁeld as a 2-D Gauss-Mark o v r andom ﬁeld (GMRF), which is suitable f or many physical pro cesses, and consid er the K ullbac k-Leibler information (KL I ) and mutual information (MI) as our information measur es [6, 7]. Our appr oac h for calculating the total obtainable information October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 4 is based on the large deviations prin ciple (LDP). Under a stationarit y assump tion, the amoun t of information from a sensor no d e b ecomes indep en den t of sensor lo cation as the net w ork size gro w s, and th e total amoun t of information is appro ximately giv en by the pro duct of the n umber of sensor nod es and the asymptotic inf ormation rate or asymp totic p er-no de information. (Th us, the units of these qu an tities is nats/no de.) T o quanti fy the inf ormation con tent, we ﬁrst derive closed-form expressions f or the asymptotic p er-no de KLI and MI for s tationary Gaussian ﬁelds in a general d -dimensional ( d -D) lattice in the sp ectral d omain, and then app ly these results to the 2-D case. W e do so by exploiting the sp ectral structur e of d -D s tationary Gaussian signals and the r elationship b et ween the eigen v alues of the blo ck circulan t approximat ion to a blo c k T oep litz matrix describing the d -D correlatio n structur e. Ho wev er, the general expressions obtained in this wa y r ender the in v estigation of the ﬁeld correlatio n and SNR diﬃcult. T o address this problem, w e adop t the c onditional autor e gr ession (CAR) mo del , whic h is a generalization of the autoregressiv e (AR) mo del of classical time series analysis. W e furth er inv estigate the prop erties of the asymp totic p er-n o de KLI and MI as functions of the ﬁeld corr elation and the measuremen t SNR under th e sy m metric ﬁr st ord er cond itional au toregression (SFCAR) mo d el, whic h captures the 2-D co rrelation on the plane eﬀectiv ely . In this case, the asymptotic p er-no de KLI and MI are giv en explicitly in terms of the SNR and the ﬁeld correlation. The b eha vior of the asymptotic p er-no de K LI and MI as functions of correlation strength is seen to d ivide in to tw o regions d ep endin g on the v alue of the S NR. At high SNR, uncorr elated observ ations maximize the p er-no de information for a giv en SNR, whereas there is n on-zero optimal correlation at lo w SNR. Int erestingly , it is seen that there is a discont inuit y in the optimal correlation strength as a function of SNR. In the p erfectly correlated case, the asymptotic p er-no d e KLI and MI are zero as exp ected. As a fun ction of SNR, the asymp totic p er-no d e information in creases as log S NR for a giv en correlation s tr ength at high SNR. A t lo w SNR, the t w o information measur es show diﬀeren t rates of conv ergence to zero. Based on the d eriv ed expressions for asymptotic p er-no de information and their prop erties under the S FCAR and corr esp onding correlation function, we then inv estigate the fund amen tal b ehavio r of large scale ad ho c sensor netw orks deplo y ed o ver correlated random ﬁelds for statistical inference. Sp eciﬁcally , we examine the total information [nats] (ab out th e und erlying physic al pro cess) obtainable f rom the net w ork and the energy eﬃciency [nats/J] deﬁned as the ratio of total gathered information to the requir ed energy as the co v erage, density and energy v ary . W e October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 5 assume that sensors are lo cated on a 2-D lattice and all sensor no des in the net w ork deliv er the measured data to a fus ion cen ter in th e cen ter of the 2-D lattice via minimum hop routing witho ut in-network data fusion . Under these assumptions, we ha v e the follo wing results on the trade-oﬀs among the information, co verag e, den s it y and energy , and the resu lts p r o vide guidelines for the design of sensor n etw orks for statistic al inference ab out man y inte resting physica l pro cesses that can b e mo delled as 2-D correlated rand om ﬁ elds : (1) When the sensor n o de densit y is ﬁxed , th e amount of total inf orm ation increases linearly with resp ect to (w.r.t.) the co v erage area, and the energy eﬃciency d ecreases to zero with rate Θ  area − 1 / 2  as the co v erage area increases. F ur ther, in this case the amount of information p er sensor no de diminish es to zero as the net w ork s ize gro ws with ﬁxed energy p er no de. (2) As the sensor spacing d n increases, the p er-n o de inf ormation con v erges to its limit D with rate D − √ d n e − αd n for a giv en diﬀusion rate α . Hence, the p er-no d e information saturates almost exp onenti ally as we increase the sensor spacing. (3) When the co v erage area is ﬁ xed and the no de den s it y increases, the p er-no de information is in v ersely pr op ortional to the no de den sit y for an y nont rivial diﬀu sion rate. Hence, the total amoun t of inform ation fr om a give n area is upp er b ounded unless th e r andom ﬁeld is spatially white. (4) As the total energy E t consumed in the n et work increases, the total information obtainable from the net wo rk is giv en by Θ  E 2 / 3 t  for ﬁ xed no d e dens it y and increasing cov erage, whereas the total information increases only with rate of O (log E t ) for ﬁxed no de densit y and ﬁ xed co verage . B. R elate d Work Large deviations analysis of Gaussian pro cesses in Gaussian noise has b een considered p revi- ously , e.g., [8–13]. Ho we ve r, most w ork in this area considers only 1-D s ignals or time series. A closed-form expr ession for the asymp totic KL I rate wa s ob tained and its pr op erties were in- v estigated for 1-D hidden Gauss-Mark o v random pro cesses in [12]. Large deviations analyses w ere u sed to examine th e issues of optimal sensor den s it y and optimal sampling in a 1-D s ignal mo del in [2] and [4]. F or a 2-D setting, an error exp onen t wa s obtained f or the detection of 2-D GMRFs in [14], w here the sensors are lo cated randomly and the Mark o v graph is based on the nearest neighbor dep end ency enabling a lo op-free graph. Our work here fo cus es on th e analysis of the fundamenta l b ehavi or of 2-D sensor netw orks deplo y ed for statistical in f erence via new October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 6 large deviations results for general d -D and 2-D stationary Gaussian random ﬁelds and their application to 2-D SFCAR GMRFs, whic h enable u s to inv estigate the imp act of ﬁ eld correla- tion and measur emen t SNR on the information and the fundamental b eha vior of ad ho c sen s or net w orks for statistical inference with preliminary p resen tation of the work in [15]. C. N otation and O r ganization W e will make use of standard n otational con v en tions. V ecto rs and matrices are written in b oldface with matrices in capitals. All v ectors are column vecto rs. F or a matrix A , A T indicates the transp ose and A ( i, j ) denotes the ( i, j )-th elemen t of A . W e reserv e I m for the iden tit y matrix of size m (the subs cript is included only when necessary). F or a random v ector x , E j { x } is the exp ectation of x under probabilit y den s it y p j , j = 0 , 1. The notation x ∼ N ( µ , Σ ) means that x is Gaussian distribu ted with mean vect or µ and co v ariance matrix Σ . F or a set A , |A| denotes the cardinalit y of A . The pap er is organized as follo ws. The bac kground and signal mo d el are describ ed in Section I I. In Section I I I, the closed-form expressions for the asymptotic KLI and MI rates are obtained in th e sp ectral domain, and their prop erties are inv estigated as functions of the correlation and the SNR under the sy m metric ﬁ rst order CAR mo del. The trad e-oﬀs related to ad ho c sensor net w orks d eplo yed for statistical inf er en ce are presen ted in Section IV, follo wed by conclusions in Section V. I I. Back ground and Signal Mo del W e assume that sensors are distrib uted ov er a 2-D area and eac h sensor measur es th e underlying signal ﬁeld at its lo cation. T o sim p lify the problem and gain insights into b eha vior in 2-D, we assume that sensors are lo cated on a 2-D square lattice I n ∆ = { ( i, j ) , i = 0 , 1 , · · · , n − 1 , and j = 0 , 1 , · · · , n − 1 } , (1) where the distance b et w een tw o adjacen t no d es ( i, j ) and ( i + 1 , j ) is d n , as sh o wn in Fig. 2. (W e will use ij to denote ( i, j ) when there is no am b iguit y of notation.) W e mo d el the 2-D signal ﬁeld { X ij , ij ∈ I n } (or simply { X ij } ) sampled by sensors as a GMRF ∗ w.r.t. an undirected graph in whic h a no de corresp onds to a s en sor no de or its s ignal sample. W e assume that eac h sensor has ∗ The Mark o v depen d ence structure ma y be restrictiv e. How ever , it is a meaningful model capturing 2-D spatial correlation structure and allo wing further analysis. October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 7 Gaussian measuremen t noise. The noisy measurement Y ij of Sensor ij on the 2-D lattice I n is then giv en b y Y ij = X ij + W ij , ij ∈ I n , (2) where { W ij } represen ts indep endent and iden tically distrib uted (i.i.d.) N (0 , σ 2 ) noise with a kno wn v ariance σ 2 , and the GMRF { X ij } is assum ed to b e in dep end en t of the measurement noise { W ij } . Thus, the observ ation samples form a 2-D hidden GMRF. † In the follo w ing, we brieﬂy r eview results on GMRFs r elev an t to our fur ther dev elopmen t. Deﬁnition 1 (Und irected graph) An u ndirected lab elled graph G is a collection ( N , E ) of no des and edges, where N = { 1 , 2 , · · · , N } is the set of n o des in the graph, and E is the set of edges { ( l, m ) : l, m ∈ N and l 6 = m } . There exists an und irected edge b et w een t w o no d es l and m if and only if ( l , m ) ∈ E . W e will use the terms no de, sample and sensor interc hangeably h ereafter. Deﬁnition 2 (GMRF) A Gaussian r andom vect or x = [ X 1 , X 2 , · · · , X N ] T ∈ R N with mean v ector µ and co v ariance matrix Σ > 0 is a GMRF w.r.t. a lab elled graph G = ( N , E ) if X l and X m are indep end en t give n X − lm if and only if there exists no edge b et we en no d es l and m , where X − lm ∆ = { X k , k ∈ N and k 6 = l, m } . P S f r a g r e p l a c e m e n t s ( i, j ) X ij X ij W ij measuremen t noise Y ij Y ij Sensor ij d n d n x y z k k k k Fig. 2 Sensors on a 2-D la ttice: hidden Marko v structure Note that a GMRF is d eﬁned usin g cond itional indep endence on a graph. Ho we ve r, its distri- bution is easily c haracterized by the mean µ and th e pr ecision matrix Q ( ∆ = Σ − 1 ), and is giv en † In this pap er, w e fo cus primarily on the spatial correlation structure of 2- D sensor ﬁ elds, and the signal ev olution o ver time is not considered. October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 8 b y p ( x ) = (2 π ) − N/ 2 | Q | 1 / 2 exp  − 1 2 ( x − µ ) T Q ( x − µ )  , (3) and Q lm 6 = 0 if and only if ( l, m ) ∈ E f or all l 6 = m , i.e., Q lm = 0 ⇐ ⇒ X l ⊥ X m |X − lm . (4) Note that the co v ariance matrix Σ is completely d ense in general wh ile the precision matrix Q has n onzero element s Q lm only w h en there is an edge b et ween no d es l and m in th e Marko v random ﬁeld. Hence, when the graph is n ot f ully conn ected, the p recision matrix is s p arse [16]. The 2-D ind exing sc heme ( i, j ) in (1) and (2) can pr op erly b e con verted to a 1-D sc heme to ap p ly Deﬁnitions 1 and 2. F rom here on, we again use the 2-D indexing sc heme for conv enience. Deﬁnition 3 (S tationarit y) A GMRF { X ij } on a 2-D inﬁn ite lattice I ∞ is said to b e (second order) stationary if the mean v ector is constan t and the co v ariance b etw een samples X ij and X i ′ j ′ dep end s only on the diﬀerence of the n o de index, i.e., Co v( X ij , X i ′ j ′ ) = E { ( X ij − µ )( X i ′ j ′ − µ ) } = c ( i − i ′ , j − j ′ ) for some function c ( · , · ), where µ is the mean of the s tationary ﬁeld. Without loss of generalit y , we assume that the signal GMRF { X ij } is zero-mean. ‡ F or a 2-D zero-mean and s tationary GMRF { X ij } , th e co v ariance { γ ij } is d eﬁned as γ ij ∆ = E { X i ′ j ′ X i ′ + i,j ′ + j } = E { X 00 X ij } , (5) whic h d o es n ot dep end on i ′ or j ′ due to the stationarit y . The sp ectral den sit y fu nction of a stationary GMRF { X ij } on I ∞ with co v ariance γ ij is d eﬁned as f ( ω 1 , ω 2 ) = 1 (2 π ) 2 X ij ∈I ∞ γ ij e − ι ( iω 1 + j ω 2 ) , (6) where ι = √ − 1 and ( ω 1 , ω 2 ) ∈ [ − π , π ) 2 . Note that (6) is a 2-D extension of the con v en tional 1-D F ourier trans f orm. W e can express { γ ij } from the sp ectral density function via the inv erse transform γ ij = Z π − π Z π − π f ( ω 1 , ω 2 ) e ι ( iω 1 + j ω 2 ) dω 1 dω 2 . (7) ‡ Of course, if a stationary GMRF has a kno wn and n on-zero mean, the k now n mean can be subtracted to yield a zero-mean ﬁeld. October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 9 A stationary GMRF can b e implicitly sp eciﬁed b y a conditional autoregressiv e (CAR) mo del, whic h is a natural generalization of the autoregressiv e (AR) mo del arising in 1-D time series and whic h p ro vides an eﬃcien t to ol for capturin g the spatial correlation structure of th e sensor ﬁeld considered here. Deﬁnition 4 (Th e conditional autoregression [16]) A zero-mean C AR GMRF is deﬁ ned by a set of fu ll conditional normal d istributions with m ean and p recision: E { X ij |X − ij } = − 1 θ 00 X i ′ j ′ ∈I ∞ \{ 00 } θ i ′ j ′ X i + i ′ ,j + j ′ , (8) and E − 1 { X 2 ij |X − ij } = θ 00 > 0 , (9) where X − ij denotes the set of all v ariables except X ij . Note in (8) that the th e conditional mean of X ij giv en all other nod e v ariables dep ends on n o des ( i + i ′ , j + j ′ ) such that θ i ′ j ′ 6 = 0, and the relationship b et we en th e CAR m o del of (8) and (9) and the p recision matrix is giv en b y Q ( i,j ) , ( i + i ′ ,j + j ′ ) = θ i ′ j ′ . (10) Hence, the Mark o v dep endence structure on the graph is easily captured by the CAR mo del through (4), and { θ i ′ j ′ } directly r ep resen t th e connectivit y of the Mark o v graph. The or em 1 (Sp ectrum of a CAR mo d el [16]) The GMRF deﬁned by th e CAR mo del of (8) and (9) is a zero-mean stationary Gaussian pro cess on I ∞ with the sp ectral d ensit y fu nction f ( ω 1 , ω 2 ) = 1 (2 π ) 2 1 P ij ∈I ∞ θ ij exp( − ι ( iω 1 + j ω 2 )) , (11) if |{ θ ij 6 = 0 }| < ∞ , θ ij = θ − i, − j , θ 00 > 0 , (12) and { θ ij } is s u c h that f ( ω 1 , ω 2 ) > 0 , ∀ ( ω 1 , ω 2 ) ∈ [ − π , π ) 2 . (13) Henceforth, w e assume that the 2-D sto c hastic signal { X ij } in (2) is give n by a s tationary GMRF deﬁned by the CAR mo del of (8) and (9) satisfying (12) and (13) as n → ∞ . October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 10 The SNR of the observ ation Y ij in (2) is well deﬁned du e to the stationarit y as n → ∞ , and is giv en by SNR = E { X 2 ij } E { W 2 ij } = P σ 2 , ∀ ij, (14) where the signal p ow er is constant ov er ( i, j ) ∈ I ∞ and is giv en, using the in v erse F ourier transform of (6), by P = γ 00 = Z π − π Z π − π f ( ω 1 , ω 2 ) dω 1 dω 2 . (15) I I I. Asymp totic In forma tion Ra tes : Closed -F orm Exp ressions and I mp act of Correla tion and Signal-to-Noise Ra tio In this section, w e d eriv e closed-form expr essions for the asymptotic KLI and MI rates under the 2-D CAR GMRF mod el discussed in the previous section. W e further inv estigate the prop er- ties of the asymptotic information rates under a symmetric correlatio n assumption. F or the MI, the s ignal mod el (2) is directly app licable, whereas for the KLI the probabilit y densit y functions of the n ull (noise-only) and alternativ e (signal-plus-noise) distribu tions are giv en by p 0 ( Y ij ) : Y ij = W ij , ij ∈ I n , and (16) p 1 ( Y ij ) : Y ij = X ij + W ij , ij ∈ I n , (17) resp ectiv ely . The asymptotic KLI r ate K is deﬁned as K = lim n →∞ 1 |I n | log p 0 p 1 ( { Y ij , ij ∈ I n } ) almost surely (a.s.) u nder p 0 , (18) where p 0 and p 1 are give n b y (16) and (17), resp ectiv ely . Un der a Neyman-P earson detection form ulation, the miss probability P M deca ys exp onentia lly in many cases, includin g (16) and (17), and the error exp onent is deﬁned as the exp onent ial d eca y rate lim |I n |→∞ − 1 |I n | log P M , (19) where |I n | is the total num b er of samples in I n . It is known that the error exp onent is giv en b y the asymptotic K LI rate K deﬁned in (18) in this case [17 ]. Hence, a larger KLI rate (or p er-no de KLI) implies b etter detection p erf orm ance with a giv en net w ork size, or a smaller n et work size required f or a giv en lev el of p erform ance. While the asymptotic KLI rate determines the error exp onen t for Neyman-P earson detection, the asymptotic MI rate is in terp r eted as the amount of uncertain t y redu ction ab out the hid - den signal ﬁeld r esulting from one observ ation sample, in the large samp le size r egime. The October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 11 asymptotic MI rate I is giv en by I = lim n →∞ 1 |I n | I ( { X ij , ij ∈ I n } ; { Y ij , ij ∈ I n } ) , = lim n →∞ 1 |I n | [ H ( { X ij , ij ∈ I n } ) − H ( { X ij , ij ∈ I n }|{ Y ij , ij ∈ I n } )] . (20) It is sho wn in the sequel that the asymptotic KLI rate is smaller than the asymp totic MI r ate and that the t w o in f ormation measur es con v erge when SNR increases. T h us, at high SNR the t w o information measures are equiv alent . A. Asymptotic Information R ates in Gener al d -D imension While the 2-D results are relev an t to our analysis of fun damen tal tr ade-oﬀs in planar sensor net w orks, it is of theoretical in terest to in vestig ate the stati stical pr op erties of statio nary Gaussian random ﬁelds in general higher dimens ion. I n this section, we ﬁ rst deriv e close d-form exp ressions for th e asymp totic KLI and MI rates for stationary Gaussian r andom ﬁelds in d -D, and then apply the resu lts to the 2-D case. F or a stationary d -D Gaussian rand om ﬁ eld { Y i , i ∈ Z d } , where Z is the set of all in tegers, the auto co v ariance fu nction und er p 1 is give n by γ h = E 1 { Y i Y i + h } , h = ( h 1 , h 2 , · · · , h d ) ∈ Z d , (21) and the corresp onding F ourier transform (i.e., the p o w er sp ectral d en sit y) and its inv erse are giv en by f 1 ( ω ) = 1 (2 π ) d X h ∈ Z d γ h e − ι h · ω , ω = ( ω 1 , ω 2 , · · · , ω d ) ∈ [ − π , π ) d , (22) and γ h = Z e ι h · ω f 1 ( ω ) d ω , (23) resp ectiv ely , where the in tegration is ov er ω ∈ [ − π , π ) d , and h · ω denotes the inner pr o duct b et we en h and ω . Note that (21), (22) and (23) are the extensions of (5), (6) and (7), resp ectiv ely , to d -D. The null and alternativ e distributions arising in the KLI in d -D are giv en b y    p 0 ( Y i ) : Y i = W i , i ∈ D n , p 1 ( Y i ) : Y i = Y (1) i , i ∈ D n , (24) where { W i } are i.i.d. Gaussian from N (0 , σ 2 ), { Y (1) i } is a stationary d -D Gaussian rand om ﬁ eld with sp ectrum f 1 ( ω ) § , and D n ∆ = [0 , 1 , · · · , n − 1] d . (25) § Note that { Y (1) i } need not b e a hidden Mark o v ﬁeld. October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 12 Based on the previous wo rk [18], we further exp loit the r elationship b et w een the eigen v alues of blo c k circulant and blo c k T o eplitz m atrices representing correlation structure in d -D and the i.i.d. null distribution, an d obtain the KLI for (24) give n by the follo wing theorem. The or em 2 (Asymptotic KLI rate in d -D) Sup p ose that A.1 the alternativ e sp ectrum f 1 ( ω ) has a p ositiv e lo wer b ou n d, and A.2 ∃ M < ∞ such that ∀ k = 1 , 2 , · · · , d, P h ∈ Z d (1 + | h k | ) | γ h | < M . Then, the asymptotic KL I rate K for (24) is giv en b y K = 1 (2 π ) d Z [ − π ,π ) d  1 2 log (2 π ) d f 1 ( ω ) σ 2 − 1 2  1 − σ 2 (2 π ) d f 1 ( ω )  d ω (26) = 1 (2 π ) d Z [ − π ,π ) d D ( N (0 , σ 2 ) ||N (0 , (2 π ) d f 1 ( ω ))) d ω , (27) where D ( ·||· ) denotes the Kullbac k-Leibler distance. Pr o of: See App endix I. Theorem 2 is an extension to general d -D of th e asymptotic KLI r ate in 1-D obtained in [12], and sho ws that the fr equency binning in terpretation of (27) holds in the general d -D case un der some regularit y conditions on the alternativ e sp ectrum. Note th at the in tegrand in (27) is the Kullbac k-Leibler information b et ween t wo zero-mean Gaussian distribu tions with v ariances σ 2 and (2 π ) d f 1 ( ω ), resp ectiv ely . F or eac h d -D frequen cy segment d ω , the sp ectra can b e th ou ght of as b eing ﬂat, i.e., th e signals are ind ep endent, and Stein’s lemma [19] can b e applied for the segmen t. T he o v erall KLI is the sum of con tributions from eac h bin . The s m o othness of the sp ectrum f 1 ( ω ) is a su ﬃcien t condition for Ass u mption A .2 for second-order stationary ﬁelds, and th us the frequency b inning in T heorem 2 is v alid for a wide class of sp ectra. Th eorem 2 follo ws from the fact that K is giv en by the almost-sure limit of the normalized log-lik eliho o d ratio in (18) and that w e ha v e Gaussian distribu tions for p 0 and p 1 . That is, K is giv en by the almost sure limit K = lim n →∞ 1 |D n |  1 2 log det( Σ 1 , |D n | ) det( Σ 0 , |D n | ) + 1 2 y T |D n | ( Σ − 1 1 , |D n | − Σ − 1 0 , |D n | ) y |D n |  under p 0 , (28) where y |D n | is a ve ctor consisting of |D n | observ ation samples { Y i , i ∈ D n } with element s arranged in lexicographic order; for example, in 2-D y |I n | = [ y 1 , · · · , y |I n | ] T ∆ = [ Y 00 , Y 10 , · · · , Y n − 1 , 0 , Y 01 , · · · , Y n − 1 ,n − 1 ] T , (29) October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 13 and Σ 0 , |D n | and Σ 1 , |D n | are the co v ariance matrices of y |D n | under p 0 and p 1 , resp ectiv ely . Note that the log-lik eliho o d r atio in (28) consists of tw o terms: one is a deterministic term and the other is a quadr atic r andom term. The o verall conv ergence follo ws from the conv ergence of eac h of the tw o terms . Not e that the deterministic term in (28) is simply the mutual in formation b et we en { X i , i ∈ D n } and { Y i , i ∈ D n } for the mo d el Y i = X i + W i , i ∈ D n . (30) Using the con v ergence of the ﬁr s t term in th e r igh t-handed side (RHS) of (28), the asymp totic MI rate I for d -D is give n by I = 1 (2 π ) d Z [ − π ,π ) d 1 2 log σ 2 + (2 π ) d f ( ω ) σ 2 d ω , (31) where f ( ω ) is the sp ectrum of the signal { X i } . Th is is simp ly a d -D extension of the 1-D MI rate in sp ectral form [20], and sh o ws the v alidit y of the log (1+ SNR) formula and frequency binning approac h in general d -D un der some regularity cond itions on the sp ectrum; a su ﬃcien t condition is pro vided in Theorem 2. Applying the d -D results to the 2-D hidden GMRF mo del of (16) an d (17), w e ha v e the follo win g corollary for 2-D. Cor ol lary 1 (Asymptotic information r ates in 2-D) Assumin g that the conditions (12) and (13) hold, the asymptotic KLI and MI rates for th e hidden CAR GMRF mo del with (16) and (17) are giv en b y K = 1 4 π 2 Z π − π Z π − π  1 2 log σ 2 + 4 π 2 f ( ω 1 , ω 2 ) σ 2 − 1 2  1 − σ 2 σ 2 + 4 π 2 f ( ω 1 , ω 2 )  dω 1 dω 2 , (32) and I = 1 4 π 2 Z π − π Z π − π 1 2 log σ 2 + 4 π 2 f ( ω 1 , ω 2 ) σ 2 dω 1 dω 2 , (33) where f ( ω 1 , ω 2 ) is the 2-D sp ectrum of the signal GMRF { X ij , ij ∈ I ∞ } deﬁn ed in (11). Pr o of: See App endix I. Comparing (32) and (33), we note that the asymptotic K LI rate is str ictly less than th e asymptotic MI rate f or an y p ositiv e s ignal sp ectrum , and that the t wo information measures con v erge with a ﬁxed oﬀset of -1/2 as the SNR increases without b oun d since σ 2 σ 2 +4 π 2 f ( ω 1 ,ω 2 ) → 0 in (32) as SNR → ∞ . Hence, the t w o information m easur es can b e equiv alen tly used at h igh SNR. October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 14 B. Symmetric First-Or der Conditional Autor e g r ession In the previous section, we ha ve deriv ed closed-form expressions for the asym ptotic KLI and MI rates for hidden CAR GMRFs with general 2-D s p ectra deﬁned in (11) in the sp ectral domain. Ho wev er, these general sp ectral expressions rend er fu rther analysis infeasible. T o in v estigate the impact of the ﬁeld correlation and the SNR on the information rates, we fu rther adopt the symmetric ﬁ rst order conditional autoregression (S FCAR) mo del, describ ed by the conditions E { X ij |X − ij } = λ κ ( X i +1 ,j + X i − 1 ,j + X i,j +1 + X i,j − 1 ) , (34) and E − 1 { X 2 ij |X − ij } = κ > 0 , (35) where 0 ≤ λ ≤ κ 4 . ¶ Note that the parameters in (8) and (9) for this mo d el are giv en by θ 00 = κ , θ 1 , 0 = θ − 1 , 0 = θ 0 , 1 = θ 0 , − 1 = − λ and all other θ ij = 0. In this m o del, the correlation is P S f r a g r e p l a c e m e n t s − λ − λ − λ − λ κ ( i, j ) ( i, j + 1) ( i − 1 , j ) ( i + 1 , j ) ( i, j − 1) Fig. 3 Symmetric first order co nditional autoregression model symmetric f or eac h set of four neigh b oring no des, as seen in Fig. 3. The S FCAR mo del is a simple y et meaningful extension of the 1-D ﬁrst order autoregression (AR) mo d el whic h has the conditional causal dep endency only on the previous sample. Here in the 2-D SFCAR we hav e the conditional dep end ency on four neighborin g no des in the four (planar) directions. By Theorem 1 the sp ectrum of the S F CAR is give n by f ( ω 1 , ω 2 ) = 1 4 π 2 κ (1 − 2 ζ cos ω 1 − 2 ζ cos ω 2 ) , ( 36) ¶ This is a suﬃcien t condition to satisfy (12) and (13). October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 15 where we deﬁne the e dge dep endenc e f actor ζ as ζ ∆ = λ κ , 0 ≤ ζ ≤ 1 / 4 . (37) Note that for the range of 0 ≤ ζ ≤ 1 / 4 the 2-D sp ectrum (36) is alw a ys n on -n egativ e and th e conditions (12) and (13) are satisﬁed. Note also that ζ = 0 corresp onds to the i.i.d. case whereas ζ = 1 / 4 corresp ond s to the p erfectly correlated case, i.e., X ij = X i ′ j ′ for all i, j, i ′ , j ′ . Hence, the correlatio n strength can b e captured in this single quantit y ζ for 2-D SF CAR signals: large r ζ implies stronger correlation. T he p ow er of the SF CAR signal is obtained usin g the inv erse F ourier tr ansform via the relation (6), and is giv en b y [21] P = γ 00 = 2 K (4 ζ ) π κ ,  0 ≤ ζ ≤ 1 4  , (38) where K ( · ) is the complete elliptic in tegral of the ﬁ rst kind. The SNR is giv en b y SNR = P σ 2 = 2 K (4 ζ ) π κσ 2 . (39) Using (32), (36) and (39), w e no w obtain the asymptotic K LI and MI r ates in the SC F A R signal case, den oted b y K s and I s and giv en in the follo w ing corollary to Corollary 1. Cor ol lary 2: F or the hid den 2-D S F CAR signal mo del the asymptotic p er-no d e KLI K s is giv en by K s = 1 4 π 2 Z π − π Z π − π  1 2 log  1 + SNR (2 /π ) K (4 ζ )(1 − 2 ζ cos ω 1 − 2 ζ cos ω 2 )  − 1 2   1 − 1 1 + SNR (2 /π ) K (4 ζ )(1 − 2 ζ cos ω 1 − 2 ζ cos ω 2 )    dω 1 dω 2 , (40) and the asymp totic p er-no de MI I s is giv en by I s = 1 4 π 2 Z π − π Z π − π 1 2 log  1 + SNR (2 /π ) K (4 ζ )(1 − 2 ζ cos ω 1 − 2 ζ cos ω 2 )  dω 1 dω 2 . (41) Pr o of: The r esult follo w s u p on su bstitution of (36) and (39) in to (32) and (33), resp ectiv ely .  Note that the SNR for the hidd en SFCAR mo d el is dep end en t on correlation th r ough ζ (see (39)). How eve r, the SNR and correlation are separated in the expressions (40) and (41) for the asymptotic p er-no de information, wh ic h enables us to in v estigate the eﬀects of eac h term on the p er-sample in formation separately . October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 16 B.1 Pr op erties of the asymptotic p er-no de KLI and MI for the hidden SFCAR mo del First, it is readily s een fr om Corollary 2 that the asymptotic p er-no d e KLI K s and MI I s are con tin uously diﬀerenti able fu nctions of the edge dep endence factor ζ (0 ≤ ζ ≤ 1 / 4) for a give n SNR sin ce f : x → K ( x ) is a contin uously diﬀerenti able C ∞ function f or 0 ≤ x < 1 [22]. No w w e examine the asymptotic b eha vior of K s and I s as functions of ζ . The v alues of K s at the extreme correlations are give n b y noting that the v alues of the complete elliptic inte gral at the t w o extreme correlation p oints K (0) = π 2 and K (1) = ∞ . Therefore, in the i.i.d. case (i.e., ζ = 0), C orollary 2 r educes to Stein’s lemma [19] as exp ected, and K s is giv en b y K s (0) = 1 2 log(1 + SNR) − 1 2  1 − 1 1 + S NR  (42) = D ( N (0 , 1) ||N (0 , 1 + SNR)) . (43) F or th e p erfectly correlated case ( ζ = 1 / 4), on the other hand , K s = 0. In fact, in this case as w ell as in the i.i.d. case, the t wo -dimensionalit y is ir relev an t. T he kn o wn result in the 1-D case [12] is applicable. With r egard to I s , we ha v e similar b eha vior at the extreme correlations. In the i.i.d. case, the m utual information is giv en by the we ll kno wn formula I s (0) = 1 2 log(1 + SNR) , (44) whereas w e hav e I s = 0 in the p erfectly correlated case. Thus, b oth information measures are zero at p erfect correlati on ( ζ = 1 / 4). The limiting b eha vior of the asymptotic information rates near th e extreme correlation v alues is giv en b y T a ylor’s theorem. Due to the diﬀerentia bilit y of K s and I s w.r.t. ζ , w e h a ve K s ( ζ ) = c 1 · ( ζ − 1 / 4) + o ( | ζ − 1 / 4 | ) , (45) and I s ( ζ ) = c ′ 1 · ( ζ − 1 / 4) + o ( | ζ − 1 / 4 | ) , (46) in a neighborh o o d of ζ = 1 / 4 for some constan ts c 1 and c ′ 1 as ζ → 1 / 4. Similarly , we also ha v e the linear limiting b ehavio r for K s and I s in a neigh b orho o d of ζ = 0 with non-zero limiting v alues, D ( N (0 , 1) ||N (0 , 1 + SNR)) an d 1 2 log(1 + SNR), r esp ectiv ely , as ζ → 0. That is, K s ( ζ ) = K s (0) + c 2 ζ + o ( ζ ) , (47) October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 17 and I s ( ζ ) = I s (0) + c ′ 2 ζ + o ( ζ ) , (48) for some c 2 and c ′ 2 , as ζ → 0. 0 0.05 0.1 0.15 0.2 0.25 0.45 0.5 0.55 0.6 0.65 0.7 0.75 ζ Kullback−Leibler information SNR = 10 dB 0 0.05 0.1 0.15 0.2 0.25 0.06 0.07 0.08 0.09 0.1 0.11 ζ Kullback−Leibler information SNR = 0 dB (a) (b) 0 0.05 0.1 0.15 0.2 0.25 0.028 0.03 0.032 0.034 0.036 0.038 ζ Kullback−Leibler information SNR = −3 dB 0 0.05 0.1 0.15 0.2 0.25 0.015 0.0155 0.016 0.0165 0.017 0.0175 0.018 0.0185 0.019 ζ Kullback−Leibler information SNR = −5 dB (c) (d) Fig. 4 K s as a function of ζ : (a) SNR = 10 dB, (b) SNR = 0 dB, (c) SN R = -3 dB, (d) SNR = -5 dB F or intermediate v alues of correlation, we ev aluate (40) and (41) for sev eral diﬀeren t SNR v alues, as sho wn in Fig. 4. It is seen that, at high SNR, K s decreases monotonically as ζ increases. Hence, i.i.d. observ ations yield the largest p er-no d e information for a giv en v alue of SNR when SNR is large, as in the 1-D case [12] . As w e decrease th e SNR, it is seen that a s econd mo de grows near ζ = 1 / 4, i.e., in the strong correlation region. As w e decrease the SNR further, October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 18 the v alue of ζ of the second mo d e shifts to w ard 1 / 4, and th e v alue of th e second mo de exceeds that of the i.i.d. case. Hence, there is a discontin uity in the optimal correlation as a fun ction of S NR in the 2-D case even if the maximal K s itself is con tinuous, as seen in Fig. 5. Th at is, th ere is a ph ase transition for optimal correlation w.r.t. SNR: ab o v e a certain SNR v alue i.i.d. observ ations yield the b est p erformance, whereas b elo w that SNR p oin t su ddenly strong correlation is preferred . This is not the case for 1-D Gauss-Mark o v time series, where th e optimal correlation maximizing the information rate is con tinuous w.r.t. SNR. Although it is not sho wn here, the p er-no d e MI I s exhibits similar b ehavio r as a fu nction of the edge d ep enden ce factor ζ . −10 −8 −6 −4 −2 0 2 4 0 0.05 0.1 0.15 0.2 0.25 SNR [dB] Optimal ζ P S f r a g r e p l a c e m e n t s ( i , j ) X i j W i j Y i j S e n s o r i j Fig. 5 Optimal ζ maximizing K s vs. SNR With r egard to K s and I s as f u nctions of SNR, it is str aigh tforward to see from (40) that they are con tinuously diﬀerentia ble functions, and the b eha vior of K s and I s with resp ect to SNR is giv en by the follo wing theorem. The or em 3 (P er-no de information vs. SNR) Th e asymptotic p er-no d e KLI K s for the h id den SF CAR mo del is conti nuous and monotonically increasing as SNR increases for a giv en edge dep end ence factor ζ ∈ [0 1 / 4]. Moreo v er, K s increases w ith rate 1 2 log SNR as S NR → ∞ . As SNR d ecreases to zero, on th e other hand, K s con v erges to zero and the r ate of con v ergence is giv en by K s (SNR) = c 3 · SNR 2 + o (SNR 2 ) , (49) October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 19 as SNR → 0, wh ere c 3 is giv en b y c 3 = 1 2 6 K 2 (4 ζ ) Z π − π Z π − π 1 (1 − 2 ζ cos ω 1 − 2 ζ cos ω 2 ) 2 dω 1 dω 2 . (50) The p er-no d e MI I s has similar prop erties as a fun ction of SNR, i.e., it is a contin uous an d monotonically increasing function of SNR. A t high SNR, it increases with rate 1 2 log SNR, wh ereas it decreases to zero w ith rate of con v ergence I s (SNR) = c ′ 3 · S NR + o (SNR ) , (51) as SNR → 0, wh ere c ′ 3 is giv en b y c ′ 3 = 1 2 3 π K (4 ζ ) Z π − π Z π − π 1 1 − 2 ζ cos ω 1 − 2 ζ cos ω 2 dω 1 dω 2 . (52) Pr o of: See the App end ix I. −5 0 5 10 15 20 25 30 0 0.5 1 1.5 2 2.5 3 3.5 SNR [dB] Per−sample information Kullback−Leibler information K s Mutual information I s P S f r a g r e p l a c e m e n t s ( i , j ) X i j W i j Y i j S e n s o r i j Fig. 6 K s and I s as functions of SNR ( ζ = 0 . 1 ) Note that the limiting b eh avior as SNR → 0 is diﬀerent for K s and I s ; K s deca ys to zero quadratically while I s decreases linearly . Fig. 6 sho ws K s and I s with resp ect to SNR for ζ = 0 . 1. The log SNR b eha vior is eviden t at high SNR for b oth information measures. Note that K s and I s increase with th e same slop e in the logarithmic scale with oﬀset 1 / 2. This is easily seen from (40 ) and (41) b ecause the second term in the in tegrand of (40) con v erges to -1/2, and th us K s → I s − 1 2 as SNR increases. Ho w ev er, the oﬀset is n egligible as SNR increases. It is easy October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 20 to see from (40 ) and (41) that f or a giv en edge dep endence factor ζ the conv ergence b et w een th e t w o information measures is c haracterized b y K s I s = 1 + O  1 log SNR  as SNR → ∞ . IV. Ad Hoc Sensor Networks: Fundament al Trad e-Offs among Informa tion, Co verage, Density and Energy Using the results of the previous sections, we no w answe r the fundamental questions, raised in Section I, concerning planar ad ho c sensor n etw orks d ep lo yed o v er corr elated r andom ﬁ elds for statistica l inference under the 2-D hidden SF C AR GMRF mo d el. W e ﬁrst deriv e relev ant p h ysical correlation p arameters for the S FCAR from the corresp onding con tin uous-ind ex stochastic mo d el. Once the ph ysical correlatio n parameters for the SFCAR are obtained, the analysis of inf ormation obtainable from an ad ho c sensor n et w ork and related trade-oﬀs is straigh tforward. A. Physic al Corr elation Mo del W e ﬁrst derive h o w the p h ysical correlation is related to the edge dep enden ce factor ζ in the 2-D S FCAR mo del. Th e edge correlation co eﬃcien t ρ is deﬁned as ρ ∆ = γ 01 γ 00 = γ 10 γ 00 , (0 ≤ ρ ≤ 1) , (53) due to the s patial symmetry , wh ere γ ij = E { X 00 X ij } . ρ represent s the correlation strength b et we en the signal samp les of tw o adjacen t sensor no des connected by the Mark o v d ep endence graph deﬁned by the SF CAR mo del. The edge correlation co eﬃcien t ρ is obtained using the follo win g relationship [21]: κγ 00 = 1 + 4 ζ κγ 01 ⇒ γ 01 = κγ 00 − 1 4 κζ , (54) and by substituting (38) and (54) in to (53), w e h a v e ρ = (2 /π ) K (4 ζ ) − 1 (2 /π )(4 ζ ) K (4 ζ ) =: g − 1 ( ζ ) . (55) Note that the correlation co eﬃcien t ρ is n ot dep end en t on the p o w er f actor κ in (35), as exp ected, ev en though γ 00 and γ 01 are. Note that f unction g − 1 : ζ → ρ is a con tin uous and diﬀeren tiable C 1 function on the domain 0 ≤ ζ ≤ 1 / 4 due to the con tinuous diﬀeren tiabilit y of K ( x ) for 0 ≤ x < 1, and g − 1 (1) = lim x → 1 (2 /π ) K ( x ) − 1 (2 /π ) xK ( x ) = 1 by K (1) = ∞ . Note also that g − 1 (0) = 0 since K (0) = π / 2. Th us, the inv erse mapping g : ρ → ζ from the edge correlation factor ρ to the edge dep end ence factor ζ , which maps zero and one to zero and 1/4, resp ectiv ely , b eha v es as shown in Fig. 7 (a). October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 21 0 0.2 0.4 0.6 0.8 1 0 0.05 0.1 0.15 0.2 0.25 ρ ζ 0 0.5 1 1.5 2 2.5 3 0 0.5 1 d n ρ α = 1 (a) (b) Fig. 7 (a) edge dependence f a ctor ζ vs. edge correla tion coefficient ρ and (b) ρ vs. edge length d n No w w e consider the correlation co eﬃcient ρ as a fun ction of the sensor spacing d n . In general, the correlat ion fun ction h : d n → ρ is a p ositiv e and monotonically decreasing function of d n with h (0) = 1 and h ( ∞ ) = 0. It is well kno wn that for the 1-D ﬁrst ord er AR signal a corresp ondin g underlying (con tin uous-ind ex) ph ysical mo del is giv en by the Or nstein-Uhlen b eck pro cess ds ( x ) dx = − As ( x ) + B u ( x ) , (56) and its discrete-time equiv alen t is giv en by    s i +1 = as i + u i , a = E { s i s i − 1 } / E { s 2 i } = e − Ad n , (57) where A ≥ 0, B ∈ R , s i = s ( id n ), and th e inp ut pro cesses u ( x ) and u i are zero-mean wh ite Gaussian pro cesses. Here, d n is the spacing b et w een t wo adjacen t signal samp les. F or the 2-D SF CAR signal, ho w ev er, the same sto c hastic diﬀerentia l equation is n ot applicable. Note that the dep end ence in the signal in (56) and (57) is only on the past in 1-D space, whereas the signal (34) has sym metric dep endence in all four direction in the plane. The S F CAR signal is giv en by the solution of a second-order diﬀerence equation X ij = ζ ( X i +1 ,j + X i − 1 ,j + X i,j +1 + X i,j − 1 ) + ǫ ij , (58) October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 22 and the corresp ond in g con tinuous-index physic al mod el is giv en by the sto chastic L aplac e e quation [23]. "  ∂ ∂ x  2 +  ∂ ∂ y  2 − α 2 # X ( x, y ) = ǫ ( x, y ) , (5 9) where α ( ≥ 0) is the ph ysical diﬀusion rate, and ǫ ij and ǫ ( x, y ) are 2-D white zero-mea n Gaussian p ertur b ations. Note that the solution of (59) is circularly symmetric, i.e., it dep en ds only on r = p x 2 + y 2 , and samples of the solution X ( x, y ) of (59) on lattice I n do n ot form a discrete- index SF CAR GMRF. How ev er, (59) is still th e con tinuous-index counte rpart of (58), and we use its correlation function for the SFCAR mo del. The corr elation fu nction corresp onding to (59) is given by [23] ρ = h ( d n ) = αd n K 1 ( αd n ) , (60) where K 1 ( · ) is the mo d iﬁed Bessel fun ction of the second kind . Fig. 7 (b) sho ws the correlation function w.r.t. d n for α = 1. T he asymptotic b ehavio r of K 1 ( x ) is given b y    K 1 ( x ) → p π 2 x e − x as x → ∞ , K 1 ( x ) → 1 x as x → 0 . (61) The correlation function (60) can b e rega rded as the represen tativ e correlation in 2-D, s im ilar to the exp onen tial correlation function e − Ad n in 1-D. Both fun ctions decrease monotonically w.r.t. d n . How eve r, the 2-D correlatio n fu nction is ﬂat at d n = 0 [23], i.e.,  dρ dd n  d n =0 = 0 , (62) and it deca ys with rate √ d n e − αd n as d n → ∞ . Note that the 2-D correlation fu nction h as √ d n in fr on t of the exp on ential deca y as d n → ∞ . Ho w ev er, this p olynomial term is not signiﬁcan t and the exp onenti al deca y is dominant for large d n . Thus, we hav e ζ = g ( h ( d n )) , and for giv en physic al p arameters (with a slight abuse of n otation), K s (SNR , ζ ) = K s (SNR , g ( h ( d n ))) = K s (SNR , d n ) , and I s (SNR , ζ ) = I s (SNR , g ( h ( d n ))) = I s (SNR , d n ) . W e will use the argumen ts SNR, ζ an d d n for K s and I s prop erly as needed for exp osition. October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 23 B. Sc aling L aws in A d Ho c Sensor Networks over Corr elate d R andom Fields In this section, w e in v estigate the fun damen tal b eha vior of wireless ﬂat multi-hop ad ho c sensor net works dep loy ed for stati stical inference based on the 2-D hidden SF CAR mo del and the corresp ondin g correlation fu nctions (55) and (60). W e consider s everal criteria for determining the eﬃciency of the s en sor net w ork. Sp eciﬁcally , we consider the total amount of inform ation [nats] obtainable from the net w ork and the ener gy eﬃciency η of a sen sor net w ork, deﬁned as η = total gathered information I t total required energy E t [nats/J] , (63) where the gathered information is ab out the u nderlying physical pro cess. In the follo win g, we summarize the assumptions for the planar ad ho c sensor netw ork that w e consider. (A.1) n 2 sensors are lo cated on the grid I n with spacing d n , as sho wn in Fig. 2, and a f u sion cen ter is lo cated at th e cen ter ( ⌊ n/ 2 ⌋ , ⌊ n/ 2 ⌋ ). Th e net work size is L × L , where L = nd n . Thus, the n o de densit y µ n on I n is giv en by µ n = n 2 L 2 = n 2 ( nd n ) 2 . (64) (A.2) The observ ations { Y ij } of sensor n o des form a 2-D hidd en (discrete- index) SF CAR GMRF on the lattic e for eac h d n > 0, and the edge dep endence factor is giv en by th e correlatio n functions (55) and (60). (A.3) The fusion center gathers the measur emen ts fr om all no des u sing minimum h op routing. Note that the links in Fig. 2 are not only the Mark o v dep endence edges bu t also the routing links. The min imum hop r outing r equ ires a h op count of | i − ⌊ n/ 2 ⌋| + | j − ⌊ n/ 2 ⌋| to delive r Y ij to the fusion cen ter. (A.4) The comm unication energy p er link is give n b y E c ( d n ) = E 0 d ν n , where ν ≥ 2 is th e propagation loss factor of the wireless c h annel. (A.5) Sensing requires energy , and the sensing energy p er no de is denoted b y E s . Moreo v er, we assume that the me asur ement SNR in (14) is linearly increasing w.r.t. E s , i.e., SNR = β E s for some constant β . R emark 1: Assu mption (A.2) facilitates the analysis. Sin ce discrete samples of a con tin uous- index GMRF d o not form a discr ete-index GMRF almost surely , we assu me that for eac h d n sensor samples on I n form a discrete-index SF CAR GMRF, and matc h the correlation b et w een t w o neigh b oring n o des with the physically meaningful corr elation fun ction (60). October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 24 R emark 2: In Assumption (A.3) w e assu me that there is no d ata fu sion du ring the information gathering, i.e., n o in -net w ork data fu sion. The f u sion cente r collects the r aw measurements from all sensors. R emark 3: W e can also consider a r outing graph diﬀerent from the Mark o v dep endence graph in Fig. 2. F or example, sens ors not d irectly connected to the transm itting no d e via the Mark ov dep end ence edge can deliv er the data to the fusion cen ter. Ho wev er, this resu lts in a redu ced n umber of hops with a larger h op length, and the corresp onding r outing path consumes more energy . Thus, Assumption (A.3) of minim um hop routing via the Marko v dep end ence ed ge ensures least energy consu m ption with a minim um hop routing str ategy . R emark 4: Assu mption (A.5) do es not imply that w e can increase the p o w er of the underlying signal, b ut it means th at we can increase the SNR of eﬀectiv e s en sor samp les. S upp ose that E 1 joules are required for one sensing to obtain one sample Y ij (1) = X ij (1)+ W ij (1) at lo cation ij and the measuremen t S NR of this sample is SNR 1 . No w assume that w e hav e M identica l subsensors at location ij and obtain M samples with one sample p er eac h subsensor, requiring M · E 1 joules, and we tak e an av erage of M samples at lo cation ij , yielding Y ij = (1 / M ) P M m =1 Y ij ( m ) w here Y ij ( m ) denotes the sample at th e m th sub sensor at lo cation ij . The measurement SNR of the eﬀectiv e sample Y ij is giv en b y M · SNR 1 assuming th at the measurement noise is i.i.d. across the sub sensors. Thus, th e eﬀectiv e measuremen t SNR at eac h sensor can b e increased linearly w.r.t. the sens in g energy . Ho wev er, this linear SNR mo del is an op timistic assumption sin ce the ob s erv ation SNR ma y saturate as the sensing energy is increased without b ound in practical situation. F rom here on, we consider v arious asymptotic scenarios and in v estigate the fun d amen tal b e- ha vior of ad ho c s en sor n et works deplo y ed o v er correlated random ﬁelds for statistical inference under assumptions (A.1) - (A.5) . Our asymptotic analysis in the previous sections enables u s to calculate the total information I t for large sen s or net wo rks. The total amount of information is give n approximat ely by the pro d uct of the num b er of s en sor n o des in the net wo rk and the asymptotic p er-no d e information K s or I s , i.e., I t = n 2 K s (SNR , d n ) or I t = n 2 I s (SNR , d n ) , (65) for KLI or MI, resp ectiv ely . The total energy E t required for data gathering via th e m inim um October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 25 hop routing is giv en b y E t = n 2 E s + E c ( d n ) n − 1 X i =0 n − 1 X j =0 ( | i − ⌊ n/ 2 ⌋| + | j − ⌊ n/ 2 ⌋| ) , =    n 2 E s + 1 2 n ( n − 1)( n + 1) E c ( d n ) if n o dd , n 2 E s + 1 2 n 3 E c ( d n ) if n even . (66) First, w e consider an inﬁnite area mo del with ﬁ xed d ensit y . In this case, the num b er of sensor no des p er u nit area is ﬁxed and th e total area increases without b ound as we increase n . T he b ehavio r of the information vs. area and en er gy in this case is giv en in the f ollo wing theorem. The or em 4 (Fixed densit y and inﬁnite area) F or an ad ho c sensor net w ork w ith a ﬁxed and ﬁnite no de densit y and ﬁxed sensing energy p er no de, the total amoun t of information increases linearly w.r.t. area, but the amount of gathered inf orm ation p er un it energy deca ys to zero with rate η = Θ  area − 1 / 2  , (67) for an y non-trivial diﬀusion rate α , i.e. , 0 < α < ∞ , as we increase the area. F urther, in this case the total amount of inf ormation obtainable from the netw ork as a function of total consu med energy increases with rate of T otal in formation I t = Θ  E 2 / 3 t  , (68) for an y propagation loss factor ν > 0, as the total energy E t consumed by the netw ork increases without b ound, i.e., E t → ∞ . Pr o of: See App endix I. Theorem 4 enables u s to in v estigate the asymptotic b eha vior of ad h o c sensor net w orks with ﬁxed av ailable energy p er no de. F rom the detecti on p ersp ectiv e the error pr obabilit y is giv en b y P M ∼ e − I t ( E t ( N t ( A ))) , (69) for large net works, wher e N t ( A ) repr esen ts the tota l n um b er of sensor no des in the net wo rk with co verage area A . No w consider that eac h no de h as a ﬁxed amount of energy d enoted b y ¯ E ( < ∞ ). Then, the total energy in the net w ork is giv en by E t = N t ( A ) ¯ E . (70) October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 26 Note in this case that the tota l energy a v ailable in the net wo rk increases linearly w.r.t. th e num b er of sensor no des. The asymptotic b eha vior of ad ho c net wo rks with ﬁxed p er-no de energy is giv en b y the follo w ing corollary to Theorem 4. Cor ol lary 3: F or an ad ho c sensor n et w ork w ith a ﬁxed and ﬁnite no de density and ﬁxed p er- no de sensing ener gy , the inf orm ation amount p er sensor no d e diminishes to zero as the netw ork size gro ws, i.e., lim N t ( A ) →∞ − 1 N t ( A ) log P M ( E t ( N t ( A ))) = lim N t ( A ) →∞ O ( N t ( A ) − 1 / 3 ) = 0 , (71) if eac h sensor h as a ﬁ nite amoun t of a v ailable energy . Pr o of: Sub s titute (68), (69) and (70) into I t , P M and E t , resp ectiv ely . Corollary 3 states that a non-zero p er-no de information is not ac hiev able as th e co ve rage increases without in-net w ork data fu sion in the case that eac h no de has only a ﬁxed amount of energy , whic h is the case in most net wo rk design w ith ﬁxed amoun t of battery . In this case, the p er-no de information scales with O ( N − 1 / 3 t ) as the net wo rk size grows. Th is result is b y the comm u nication energy required f or ad ho c routing without in-net wo rk data fusion. Note f rom (66) that for the ﬁxed den sit y and increasing area mo del the sensing ener gy increases quadratically with n w hile the comm unication energy without in-net w ork data fusion in creases cubically with n since d n is ﬁxed w.r.t. n . Hence, for ad ho c sensor net wo rks with large co v erage areas the communication energy dominates the s ensing energy , and b oth the energy eﬃcie ncy for information and the p er- no de in f ormation un der ﬁxed p er-no d e energy constraint diminish to zero b ecause of the slo w er increasing rate of the tota l inform ation amoun t than that of the comm unication energy required for ad ho c rou tin g without in -n et work data fusion. This d iminishing energy eﬃciency and p er-no de information under ﬁ x ed p er-no d e energy co n- strain t can b e ﬁxed w ith in-network data fusion . S u pp ose that in -net work data fusion is p er- formed so that eac h no de needs to deliv er (aggreg ated) data only to the neigh b oring no de along the minim um h op r oute to the fu sion center in Fig. 2. In this case the num b er of transmission asso ciated with one no de is just one and the total n um b er of transmission in the net wo rk is giv en b y Θ ( n 2 ). So, the comm unication energy as w ell as the sensing ener gy increases quadratically with n . S ince the total amount of information also increases qu ad r atically with n , the tota l October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 27 amoun t of in formation as a fu nction of total energy is giv en, u nder this aggregation scenario, b y I t = Θ( E t ) , (72) as we in crease the area, and a non-zero energy eﬃciency and a non-zero p er-no d e inf ormation under ﬁ xed p er-no de energy constrain t are ac h iev ed. Thus, in-net wo rk data fusion is essent ial for energy-eﬃciency in large sensor n et works. Next, we consid er the case in whic h the no d e densit y diminishes, i.e., d n → ∞ . Es p ecially , this case is of inte rest at high SNR since at h igh SNR less correlated samples yield larger p er-no de information, as seen in Section I I I-B.1. Ho we v er, the p er-no de inform ation is u pp er b oun ded as d n → ∞ , and the asymp totic b eha vior is giv en by the follo wing theorem. The or em 5: As d n → ∞ , the p er-no de information K s and I s con v erge to D ( N (0 , 1) ||N (0 , 1 + SNR)) and 1 2 log SNR, resp ective ly , and the con ve rgence rates are give n by K s ( d n ) = D ( N (0 , 1) ||N (0 , 1 + SNR)) − c 4 p d n e − αd n + o  p d n e − αd n  (73) and I s ( d n ) = 1 2 log(1 + SNR) − c ′ 4 p d n e − αd n + o  p d n e − αd n  , (74) with p ositiv e constan ts c 4 and c ′ 4 . Pr o of: See App endix I. Theorem 5 explains ho w muc h gain in inf ormation is obtained from less correlated observ ation samples by making the sen s or spacing larger. Fig. 8 sh o ws the p er-no d e KLI K s and the com- m unication energy E c for eac h link as functions of d n for α = 1, c 4 = 1 and 10 dB SNR. The ga in in in formation is given by √ d n e − αd n for large d n , whereas the required p er-link comm unication energy incr eases without b ound , i.e., E c ( d n ) = E 0 d ν n ( ν ≥ 2). Since the exp onenti al term is dominan t in the gain as d n increases, the information gained by increasing the sensor spacing d n decreases almost exp onentia lly fast, and n o signiﬁcant gain is obtained by increasing the s ensor spacing further after some p oin t. Hence, it is not eﬀectiv e, in terms of energy eﬃciency , to increase the sensor sp acing to o muc h to obtain less correlated samples at high S NR. F rom Theorem 5 we ha v e seen that increasing the sensor spacing is not so eﬀectiv e in terms of th e information gain p er u nit of consumed energy since the p er-link communicatio n energy increases w ithout b ound . On the other h and, the p er-link comm unication energy can b e made October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 28 1 2 3 4 5 6 7 8 9 10 0.3 0.4 0.5 0.6 0.7 0.8 Per−sensor information K s 1 2 3 4 5 6 7 8 9 10 0 2 4 6 8 10 d n Per−link communication energy E c P S f r a g r e p l a c e m e n t s E ( R ) R R 0 R c I A M u t u a l i n f o r m a t i o n Fig. 8 Per-node informa tion and per-link communica tion energy w.r.t. sensor sp acing d n (SNR = 10 dB, α = 1 , c 4 = 1 ) arbitrarily s m all by decreasing the sensor s p acing. T o inv estigate the eﬀect of diminishin g com- m unication energy E c as d n → 0, we no w consid er the asymptotic case in whic h the no d e densit y go es to inﬁnity f or a ﬁ xed co v erage area. In this case, the p er-no de information deca ys to zero as d n → 0 since ζ → 1 / 4 as d n → 0, and K s ( ζ ) and I s ( ζ ) con v erge to zero as ζ → 1 / 4, as sh o w n in Section I I I-B.1. The asymptotic b eha vior in this case is giv en b y the follo wing theorem. The or em 6 (Inﬁn ite densit y mo del) F or th e inﬁn ite d en sit y mo d el with a ﬁ xed co ve rage area S w ith non trivial diﬀusion rate α , th e p er-no d e inform ation deca ys to zero with co nv ergence rate K s = c 5 µ − 1 n + o  µ − 1 n  , (75) for some constan t c 5 as the no d e densit y µ n → ∞ . Hence, the amoun t of total information from the co ve rage area con v erges to the constant c 5 S as µ n → ∞ . F urthermore, in th e case of n o sensing energy , a non-zero energy eﬃciency η is ac hiev able if the propagation loss factor ν = 3, and ev en an inﬁnite energy eﬃciency k is ac hiev able u nder Assumption (A.4) if ν > 3 as µ n → ∞ . k Of course, this is und er Assumption (A.4) for any d n > 0. In reality , Assumption (A. 4) is val id for d n ≥ d min for some d min > 0. October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 29 I s has similar b ehavio r. Pr o of: See App endix I. R emark 5: Th e ﬁ nite total in f ormation for the inﬁnite d ensit y and ﬁxed area mo d el follo w s our intuitio n. The maxim um information pro vided by the samp les fr om the cont inuous-index random ﬁeld do es not exceed the information b etw een X ( x, y ) and Y ( x, y ) except in the case of spatially w h ite ﬁelds. Here, the relev ance of (62) in 2-D is evid ent. F r om (62) we ha v e K s, 2 − D ( ζ ( ρ ( d n ))) = c 6 · d 2 n + o ( d 2 n ) , (76 ) as d n → 0 since h : d n → ζ has slop e zero at d n = 0 and K s is a contin uous and diﬀerent iable function of ζ . In the 1-D case, it is sho wn in [12] that K s, 1 − D is also a con tinuous and diﬀerenti able function of a = e − Ad n for 0 ≤ a ≤ 1 with K s, 1 − D | a =1 = 0. How ev er, the exp onenti al correlation e − Ad n has a nonzero slop e at d n = 0, and thus we h a v e K s, 1 − D ( a ( d n )) = c ′ 6 · d n + o ( d n ) , (77) as d n → 0. The num b er of no d es in the s p ace is giv en b y Θ( n 2 ) and Θ( n ) for 2-D and 1-D, resp ectiv ely , and d n = L/n in b oth cases. Hence, the total amoun t of information from the co verage sp ace (giv en by the pr o duct of the p er-no de inf ormation and the n umber of n o des in the space) con verge s to a constan t b oth in 1-D and 2-D as the n o de density increases. Thus, an y prop er 2-D correlation fun ction w.r .t. the sample distance should hav e a ﬂat top at a distance of zero. R emark 6: It is common that the propagation loss factor ν > 3 for near ﬁeld p ropagation (i.e., d n → 0). Hence, in ﬁnite energy eﬃciency is th eoretically ac hiev able un der Ass u mption (A. 4) as we increase the no de d ensit y f or a ﬁxed area assu ming that only comm unication energy is required. Note th at th e total amount of information con v erges to a constan t as we increase the no de densit y . So, the inﬁnite energy eﬃciency is ac hiev ed by dim in ishing communicatio n en er gy as d n → 0. R emark 7: Cons id ering the sensing energy , inﬁnite energy eﬃciency is not feasible ev en theo- retically since w e h a v e in this case E t = n 2 E s + Θ( n 3 − ν ) , (78) and η = c 5 S + o (1) n 2 E s + Θ( n 3 − ν ) , ν ≥ 2 , (79) October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 30 as n → ∞ for ﬁxed co verag e area. In this case the sensing energy n 2 E s is the dominant factor for lo w energy eﬃciency , and the energy eﬃciency d ecreases to zero with rate O  µ − 1 n  . Th us, it is critical for densely deploy ed sensor net wo rks to min im ize the sensing energy or p ro cessing energy for eac h sensor. In the inﬁnite densit y m o del, we hav e observed that energy is an imp ortan t factor in eﬃcie ncy . No w we inv estigate the c hange of total information w.r.t. energy . There are man y p ossible wa ys to in v est energy in the net w ork. One simple wa y is to ﬁ x the no d e d ensit y and co v erage area and to increase the sensing energy . W e assume that the netw ork size is suﬃciently large so that our asymptotic analysis is v alid. The energy-asymptotic b eh avior in this case is giv en in the follo win g theorem un der Assumptions (A.1)-(A .5) . The or em 7: As we increase the total energy E t consumed by a sensor net wo rk (including b oth sensing and comm unication) with a ﬁxed no d e d ensit y and ﬁxed area, the total information increases with rate T otal in formation I t = O (log E t ) (80) as E t → ∞ . Pr o of: See App endix I. Theorem 7 suggests a guideline for inv esting the excess energy . It is not eﬃcien t in terms of th e total amount of gathered inform ation to in v est energy to imp ro v e the qu alit y of sensed samples from a limited area. Th is only provi des an increase in total information at a logarithmic rate. Note in Theorem 4 that the in f ormation gain is giv en by I t = Θ( E 2 / 3 t ) (81) as we increase the co v erage area with ﬁxed densit y and sens in g energy even with ou t in-net w ork data f usion. Th us, the energy should b e sp ent to increase the num b er of samples by enlarging the co verage area even if it yields less acc urate samples. In this w a y , we can ac h iev e the information increase with rate at least Θ( E 2 / 3 t ) wh ic h is m uc h faster th an the logarithmic in crease obtained b y increasing the s en sing energy . C. O ptimal No de Density In the previous section, we inv estigated the asymptotic b ehavio r of the total inform ation obtainable fr om the net wo rk and the energy eﬃciency as the co v erage, density or energy c han ge. October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 31 W e now consider another imp ortant problem in sens or net wo rk d esign for statistical inference ab out und erlying random ﬁelds, namely , th e optimal dens ity problem. Here, w e are giv en a ﬁxed co verage area, an d are in terested in determining an optimal n o de densit y . The total amoun t of information gathered from the net wo rk increases monotonically (eve n if it has an u pp er b ound ) as we increase the no de densit y , as shown in Th eorem 6. Hence, the problem cannot b e prop erly form ulated without some constraint. W e consider a tota l energy constrain t in whic h a ﬁxed amoun t of energy is av ailable to the entire net w ork for b oth sensin g and comm unication. Thus, w e consider the f ollo wing p roblem. Pr oblem 1 (O p timal dens ity) Giv en a ﬁxed co v erage area with size L × L and total a v ailable energy E t , ﬁnd the densit y µ n that maximize s the tota l information I t obtainable f rom th e sensor net w ork. The ab o v e optimization problem can b e solv ed using ou r analysis based on the large deviations principle assu ming the asymptotic result is still v alid in th e lo w density case, and the optimal densit y for the KLI measure is giv en by µ ∗ n = arg max µ n L 2 µ n K s (SNR( E t , µ n ) , d n ( µ n )) , (82) s.t. n 2 E s ( µ n ) + 1 2 n ( n − 1)( n + 1) E c ( d n ( µ n )) ≤ E t , (83) where the s ensing energy E s as w ell as n an d d n are fu nctions of the n o de dens ity µ n . F rom µ n (= n 2 /L 2 ), w e ﬁr st calculate n and then d n = L/n . (Here, the quan tization of n to the n earest in teger is not p erformed.) With the determined d n , E c ( d n ) is obtained from the propagation parameters E 0 and ν , and then E s ( µ n ) is obtained from the constraint (83). When E s ( µ n ) is d etermined, the m easur emen t SNR is calculated usin g Assu mption (A . 5) , i.e., SNR = β E s , and ﬁ nally we ev aluate the p er-no de information K s (SNR , ζ ( ρ ( d n ))) and I s (SNR , ζ ( ρ ( d n ))) from Corollary 2. Fig. 9 sh o ws the tota l information obtainable from a 2 meter × 2 meter area as we c hange the no de densit y µ n with a ﬁxed tota l energy budget of E t joules. Other parameters that we use are giv en by α = 10 0 , β = 1 , E 0 = 0 . 1 and ν = 2 . Here, the v alues of E t , E 0 and β are selec ted so that the minim um and maxim um p er-no de sensing SNRs are roughly -10 to 10 d B for maxim um and minimum densities, resp ectiv ely . Th e October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 32 0 20 40 60 80 100 120 0 5 10 15 20 µ n [nodes/m 2 ] Total Kullback−Leibler information E t =50 [J] E t =100 [J] E t =150 [J] 0 20 40 60 80 100 120 0 10 20 30 40 50 60 µ n [nodes/m 2 ] Total mutual information [nats] E t =50 [J] E t =100 [J] E t =150 [J] (a) (b) Fig. 9 (a) tot al KL I vs. density and (b) tot a l MI vs. density diﬀusion rate α = 100 is chosen for the edge correlation co eﬃcien t ρ to range from almost zero to 0.6 as the no de d ensit y v aries. It is seen in the ﬁgur e that there is an optimal d en sit y f or eac h v alue of E t under either information measure. It is also seen that the total KL I is sensitiv e to the densit y c hange wher eas the total MI is less sensitiv e. T he existence of the optimal density is explained as follo ws. At lo w densities, w e ha v e only a few sensors in the area. So, the energy for comm unication is n ot large due to the small n umber of comm unicating n o des (see (108) b elo w) and most of the ener gy is allocated to sensing. Here, th e p er-no d e sensing energy is ev en higher du e to the small num b er of sensors. Ho w ev er, the p er-no d e information increases only logarithmically w.r.t. the s en sing energy or SNR by Th eorem 7, and this logarithmic gain cannot comp ensate for the loss in the num b er of sensors. Hence, lo w d en sit y yields very p o or p erformance, and large gain is ob tained initially as w e increase the densit y from v ery lo w v alues, as s een in Fig. 9. As we fur ther in crease the density , on the other h an d , the p er-no de sensin g energy or S NR decreases d ue to the increase in the o verall communicat ion and the increase in the num b er of sensor no d es, and the measurement SNR is in the lo w SNR regime ev entually , where (49) and (51) h old. F r om (66), w e h a v e E s ( µ n ) = β − 1 SNR = O ( n − 2 ) (84) October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 33 for ﬁx ed E t and E c = E 0 ( L/n ) 2 , as n → ∞ . By the quad r atic deca ying b eha vior of K s at lo w SNR giv en b y (49), the total Kullbac k-Leibler information is giv en b y T otal K LI = L 2 µ n K s = O ( n 2 n − 4 ) = O ( n − 2 ) = O ( µ − 1 n ) . By (51), on the other hand , the mutual inf orm ation deca ys linearly as SNR decreases to zero, and the total mutual information is giv en by T otal MI = L 2 µ n I s = O ( n 2 n − 2 ) = O (1) . This explains the initial fast deca y after the p eak in Fig. 9 (a) and ﬂ at curv e in Fig. 9 (b ). In the ab o v e equatio ns, ho wev er, the eﬀect of ζ on K s and I s is not considered. As the no d e densit y increases, the sen sor spacing d ecreases an d the ed ge dep endence f actor ζ increases for a giv en diﬀusion rate α . The b ehavi or of the p er-no d e information as a function of ζ is sho wn in Fig. 4. Note in Fig. 4 that the p er-no de inform ation has a s econd lob e at strong correlation at lo w SNR while at high S NR it decreases monotonically as the correlation b ecomes strong. The b eneﬁt of sample correlation is eviden t in the low energy case ( E t = 50[J]) in 9 (a); the second p eak around µ n = 95 [no des/ m 2 ] is observ ed. Note that the second p eak is not v ery signiﬁcan t. Since the p er-n o de information deca ys to zero as ζ → 1 / 4 ev en tually , the total amount of information decreases ev en tually , as seen in th e right corner of the ﬁgur e, as w e increases the n o de density further. V. Conclus ion and Discuss ion In this pap er, w e hav e consid ered the d esign of sensor netw orks f or statistical inference ab out correlated random ﬁelds in a 2-D setting. T o quantify the information from th e sensor net w ork, w e ha v e us ed a sp ectral domain app roac h to derive closed-form expressions for asymptotic KLI and MI rates in general d -D and in 2-D in particular, and ha ve adopted the 2-D hid den CAR GMRF f or our signal mo del to capture th e spatial correlatio n and measuremen t noise for samples in a 2-D sensor ﬁeld. Un d er the ﬁ rst order symmetry assum p tion, w e ha v e further obtained the asymptotic information rates explicitly in terms of the S NR and the edge dep endence facto r, and ha v e in v estigated the p rop erties of the asymptotic information rates as functions of SNR and correlation. Based on these LDP resu lts, we hav e then analyzed the asymptotic b ehavio r of ad ho c s en sor net w orks deplo y ed o v er 2-D correlated random ﬁ elds for s tatistica l inf erence. Under the S F CAR GMRF mo del, w e h a v e obtained fu n damen tal scal ing la ws for total information and October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 34 energy eﬃciency as th e co v erage, n o de densit y and consumed energy c hange. The r esults pro vide guidelines for sensor n et w ork design for statistical inference ab out 2-D correlated rand om ﬁelds suc h as temp erature, h umidity , or den s it y of a gas on a certain area. In closing, w e discus s sev eral issues r elated to some of th e assumptions we ha ve us ed to simplify our analysis. First, of course, sensors in a r eal net w ork may not b e lo cated on a 2-D grid. Ho wev er, w e conjecture that similar scaling b eha viors w.r.t. the co v erage, den s it y and energy are v alid for randomly and unif orm ly deplo yed sensors. Secondly , the s patial Mark o v assumption ma y b e restrictiv e. Ho wev er, it is a minimal mo del that captures the t wo dimen s ionalit y of the signal correlation stru cture in all planar directions and allo ws analysis to b e tractable. And, ﬁnally we ha ve not considered the temp oral evo lution of the sp atial signal ﬁeld. In case of i.i.d. temp oral v ariation, the results here can b e applied directly w ith ou t mo diﬁ cation. When the signal v ariation o v er time is correlated, th e mo diﬁcation to spatio-temp oral ﬁelds is required. Appendix I Pr o of of The or em 2 The asymptotic KLI rate K is giv en by the almost-sure limit K = lim n →∞ 1 |D n | log p 0 p 1 ( { Y i , i ∈ D n } ) , (85) ev aluated un der p 0 [24]. W e consider the follo w in g ind ex m apping from d -D to 1-D in lexico- graphic order: l = f id ( i ) , ( i ∈ [0 , 1 , · · · , n − 1] d ) , (86) and the corresp onding observ ation ve ctor y |D n | generated from { Y i , i ∈ D n } . Th en , y |D n | is a zero-mean Gaussian ve ctor with the co v ariance matrices Σ 0 , |D n | and Σ 1 , |D n | under p 0 and p 1 , resp ectiv ely . Hence, the asymptotic K L I rate is giv en b y K = lim n →∞ 1 |D n |  1 2 log det( Σ 1 , |D n | ) det( Σ 0 , |D n | ) + 1 2 y T |D n | ( Σ − 1 1 , |D n | − Σ − 1 0 , |D n | ) y |D n |  , (87) under p 0 . No w we consider the terms on the RHS of (87). First, we consider log det( Σ 0 , |D n | ). Since Σ 0 , |D n | = σ 2 I n d under the assumption of an i.i.d. n ull distribution, we simply ha v e 1 |D n | log det Σ 0 , |D n | = 1 n d log det( σ 2 I n d ) = log σ 2 . (88) Next we consider the term 1 |D n | y T |D n | Σ − 1 0 , |D n | y |D n | . S ince y |D n | is i.i.d. Gaussian, d -D is irrelev ant October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 35 in this case, the kno wn result from [25, Prop osition 10.8.3 ] is applicable, and w e hav e 1 |D n | y T |D n | Σ − 1 0 , |D n | y |D n | → 1 almost surely , (89) assuming that the random v ector y |D n | is generated from the d istribution p 0 . No w we consider the term 1 |D n | log det Σ 1 , |D n | . This is the entrop y r ate of a d -D Gaussian p ro cess, and the con v ergence b ehavio r of this term is s tudied in [18]. It is shown in [18, p. 391] un der the assum p tion in Theorem 2 that we ha v e      log det Σ 1 , |D n | − |D n | (2 π ) d Z [ − π ,π ) d log((2 π ) d f 1 ( ω )) d ω      = O  |D n | n  . Applying this result, we ha v e 1 |D n | log det Σ 1 , |D n | → 1 (2 π ) d Z [ − π ,π ) d log((2 π ) d f 1 ( ω )) d ω . (90) Finally , we consider the rand om term 1 |D n | y T |D n | Σ − 1 1 , |D n | y |D n | . ∗∗ By Lemma 2 in App endix I I, we ha v e 1 |D n | y T |D n | Σ − 1 1 , |D n | y |D n | → 1 (2 π ) d Z [ − π ,π ) d σ 2 (2 π ) d f 1 ( ω ) d ω , (91) almost surely as n → ∞ . Com bining (87) - (91), w e ha v e K = 1 (2 π ) d Z [ − π ,π ) d  1 2 log (2 π ) d f 1 ( ω ) σ 2 − 1 2  1 − σ 2 (2 π ) d f 1 ( ω )  d ω . (92) Since D ( N (0 , σ 2 0 ) ||N (0 , σ 2 1 )) = 1 2 log σ 2 1 σ 2 0 − 1 2  1 − σ 2 0 σ 2 1  , (93) (92) is given by K = 1 (2 π ) d Z [ − π ,π ) d D ( N (0 , σ 2 ) ||N (0 , (2 π ) d f 1 ( ω ))) d ω . (94)  Pr o of of Cor ol lary 1 F or the 2-D hid den mo del we ha v e f 1 ( ω 1 , ω 2 ) = (2 π ) − 2 σ 2 + f ( ω 1 , ω 2 ) , (95 ) ∗∗ The proof given in [25] and [26] for the con vergence of this term for the 1-D index case is n ot ap p licable for general d -D, n or is t h e almo st-sure conv ergence of the term sho wn in [18], where the converge nce of t he term in probabilit y to an integral in vol ving the perio dogram wa s shown. Thus, w e pro ve the almost-sure con vergence of the term in Lemma 2 separately in App endix II . October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 36 where f ( ω 1 , ω 2 ) is the CAR sp ectrum (11) in 2-D satisfying (12) and (13). First, f 1 ( ω 1 , ω 2 ) has a p ositive lo wer b oun d, and thus satisﬁes Assumption A .1 in Theorem 2. It is also kno wn in [27] that if k = ( k 1 , · · · , k d ) ∈ N d and if f 1 ( ω ) is of class C k (i.e., diﬀerent iable up to the k d -order w.r.t. ω d ), then lim su p h →∞ h k 1 1 h k 2 2 · · · h k d d | γ h | < ∞ , (96) where N is the set of all n atural num b ers, and h → ∞ means that at least one co ordinate tend s to inﬁnity . Under the condition (12) and (13), the h idden CAR sp ectrum f 1 ( ω 1 , ω 2 ) in (95) is C ( ∞ , ∞ ) , i.e., s mo oth b oth in ω 1 and ω 2 . Th is ensur es that Assumption A.2 in Theorem 2 is satisﬁed, and the corollary follo w s by subs tituting (95) and d = 2 in to (26).  Pr o of of The or em 3 The con tinuit y is straigh tforw ard. The monotonicit y is sh o wn as follo ws. Let s = 1 + SNR g ζ ( ω ) where g ζ ( ω ) = ((2 /π ) K (4 ζ )(1 − 2 ζ cos ω 1 − 2 ζ cos ω 2 )) − 1 . Then, the partial deriv ativ e of K s w.r.t. SNR is giv en b y ∂ K s ∂ SNR = 1 (2 π ) 2 Z ω ∈ [ − π ,π ) 2 ∂ ∂ s  1 2 log s + 1 2 s − 1 2  ∂ s ∂ SNR d ω , (97) where ∂ ∂ s  1 2 log s + 1 2 s − 1 2  = 1 2 s − 1 s 2 = 1 2 SNR g ζ ( ω ) s 2 ≥ 0 , (98) and ∂ s ∂ SNR = g ζ ( ω ) ≥ 0 (99) for 0 ≤ ζ ≤ 1 / 4 . Hence, ∂ K s ∂ SNR ≥ 0 , (100) and K s increases monotonically as SNR increases for a giv en ζ (0 ≤ a ≤ 1 / 4). As SNR → ∞ , w e h a v e K s ≈ 1 (2 π ) 2 Z ω ∈ [ − π ,π ) 2 1 2 log(SNR g ζ ( ω )) d ω , = 1 2 log SNR + 1 (2 π ) 2 Z ω ∈ [ − π ,π ) 2 1 2 log( g ζ ( ω )) d ω . Th us, w e ha v e 1 2 log SNR b eha vior at high SNR. F or (49) and (51), tak e th e T a ylor expansion aroun d SNR = 0 to obtain log(1 + SNR g ζ ( ω )) = SNR g ζ ( ω ) − SNR 2 g 2 ζ ( ω ) / 2 + · · · , October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 37 1 1 + SNR g ζ ( ω ) = 1 − SNR g ζ ( ω ) + SNR 2 g 2 ζ ( ω ) − · · · , and then in tegrate.  Pr o of of The or em 4 In this case, the edge length d n = d for all n , and thus the asymptotic p er-sensor information K s ( d n ) or I s ( d n ) do es not c h ange with n . Considering the Kullback- Leibler information, w e ha v e I t = n 2 K s ( d ), an d area = Θ( n 2 ). Hence, the total information is linear w.r.t. area. T he total energy E t required for d ata gathering is giv en by E t = n 2 E s + E c ( d ) n − 1 X i =0 n − 1 X j =0 ( | i − ⌊ n/ 2 ⌋| + | j − ⌊ n/ 2 ⌋| ) , = n 2 E s + Θ( n 3 ) E c ( d ) , (101) where the ﬁr st term is the sensing energy and the second term is the energy consumed for comm unication. The energy eﬃciency is giv en by η = n 2 K s ( d ) n 2 E s + Θ( n 3 ) E c ( d ) = Θ  1 n  , (102) as n → ∞ . Sin ce area = Θ( n 2 ), (67) follo ws. F or the second statemen t we hav e E t = Θ( n 3 ). Th e total information is giv en by n 2 K s (SNR , d ). Since K s is ﬁ xed, the total information is Θ( n 2 ) as n → ∞ , and w e h a v e (68).  Pr o of of The or em 5 The pro of is b y th e asymptotic b eha vior of the mo d iﬁ ed Bessel fun ction K 1 ( · ) of the second kind and T a ylor expansion of K s (as a f unction of ζ ) and ζ (as a function of ρ ), whic h is allo w ed b ecause of their con tinuous diﬀerent iabilit y . F rom (60) and (61) we ha ve ρ ( d n ) = r π 2 αd n e − αd n + o  αd n e − αd n  (103) as d n → ∞ . F rom the contin uous diﬀeren tiabilit y of K s as a f unction of ζ in (47) and ζ as a function of ρ , we ha v e K s = D ( N (0 , 1) ||N (0 , 1 + SNR)) − c 2 ζ + o ( ζ ) , = D ( N (0 , 1) ||N (0 , 1 + SNR)) − c 2 ( c 7 ρ + o ( ρ )) + o ( c 7 ρ + o ( ρ )) , = D ( N (0 , 1) ||N (0 , 1 + SNR)) − c 2 c 7 ρ + o ( ρ ) , October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 38 for some c 2 , c 7 > 0. Ap p lying (103) to the ab o v e equation, we hav e (73). The pro of for the m utual information I s is similar.  Pr o of of The or em 6 Consider a ﬁx ed area with size L × L and a lattice I n on it. The sensor spacing d n for n is giv en b y d n = L n . By (62), w e h a v e ρ ( d n ) = 1 + c 8 · d 2 n + o ( d 2 n ) (104) for some constan t c 8 . By the con tin uous diﬀerent iabilit y of K s (as a function of ζ ) and ζ (as a function of ρ ), we ha ve ζ = 1 4 + c 9 · (1 − ρ ) + o ((1 − ρ ) 2 ) , and K s = c 1 · ( ζ − 1 / 4) + o ( ζ − 1 / 4) , for some constan t c 9 . S ubstituting (104) in to the ab ov e equations give s K s = c 10 · d 2 n + o ( d 2 n ) , (105) for some constan t c 10 . The no de den sit y is giv en by µ n = n 2 L 2 = d − 2 n . (10 6) Substituting (10 6) into (105) yields (75). The total amoun t of information p er un it area is given b y µ n K s = c 5 + o (1) , (107) and it con v erges to c 5 as n → ∞ . T o calculate the energy eﬃciency , w e ﬁrs t calculate the total comm un ication energy consu med b y the minimum hop routing, give n by E ′ t = E c ( d n ) n − 1 X i =0 n − 1 X j =0 ( | i − ⌊ n/ 2 ⌋| + | j − ⌊ n/ 2 ⌋| ) , = Θ ( n 3 ) E c ( d n ) = E 0 L ν n − ν Θ( n 3 ) , = Θ ( n 3 − ν ) , (108) October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 39 as n → ∞ (i.e., µ n → ∞ ). Here, E ′ t denotes the tota l energy considering only the comm un ication energy . T he energy eﬃciency in this case is give n by η ′ = µ n K s E ′ t [nats /J /m 2 ] . (109) Applying (107) and (108) to the ab o v e equation, we ha v e the claims.  Pr o of of The or em 7 Note that E t = n 2 E s + Θ( n 3 ) E c ( d n ) . In this case, n and d n are ﬁxed, and Theorem 3 is dir ectly applicable. Since the num b er of no des and communicat ion energy are ﬁ xed, the sen s ing energy increases linearly with the total ener gy E t . By Assumption (A.5) , the measurement SNR increases linearly with the sensing energy . Applying Th eorem 3 yields (80).  Appendix I I T o p ro v e Lemma 2 (this w ill b e stated b elo w), we brieﬂy in tro du ce some relev ant preliminary results. Deﬁnition 5 (Matrix norms [18, 28]) Let A b e an n × n matrix w ith sin gular v alue decomp o- sition A = USV T = n X i =1 s i u i v T i , (110) where U and V are un itary matrices with columns u i and v i , resp ectiv ely , and S = d iag( s 1 , s 2 , · · · , s n ) with nonn egativ e elemen ts s 1 ≥ s 2 ≥ · · · s n ≥ 0. The op e r ator norm of k A k is deﬁned as k A k = s 1 = sup x 6 = 0 k Ax k / k x k , (111) where k x k d enotes the 2-norm of x . On the other hand, the tr ac e class norm of A is deﬁned as k A k 1 = X i s i . ( 112) Note that if A is a symmetric matrix with eigen v alues { λ i } , then k A k 1 = X i | λ i | . (113) October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 40 R emark 8 (The co v ariance matrix and its circulan t app r o ximation) Using vecto r notation, the co v ariance matrix of the v ector y |D n | in (29) under p 1 is giv en b y Σ 1 , |D n | = E 1 { y |D n | y T |D n | } = [ σ f − 1 id ( i ) ,f − 1 id ( j ) ] , σ f − 1 id ( i ) ,f − 1 id ( j ) = γ i − j , i , j ∈ D n , (114) where γ h is deﬁned in (23) and f id is d eﬁned in (86). With s ligh t abuse of n otation, w e use σ ij for σ f − 1 id ( i ) ,f − 1 id ( j ) for the sak e of exp osition. The circulan t app ro ximation C |D n | to Σ 1 , |D n | is obtained by treati ng D n as a high dimens ional torus with opp osite end s b eing n eigh b ors, and C |D n | is give n by C |D n | = [ c ij ] , c ij = γ π ( i − j ) , i , j ∈ I n , (115) where the m apping π : Z d → Z d is deﬁned as π ( h ) = π ( h 1 , h 2 , · · · , h d ) = ( h ′ 1 , h ′ 2 , · · · , h ′ d ) , (116) and h ′ k = h k I ( | h k | ≤ n / 2) + ( n − | h k | ) I ( | h k | > n / 2) , k = 1 , · · · , d. †† (117) Here, I ( · ) is the indicator function. Note that Σ 1 , |D n | is a blo c k T o eplitz matrix, while C |D n | is a blo c k circulan t matrix. It is known that the eigen v alues of the blo c k circulan t matrix C |D n | are giv en by λ i = X h ∈D n γ π ( h ) e ι h · ω i , (118) for i = ( i 1 , · · · , i d ) ∈ D n , where ω i = ( ω i 1 , ω i 2 , · · · , ω i d ) =  2 π i 1 n , 2 π i 2 n , · · · , 2 π i d n  . (119) Deﬁne the p erio dic ap p ro ximate sp ectral density b y f c n ( ω ) = (2 π ) − d X h ∈D n γ π ( h ) e ι h · ω . (120) Then, the eigen v alues of C |D n | are giv en b y λ i = (2 π ) d f c n ( ω i ) , i ∈ D n . (121) †† The distinction of ev en and o dd n will not b e considered for simplicit y , as this is merely a tec hnical issue. I n either case, the asymptotic b ehavio r is the same. October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 41 F urther, it is shown in [18, Lemma 4.1.(c)] that th e p er io dic app ro ximate sp ectral density con- v erges un iformly to the true sp ectral density f 1 ( ω ), i.e., sup ω ∈ [ − π ,π ) d | f c n ( ω ) − f 1 ( ω ) | → 0 , (122) as n → ∞ . L emma 1: Und er the assu m ption of Th eorem 2, we ha v e (a) f c n ( ω ) is unif orm ly con tinuous for suﬃcien tly large n . (b) sup ω ∈ [ − π ,π ) d     1 f c n ( ω ) − 1 f 1 ( ω )     → 0 as n → ∞ . (123) (c) 1 /f c n ( ω ) is unif orm ly con tin uous for su ﬃcien tly large n . Pr o of of L emma 1 (a) By assumption, f 1 ( ω ) is con tin uous on the compact domain [ − π , π ] d . By the uniform con ti- n uit y theorem, f 1 ( ω ) is unif orm ly con tinuous. F or any ǫ > 0, || ω − ω ′ || < δ imples   f c n ( ω ) − f c n ( ω ′ )   ≤   f c n ( ω ) − f 1 ( ω ) + f 1 ( ω ) − f 1 ( ω ′ ) + f 1 ( ω ′ ) − f c n ( ω ′ )   , ≤ | f c n ( ω ) − f 1 ( ω ) | + | f 1 ( ω ) − f 1 ( ω ′ ) | + | f 1 ( ω ′ ) − f c n ( ω ′ ) | , ≤ ǫ/ 3 + ǫ/ 3 + ǫ/ 3 , for s u ﬃcien tly large n . The con ve rgence of the ﬁrst and third terms is by (122) and that of the second term is by the uniform contin uity of f 1 ( ω ). (b) Since the sp ectrum f 1 ( ω ) has a p ositive lo w er b ound b y assump tion, its in ve rse 1 /f 1 ( ω ) is b ound ed from ab o ve . In add ition, du e to (122) there exists M 1 > 0 such that 1 f 1 ( ω ) ≤ M 1 and 1 f c n ( ω ) ≤ M 1 , (124) for all ω ∈ [ − π , π ) d and for suﬃcien tly large n . Th en, for any ǫ > 0     1 f c n ( ω ) − 1 f 1 ( ω )     =     1 f c n ( ω ) 1 f 1 ( ω )     | f c n ( ω ) − f 1 ( ω ) | , (125) ≤ ǫ M 2 1 (126) for all ω ∈ [ − π , π ) d and for suﬃcien tly large n , by (122) and (124). (c) F or any ǫ > 0, || ω − ω ′ || < δ implies     1 f c n ( ω ) − 1 f 1 ( ω ′ )     ≤     1 f c n ( ω ) − 1 f 1 ( ω ) + 1 f 1 ( ω ) − 1 f 1 ( ω ′ ) + 1 f 1 ( ω ′ ) − 1 f c n ( ω ′ )     , October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 42 ≤     1 f c n ( ω ) − 1 f 1 ( ω )     +     1 f 1 ( ω ) − 1 f 1 ( ω ′ )     +     1 f 1 ( ω ′ ) − 1 f c n ( ω ′ )     , ≤ ǫ / 3 + ǫ/ 3 + ǫ / 3 , for suﬃcient ly large n . The conv ergence of the ﬁr st and third terms is by (123) and that of the second term is by the u niform con tin uit y of 1 /f 1 ( ω ). (The u niform contin uity of 1 /f 1 ( ω ) is ob vious d ue to the uniform conti nuit y and strict p ositivit y of f 1 ( ω ).)  L emma 2: Und er the conditions of Th eorem 2, we ha v e 1 |D n | y T |D n | Σ − 1 1 , |D n | y |D n | → 1 (2 π ) d Z [ − π ,π ) d σ 2 (2 π ) d f 1 ( ω ) d ω , almost surely . Pr o of of L emma 2 First, it is shown in [18, Lemma 4.1.(a)] that |D n | − 1 || Σ 1 , |D n | − C |D n | || 1 = O  1 n  , (127) as n → ∞ . Let { λ |D n | ( i ) , i = 1 , 2 , · · · , |D n |} b e the eig env alues of |D n | − 1 ( Σ 1 , |D n | − C |D n | ), where |D n | = n d for d -D. Then, by (113) and (127) w e ha v e n d X i =1 | λ |D n | ( i ) | = O  1 n  . (128) Since th e con v ergence of the eigen v alues of the blo c k T o eplitz matrix Σ 1 , |D n | and its blo c k circulan t appro ximation C |D n | is uniform (Th e eigenv alues of these matrices are the samples of the corresp ondin g sp ectra for suﬃcient ly large n ; see (121) and (122) .), min i | λ |D n | ( i ) | and max i | λ |D n | ( i ) | h a ve the s ame con ve rgence rate, i.e., there exist M 2 , M 3 and r n suc h that M 2 r n ≤ min i | λ |D n | ( i ) | ≤ max i | λ |D n | ( i ) | ≤ M 3 r n . (129) By (128) and (129) we ha v e r n = O  1 n d +1  . (130) Since the sp ectra f 1 ( ω ) and f c n ( ω ) h a v e p ositive low er b ounds by assum p tion, th eir inv erses 1 /f 1 ( ω ) and 1 /f c n ( ω ) are b ound ed from ab ov e. Hence, the eigen v alues of Σ − 1 1 , |D n | and C − 1 |D n | are b ound ed from ab o v e sin ce the eigen v alues of these matrices are the samples of the corresp ondin g in v erse sp ectra f or suﬃcient ly large n , and th us w e hav e || Σ − 1 1 , |D n | || < M 1 and || C − 1 |D n | || < M 1 (131) October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 43 for all su ﬃcien tly large n . No w consider the err or b etw een t wo qu adratic terms.    |D n | − 1 y T |D n | Σ − 1 1 , |D n | y |D n | − |D n | − 1 y T |D n | C − 1 |D n | y |D n |    =    |D n | − 1 y T |D n |  Σ − 1 1 , |D n | − C − 1 |D n |  y |D n |    , =    |D n | − 1 y T |D n | C − 1 |D n |  C |D n | − Σ 1 , |D n |  Σ − 1 1 , |D n | y |D n |    , ( a ) ≤ C M 2 1 |D n | X i =1 | λ i | y 2 i , ( b ) ≤ C M 2 1 M 3 r n |D n | X i =1 y 2 i , ( c ) ≤ C M 2 1 M 3 O  1 n  1 n d n d X i =1 y 2 i , ( d ) → 0 a.s. (132) for some C > 0. Here, step (a) is by (131) and the deﬁn ition of the trace class norm (113), step (b) is b y (129), and step (c) is by (130). S tep (d) is by the strong la w of large n umb ers (SLLN) on the sample mean of y 2 i . Since { y i } is i.i.d. N (0 , σ 2 ) u nder p 0 , 1 n 2 P n 2 i =1 y 2 i → σ 2 almost surely . Th us, the quadratic form using the blo ck circulant app ro ximation con v erges almost s urely to that based on the true co v ariance matrix. W e next consider the asymp totic b eh a vior of |D n | − 1 y T |D n | C − 1 |D n | y |D n | . Since C |D n | is a blo c k circulan t matrix, the eigendecomp osition is giv en b y [29, 30] C |D n | = W |D n | Λ |D n | W H |D n | , (133) where W |D n | is the d -dimensional discrete F ourier transform (DFT) m atrix w hic h is unitary , and Λ |D n | = diag( λ 0 , ··· , 0 , · · · , λ n − 1 , ··· ,n − 1 ) . (134) The inv erse of C |D n | is giv en by C − 1 |D n | = W |D n | Λ − 1 |D n | W H |D n | . (135) Deﬁne ¯ y |D n | = W H |D n | y |D n | . (136) October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 44 Then, ¯ y |D n | is a v ector of i.i.d. Gaussian random v ariables since W |D n | is u nitary and y |D n | is a v ector w ith i.i.d. Gaussian elemen ts under p 0 . T h us, |D n | − 1 y T |D n | C − 1 |I n | y |D n | is give n by S n = |D n | − 1 y T |D n | C − 1 |D n | y |D n | = |D n | − 1 ¯ y T |D n | Λ − 1 |D n | ¯ y |D n | , = 1 n d X i ∈D n ¯ Y 2 i λ i , (13 7) = 1 n d n − 1 X i 1 =0 · · · n − 1 X i d =0 ¯ Y 2 i 1 , ··· ,i d λ i 1 , ··· ,i d , (138) where { ¯ Y i , i ∈ D n } is i.i.d. zero-mean Gaussian with v ariance σ 2 . F or suﬃcien tly large n , ﬁx K (0 < K < n ) and divide th e indices of eac h d imension such that I = [0 , 1 , · · · , n − 1] = I (0) ∪ I (1) ∪ · · · I ( K − 1) , I ( i ) ∩ I ( j ) = φ if i 6 = j, and |I (0) | = · · · = |I ( K − 2 | = ⌊ n /K ⌋ , |I ( K − 1) | = n − ( K − 1) |I (0) | . Then, (138) is giv en by S n = 1 K d K − 1 X j 1 =0 · · · K − 1 X j d =0   1 |I ( j 1 ) | · · · |I ( j d ) | X i 1 ∈I ( j 1 ) · · · X i d ∈I ( j d ) ¯ Y 2 i 1 , ··· ,i d λ i 1 , ··· ,i d   . (139) No w let i 1 , · · · , i d ( j 1 , · · · , j d ) denote the index represen ting the cen ter of the ( j 1 , · · · , j d ) th h yp er - cub e. Then, by (12 1) w e h a v e 1 λ i 1 , ··· ,i d ( j 1 , ··· ,j d ) = 1 (2 π ) d 1 f c n ( ω j ) , (140) ω j = ( ω j 1 , · · · , ω j d ) =  2 π j 1 K , · · · , 2 π j d K  , (141) and 1 (2 π ) d 1 f c n ( ω j ) − ǫ ′ ≤ 1 λ i 1 , ··· ,i d ≤ 1 (2 π ) d 1 f c n ( ω j ) + ǫ ′ (142) for all ( i 1 , · · · , i d ) in the ( j 1 , · · · , j d ) th h yp er cu b e. Here, ǫ ′ ( > 0) is indep en d en t of ( j 1 , · · · , j d ) since 1 /f c n ( ω ) is unif orm ly contin uous o ve r ω ∈ [ − π , π ) d b y Lemma 1 (c). Applying (142) to (139), we ha ve V n − ǫ ′ n d X i ∈D n ¯ Y 2 i ≤ S n ≤ V n + ǫ ′ n d X i ∈D n ¯ Y 2 i , (14 3) where V n = 1 K d K X j 1 =1 · · · K X j d =1 1 (2 π ) d 1 f c n ( ω j )   1 |I ( j 1 ) | · · · |I ( j d ) | X i 1 ∈I ( j 1 ) · · · X i d ∈I ( j d ) ¯ Y 2 i 1 , ··· ,i d   . (144) October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 45 By the S LLN for the sample m ean of ¯ Y 2 i , we ha v e σ 2 − ǫ ′′ ≤ 1 |I ( j 1 ) | · · · |I ( j d ) | X i 1 ∈I ( j 1 ) · · · X i d ∈I ( j d ) ¯ Y 2 i 1 , ··· ,i d ≤ σ 2 + ǫ ′′ , ( 145) almost surely for suﬃcientl y large n giv en K . Thus, V n is give n by ( σ 2 − ǫ ′′ ) Z n ≤ V n ≤ ( σ 2 + ǫ ′′ ) Z n , ( 146) where Z n = 1 K d K X j 1 =1 · · · K X j d =1 1 (2 π ) d f c n ( ω j ) . (147) No w we tak e K → ∞ , and the Riemann sum Z n con v erges to Z n → 1 (2 π ) d Z − [ π ,π ) d 1 (2 π ) d f 1 ( ω ) d ω (148) b y Lemma 1 (b) and (c). Since ǫ ′ and ǫ ′′ can b e made arbitrarily small b y making n and K large, and 1 (2 π ) d R − [ π ,π ) d 1 (2 π ) d 1 f 1 ( ω ) d ω < M 4 for some M 4 > 0 and n − d P i ∈D n ¯ Y i → σ 2 a.s., we hav e by (143), (146) and (148), th at |D n | − 1 y T |D n | C − 1 |D n | y |D n | → (2 π ) − d Z ω ∈ [ − π ,π ) 2 σ 2 (2 π ) d f 1 ( ω ) d ω , (149) almost surely as n → ∞ . By (132) and (149) we ha v e |D n | − 1 y T |D n | Σ − 1 1 , |D n | y |D n | → (2 π ) − d Z ω ∈ [ − π ,π ) 2 σ 2 (2 π ) d f 1 ( ω ) d ω , (150) almost surely as n → ∞ . T his concludes the pro of.  Referen ces [1] D. Estrin, D. Culler, K. Pister and G. S u khatme, “Connecting the p hysical w orld with perva sive net w orks,” IEEE Pervasive Computing , vol. 1, no. 1, pp. 59-69, Jan.-Mar. 2002. [2] J.-F. Chamberland and V. V. V eerav alli, “Ho w dense should a sensor netw ork be for detection with correlated observ ations?,” IEEE T r ansactions on Inf ormation The ory , vol. 52, no. 11, pp . 5099 - 5106, Nov. 2006. [3] M. Dong, L. T ong and B. M. Sadler, “Impact of data retriev al pattern on homogeneous signal ﬁeld recon- struction in d ense sensor netw ork s,” IEEE T r ansactions on Signal Pr o c essing , vol . 54, n o. 11, pp. 4352 - 4364, No v. 2006. [4] Y. Sung, X. Zhang, L. T ong and H. V . Poor, “Sensor conﬁguration and activ ation for ﬁeld detection in large sensor arra ys,” IEEE T r ansactions on Signal Pr o c essing , vol. 56, no. 2, pp. 1354 - 1365, F eb . 2008. [5] L. S. Pillutla and V. Krishnamurth y , “Mutu al information and energy tradeoﬀ in correlated wireless sensor netw orks,” in Pr o c. 2008 IEEE I nternational Confer enc e on Comm uni c ations (IC C) , Beiging, China, May 2008. October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 46 [6] F. Liese and I. V a jda, “On dive rgence and informations in statistics and information theory ,” I EEE T r ans- actions on Information The ory , vol. 52, no. 10, p p. 4394 - 4412, Oct. 2006. [7] S. Ku llbac k, Information The ory and Statistics . Dov er Publications, Inc., Mineola, NY, 1997. [8] M. D. Donsker and S . R. S. V aradh an , “Large d eviations for stationary Gaussi an process,” Comm uni c ations in Mathematic al Physics , vol . 97, pp. 187 - 210, 1985. [9] G. R. Benitz and J. A . Bucklew, “Large d eviation rate calculations for nonlinear detectors in Gaussian noise,” IEEE T r ansactions on Information The ory , vol. 36, no. 2, p p . 358 - 371, Mar. 1990. [10] W. Bryc and A. Demb o, “Large deviations for quadratic fun ctionals of Gaussian pro cesses,” Journal of The or etic al Pr ob ability , vol. 10, no. 2, pp. 307 - 332, 1997. [11] B. Bercu, F. Gam b oa and A. Rouault, “Large deviations for quadratic forms of stationary Gaussian processes,” Sto chastic Pr o c esses and Their Applic ations , vol . 71, pp. 75 - 90, 199 7. [12] Y. Sung, L. T on g and H. V. Poor, “Neyman-Pearson detection of Gauss-Mark o v signals in noise: Closed-form error exp onent and prop erties,” IEEE T r ansactions on Information The ory , v ol. 52, n o. 4, pp. 1354 - 1365 , Apr. 2006. [13] A. S. Leong, S. Dey and J. S . Ev ans, “Error exp onents for Neyman-Pea rson detection of Marko v c hains in noise,” IEEE T r ansactions on Signal Pr o c essing , vol. 55, no. 10, pp. 5097 - 5103, Oct. 2007. [14] A. Anandk umar, L. T ong and A. Swami , “Detection of Gauss-Mark ov random ﬁeld on nearest-neighbor graph,” in Pr o c. 2007 IEEE International Confer enc e on A c oustics, Sp e e ch, and Signal Pr o c essing (ICASSP’07) , Honolulu, HI, Apr. 2007. Oct. 2007. [15] Y. Su ng, H . V. Poor and H. Y u, “Informatio n, energy and den sit y for ad hoc sensor net wo rks o ver correlated random ﬁelds: L arge deviations analysis,” in Pr o c. 2008 I nternational Symp osium on Information The ory (ISIT) , T oronto, Onta rio, Canada, Jul. 2008. [16] H. Rue and L. Held, Gaussian Markov R andom Fields: The ory and Applic atons . Chapman & Hall/CR C, New Y ork, 2005 . [17] R. R. Bahadu r, S. L. Zabell and J. C. Gupta, “Large d eviations, tests, and estimates,” in I. M. Chakrav arti, ed., Asympto tic The ory of Statist ic al T ests and Estimation: In Honor of Wassily Ho eﬀding , Academic Press, New Y ork, 1980 . [18] J. T. Kent and K. V. Mardia, “S p ectral and circulant approximations to the likelihoo d for stationary Gaussian random ﬁelds,” Journal of Statistic al Planning and Infer enc e , vol. 502, no. 3, p p. 379 - 394, 1996. [19] R. R. Bahadur, “Some Limit Theorems in Statistics,” in Conf er enc e Bo ar d of the Mathematic al Scienc es R e gional Confer enc es Series in Applie d Mathematics, SIAM , no. 4, Philadelphia, P A , 1971. [20] T. Co ver and J. Thomas, Elements of Information The ory . John Wiley & Sons, Inc., New Y ork, 1991 . [21] J. Besag, “On a system of tw o-dimensional recurrence equ ations,” Journal of the R oyal Statistic al So ciety. Series B , vol. 43, no. 3, pp. 302 - 309, 1981 . [22] A. Erd ´ elyi, Higher T r ansc endental F unctions, V ol. I I. McGra w-Hill, N ew Y ork, 195 3. [23] P . Whittle, “On stationary p rocesses in the plane,” Biometrika , vol. 41, no. 3, pp. 434 - 449, Dec. 1954. [24] I. V a jda, The ory of Stat istic al Infer enc e and Information . Kluw er A cademic Publishers, Dordrech t, 1989 . [25] P . J. Brockw ell an d R. A. D a vis, Time Series: The ory and Metho ds . 2nd Edition, Springer, N ew Y ork, 1991. October 30, 2018 DRAFT TO APPEAR IN IEEE TRANS. ON INFORMA TION THEOR Y , JUNE 2009 47 [26] E. J. H annon, “The asymptotic theory of linear time-series m o dels,” Journal of Applie d Pr ob ability , vol. 10, no. 1, pp. 130 - 145, Mar. 1973. [27] X. Guyon, R andom Fields on a Network: Mo deling, Statistics and Applic ations . Springer-V erlag, New Y ork, 1995. [28] I. Gohberg, S. Goldberg and M. A. Kaashoek, Basic Classes of Line ar Op er ators . Birkh¨ auser, Basel, Switzer- land, 2004. [29] R. J. Martin, “A note on the asympt otic eigenv alues and eigenv ectors of th e disp ersion matrix of a second- order stationary process on a d - d imensional lattice,” Journal of Applie d Pr ob abili ty , vol. 23, no. 2, pp. 529 - 535, Jun. 1986. [30] P . J. Davis, Cir culant M atric es . Chelsea Pu blishing, New Y ork, 1994. October 30, 2018 DRAFT

How Much Information can One Get from a Wireless Ad Hoc Sensor Network over a Correlated Random Field?

Original Paper

Comments & Academic Discussion

Leave a Comment

Original Paper

Related Papers

Comments & Academic Discussion

Leave a Comment