On The Positive Definiteness of Polarity Coincidence Correlation Coefficient Matrix

1 On The Positi v e Defin iteness of Polarity Coincidence Correlation Coef ficient Matrix F . Haddadi, Student Member , IEEE, M. M. Nayebi, Senior Member , IEEE, and M. R . Aref Abstract — Polarity coincidence co rrelator (PCC), when used to estimate the covariance matr ix on an element-by- element basis, may not yield a positive semi-definite (PSD) estimate. Devlin et al. [ 1], claimed that element-wise PCC is not guaranteed to be PSD in dimensions p > 3 for real signals. However , no j ustification or proof was available on this is sue. In this letter , i t is prov ed that f or real signals with p ≤ 3 a nd for complex signals with p ≤ 2 , a PSD estimat e is guaranteed. Counterexamples a re presented for higher dimensions which yield in valid covariance e stimates. Index T erms — Polarity coincidence correlator , element- wise covariance estima te, po sitive semi-definite. I . I N T RO D U C T I O N A N D P R E L I M I NA R I E S P OLARITY coinc idence correlator (PCC) is a rob u st and nonparametric estimator of bivari ate correlation [1], [2]. It is also a fast and lo w-cost estimator for applications with extraordinary computational complex- ity . R adio astronomy is an instance in which P CC is by far the most f avorable co rrelator [3]. Several researche rs hav e in vestigated the statistical error of PCC as an es timate of biv a riate correlation [4], [5]. In multiv ariate case, using PCC to estimate elements of the cov ariance matrix does not guarantee a PSD matrix estimator [1], [6]. De vlin et al. [1, S ec. 4.4], referring to a p ersonal co mmunication, claim that element-wise PCC (or ”quadrant correlation”), may yield an in valid cov ariance estimate for p > 3 and real signals. In this letter , we p rove t hat for real s ignals with p ≤ 3 , and c omplex signals wit h p ≤ 2 , PCC estimate is PSD. For higher dimensions , c ounterexamples are presented which yield inv alid covariance estimates . Let x and y be two zero-mean real ran dom variables with correlation coefficient r distributed with elliptical symmetry . It is well k nown that [ 6]: r = sin  π 2 E { sgn ( x ) sgn ( y ) }  (1) This work was supported by Advanced Communication Re- search Institute (ACRI), Sharif Univ ersi ty of T echnolog y , T ehran, Iran. Authors are with the Department of Electrical Engineering, Sharif Uni versity of T echn ology , T ehran, Iran (e-mails: farzanhad- dadi@yahoo .com, Nayebi@sharif.edu, and Aref@sharif.edu). where sgn ( x ) =  +1 : x ≥ 0 − 1 : x < 0 (2) Using (1), an estimate of r from N iid observati ons x i , y i , i = 1 , . . . , N is given by ˆ r = sin  π 2 1 N N X i =1 s xi s y i  . (3) where s xi = sgn ( x i ) . In the c omplex case, we ca n d efine the comp lex sign function as sgn c ( x ) , sgn ( ℜ [ x ]) + j sgn ( ℑ [ x ]) , where ℜ [ x ] and ℑ [ x ] are real and imagina ry parts of x , re spectively . In the Ap pendix, it is s hown that ℜ [ r ] = sin  π 4 E  ℜ [ sgn c ( x ) sgn ∗ c ( y )]   ℑ [ r ] = sin  π 4 E  ℑ [ sgn c ( x ) sgn ∗ c ( y )]   (4) where ( · ) ∗ denotes complex conjug ate. Similar to (3), an estimate for the complex case is obtaine d by rep lacing expectation with the av e rage as ˆ r R = sin  π 4 1 N N X i =1 [ s xiR s y i R + s xiI s y i I ]  ˆ r I = sin  π 4 1 N N X i =1 [ s xiI s y i R − s xiR s y i I ]  (5) where ( · ) R and ( · ) I denote real a nd imagina ry parts, respectively . I I . M A I N R E S U LT Let R p × p be the covari ance matrix of p random signals with unit d iagonal eleme nts and of f-diago nal elements r ij : i, j = 1 , · · · , p . For p = 2 cas e, a valid correlation estimate should satisfy | ˆ r | ≤ 1 . For the real case of (3), | ˆ r | = | sin( · ) | ≤ 1 . For the c omplex case, regarding (5) define α and β such that ˆ r R = sin( α ) and ˆ r I = sin( β ) . Then α + β = π 4 N N X i =1 [ s xiR ( s y i R − s y i I ) + s xiI ( s y i R + s y i I )] (6) and it can be eas ily c hecked that the a r gument of sum- mation in (6) belong s to {± 2 } . This yields α + β ≤ π 2 . In 2 N a 1 N a 2 N a 3 x y z Fig. 1. Polarity coincidence diagram of x , y , z . Black stri ps denote the pack ed positions of polarity coincidences of each signal with signal x . The strips lengths N a i are the numbe r of polarity coincidences. the same manner , we ca n s how that ± α ± β ≤ π 2 which giv e s | α | + | β | ≤ π 2 . Now it is straightforward to see that | ˆ r | 2 = sin 2 ( α ) + sin 2 ( β ) ≤ sin 2 ( | α | ) + sin 2 ( π 2 − | α | ) = 1 . For p = 3 and real signals, we calculate t he v a lid range of the elements of a 3 × 3 cov ariance matrix. Then we show that P CC es timate lies in this range. A. V alid Range of Covarianc e Let R ∈ R 3 × 3 be a c ovari ance matrix with unit diag- onal elements. V alid range of r 23 should be calcu lated when r 12 , r 13 ∈ [ − 1 , +1] are fixed. It c an b e read ily shown that | R | ≥ 0 impli e s tha t | r 23 − r 12 r 13 | ≤ q  1 − r 2 12  1 − r 2 13  . (7) B. PCC Co variance Estimate Assume random sign seque nces s x , s y , s z with length N . Con sider the positions of po larity coinc idence with s x as black positions or ” + ” and elsewhere as white or ” − ”. Obviously all of the positions in s x is ” + ” and ( s y i , s z i ) have four states of { ++ , + − , − + , −−} . Since the permutation of the samples does not af fect the estimate in (3 ), put the sample s of s x , s y , s z from left in the order o f { + + − , + + + , + − + , + − − } as in Fig. 1. Th en a ny rando m sign sequ ences of s x , s y and s z can be replace d by the mode l in Fig. 1 with ap propriate strip lengths N a i (with a 1 = 1 ) a nd relati ve positions o f strips. Let R s be the c ovari a nce matrix of s x , s y , s z with elements r sik , i, k = 1 , 2 , 3 . Th e maximum of r s 12 = +1 occurs in a 2 = 1 and the minimum o f r s 12 = − 1 in a 2 = 0 . In fact, r s 12 = 1 N P N i =1 s xi s y i = 1 N [ N a 2 − ( N − N a 2 )] = 2 a 2 − 1 , in othe r words a i = 1 + r s 1 i 2 . (8) r s 12 and r s 13 are de termined by the v alues of a 2 and a 3 , r sii = 1 , an d the poss ible range of r s 23 should be ca lculated. r s 23 depend s on the number of p olarity coincidenc es of y a nd z wh ich is maximum when the T ABL E I C O U N T E R E X A M P L E S F O R R E A L A N D C O M P L E X D A TA Real Case Complex Case s x + + + + s y + + − − s z + + + − s w + + − + s x ++ ++ s y ++ − + s z ++ −− strip of z is in the left corner , and minimum when it is in the right corner . After some calculations , the range of r s 23 is found a s | r s 12 + r s 13 | − 1 ≤ r s 23 ≤ 1 − | r s 12 − r s 13 | . (9) It shou ld be noted tha t the effect of fin ite N is the quantization of the accessible values. Now , it c an be readily verified that sin  π 2 (1 − | r s 12 − r s 13 | )  = r 12 r 13 + q  1 − r 2 12  1 − r 2 13  (10) and sin  π 2 ( | r s 12 + r s 13 | − 1)  = r 12 r 13 − q  1 − r 2 12  1 − r 2 13  . (11) Therefore, ˆ r 23 = sin  π 2 r s 23  satisfies (7). This, besides | ˆ r 12 | < 1 and | ˆ r 13 | < 1 ca n be used to show that | ˆ R | ≥ 0 (as in (7)) and the asse rtion is proved that for p = 3 and real data, P CC estimate is a v a lid cov ariance matrix. I I I . C O U N T E R E X A M P L E S In this section, so me coun terexamples are p resented to show that PCC cov a riance estimate is not guaranteed to be PS D in d imensions p > 3 f or rea l s ignals and p > 2 for complex signals. In real data c ase with p = 4 and number o f ob servations N = 4 , the real sign seque nces in T able I resu lts in an in valid covariance estimate. After simple comp utations, we will hav e r s 12 = r s 34 = 0 and r s 13 = r s 14 = r s 23 = r s 24 = 0 . 5 . The c ov ariance estimate will be ˆ R 1 =     1 0 0 . 7 0 . 7 0 1 0 . 7 0 . 7 0 . 7 0 . 7 1 0 0 . 7 0 . 7 0 1     with eige n values [ − 0 . 4 , 1 , 1 , 2 . 4 ] . Then ˆ R 1 , with a n eg- ati ve eigen value, is not a valid covariance matrix. W e c an augment this example to gi ve a c ounter- example for dimension p = 5 . Re peat each sign twice to have fou r sign als with number of ob servations 2 N . 3 Note that the cov ariance matrix does not change. N ow , add a new signal with alternating sign in eac h sample. The covariance es timate will be ˆ R aug =  ˆ R 1 0 0 T 1  . where 0 is the 4 × 1 vector of zeros . As a con seque nce of the structure of ˆ R aug , eigenv alues of ˆ R 1 are also eigen values of ˆ R aug . Therefore, ˆ R aug is an in v a lid co- variance ma trix. This proced ure can c ontinue to produce counterexamples for high er dimensions in real d ata case. In case of c omplex signa ls, p = 3 and N = 2 , a counterexample is giv en in T able I, where ” − + ” d enotes − 1 + j . The resulting es timate is ˆ R =   1 0 . 7 − j 0 . 7 0 0 . 7 + j 0 . 7 1 0 . 7 − j 0 . 7 0 0 . 7 + j 0 . 7 1   with eige n values [ − 0 . 4 , 1 , 2 . 4 ] which make ˆ R an in valid covari ance matrix. Augmentation of the complex signal set for higher dimensions is similar to the real case, except that the new added sign al alternates between ” ++ ” and ” − − ”. I V . A P P L I C AT I O N S O F T H E R E S U L T S In this section, we disc uss the p ractical usefulnes s of the main resu lts of this letter which focus es on low number of se nsors. In the signal proces sing c ontext, covari ance es timation often arises in the multi-sensor applications where parameters of interest are f u nctions of the true data covari ance matrix. Although PC C e stimate of the c ov ariance matrix exhibits attractiv e features suc h as robustness a nd extremely low complexity , it c annot be guaranteed to be PSD in the applications with lar ge number of s ensors. Selection of the number of s ensors in an app lication depend s on both nature of the problem a nd practical limitations. In theory , more se nsors a lways res ults in a better estimate, as proved in many cases such as direction finding through examination of the Cramer -Rao bounds [7]. In practice, complexity issues usually limit the number of senso rs. Large arrays are used when ev er performance be of the main importance regardless of the cos t. In such case s as DO A estimation in military en vironmen ts (rada r and sonar), thousands of sensors are not uncommo n. Ne vertheless , most low-cost ci vil applications use very fe w sens ors. In the follo wing, we consider some of these applications. A. MIMO Communication S ystems Multiple anten na sys tems are an integral p art of the most n ew wireless communica tion systems increasing user and d ata capacity (e.g. UMTS/W -CDMA, 802.11n WLAN, 60 GHz WP AN). Multiple an tennas c an provide div e rsity gain and/or better a ntenna ga in through bea m- forming in base station and/or handse t. Be amformers (e.g. con ventional or Capon) usu ally uti lize an es timate of the a rray covariance matrix [8], that may be obtained using PCC a s a power- saving estimator . It is well known that p erformance i mprovement due to di versity gain re- duces as the number of a ntennas increase s. This, besides space limit o n the hand set and cou pling pheno mena hav e resulted in the prev alenc e of MI MO sys tems wit h very few (usua lly 2 to 4) antennas [9], [10]. B. Blind S ource Separation (BSS) BSS has found numerous p otential ap plications in the field of audio signal process ing [11]. An array of microphones is use d to gather multiple sign al mixtures and diverse methods are used to extr act signals from these obse rvations. A lar ge class of BS S me thods use real-valued inter -sensor c ovari a nces with different time lags to es timate the mixing matrix an d de sired signals (e.g. SOBI [12], J ADE [13]). This also include s in- put s ignals whitening as a preproces sing that con verts the co n voluti ve source se paration problem to a simpler independ ent comp onent a nalysis (ICA) problem. This family of two-step a lgorithms is k nown as AMUSE (Algorithm for MUltiple Source Extraction). PCC, as a fast correlator , can make real-time operation more feasible in these methods . For realistic situations where we have fe wer sens ors than s ources, u nderdetermined methods are propose d [14]. Many methods are presented for the spe cial case of 2 sensors and multiple sources (e.g. DUET [15 ], and [16]), an d also quite few sensors are co mmon to many realizations of the methods [12], [14]. A P P E N D I X C O M P L E X P C C Let x, y be two z ero-mean, un it-v arianc e, a nd c ircu- larly symmetric complex random v a riables with indepen- dent rea l an d imagina ry p arts. T o prove (4), w e e xpand the expectation as E { sgn c ( x ) sgn ∗ c ( y ) } = E  [ sgn ( x R y R ) + sgn ( x I y I )] + j [ sgn ( x I y R ) − sgn ( x R y I )]  . (12) Furthermore, E { xy ∗ } = r implies that E { x R y R + x I y I } = r R E { x I y R − x R y I } = r I . (13) 4 Circular symmetry of x a nd y yields E { x R y R } = E { x I y I } = r R / 2 E { x I y R } = − E { x R y I } = r I / 2 (14) and E { x 2 R } = E { x 2 I } = E { y 2 R } = E { y 2 I } = 1 2 . Then the co rrelation coefficients will be Cor ( x R , y R ) = C or ( x I , y I ) = r R Cor ( x I , y R ) = − Co r ( x R , y I ) = r I . (15) Substituting (15) and (1) into (12) giv e s E { sgn c ( x ) sgn ∗ c ( y ) } = 4 /π [ sin − 1 ( r R ) + j sin − 1 ( r I ) ] (16) which implies (4). R E F E R E N C E S [1] S. J. Devlin, R. Gnanadesika n, and J. R . K ettenring, ”Rob ust estimation and outlier detection with correlation coef ficients”, Biometrika , vol. 62, No. 3, pp. 531-545, March 1975. [2] S. S. W olf f, J. B . T homas, and T . R. W illi ams, ”The polarity- coincidence correlator: a nonparametric detection device”, IRE T rans. Information Theory , vol. 8, No. 1, pp. 5-9, Jan. 1962. [3] P . C. Egau, ”Correlati on systems in radio astronomy and related fields”, IEE Pr oc. , vo l. 131, Pt. F , No. 1, pp. 32-39, Feb. 1984. [4] K. J. Gabriel, ”Comparison of three correlation coefficient estimators for gaussian stationary processes ”, IEEE T rans. Acoustic, Speech , Signal Pro c. , vol. ASSP-31, No. 4, pp. 1023- 1025, Aug. 1983. [5] G. Jacovitti and R. Cusani, ”Performance of normalized co- rrelation estimators for complex processes”, IEEE T rans. Signal Pr oc. , vol. 40, N o. 1, pp. 114-128 , Jan. 1992. [6] S. V isuri, V . Koi vunen, and H. Oja, ”Sign and rank cov ariance matrices”, J. Stat. Plan. Infer . , vol. 91, No. 2, pp. 557-575, Feb . 2000. [7] P . Stoica and A. Nehorai, ”MUSIC, maximum likelihood and Cramer-Rao bound ”, IEEE Tr ans. Acoustic Speech Sig. Pr oc. , vol 37, No. 5, pp. 720-741 , May 1989. [8] B. D. V an V een and K. M. Buckley , ”Beamforming: a versatile approach to spatial filtering”, IEE E A SSP Mag. , pp. 4-24, Apr . 1988. [9] H. T . Hui, ”P ractical dual-helical an tenna array for diversity- MIMO receiving antennas on mobile handsets”, I EE Proc. Micr ow . Antennas Pr opag . , vol. 152, No. 5, pp. 367-372, Oct. 2005. [10] D. Browne , M. Manteghi, M. P . Fi tz, and Y . Rahmat-samii, ”Experiments wi th compact antenna arrays for MIMO radio communications” , IEEE T rans. Ant. P r opag. , vol. 54, No. 11, pp. 3239-3259, Nov . 2006. [11] K. T orkk ola, ”Blind separation for audio signals - are we there yet?”, in Proc . W orkshop on Independen t Component Analysis and Blind Signal Separation , Aussois, France, Jan. 11-15, 1999. [12] A. Belouchrani, K. Abed-Mera im, J.-F . Cardoso, and E . Moulines, ”A blind source separation technique using second- order st atistics”, IEEE T rans. Signal Pr oc. , vol. 45, No. 2, pp. 434-444, Feb . 1997. [13] J. F . Cardoso, ”High-order contrast for independen t component analysis”, Neural Compu tation , vol. 11, No. 1, pp. 157-19 2, 1999. [14] S. Winter , W . Kellermann, H. Saw ada, and S. Makino, ”MAP- based underdetermine d blind source separation of con volutiv e mixtures by hierarchical clustering and l1-norm minimization”, EURASIP J. Advances Signal Proc. ,pp. 1-12, 2007. [15] O. Yilmaz and S. Rickard, ”Bli nd separation of speech mixtures via t ime-frequenc y masking”, IEEE Tr ans Signal Pro c. , vol. 52, No. 7, pp. 1830-18 47, July 2004. [16] P . Comon, ”Blind identification and source separation in 2 × 3 under -determined mixtures”, IEE E Tr ans. Signal Pr oc. , vol. 52, No. 1,pp. 11-12, Jan. 2004.