Instantaneous and lagged measurements of linear and nonlinear dependence between groups of multivariate time series: frequency decomposition

Cite as: “RD Pascual-Marqui: Instantaneous and lagged measurements of linear and nonlinear dependence between groups of multivariate time series: freq uency decomposition. arXiv:0711. 1455 [stat.ME], 2007-November-09, http:/ /arxiv.org/abs/0711.1455 ” Page 1 of 18 Instant aneous and lagge d measur ements of linear and nonlinear dependenc e betwe en gr oups of multivariat e time series: f requency dec omposition Roberto D. Pascual-Marqui The KEY Institute for Brain-Mind Research University Hospital of Psychiatry Lenggstr. 31, CH-8032 Zurich, Switzerland pascualm at key.uzh.ch www.keyinst.uzh.ch/loreta 1. Abstract Measures of linear dependence (coher ence) and nonlinear dependence (phase synchronization) between any number of multivar iate time series are defined. The measures are expressed as the sum of lagged depend ence and instantaneous dependence. The measures are non-negative, and take the value zero only when there is independence of the pertinent type. These measures are defined in the frequency domain and are applicable to stationary and non-stationary time series. Thes e new results extend an d refine significantly those presented in a previous technical re [stat.ME], http://arxiv.org/abs/0706.1776 ), an d have been largely motivated by the seminal paper on linear feedback by Geweke (198 2 JASA 77:304-313). One important field of application is neurophysiology, where the time se ries consist of el ectric neuronal activity at several brain locations. Coherence and pha se synchronization are interpreted as “connectivity” between locations. However, any measure of dependence is highly contaminated with an instantaneous, non- physiological contribution due to volume conduction and low spatial resolution. The new techniques remove this confounding factor considerably. Moreover, the measures of depend ence can be applied to any number of brain areas jointly, i.e. distributed cortical netw orks, whose activity can be estimated with eLORETA (Pascual-Marqui 2007, arXiv:0710.3341 [math-ph], http://arxiv.org/abs/07 10.3341 ). 2. Introduction This study extends and refines significan tly the results presented in a previous technical report (Pascual-Marqui 2007a). Some results from that previous paper will be repeated here for the sake of completeness. 2.1. The discrete Fourier transform for multivariate time s eries The terms “multivariate time series”, “multi ple time se ries”, and “vector time series” have identical meaning in this paper. Cite as: “RD Pascual-Marqui: Instantaneous and lagged measurements of linear and nonlinear dependence between groups of multivariate time series: freq uency decomposition. arXiv:0711. 1455 [stat.ME], 2007-November-09, http:/ /arxiv.org/abs/0711.1455 ” Page 2 of 18 For general notation and definitions, see e.g. Brillinger (1981) for stationary multivariate time series analysis, and see e.g. Mardia et al (1979) for general multivariate statistics. Let 1 p jt × ∈ X  and 1 q jt × ∈ Y  denote two stationary multivariate time series, for discre te time 0... 1 T tN =− , with 1... R j N = denoting the j -th time segment. The dis crete Fourier transforms are denoted as 1 p j ω × ∈ X  and 1 q j ω × ∈ Y  , and defined as: Eq. 1 1 2 0 T T N it N jj t t e πω ω − − = = ∑ XX Eq. 2 1 2 0 T T N it N jj t t e πω ω − − = = ∑ YY for discrete frequencies 0... 1 T N ω =− , and where 1 i =− . It will be assumed throughout that ω X and ω Y each have zero mean. 2.2. Classical cross-spectra Let: Eq. 3 * 1 1 R N jj j R N ωω ω = = ∑ XX SX X Eq. 4 * 1 1 R N jj j R N ωω ω = = ∑ YY SY Y Eq. 5 * 1 1 R N jj j R N ωω ω = = ∑ XY SX Y Eq. 6 ** 1 1 R N jj j R N ωω ω ω = == ∑ YX XY SS Y X denote complex valued covariance matrices, wh ere the superscript “*” denotes vector/matrix transposition and complex conjugation. Note that ω XX S and ω YY S are Hermitian matrices, satisfying * = SS . When multiplied by the factor () 1 2 T N π − , these matrices correspond to the classical cross-spectral density matrices. 2.3. Phase-information cross-spectra The discrete Fourier transforms in Eq. 1 and Eq. 2 contain both phase and amplitude information, which carries over to the covar iance matrices in Eq. 3, Eq . 4, Eq. 5, and Eq. 6. This means that for the analysis of phase info rmation, the amplitudes must be factored out by an appropriate normalization method. This is achieved by using the following definition for the normalized complex-valued discrete Fourier transform vector: Eq. 7 () 12 * jj j j ωω ω ω − = XX X X  and: Cite as: “RD Pascual-Marqui: Instantaneous and lagged measurements of linear and nonlinear dependence between groups of multivariate time series: freq uency decomposition. arXiv:0711. 1455 [stat.ME], 2007-November-09, http:/ /arxiv.org/abs/0711.1455 ” Page 3 of 18 Eq. 8 () 12 * jj j j ωω ω ω − = YY Y Y  Note that this normalization operation, although deceivingly simple, is a highly nonlinear transformation. The corresponding covariance matrices containing phase information (without amplitude information) are: Eq. 9 * 1 1 R N jj j R N ωω ω = = ∑ XX SX X   Eq. 10 * 1 1 R N jj j R N ωω ω = = ∑ YY SY Y   Eq. 11 * 1 1 R N jj j R N ωω ω = = ∑ XY SX Y   Eq. 12 ** 1 1 R N jj j R N ωω ωω = == ∑ YX XY SS Y X    Note that the normalization used in Eq. 7 and Eq. 8 will be the basis for the analysis of phase synchronization between the multivariate time series X and Y . Note that ω XX S  and ω YY S  are Hermitian matrices. When multiplied by the factor () 1 2 T N π − , these matrices correspond to what is de fined here as the phase-information cross- spectra. 2.4. Instantaneous, zero-phase, zero-lag covariance The instantaneous, zero-phase, zero-lag covariance matrix corresponding to a multivariate time series at frequency ω , is simply the real part of the Hermitian covariance matrix at frequency ω , i.e . () Re ω S . To justify this, consider the multivariate time series 1 p jt × ∈ X  , for discrete time 0... 1 T tN =− , with 1... R j N = denoting the j -th time segment. In a first step, filter the time ser i es to leave exclusively the frequency ω component. Denote the filtered time series as () Filtered jt ω X . Note that, by construction, the spectral density of () Filtered jt ω X is zero everywhere except at frequency ω . In a second step, compute the instantaneou s, zero-lag, zero phase shifted, time domain, symmetric covariance matrix for the filtered time series () Filtered jt ω X at frequency ω : Eq. 13 () () 11 1 t R N N T Filtered Filtered r r jt jt jt TR NN ωω ω × == =∈ ∑∑ AX X  Cite as: “RD Pascual-Marqui: Instantaneous and lagged measurements of linear and nonlinear dependence between groups of multivariate time series: freq uency decomposition. arXiv:0711. 1455 [stat.ME], 2007-November-09, http:/ /arxiv.org/abs/0711.1455 ” Page 4 of 18 Finally, by making use of Parseval’s theorem for the filtered time series, the following relation holds: Eq. 14 () 2 Re 2 T N ωω = XX SA where () Re ω XX S denotes the real part of ω XX S given by Eq. 3 above. These arguments apply identically to the no rmal ized time series, as in Eq. 7 to Eq. 12 above, when considering the phase-informa tion cross-spectra. This means that the instantaneous, zero-phase, zero-lag covari ance matrix corresponding to a normalized multivariate time series X at frequency ω , is simply the real part of the phase-information Hermitian covariance matrix at frequency ω , i.e . () Re ω XX S  . The section entitled “Appendix 1” gives a brief descriptio n of the problems that arise in neurophysiology, where any measure of depe ndence is highly contaminated with an instantaneous, non-physiological contribution due to volume conduction and low spatial resolution. 3. Measures of linear dependence (coherence-type) between two multivariate time series The definitions presented here are largely motivated by the seminal paper on linear feedback by Geweke (1982). The measure of linear dependence between time series X and Y at frequency ω is defined as: Eq. 15 () , ln T F ω ω ωω ωω ω ⎛⎞ ⎜⎟ ⎝⎠ = ⎛⎞ ⎜⎟ ⎝⎠ YY XX XY YY YX XY XX S0 oS SS SS where M denotes the determinant of M . The matrix in the numerator of Eq. 15 is a block- diagonal matrix, with 0 denoting a matrix of zeros, which in this case is of dimension qp × . This measure of linear dependence is expressed as the sum of the lagged linea r dependence () F ω XY  and instantaneous linear dependence () F ω XY i : Eq. 16 () () () , FF F ωω ω =+ XY X Y X Y i The measure of instantaneous linear dependence is defined as: Eq. 17 () Re ln Re T F ω ω ωω ωω ω ⎛⎞ ⎜⎟ ⎝⎠ = ⎛⎞ ⎜⎟ ⎝⎠ YY XX XY YY YX XY XX S0 oS SS SS i where () Re M denotes the real part of M . Cite as: “RD Pascual-Marqui: Instantaneous and lagged measurements of linear and nonlinear dependence between groups of multivariate time series: freq uency decomposition. arXiv:0711. 1455 [stat.ME], 2007-November-09, http:/ /arxiv.org/abs/0711.1455 ” Page 5 of 18 Finally, the measure of lagged linear dependence is: Eq. 18 () () () , Re Re ln T T FF F ωω ω ωω ω ωω ω ωω ω ωω ω ⎧ ⎫ ⎛⎞ ⎛ ⎞ ⎪ ⎪ ⎨ ⎬ ⎜⎟ ⎜ ⎟ ⎝⎠ ⎝ ⎠ ⎪ ⎪ ⎩⎭ =− = ⎧ ⎫ ⎛⎞ ⎛ ⎞ ⎪ ⎪ ⎨ ⎬ ⎜⎟ ⎜ ⎟ ⎝⎠ ⎝ ⎠ ⎪ ⎪ ⎩⎭ YY YX YY XY XX XX XY X Y X Y YY YX YY XY XX XX SS S 0 SS o S SS S 0 SS o S i All three measures are non-negative. They take the value zero only when there is independence of the pertinent type (lagged, instantaneous, or both). Not that the measure of linear d ependence () , F ω XY i n E q . 1 5 c a n b e i n t e r p r e t e d a s follows: Eq. 19 () () () 2 ,, 1e x p F ρ ωω =− − XY XY where () , ρ ω XY was defined as the general coherence in Pascual-Marqui (2007a; see Eq. 7 therein): Eq. 20 () 1 22 , 1 G ωω ω ω ω ρω ρ − − == − YY YX XX XY XY YY SS S S S Some relevant literature that motivated the definition of the general coherence () 2 , ρ ω XY in the previous study (Pascual-Marqui 20 07a) follows. In the case of real-valued stochastic variables, Mardia et al (1979) re view several “measures of correlation between vectors”. In particular, Kent (1 983) proposed a general measure of correlation that is closely related to the vector alienation coefficient (Hot e l l i n g 1 9 3 6 , M a r d i a e t a l 1 9 7 9 ) . T h i s m e a s u r e of general coherence is also equivalent to th e coefficient of determination as defined by Pierce (1982). All these definitions can be straightforwardly generalized to the complex valued domain. In order to illustrate and further motivate these measures of linear dependence, a detailed analysis for the simple case of two univariate time series is presented. In the case that the two time series are un ivariate, the measure of linear dependence () , F ω XY in Eq. 15 is: Eq. 21 () () () () 2 , 22 ln ln 1 Re Im yy xx yy xx yx yx ss F ss s s ωω ωω ω ω ω ρ == − − ⎡⎤ ⎡ ⎤ −− ⎣⎦ ⎣ ⎦ XY where: Eq. 22 () () () 22 2 Re I m yx yx yy xx ss ss ωω ωω ρ ⎡⎤ ⎡ ⎤ + ⎣⎦ ⎣ ⎦ = In Eq. 22, ρ is the ordinary squared coherence (see e. g. Equation 3 in Nolte et al 2004). The measure of instantaneous linear dependence is: Eq. 23 () () 2 ln Re yy xx yy xx yx ss F ss s ωω ωω ω ω = ⎡⎤ − ⎣⎦ XY i Cite as: “RD Pascual-Marqui: Instantaneous and lagged measurements of linear and nonlinear dependence between groups of multivariate time series: freq uency decomposition. arXiv:0711. 1455 [stat.ME], 2007-November-09, http:/ /arxiv.org/abs/0711.1455 ” Page 6 of 18 Note that we can define the instantaneous coherence () ρ ω XY i as: Eq. 24 () () () 2 ln 1 F ω ρ ω =− − XY XY ii In general, this gives: Eq. 25 () () 2 Re 1e x p 1 Re T F ωω ωω ω ω ρω ω ⎛⎞ ⎜⎟ ⎝⎠ =− − =− ⎡⎤ ⎣⎦ ⎛⎞ ⎜⎟ ⎝⎠ YY YX XY XX XY XY YY XX SS SS S0 oS ii and in the case of univariate time series it simplifies to: Eq. 26 () () 2 2 Re yx yy xx s ss ω ωω ρω ⎡⎤ ⎣⎦ = XY i which, not surprisingly, is directly related to the real part of the complex valued coherency. Finally, in the particular case of univariate time series, the measure of lagged linear dependence is: Eq. 27 () () () () () () () () () () () , 22 2 2 22 2 ln ln Re Im Re Re ln Re Im ln 1 yy xx yy xx yy xx yx yx yy xx yx yy xx yx yy xx yx yx FF F ss ss ss s s ss s ss s ss s s ωω ωω ωω ω ω ωω ω ωω ω ωω ω ω ωω ω ρω ⎧ =− ⎫ ⎪⎪ ⎪⎪ =− ⎪⎪ ⎡ ⎤⎡ ⎤ ⎡ ⎤ −− − ⎪⎪ ⎣ ⎦⎣ ⎦ ⎣ ⎦ ⎪⎪ ⎨⎬ ⎡⎤ ⎡⎤ − ⎪⎪ ⎣⎦ ⎢⎥ = ⎪⎪ ⎢⎥ ⎡⎤ ⎡ ⎤ −− ⎪⎪ ⎢⎥ ⎣⎦ ⎣ ⎦ ⎣⎦ ⎪⎪ ⎪⎪ =− − ⎩⎭ XY X Y X Y XY i  with: Eq. 28 () () () 2 2 2 Im Re yx yy xx yx s ss s ω ωω ω ρω ⎡⎤ ⎣⎦ = ⎡⎤ − ⎣⎦ XY  In Eq. 28, for the particular case of univariate time series, () 2 ρ ω XY  is equal to the “zero-lag removed general coherence” GL ρ defined in Pascual-Marqui (2007a). In our previous related study (Pascual-M arqui 2007a), the general definition given there for the “zero-lag removed coherence” (see Eq. 22 therein) was: Eq. 29 2 1 Re GL ωω ωω ωω ωω ρ ⎛⎞ ⎜⎟ ⎝⎠ =− ⎛⎞ ⎜⎟ ⎝⎠ YY YX XY XX YY YX XY XX SS SS SS SS The new definition given here for the lagge d coherence follows from the relation: Eq. 30 () () () 2 ln 1 F ω ρ ω =− − XY XY  which gives: Cite as: “RD Pascual-Marqui: Instantaneous and lagged measurements of linear and nonlinear dependence between groups of multivariate time series: freq uency decomposition. arXiv:0711. 1455 [stat.ME], 2007-November-09, http:/ /arxiv.org/abs/0711.1455 ” Page 7 of 18 Eq. 31 () () 2 1e x p 1 Re Re T T F ωω ω ωω ω ωω ω ωω ω ρω ω ⎧ ⎫ ⎛⎞ ⎛⎞ ⎪ ⎪ ⎨ ⎬ ⎜⎟ ⎜⎟ ⎝⎠ ⎝⎠ ⎪ ⎪ ⎩⎭ ⎡⎤ =− − =− ⎣⎦ ⎧ ⎫ ⎛⎞ ⎛⎞ ⎪ ⎪ ⎨ ⎬ ⎜⎟ ⎜⎟ ⎝⎠ ⎝⎠ ⎪ ⎪ ⎩⎭ YY YX YY XY XX XX XY XY YY YX YY XY XX XX SS S 0 SS o S SS S 0 SS o S  Both definitions (Eq. 29 and Eq. 31) are id entical for the case of two univariate time series. However, they are different for the mu ltivariate case. Whereas the old definition in Eq. 29 lumps together all variables from X and Y , the new definition given here in Eq. 31 conserves the multivariate structure of the tw o multivariate time series. Th e improvement of the new lagged coherence in Eq. 31 is that it measures the lagged linear dependence between the two multivariate time se ries without being a ffected by the covariance structure within each multivariate time series. The shortcoming of the old definition from our previous study (Pascual-Marqui 2007a), shown in Eq. 29, is that it is contaminated by the dependence structures of the univar iate time series within X and within Y . Another point worth stressing is the asymme try in the results for the instantaneous coherence () 2 ρ ω XY i (Eq. 26) and the lagged coherence () 2 ρ ω XY  (Eq. 28). While the instantaneous coherence is the real part of the complex valu ed coherency, the lagged coherence is not the imaginary part of the complex va lued coherency. Ideally, the lagged coherence is a measure that is not affected by instantaneous dependence, whereas the imaginary part of the complex valued coherenc y ( N o l t e e t a l 2 0 0 4 ) i s m o r e a f f e c t e d b y instantaneous dependence (Pascual-Marqui 2007a). This makes the lagged coherence (Eq. 31) a much more adequate measure of electrop hysiological connectivity, because it removes the confounding effect of instantaneous de pendence due to volume conduction and low spatial resolution (Pascual-Marqui 2007a). Note that the measures of linear dependence defined by Eq. 15, Eq . 17, and Eq. 18 each have the form of a ratio of variances, which co mpares the residuals of different models (i.e. different dependent and independent variables) . Under the assumption that the time series are wide-sense stationary, large sample distri bution theory can be used to test the nu ll hypothesis that a given measure of linear dependence is zero. Following the same methodology as in Geweke (1982), the asymptotic distributions are: Eq. 32 () () ( ) () () ( ) () () ( ) 2 0, , 2 0 2 0 ˆ :0 , 2 ˆ :0 , ˆ :0 , a T a T a T Under H F N F pq Under H F N F pq Under H F N F p q ωω χ ωω χ ωω χ ⎧⎫ = ⎪⎪ ⎪⎪ = ⎨⎬ ⎪⎪ ⎪⎪ = ⎩⎭ XY XY XY XY XY XY  ii ∼ ∼ ∼ 4. Measures of linear dependence (coherence-type) between groups of multivariate time series Consider the case of three multivariate time ser ies 1 p jt × ∈ X  , 1 q jt × ∈ Y  , and 1 r jt × ∈ Z  , for discrete time 0.. . 1 T tN =− , with 1... R j N = denoting the j -th time segment. Cite as: “RD Pascual-Marqui: Instantaneous and lagged measurements of linear and nonlinear dependence between groups of multivariate time series: freq uency decomposition. arXiv:0711. 1455 [stat.ME], 2007-November-09, http:/ /arxiv.org/abs/0711.1455 ” Page 8 of 18 The measures of linear dependence betw een the three multivariate time se ries are related in the usual way: Eq. 33 () () () ,, FF F ωω ω =+ XYZ X Y Z X Y Z  i i and are given by: Eq. 34 () ,, ln F ω ω ω ωω ω ωω ω ωω ω ω ⎛⎞ ⎜⎟ ⎜⎟ ⎜⎟ ⎝⎠ = ⎛⎞ ⎜⎟ ⎜⎟ ⎜⎟ ⎝⎠ YY XX ZZ XYZ YY YX YZ XY XX XZ ZY ZX ZZ S0 o 0S 0 00 S SS S SS S SS S Eq. 35 () Re ln Re F ω ω ω ωω ω ωω ω ωω ω ω ⎛⎞ ⎜⎟ ⎜⎟ ⎜⎟ ⎝⎠ = ⎛⎞ ⎜⎟ ⎜⎟ ⎜⎟ ⎝⎠ YY XX ZZ XYZ YY YX YZ XY XX XZ ZY ZX ZZ S0 o 0S 0 00 S SS S SS S SS S ii and: Eq. 36 () Re Re ln F ωω ω ω ωω ω ω ωω ω ω ωω ω ω ωω ω ω ωω ω ω ω ⎛⎞ ⎛ ⎞ ⎜⎟ ⎜ ⎟ ⎜⎟ ⎜ ⎟ ⎜⎟ ⎜ ⎟ ⎝⎠ ⎝ ⎠ = ⎧⎫ ⎛⎞ ⎛ ⎞ ⎪⎪ ⎜⎟ ⎜ ⎟ ⎨⎬ ⎜⎟ ⎜ ⎟ ⎪⎪ ⎜⎟ ⎜ ⎟ ⎝⎠ ⎝ ⎠ ⎩⎭ YY YX YZ YY XY XX XZ XX ZY ZX Z Z ZZ XYZ YY YX YZ YY XY XX XZ XX ZY ZX Z Z ZZ SS S S 0 o SS S 0 S 0 SS S 0 0 S SS S S 0 o SS S 0 S 0 SS S 0 0 S  Coherences for each type of measure of li near dependence in Eq. 33 are defined by the general relation (see e.g. Pierce 1982): Eq. 37 () () 2 1e x p F ρ ωω =− − ⎡⎤ ⎣⎦ As previously argued, under the assumption that the time series are wide-sense stationary, large sample distribution theory ca n be used to test the null hypothesis that a given measure of linear dependence is zero. In this case, the asymptotic distributions are: Eq. 38 () () ( ) () () ( ) () () ( ) 2 0, , , , 2 0 2 0 ˆ :0 , 2 2 2 ˆ :0 , ˆ :0 , a T a T a T Under H F N F pq pr qr Under H F N F pq pr qr Under H F N F pq pr qr ωω χ ωω χ ωω χ ⎧⎫ =+ + ⎪⎪ ⎪⎪ =+ + ⎨⎬ ⎪⎪ ⎪⎪ =+ + ⎩⎭ XYZ XYZ XYZ XYZ XYZ XY Z   ii ii ∼ ∼ ∼ The generalization of these definitions to a n y number of multivaria te time series is straightforward. It is important to emphasize here that these measures of linear dependence for groups of multivariate time series can be appl ied in the field of neurophysiology. In this Cite as: “RD Pascual-Marqui: Instantaneous and lagged measurements of linear and nonlinear dependence between groups of multivariate time series: freq uency decomposition. arXiv:0711. 1455 [stat.ME], 2007-November-09, http:/ /arxiv.org/abs/0711.1455 ” Page 9 of 18 case, the time series consist of electric neuron al activity at several brain locations, and the measures of dependence are interpreted as “connectivity” between locations. When considering several brain locations, these new me asures can be used to tes t for the existence of distributed cortical networks, whose activity can be estimated with exact low resolution brain electromagnetic tomography (Pascual-Marqui 2007b). 5. Measures of linear dependence (coherence-type) between all univariate time series A particular case of interest consists of measuring the linear dependence between all the univariate time series that form part of th e vector time series. For instance, consider the vector time series 1 p jt × ∈ X  . Then the measures of line ar dependence between all “ p ” univariate time series of X are: Eq. 39 () () () , FF F ωω ω =+ XX X X X X i Eq. 40 () () , ln Diag F ω ω ω = XX XX XX S S Eq. 41 () () () ln Re Diag F ω ω ω = XX XX XX S S i Eq. 42 () () () () , Re ln FF F ω ω ωω ω =− = XX XX X X X X XX S S i Coherences for each type of measure of li near dependence in Eq. 39 are defined by the general relation (see e.g. Pierce 1982): Eq. 43 () () 2 1e x p F ρ ωω =− − ⎡⎤ ⎣⎦ In Eq. 40 and Eq. 41, the notation () Di ag M denotes a diagonal matrix formed by the diagonal elements of M . Note that for Hermitian matrices, such as ω XX S , the diagonal elements are pure real, which implies that: Eq. 44 () () () () () Re Re Diag Diag Diag ωω ω == XX XX XX SS S As a consistency check, it can easily be verified that when these definitions are applied to a vector time series with 2 compon ents, the same results are obtained as in the case of two univariate time seri es (Eq. 21, Eq. 23, and Eq. 27). Under the assumption that the time series are wide-sense stationary, large s ample distribution theory can be used to test the null hypothesis that a given measure of linear dependence is zero. In this case, the asymptotic distributions are: Eq. 45 () () ( ) () () () ( ) () () () ( ) () 2 0, , 2 0 2 0 ˆ :0 , 1 ˆ :0 , 1 2 ˆ :0 , 1 2 a T a T a T Under H F N F p p Under H F N F p p Under H F N F p p ωω χ ωω χ ωω χ ⎧⎫ =− ⎪⎪ ⎪⎪ ⎪⎪ =− ⎨⎬ ⎪⎪ ⎪⎪ =− ⎪⎪ ⎩⎭ XX XX XX XX XX XX  ii ∼ ∼ ∼ Cite as: “RD Pascual-Marqui: Instantaneous and lagged measurements of linear and nonlinear dependence between groups of multivariate time series: freq uency decomposition. arXiv:0711. 1455 [stat.ME], 2007-November-09, http:/ /arxiv.org/abs/0711.1455 ” Page 10 of 18 As a further consistency check, note that the test () 0 :0 HF ω = XX i corresponds to the classical case of testing if a real-valued correlati on matrix is the identi ty matri x. The statistic given above is precisely the log-likelihood ratio statistic, which is asymptotically chi-square with the specified degrees of freedom ( Kullback 1967 ). 6. Measures of nonlinear depende nce (phase synchronization type) between two multivariate time series The term “phase synchronization” has a very rigorous physics definition (see e.g. Rosenblum et al 1996). The basic idea behind this definition has been adapted and used to great advantage in the neurosciences (Tass et al 1998, Quian-Quiroga et al 2002, Pereda et al 2005, Stam et al 2007), as in for example, the analysis of pairs of time series of measured scalp electric potentials differences (i.e. EEG: electroencephalogram). Other equivalent descriptive names for “phase synchronization” that appear in the neurosciences are phase locking, phase locking value, phase locking index, phase coherence, and so on. An informal definition for the statistical “phase synchronizatio n” model will now be giv en. In or de r to sim pl if y th is inf or ma l d efi nit ion ev en fur the r, it wil l b e a ssu me d th at the re are two univariate stationary time series (i. e. 1 pq == ) of interest. At a given discrete frequency ω , the sample data in the frequency domain (using the discrete Fourier transform) is denoted as , jj xy ωω ∈  , with 1 ... R j N = denoting the j -th time segment. If the phase difference x y jj j ϕϕ ϕ Δ= − is “stable” over time segments j , regardless of the amplitudes, th en there is a “connection” between the location s at which the meas urements were made. A measure of stability of phase difference is precis ely “phase synchronization”. It can as well be defined for the non-stationary case, using co ncepts of time-varying instantaneous phase, and defining stability over time (instead of stability over time segments). In the case of univariate time series, i.e. 1 pq == , phase synchronization can be viewed as the modulus (absolute value) of the complex valued (Hermitian) coherency between the normalized Fourier transforms. These variab les are normalized prior to the coherency calculation in order to remove from the outset any amplitude effect, leaving o nly phase information. This normalization operation is highly nonlinear. The modulus of the coherency is used as a measure for phase synchronization because it is conveniently bounded in the rang e zero (no synchronization) to one (perfect synchronization). Based on the foregoing arguments, a natural definition for the measures of nonlinear dependence (phase synchronization type) between two multivariate time se ries is exactly the same definitions as developed in the previous sections of this study, but applied to the phase-information cross-spectra (Eq. 7 to Eq. 12). The phase-information cross-spectra are based on normalized Fourier transform vectors, which is the particular requirement in this case (without amplitude information). Cite as: “RD Pascual-Marqui: Instantaneous and lagged measurements of linear and nonlinear dependence between groups of multivariate time series: freq uency decomposition. arXiv:0711. 1455 [stat.ME], 2007-November-09, http:/ /arxiv.org/abs/0711.1455 ” Page 11 of 18 For two multivariate time series, the measure of nonlinear dependence () , G ω XY is expressed as the sum of lagged nonlinear dependence () G ω XY  and instantaneous nonlinear dependence () G ω XY i : Eq. 46 () () () , GG G ωω ω =+ XY X Y X Y i with: Eq. 47 () , ln T G ω ω ωω ωω ω ⎛⎞ ⎜⎟ ⎝⎠ = ⎛⎞ ⎜⎟ ⎝⎠ YY XX XY YY YX XY XX S0 oS SS SS         Eq. 48 () Re ln Re T G ω ω ωω ωω ω ⎛⎞ ⎜⎟ ⎝⎠ = ⎛⎞ ⎜⎟ ⎝⎠ YY XX XY YY YX XY XX S0 oS SS SS   i       and: Eq. 49 () () () , Re Re ln T T GG G ωω ω ωω ω ωω ω ωω ω ωω ω ⎧ ⎫ ⎛⎞ ⎛ ⎞ ⎪ ⎪ ⎨ ⎬ ⎜⎟ ⎜ ⎟ ⎝⎠ ⎝ ⎠ ⎪ ⎪ ⎩⎭ =− = ⎧ ⎫ ⎛⎞ ⎛⎞ ⎪ ⎪ ⎨ ⎬ ⎜⎟ ⎜⎟ ⎝⎠ ⎝⎠ ⎪ ⎪ ⎩⎭ YY YX YY XY XX XX XY X Y X Y YY YX YY XY XX XX SS S 0 SS o S SS S 0 SS o S          i        In Eq. 47, Eq. 48, and Eq. 49, the Hermitia n covariance matrices are defined for the normalized discrete Fourier transform vectors (Eq. 7 to Eq. 12). All three measures are non-negative. They take the value zero only when there is independence of the pertinent type (lagged, instantaneous, or both). These measures of nonlinear dependence can be associated with measures phase synchronization ϕ as follows. The phase synchronization between two multivariate time series is: Eq. 50 () ( ) () 2 ,, 1e x p 1 T G ωω ωω ω ω ϕω ω ⎛⎞ ⎜⎟ ⎝⎠ =− − =− ⎛⎞ ⎜⎟ ⎝⎠ YY YX XY XX XY XY YY XX SS SS S0 oS         The instantaneous phase synchronization be tween two multivariate time series is: Eq. 51 () () () 2 Re 1e x p 1 Re T G ωω ωω ω ω ϕω ω ⎛⎞ ⎜⎟ ⎝⎠ =− − =− ⎛⎞ ⎜⎟ ⎝⎠ YY YX XY XX XY XY YY XX SS SS S0 oS       ii   The lagged phase synchronization between two multivariate time series is: Cite as: “RD Pascual-Marqui: Instantaneous and lagged measurements of linear and nonlinear dependence between groups of multivariate time series: freq uency decomposition. arXiv:0711. 1455 [stat.ME], 2007-November-09, http:/ /arxiv.org/abs/0711.1455 ” Page 12 of 18 Eq. 52 () () () 2 1e x p 1 Re Re T T G ωω ω ωω ω ωω ω ωω ω ϕω ω ⎧ ⎫ ⎛⎞ ⎛⎞ ⎪ ⎪ ⎨ ⎬ ⎜⎟ ⎜⎟ ⎝⎠ ⎝⎠ ⎪ ⎪ ⎩⎭ =− − =− ⎧ ⎫ ⎛⎞ ⎛ ⎞ ⎪ ⎪ ⎨ ⎬ ⎜⎟ ⎜ ⎟ ⎝⎠ ⎝ ⎠ ⎪ ⎪ ⎩⎭ YY YX YY XY XX XX XY XY YY YX YY XY XX XX SS S 0 SS o S SS S 0 SS o S                   The phase synchronization between two multivariate time series () 2 , ϕ ω XY given by Eq. 50 corresponds to the square of the “gener al phase synchronizatio n” previously defined in Pascual-Marqui (2007a; see Eq. 15 therein). In order to illustrate and further motivate these measures of nonlinear dependence, a detailed analysis for the simple case of two univariate time series is presented. In the case that the two time series are univariate, the measure of nonlinear dependence () , G ω XY in Eq. 47 is: Eq. 53 () () () () () 2 , , 22 1 ln ln 1 1R e I m xy xy G ss ωω ω ϕ ω == − − ⎡⎤ ⎡ ⎤ −− ⎣⎦ ⎣ ⎦ XY XY   with phase synchronization: Eq. 54 () () ( ) 2 22 2 * , 1 1 Re Im R N xy xy j j j R ss x y N ωω ω ω ϕω = ⎡⎤ ⎡ ⎤ =+ = ⎣⎦ ⎣ ⎦ ∑ XY    Note that by definition, du e to the normalization, 1 xx yy ss ωω ==    . In Eq. 54, , ϕ XY is the classical measure of phase synchronization. The measure of instantaneous nonlinear dependence is: Eq. 55 () () () () 2 2 1 ln ln 1 1R e xy G s ω ω ϕ ω == − − ⎡⎤ − ⎣⎦ XY XY ii  with instantaneous phase synchronization: Eq. 56 () () 2 2 Re xy s ω ϕω ⎡ ⎤ = ⎣ ⎦ XY  i which, not surprisingly, is directly related to the real part of the complex valued coherency of the normalized time series. Finally, in the particular case of univar iate time series, the measure of lagged nonlinear dependence is: Eq. 57 () () () () () () 2 2 22 1R e ln ln 1 1R e I m xy xy xy s G ss ω ωω ω ϕ ω ⎡⎤ − ⎣⎦ == − − ⎡⎤ ⎡ ⎤ −− ⎣⎦ ⎣ ⎦ XY XY      with lagged phase synchronization: Eq. 58 () () () 2 2 2 Im 1R e xy xy s s ω ω ϕω ⎡⎤ ⎣⎦ = ⎡⎤ − ⎣⎦ XY    The lagged phase synchronization between two univariate time series () ϕ ω XY  given by Eq. 58 corresponds to the “general lagged phase synchronization” (i.e. the “zero-lag Cite as: “RD Pascual-Marqui: Instantaneous and lagged measurements of linear and nonlinear dependence between groups of multivariate time series: freq uency decomposition. arXiv:0711. 1455 [stat.ME], 2007-November-09, http:/ /arxiv.org/abs/0711.1455 ” Page 13 of 18 removed” general phase synchronization)” previo usly defined in Pascual-Marqui (2007a), see Eq. 33 therein. It is worth stressing the asymmetry in the results for the instantaneous phase synchronization () 2 ϕ ω XY i (Eq. 56) and the lagged phase synchronization () 2 ϕ ω XY  (Eq. 58). While the instantaneous phase synchronization is the real part of the complex valued coherency for the normalized time seri es, the lagged phase synchronization is not the imaginary part. Ideally, the lagged phase synchroni zation is a measure that is less affected by instantaneous nonlinear dependence. In our previous related study (Pascual-Marqui 2007a), the definition given there for the “zero-lag removed general phase synchronization” (see Eq. 28 therein) was: Eq. 59 1 Re GL GL PS ωω ωω ωω ωω ρ ⎛⎞ ⎜⎟ ⎝⎠ == − ⎛⎞ ⎜⎟ ⎝⎠ YY YX XY XX YY YX XY XX SS SS SS SS             The new definition given here for the lagged phase synchronization () 2 ϕ ω XY  is given by Eq. 52. Both definitions (Eq. 52 and Eq. 59) are id entical for the case of two univariate time series. However, they are different for the mult ivariate case. Whereas the old definition in Eq. 59 lumps together all variables from X and Y, the new definition given here in Eq. 52 conserves the multivariate structure of the two multivariate time series. Th e improvement of the new lagged phase synchronization in Eq. 52 is that it measures the lagged nonlinear dependence between the two multivariate ti me series without being affected by the covariance structure within ea ch multivariate time series . The shortcoming of the old definition from our previous study (Pascual-Marqui 2007a), shown in Eq. 59, is that it is contaminated by the dependen c e s t r u c t u r e s o f t h e u n i v a r i a t e t i m e s e r i e s w i t h i n X and within Y . 7. Measures of nonlinear de pendence (phase synchronization type) between groups of multivariate time series Consider the case of three multivariate time series 1 p jt × ∈ X  , 1 q jt × ∈ Y  , and 1 r jt × ∈ Z  , for discrete time 0... 1 T tN =− , with 1... R j N = denoting the j -th time segment. The measures of nonlinear dependence between the three multivariate time series are related in the usual way: Eq. 60 () () () ,, GG G ωω ω =+ XYZ X Y Z X Y Z  i i and are given by: Cite as: “RD Pascual-Marqui: Instantaneous and lagged measurements of linear and nonlinear dependence between groups of multivariate time series: freq uency decomposition. arXiv:0711. 1455 [stat.ME], 2007-November-09, http:/ /arxiv.org/abs/0711.1455 ” Page 14 of 18 Eq. 61 () ,, ln G ω ω ω ωω ω ωω ω ωω ω ω ⎛⎞ ⎜⎟ ⎜⎟ ⎜⎟ ⎝⎠ = ⎛⎞ ⎜⎟ ⎜⎟ ⎜⎟ ⎝⎠ YY XX ZZ XYZ YY YX YZ XY XX XZ ZY ZX ZZ S0 o 0S 0 00 S SS S SS S SS S                Eq. 62 () Re ln Re G ω ω ω ωω ω ωω ω ωω ω ω ⎛⎞ ⎜⎟ ⎜⎟ ⎜⎟ ⎝⎠ = ⎛⎞ ⎜⎟ ⎜⎟ ⎜⎟ ⎝⎠ YY XX ZZ XYZ YY YX YZ XY XX XZ ZY ZX ZZ S0 o 0S 0 00 S SS S SS S SS S    ii             Eq. 63 () Re Re ln G ωω ωω ωω ωω ω ω ωω ω ω ωω ω ω ω ⎛⎞ ⎛⎞ ⎜⎟ ⎜⎟ ⎜⎟ ⎜⎟ ⎜⎟ ⎜⎟ ⎝⎠ ⎝⎠ = ⎧⎫ ⎛⎞ ⎛ ⎞ ⎪⎪ ⎜⎟ ⎜ ⎟ ⎨⎬ ⎜⎟ ⎜ ⎟ ⎪⎪ ⎜⎟ ⎜ ⎟ ⎝⎠ ⎝ ⎠ ⎩⎭ YY YY XX XX ZZ ZZ XYZ YY YX YZ Y Y XY XX XZ XX ZY ZX ZZ ZZ S0 o S0 o 0S 0 0S 0 00 S 00 S SS S S 0 o SS S 0 S 0 SS S 0 0 S                        Phase synchronization for each type of measur e of linear dependence in Eq. 60 can be defined by the general relation (see e.g. Pierce 1982): Eq. 64 () () 2 1e x p G ϕ ωω =− − ⎡⎤ ⎣⎦ The generalization of these definitions to a n y number of multivaria te time series is straightforward. It is important to emphasize here that th ese measures of nonlinear dependence for groups of multivariate time series can be a pplied in the field of neurophysiology. In this case, the time series consist of electric neuron al activity at several brain locations, and the measures of dependence are interpreted as “connectivity” between locations. When considering several brain locations, these new meas ures can be used to test for the existence of distributed cortical networks, whose activity can be estimated with exact low resolution brain electromagnetic tomography (Pascual-Marqui 2007b). 8. Measures of nonlinear dependence (phase synchronization type) between all univariate time series A particular case of interest consists of measuring the nonlinear dependence between all the univariate time series that form part of the vector time series. For instance, consider the vector time series 1 p jt × ∈ X  . In this case, since each univariate time series on its own is Cite as: “RD Pascual-Marqui: Instantaneous and lagged measurements of linear and nonlinear dependence between groups of multivariate time series: freq uency decomposition. arXiv:0711. 1455 [stat.ME], 2007-November-09, http:/ /arxiv.org/abs/0711.1455 ” Page 15 of 18 of interest, each one must be normalized. For this particular purpose we adopt the definition: Eq. 65 () 12 * jj j j Diag ωω ω ω − ⎡⎤ = ⎣⎦ XX X X  which normalizes each variable. The corresponding covariance matrix is: Eq. 66 * 1 1 R N jj j R N ωω ω = = ∑ XX SX X    Then the measures of nonlin ear dependence between all “ p ” univariate time series of X are: Eq. 67 () () () , GG G ωω ω =+ XX X X X X i Eq. 68 () , ln G ω ω =− XX XX S   Eq. 69 () () ln Re G ω ω =− XX XX S  i Eq. 70 () () () () , Re ln GG G ω ω ωω ω =− = XX XX X X X X XX S S  i  Phase synchronization for each type of measur e of linear dependence in Eq. 67 can be defined by the general relation (see e.g. Pierce 1982): Eq. 71 () () 2 1e x p G ϕ ωω =− − ⎡⎤ ⎣⎦ As a consistency check, it can easily be verified that when these definitions are applied to a vector time series with 2 compon ents, the same results are obtained as in the case of two univariate time seri es (Eq. 53, Eq. 55, and Eq. 57). 9. Conclusions 1. Previous related work (Pascual-Marqu i 2007a) was limited to measures of dependence between two multivar iate time series. This study generalizes the definitions to include measures of dependence between an y number of multivariate time series. 2. Previous measures for lagged dependence between two vector time series ( Pascual- Marqui 2007a) were inadequately affected by the dependence structure of the univariate time series within each vector time seri es. This study adequately partials out the dependence structures within each vector time series. 3. A new measure for instantaneous linear and non-linear dependence is introduced. 4. The measures of dependence introduced here have been developed for discrete frequency components. However, they can as we ll be applied to any fr equency band, defined as a set of discrete frequencies (which can ev en be disjoint). In this case, the Hermitian covariance matrices to be used in the equati ons for the measures of dependence should now correspond to the pooled matrices (i.e. the aver age Hermitian covariance over all discrete frequencies in the set defi ning the frequency band). 5. Inference methods for the measures of linear dependence are described. Cite as: “RD Pascual-Marqui: Instantaneous and lagged measurements of linear and nonlinear dependence between groups of multivariate time series: freq uency decomposition. arXiv:0711. 1455 [stat.ME], 2007-November-09, http:/ /arxiv.org/abs/0711.1455 ” Page 16 of 18 6 . A l l t h e m e a s u r e s o f d e p e n d e n c e c a n b e b a s e d o n a n y f o r m o f t i m e - v a r y i n g F o u r i e r transforms or wavelets, such as, for instance, Gabor or Morlet transforms. 7. The new measures of dependence between any number of multivariate time series can be applied to the study of brain electr ical activity, which can be estimated non- invasively from EEG/MEG recordings with methods such as eLORETA (Pascual-Marqui 2007b). When considering several brain locations jointly, these new measures can be used to test for the existence of dist ributed cortical networks. Prev ious methodology explores the connections between all possible pairs of loca tions, while the new “network approach” can test the joint dependence of several locations. Appendix 1: Zero-lag contribu tion to coherence and phase synchronization: problem description I n s o m e f i e l d s o f a p p l i c a t i o n , t h e c o h e rence or phase synchronization between two time series corresponding to two different spatia l locations is interpreted as a measure of the “connectivity” between those two locations. For example, consider the time series of scalp electric potential differences (EEG: electroencephalogram) at two locations. Th e coherence or phase synchronization is interpreted by some researchers as a measur e of “connectivity” between the underlying cortices (see e.g. Nolte et al 2004 and Stam et al 2007). However, even if the underlying cortices are not actually connected, significantly high coherence or phase synchronization migh t still occur due to the volume conduction effect: activity at any cortical area will be ob served instantaneously (zero-lag) by all scalp electrodes. As a possible solution to this problem, th e electric neuronal activity distributed throughout the cortex can be estimated from the EEG by using imaging techniques such as standardized or exact low resolution br ain electromagnetic tomography (sLORETA, eLORETA) (Pascual-Marqui et al 2002; Pascual- Marqui 2007b). At each voxel in the cortical grey matter, a 3-component vector time series is computed, corresponding to the current density vector with dipole moments along axes X , Y , and Z . This tomography has the unique properties of being linear, of having zero localization error, but of having low spatial resolution. Due to such spatial “blurring”, the time series will again suffer from non- physiological inflated values of zero-lag coherence and phase synchronization. Formally, consider two different spatial locat ions where there is no actual activity. However, due to a third truly active location, and because of low spatial resolution (or volume conductor type effect), there is so me measured activity at these locations: Eq. 72: x jt jt jt y jt jt jt ⎧⎫ =+ ⎪⎪ ⎨⎬ =+ ⎪⎪ ⎩⎭ XC Z YD Z ε ε Cite as: “RD Pascual-Marqui: Instantaneous and lagged measurements of linear and nonlinear dependence between groups of multivariate time series: freq uency decomposition. arXiv:0711. 1455 [stat.ME], 2007-November-09, http:/ /arxiv.org/abs/0711.1455 ” Page 17 of 18 where jt Z is the time series of the trul y active location; C and D are matrices determined by the properties of the low spatial resolution problem; and x jt ε and y jt ε are independent and identically distributed random white noise. In this model, although X and Y are not “connected”, coherence and phase synchronization will indicate some connect ion, due to zero-lag spatial blurring. Things can get even worse due to the zero-lag effect. Suppose that two time series are measured under two different conditions in whic h the zero-lag blurring effect is constant. The goal is to perform a statisti cal test to compare if there is a change in connectivity. Since the zero-lag effect is the same in both condit ions, then it should seemingly not account for any significant difference in coherence or phas e synchronization. However, this migh t be very misleading. In the model in Eq. 72, a simple increase in the signal to noise ratio (e.g. by increasing the norms of C and D ) will produce an increase in coherence and phase synchronization, due again to the zero-lag effect. This example shows that the zero-lag effect can render meaningless a comparison of two or more conditions. Acknowledgements I have had extremely useful discussions with G. Nolte, who pointed out a number of embarrassing inconsistencies I wrote into the first draft of the previous related technical report (Pascual-Marqui 2007a). Those disc ussions partly motivated the new methods developed in this study. References C Allefeld, J Kurths (2004); Testing for ph ase synchronization. International Journal of Bifurcation and Chaos, 14: 405-416. DR Brillinger (1981): Time series: data an alysis and theory. McGraw-Hill, New York. J Geweke (1982): Measurement of Line ar Dependence and Feedback Between Multiple Time Series. Journal of the Americ an Statistical Association, 77: 304-313. H Hotelling (1936):Relations between two sets of variables. Biometrika 28: 321-377. JT Kent (1983): Information gain an d a general measure of correlation. Biometrika, 70: 163-173. S Kullback (1967): On Testing Correlation Matrices. Applied Statistics, 16: 80-85. BFJ Manly (1997): Randomization, bootst rap and MonteCarlo methods in biology. Chapman & Hall, London. KV Mardia, JT Kent, JM Bibby (1979): Multivariate Analysis. Academic Press, London. TE Nichols, AP Holmes (2001): Nonparametric permutation tests for functional neuroimaging: a primer with exampl es. Human Brain Mapping, 15: 1-25. G Nolte, O Bai, L Wheaton, Z Mari, S Vo rbach, M Hallett (2004): Identifying true brain interaction from EEG data using the imag inary part of coherenc y. Clin Neurophysiol., 115: 2292-2307. RD Pascual-Marqui (2002): Standardized low resolution brain electromagnetic tomography (sLORETA): technical details. Methods & Findings in Experime ntal & Clinical Pharmacology, 24D: 5-12. Cite as: “RD Pascual-Marqui: Instantaneous and lagged measurements of linear and nonlinear dependence between groups of multivariate time series: freq uency decomposition. arXiv:0711. 1455 [stat.ME], 2007-November-09, http:/ /arxiv.org/abs/0711.1455 ” Page 18 of 18 RD Pascual-Marqui (2007a): Coherence and phase synchronization: generalization to pairs of multivariate time series, and removal RD Pascual-Marqui (2007b): Discrete, 3D di stributed, linear imaging methods of electric neuronal activity. Part 1: exact, zero 2007-October-17, http://arxi v.org/abs/0710.3341 . E Pereda, R Quian-Quiroga, J Bhattacharya (2 005): Nonlinear multivariate analysis of neurophysiological signals. Prog Neurobiol., 77: 1-37. DA Pierce (1982): Comment on J Geweke’s “Measurement of Linear Dependence and Feedback Between Multiple Time Series”. Journa l of the American Statis tical Association, 77: 315-316. R Quian Quiroga, A Kraskov, T Kreuz, P Gr assberger (2002): Performance of different synchronization measures in real data: a case study on electroencep halographic signals. Phys. Rev. E, 65: 041903. MG Rosenblum, AS Pikovsky, J Kurths(1 996): Phase synchronization of chaotic oscillators. Phys. Rev. Lett., 76: 1804-1807. CJ Stam, G Nolte, A Daffertshofer (2007): Ph ase lag index: Assessment of functional connectivity from multi channel EEG and MEG wi th diminished bias from common sources. Human Brain Mapping, 28: 1178-1193. P Tass, MG Rosenblum, J Weule, J Kurths, A Pikovsky, J Volkmann, A Schnitzler, HJ Freund (1998): Detection of n:m phase locking from noisy data: application to magnetoencephalography. Phys. Rev. Lett., 81: 3291-3294.