Additive Regression Model for Continuous Time Processes

ADDITIVE REGRESSION MODEL FOR CONTINUOUS TIME PR OCESSES Mohammed DEBBARH and Bertrand MAILLOT Univ ersit ´ e Paris 6 175, Rue du Chev aleret, 7501 3 P aris. debbarh@ccr.jussieu.fr. Key W ords: Additiv e regression; Con tin uous time pro cesses; Curse of dimensionalit y; Marginal in tegration. ABSTRA CT In the setting of additiv e regression mo del for con tinuous time process, w e establish the optimal uniform conv ergence ra t es and optimal asymptotic quadratic erro r of additiv e regression. T o build our estimate, we use the marginal integration metho d. 1 In tro duction and motiv ations The m ultiv ariate regression function estimation is an imp ortant problem whic h has b een extensiv ely treated for disc rete time pro cesses . It is w ell-kno wn from (11) that the additiv e regression mo dels bring out a solution to the problem of the curse of dimensionalit y in non- parametric mu ltiv a riate regression estimation, whic h is characterized b y a loss in t he rate of conv ergence of the regression function estimator when the dimension of the co v ariates increases. Additiv e mo dels allow to reac h ev en univ ar ia te rate when these mo dels fit w ell. F or contin uous time pro cesses, (2) obtained the optimal rate for the estimator of m ultiv ariate regression, whic h is the same as in the i.i.d. case. He ev en prov ed that, for pro cesses with irregular paths, it is p ossible to reac h the p ar am e tric r ate . This one, called the sup eroptimal rate, do es not dep end on the dimension of the v ariables, but the needed conditions on the pro cesses are v ery strong. That is the reason why it is relev ant to study additiv e mo dels to bring out a solution to the problem of the curse of dimensionalit y . Let Z t = ( X t , Y t ) , ( t ∈ R ) b e a R d × R -v a lued measurable sto c hastic pro cess define d on a probabilit y space (Ω , A , P ). Denote by ψ a giv en real measurable function. W e consider the additiv e regression function asso ciated to m ψ ( Y ) defined by , m ψ ( x ) = E ( ψ ( Y ) | X = x ) , ∀ x = ( x 1 , ..., x d ) ∈ R d , (1) 1 = µ + d X l =1 m l ( x l ) := m ψ, add ( x ) . (2) Let K 1 , K 2 , K 3 and K , b e k ernels resp ectiv ely defined on R , R d − 1 , R d and R d . W e denote b y ˆ f T the estimate of f , the densit y function o f the cov ariable X , (see (1)), that is, ˆ f T ( x ) = 1 T h d T Z T 0 K  x − X s h T  ds, where ( h T ) is a p ositiv e real function. In estimating the regression function define d in (1), w e use the follo wing tw o estimators ( see for exemple (3) and (5)) e m ψ, T ( x ) = Z T 0 W T ,t ( x ) ψ ( Y t ) dt with W T ,t ( x ) = K 3 ( x − X t h 1 ,T ) T h d 1 ,T ˆ f T ( X t ) , (3) and e m ψ, T ,l ( x ) := Z T 0 W l T ,t ( x ) ψ ( Y t ) dt with W l T ,t ( x ) = K 1 ( x l − X t,l h 1 ,T ) K 2 ( x − l − X t, − l h 2 ,T ) T h 1 ,T h d − 1 2 ,T ˆ f T ( X t ) , (4) where ( h j,T ) , j = 1 , 2 are p ositiv e real functions. Let q 1 , ..., q d b e d densit y functions defined in R . Setting q ( x ) = Q d l =1 q l ( x l ) and q − l ( x − l ) = Q j 6 = l q j ( x j ). T o estimate the a dditiv e comp onen ts of the regression function, we use the marginal integration metho d (see ( 6) a nd (8)). W e obt a in then η l ( x l ) = Z R d − 1 m ψ ( x ) q − l ( x − l ) d x − l − Z R d m ψ ( x ) q ( x ) d x , l = 1 , ..., d, (5) in suc h a w a y that the following tw o equalities hold, η l ( x l ) = m l ( x l ) − Z R m l ( z ) q l ( z ) d z , l = 1 , ..., d, (6) m ψ ( x ) = d X l =1 η l ( x l ) + Z R d m ψ ( z ) q ( z ) d z . (7) In view of (6) and ( 7 ), w e note that η l and m l are equal up to a n additional constan t. Therefore, η l is also an additive comp onen t, fulfilling a different iden tifiability condition. F rom (4) and (5), a natur a l estimate of this l -th comp onen t is give n b y b η l ( x l ) = Z R d − 1 e m ψ, T ,l ( x ) q − l ( x − l ) d x − l − Z R d e m ψ, T ,l ( x ) q ( x ) d x , l = 1 , ..., d, (8) 2 from which we deduce the estimate b m ψ, T ,add of t he additiv e regression function, b m ψ, T ,add ( x ) = d X l =1 b η l ( x l ) + Z R d e m ψ, T ( x ) q ( x ) d x . (9) Before stating o ur r esults, we in tro duce some additional notations a nd our assumptions. Let C 1 , ..., C d , b e d compact in terv als of R and set C = C 1 × ... × C d . F or ev ery sub- set E of R q , q ≥ 1, and a ny δ > 0, in tro duce the δ - neigh b orho o d E δ of E , namely , E δ = { x : inf y ∈E k x − y k R q < δ } , with k · k R q standing for the euclidian norm on R q . (C.1) There exists a p ositiv e constan t M suc h that | ψ ( y ) | ≤ M < ∞ . (C.2) The function m ψ is k -times contin uously differen tiable, k ≥ 1 , and sup x    ∂ k m ψ ∂ x k ℓ ( x )    < ∞ ; ℓ = 1 , ..., d. Denote b y f ℓ , ℓ = 1 , ..., d the densit y f unctions of X ℓ , ℓ = 1 , ..., d . The f unctions f and f ℓ , ℓ = 1 , ..., d, are supp osed to b e con tin uous, b ounded and ( F . 1) ∀ x ∈ C δ , f ( x ) > 0 and f ℓ ( x ℓ ) > 0 , ℓ = 1 , ..., d . ( F . 2) f is k ′ -times con tin uously differentiable on C δ , k ′ > k d. ( F . 3) F or all 0 < λ ≤ 1 ,    ∂ f ( k ) ∂ x j 1 1 ...∂ j d d ( x ′ ) − ∂ f ( k ) ∂ x j 1 1 ...∂ j d d ( x )    ≤ L k x ′ − x k λ with j 1 + ... + j d = k ′ . Where k . k is a norm o n R d and L is a p ositiv e constan t. The k ernels K 1 , K 2 , K 3 and K are assumed to fulfill the following conditions (K.1) K 1 , K 2 , K 3 and K a re con tin uous resp ectiv ely on the compact supp orts S 1 ⊂ C 1 , S 2 ⊂ C 2 × ... × C d , S 3 ⊂ C and S , (K.2) R K = 1 and R K j = 1 , j = 1 , 2 , 3 , (K.3) K 1 , K 2 and K 3 are of order k , (K.4) K is of order k ′ . 3 (K.5) K 1 is a Lipsc hitz function. The densit y functions q ℓ , ℓ = 1 , ..., d , satisfy the following assumption ( Q. 1) F or an y 1 ≤ l ≤ d , q ℓ has k con tin uous and b ounded deriv ativ es, with a com- pact supp ort included in C ℓ . There exists a set Γ ∈ B R 2 con taining D = { ( s, t ) ∈ R 2 : s = t } suc h that ( D . 1) f ( X s ,Y s ) , ( X t ,Y t ) − f ( X s ,Y s ) N f ( X t ,Y t ) exists every where for ( s, t ) ∈ Γ C , ( D . 2) A f (Γ) := sup ( s,t ) ∈ Γ C sup x , y ∈C δ ×C δ R u,v ∈ R 2 | f ( X s ,Y s ) , ( X t ,Y t ) ( x , u, y , v ) − f ( X s ,Y s ) ( x , u ) f ( X t ,Y t ) ( y , t ) | dudv < ∞ , ( D . 3) there exists ℓ Γ < ∞ and T 0 suc h that , ∀ T > T 0 , 1 T R [0 ,T ] 2 ∩ Γ dsdt ≤ ℓ Γ . W e w ork under the follow ing conditions up on the smo othing para meters h T and h j,T , j = 1 , 2, ( H . 1) h T = c ′  log T T  1 / (2 k ′ + d ) , for a fixed 0 < c ′ < ∞ , ( H . 2) h 1 ,T = c 1 T − 1 / (2 k +1) and h 2 ,T = c 2 T − 1 / (2 k +1) , for fixed 0 < c 1 , c 2 < ∞ , ( H . 2) ′ h 1 ,T = c 1 ( log( T ) T ) 1 / (2 k +1) and h 2 ,T = c 2 ( log( T ) T ) 1 / (2 k +1) , for fixed 0 < c 1 , c 2 < ∞ . Throughout this w ork, w e use the α -mixing dep endance structure where the asso ciated co efficien t is defined, for ev ery σ -fields A and B b y α ( A , B ) = sup ( A,B ) ∈ ( A , B ) | P ( A ∩ B ) − P ( A ) P ( B ) | . F or all Borelian set I in R + the σ -algebra defined b y ( Z t , t ∈ I ) is denoted b y σ ( Z t , t ∈ I ). W riting α ( u ) = sup t ∈ R + α ( σ ( Z v , v ≤ t ) , σ ( Z v , v ≥ t + u )), we use the condition ( A. 1) α ( t ) = O ( t − b ) with b > 2 d + 10 + 6+4 d k . 4 Theorem 1 Under the c on d itions ( A. 1) , ( C . 1) − ( C . 2) , ( F . 1) − ( F . 3 ) , ( K . 1) − ( K . 4) , ( Q. 1) , ( D . 1) − ( D . 3) and ( H . 1) − ( H . 2) , we have , for al l x ∈ C δ E ( b m ψ, T ,add ( x ) − m ψ ( x )) 2 = O ( T − 2 k / 2 k +1 ) . Theorem 2 Under the c on d itions ( A. 1) , ( C . 1) − ( C . 2) , ( F . 1) − ( F . 3 ) , ( K . 1) − ( K . 5) , ( Q. 1) , ( D . 1) − ( D . 3) and ( H . 1) − ( H . 2) ′ , we have sup x ∈C | b m ψ, T ,add ( x ) − m ψ ( x ) | = O   log T T  k / 2 k +1  a.s. 2 Pro ofs The pro o fs of our theorems are split into t w o steps. First, w e consider the case where the densit y is assumed to b e known. Subsequen tly , we tr eat the g eneral case when f is unkno wn. Denote b y ˆ ˆ η , e e m ψ, T ( x ) and e e m ψ, T ,l ( x ) the v ersions of ˆ η , e m ψ, T ( x ) and e m ψ, T ,l ( x ) associated t o a kno wn (formally , we replace ˆ f T b y f in the expressions (3) , (4) and e m ψ, T ,l ( x ) by e e m ψ, T ,l ( x ) in (8). In tro duce now the following quantities (see, for the discrete case (4)), w e establish the pro o f for the first comp onen t, b b m ψ, T ,add ( x ) = d X l =1 b b η l ( x l ) + Z R d e e m ψ, T ( x ) q ( x ) d x . (10) e Y ψ, T ,t = ψ ( Y t ) Z R d − 1 1 h d − 1 2 ,T K 2  x − 1 − X t, − 1 h 2 ,T  q − 1 ( x − 1 ) f ( X t, − 1 | X t, 1 ) d x − 1 , (11) G ( u − 1 ) = Z R d − 1 1 h d − 1 2 ,T K 2  x − 1 − u − 1 h 2 ,T  q − 1 ( x − 1 ) d x − 1 , (12) b α 1 ( x 1 ) = 1 T h 1 ,T Z T 0 e Y ψ, T ,t f 1 ( X t, 1 ) K 1  x 1 − X t, 1 h 1 ,T  dt, for x 1 ∈ C 1 , (13) e m T ( x 1 ) = E ( e Y ψ, T ,t    X t, 1 = x 1 ) , (14) C T = µ + Z R d − 1 d X j =2 m j ( u j ) G ( u − 1 ) d u − 1 , (15) b C T = Z R d e e m ψ, T , 1 ( x ) q ( x ) d x , (16) C = Z R m 1 ( x 1 ) q 1 ( x 1 ) dx 1 . (17) 5 The f o llo wing Lemma is of particular in terest to establish the result of theorem (1). Note that ( 19) is “ only” b e instrumen tal in the pro of of (2 0). Lemma 1 Under the as s umptions ( C . 1) − ( C . 2) , ( F . 1) − ( F . 2) , ( K . 1) , ( Q. 1) and ( H . 2) , w e have E ( ˆ C T − C T + C ) 2 = O  T − 2 k / (2 k +1)  , (18) V ar( ˆ α 1 ( x 1 )) = O  T − 2 k / (2 k +1)  , (19) E ( b b η 1 ( x 1 ) − η 1 ( x 1 )) 2 = O  T − 2 k / (2 k +1)  . (20) Pr o of: According to F ubini’s Theorem and under the additive mo del assumption, we ha v e E ( ˆ C T − C T ) = E n Z R d e e m ψ, T , 1 ( x ) q ( x ) d x − µ − Z R d − 1 d X j =2 m j ( u j ) G ( u − 1 ) d u − 1 o = Z R d E ( e e m ψ, T , 1 ( x )) q ( x ) d x − µ − Z R d − 1 d X j =2 m j ( u j ) G ( u − 1 ) d u − 1 = Z R d 1 h 1 ,t m ψ ( u ) G ( u − 1 ) Z R K 1  x 1 − u 1 h 1 ,T  q 1 ( x 1 ) dx 1 d u − µ − Z R d − 1 d X j =2 m j ( u j ) G ( u − 1 ) d u − 1 = d X j =1 Z R d 1 h 1 ,T m j ( u j ) G ( u − 1 ) Z R K 1  x 1 − u 1 h 1 ,T  q 1 ( x 1 ) dx 1 d u − Z R d − 1 d X j =2 m j ( u j ) G ( u − 1 ) d u − 1 = Z R Z R 1 h 1 ,T m 1 ( u 1 ) K 1  x 1 − u 1 h 1 ,T  q 1 ( x 1 ) dx 1 du 1 . Setting v 1 h 1 ,T = x 1 − u 1 and using a T a ylor expansion, w e get, b y ( C . 2) and ( K . 1) − ( K. 3), E ( ˆ C T − C T ) − C = Z R Z R q 1 ( x 1 ) m 1 ( x 1 − h 1 ,T v 1 ) K 1 ( v 1 ) dv 1 dx 1 − C = Z R Z R q 1 ( x 1 )[ m 1 ( x 1 − h 1 ,T v 1 ) − m 1 ( x 1 )] K 1 ( v 1 ) dv 1 dx 1 = Z R Z R q 1 ( x 1 ) h ( − h 1 ,T ) k v k 1 k ! m ( k ) 1 ( x 1 ) i K 1 ( v 1 ) dv 1 dx 1 (21) + o ( h k 1 ,T ) . 6 Under ( H . 2), it follo ws that, h E  ˆ C T − C T − C i 2 = O ( T − 2 k / (2 k +1) ) . (22) The F ubini’s theorem giv es us V ar( ˆ C T ) = 1 ( T h 1 ,T ) 2 V ar  Z T 0 ψ ( Y t ) f ( X t ) G ( X t, − 1 ) Z R K 1  x 1 − X t, 1 h 1 ,T  q 1 ( x 1 ) dx 1 dt  = 1 ( T h 1 ,T ) 2 Z t,s ∈ [0; T ] 2 Co v  ψ ( Y t ) f ( X t ) G ( X t, − 1 ) Z R K 1  x 1 − X t, 1 h 1 ,T  q 1 ( x 1 ) dx 1 ; ψ ( Y s ) f ( X s ) G ( X s, − 1 ) Z R K 1  y 1 − X s, 1 h 1 ,T  q 1 ( y 1 ) dy 1  dsdt. (23) Under ( C . 1), ( F . 1), ( K . 1) − ( K . 2) and ( Q. 1), there exists a finite constan t M 3 suc h that, for T large enough, inf  a : P  ψ ( Y t ) f ( X t ) G ( X t, − 1 ) Z R K 1  x 1 − X t, 1 h 1 ,T  q 1 ( x 1 ) dx 1 > a  = 0 o ≤ h 1 ,T M 3 . Th us, using the Billingsley’s inequalit y and the condition ( A. 1 ) , V ar( ˆ C T ) ≤ 8 M 2 3 T 2 Z s ∈ [0; T ] Z t ∈ [0 ,T − s ] α t dtds = O  1 T  . (24) Finally , b y com bining the statemen ts (22) a nd (24), w e o bta in (18 ). Pr o of o f (19). Recalling (13), w e hav e V ar( ˆ α 1 ( x 1 )) = 1 T 2 h Z [0 ,T ] 2 ∩ Γ Co v  e Y ψ, T ,t f 1 ( X t, 1 ) h 1 ,T K 1  x 1 − X t, 1 h 1 ,T  , e Y ψ, T ,s f 1 ( X s, 1 ) h 1 ,T K 1  x 1 − X s, 1 h 1 ,T  dsdt + Z [0 ,T ] 2 ∩ Γ c Co v  e Y ψ, T ,t f 1 ( X i, 1 ) h 1 ,T K 1  x 1 − X t, 1 h 1 ,T  , e Y ψ, T ,s f 1 ( X s, 1 ) h 1 ,T K 1  x 1 − X s, 1 h 1 ,T  dsdt i := A + B . (25) F or the first term, noting that, under ( C. 1) , ( F . 1), ( K. 1 ) − ( K . 2) and ( Q. 1), there ex ists a finite constan t M 4 suc h that, for T large enough, M 4 ≥ inf  a : P  e Y 2 ψ, T , 0 f 1 ( X 0 , 1 ) 2    K 1  x 1 − X 0 , 1 h 1 ,T     > a  = 0  . 7 Th us, w e hav e A ≤ 1 T 2 h 2 1 ,T Z [0 ,T ] 2 ∩ Γ E e Y ψ, T , 0 f 1 ( X 0 , 1 ) K 1  x 1 − X 0 , 1 h 1 ,T  ! 2 dsdt ≤ M 4 T 2 h 2 1 ,T Z [0 ,T ] 2 ∩ Γ Z R    K 1  x 1 − u h 1 ,T     | f 1 ( u ) | du dsdt ≤ M 4 k f k ∞ l Γ T h 1 ,T Z R | K 1 ( v ) | dv = O  1 T h 1 ,T  . (26) T o treat the second term, w e introduce the set S a ( T ) = { ( s, t ) ∈ R 2 ; | t − s | ≤ a ( T ) } , where a ( T ) = h − 1 T , we hav e B = 1 T 2 Z [0 ,T ] 2 ∩ Γ c ∩ S a ( T ) Co v  e Y ψ, T ,t f 1 ( X t, 1 ) h 1 ,T K 1  x 1 − X t h 1 ,T  , e Y ψ, T ,s f 1 ( X s, 1 ) h 1 ,T K 1  x 1 − X s h 1 ,T  dsdt + 1 T 2 Z [0 ,T ] 2 ∩ Γ c ∩ S c a ( T ) Co v  e Y ψ, T ,t f 1 ( X t, 1 ) h 1 ,T K 1  x 1 − X t h 1 ,T  , e Y T ,s f 1 ( X s, 1 ) h 1 ,T K 1  x 1 − X s h 1 ,T  dsdt := E + F . (27) Under the conditions ( C. 1 ), ( F . 1), ( K . 1) − ( K . 2) and ( Q. 1), there exis ts a constan t M 5 suc h that, f or T large enough, sup z 1 ∈ R Z ( y, z − 1 ) ∈ R × R d − 1     ψ ( y ) G ( z − 1 ) f ( z ) 1 S 1  x 1 − z 1 h 1 ,T      dy d z − 1 ≤ M 5 . Consider no w the term E , w e hav e E = 1 T 2 h 2 1 ,T Z ( s,t ) ∈ [0 ,T ] 2 ∩ Γ c ∩ S a ( T ) Z ( u, v ) ∈ R × R d Z ( y, z ) ∈ R × R d ψ ( y ) G ( z − 1 ) f ( z ) K 1  x 1 − z 1 h 1 ,T  ψ ( u ) G ( v − 1 ) f ( v ) K 1  x 1 − v 1 h 1 ,T  f Y t , X t ,Y s , X s ( u, v , y , z ) − f Y t ,bf X t ( u, v ) f Y s ,bf X s ( y , z )  dy d z dud v dsd t ≤ 2 a ( T ) M 2 5 k K 1 k 2 L 1 A f (Γ) T . (28) 8 Noting that, under the conditions ( C . 1), ( F . 1), ( K . 1) − ( K. 2) and ( Q. 1), there exists a finite constan t M 6 suc h that, for T large enough, M 6 ≥ inf  a : P  e Y ψ, T , 0 f 1 ( X 0 , 1 )    K 1  x 1 − X 0 , 1 h 1 ,T     > a  = 0  . Using the Billingsley’s inequalit y , it follow s that F ≤ 2 T 2 h 2 1 ,T Z [0 ,T ] 2 ∩ Γ c ∩ S c a ( T ) ∩{ u>v } 4 M 2 6 α ( u − v ) d udv ≤ 8 M 2 6 T h 2 1 ,T Z { t>a ( T ) } α ( t ) dt ≤ 8 M 2 6 T h 2 1 ,T La ( T ) − 1 . (29) Finally , com bining the hypothesis ( H . 2) and the statemen ts (25)and (2 9 ), we obta in (19 ) . Pr o of o f (20) . W e hav e E ( ˆ α 1 ( x 1 )) − e m T ( x 1 ) = Z R 1 h 1 ,T e m T ( u 1 ) K 1  x 1 − u 1 h 1 ,T  du 1 − e m T ( x 1 ) = Z R [ e m T ( x 1 − v 1 h 1 ,T ) − e m T ( x 1 )] K 1 ( v 1 ) dv 1 = Z R Z R d − 1 [ m ψ ( x 1 − v 1 h 1 ,T , u − 1 ) − m ψ ( x 1 , u − 1 )] G ( u − 1 ) d u − 1 K 1 ( v 1 ) dv 1 = Z R Z R d − 1  ( − h 1 ,T v 1 ) k k ! ∂ k m ψ ∂ v k 1 ( v 1 , u − 1 )  G ( u − 1 ) d u − 1 K 1 ( v 1 ) dv 1 + o ( h k 1 ,T ) . Under t he condition ( H . 2), w e obtain h E  ˆ α 1 ( x 1 ) − e m T ( x 1 ) i 2 = O  T − 2 k / (2 k +1)  . (30) Th us, w e hav e E  ˆ α 1 ( x 1 ) − e m T ( x 1 )  2 = h E  ˆ α 1 ( x 1 ) − e m T ( x 1 ) i 2 + V ar( ˆ α 1 ( x 1 )) . (31) Consequen tly , b y com bining t he fo llo wing inequalit y E ( b b η 1 ( x 1 ) − η 1 ( x 1 )) 2 ≤ 2 E  ˆ α 1 ( x 1 ) − e m T ( x 1 )  2 + 2 E ( ˆ C T − C T − C ) 2 , (32) and the statemen ts (30), (31), (19) and (32), the pro of of (20) is readily ac hiev ed. 9 2.1 Pro of of Theorem 1 Using the classical inequalit y ( a + b ) 2 ≤ 2( a 2 + b 2 ), if follo ws that, for all x ∈ C , E ( b m ψ, T ,add ( x ) − m ψ ( x )) 2 ≤ 2 E ( b b m ψ, T ,add ( x ) − m ψ ( x )) 2 + 2 E ( b m ψ, T ,add ( x ) − b b m ψ, T ,add ( x )) 2 := I 1 ( x ) + I 2 ( x ) . (33) First, consider the term I 1 , w e ha v e I 1 ( x ) = 2 E ( b b m ψ, T ,add ( x ) − m ψ ( x )) 2 ≤ 4 d d X ℓ =1 E ( b b η ℓ ( x ℓ ) − η ℓ ( x ℓ )) 2 + 4 E h Z R d ( e e m ψ, T ( x ) − m ψ, T ( x )) q ( x ) d x i 2 . (34) Arguing as in proo f o f Lemma (1), w e obt a in b C T − C T − C = Z R d e e m ψ, T ( x ) q ( x ) d x − µ − Z R d − 1 d X j =2 m j ( u j ) G ( u − 1 ) d u − 1 − Z R m 1 ( x 1 ) q 1 ( x 1 ) dx 1 , = Z R d e e m ψ, T ( x ) q ( x ) d x − µ − Z R d − 1 d X j =1 m j ( u j ) q ( u ) d u + O  h k 1 ,T  , = Z R d ( e e m ψ, T ( x ) − m ψ, T ( x )) q ( x ) d x + O  T − k / (2 k +1)  . It follow s that, E h Z R d ( e e m ψ, T ( x ) − m ψ, T ( x )) q ( x ) d x i 2 ≤ 2 E  b C T − C T − C  2 + O  T − 2 k / (2 k +1)  . (35) By com bining (34), (35), (18), and (20), we conclude that, for all x ∈ C I 1 ( x ) = O  T − 2 k / (2 k +1)  . (36) T urning our atten tion to I 2 ( x ), it holds that, E ( b m ψ, T ,add ( x ) − b b m ψ, T ,add ( x )) 2 = E " d X ℓ =1 ( b b η ℓ ( x ℓ ) − b η ℓ ( x ℓ )) + Z R d e e m ψ, T ( x ) q ( x ) d x − Z R d e m ψ, T ( x ) q ( x ) d x # 2 ≤ 4 d d X ℓ =1 E  Z R d − 1 ( e m ψ, T ,ℓ ( x ) − e e m ψ, T ,ℓ ( x )) q ( x − ℓ ) d x − ℓ  2 10 +4 d d X ℓ =1 E  Z R d ( e m ψ, T ,ℓ ( x ) − e e m ψ, T ,ℓ ( x )) q ( x ) d x  2 +2 E  Z R d e e m ψ, T ( x ) q ( x ) d x − Z R d e m ψ, T ( x ) q ( x ) d x  2 ≤ 4 d d X ℓ =1 E Z R d − 1 ( e m ψ, T ,ℓ ( x ) − e e m ψ, T ,ℓ ( x )) 2 q 2 ( x − ℓ ) d x − ℓ +4 d d X ℓ =1 E Z R d ( e m ψ, T ,ℓ ( x ) − e e m ψ, T ,ℓ ( x )) 2 q 2 ( x ) d x 2 E Z R d ( e e m ψ, T ( x ) − e m ψ, T ( x )) 2 q 2 ( x ) d x ≤ 4 d d X ℓ =1 Z R d − 1 E ( e m ψ, T ,ℓ ( x ) − e e m ψ, T ,ℓ ( x )) 2 q 2 ( x − ℓ ) d x − ℓ +4 d d X ℓ =1 Z R d E ( e m ψ, T ,ℓ ( x ) − e e m ψ, T ,ℓ ( x )) 2 q 2 ( x ) d x +2 Z R d E ( e e m ψ, T ( x ) − e m ψ, T ( x )) 2 q 2 ( x ) d x Using the decomp osition 1 /f = 1 / b f T + ( b f T − f ) / ( b f T f ), it is easily show n that for some p ositiv e constant M 1 < ∞ , w e ha v e, under ( Q. 2), for all x ∈ C and T large enough, E  e m ψ, T ,ℓ ( x ) − e e m ψ, T ,ℓ ( x )  2 ≤ M 1 E 1 h 1 ,T h d − 1 2 ,T    K 1  x ℓ − X t,ℓ h 1 ,T  K 2  x − ℓ − X t, − ℓ h 2 ,T )    × sup x ∈C | b f T ( x ) − f ( x ) | ! 2 . It’s easily seen that under o ur assumptions, fo llo wing the demonstration of Theorem 4 .9. in (2) p.112 and replacing log m b y 1, w e hav e, sup x ∈C | b f T ( x ) − f ( x ) | = O  log T T  k ′ / (2 k ′ + d )  almost surely , W e conclude that, for a ll x ∈ C , E  b b m ψ, T ,add ( x ) − b m ψ, T ,add ( x )  2 = O  log T T  2 k ′ / (2 k ′ + d )  = O  T − 2 k / (2 k +1)  . ⊓ (37 ) 11 2.2 Pro of of Theorem 2 In the next lemma w e ev aluate the difference b et w een the estimator of the additiv e regression function b b m ψ, T ,add , for contin uous time pro cess, and the estimator b b m ψ, n,add where n ∈ N . Lemma 2 F or n ∈ N lar ge enough, ther e exists a deterministic c on stant C such that for al l ω i n Ω and for al l T in [ n, n + 1[ , | b b m ψ, T ,add ( x )( ω ) − E b b m ψ, T ,add ( x )( ω ) − b b m ψ, n,add ( x )( ω ) + E b b m ψ, n,add ( x )( ω ) | < C  log( T ) T  k 2 k +1 . Pr o of: It is sufficien t to prov e that ∀ ω ∈ Ω , ∀ T ∈ [ n, n + 1[ , | b b m ψ, T ,add ( x ( ω ) − b b m ψ, n,add ( x )( ω ) | < C ′  log( T ) T  k 2 k +1 , the other part b eing a trivial consequence of this inequalit y . Moreo v er, in view of (8) and (10), w e can establish the following inequalities Z R d e e m ψ, T ( x )( ω ) q ( x ) d x − Z R d e e m ψ, n ( x )( ω ) q ( x ) d x < C 1  log( T ) T  k 2 k +1 , (38) Z R d − 1 e e m ψ, T ,l ( x )( ω ) q − l ( x − l ) d x − l − Z R d − 1 e e m ψ, n,l ( x )( ω ) q − l ( x − l ) d x − l < C ′ l  log( T ) T  k 2 k +1 , (39) Z R d e e m ψ, T ,l ( x )( ω ) q ( x ) d x − Z R d e e m ψ, n,l ( x )( ω ) q ( x ) d x < C ′′ l  log( T ) T  k 2 k +1 , (40) with l = 1 , ...d. W e just establish the first inequality , the tec hniques being the same for ( 3 9) and (40). F or fixed ω in Ω and x in R d , w e ha v e, for n large enough, y / ∈ C δ ⇒ K 3  x − y h t  q ( x ) = 0 , ∀ t ≥ n. So, b y F ubini’s Theorem Z R d e e m ψ, T ( x )( ω ) q ( x ) d x = 1 T h d 1 ,T Z [0 ,T ] Z R d ψ ( Y t ( ω )) f ( X t ( ω )) K 3 ( x − X t ( ω ) h 1 ,T ) q ( x ) d x dt = 1 T h d 1 ,T Z [0 ,n ] Z R d ψ ( Y t ( ω )) f ( X t ( ω )) K 3 ( x − X t ( ω ) h 1 ,T ) q ( x ) d x dt + O ( 1 T ) = 1 nh d 1 ,T Z [0 ,n ] Z R d ψ ( Y t ( ω )) f ( X t ( ω )) K 3 ( x − X t ( ω ) h 1 ,T ) q ( x ) d x dt + O ( 1 T ) = 1 n Z [0 ,n ] Z R d ψ ( Y t ( ω )) f ( X t ( ω )) K 3 ( u ) q ( X t ( ω ) + u h 1 ,T ) d u dt + O ( 1 T ) . 12 Denoting M 7 := sup x ∈C δ ,y ∈ R ψ ( y ) f ( x ) < ∞ , w e hav e    Z R d e e m ψ, T ( x )( ω ) q ( x ) d x − Z R d e e m ψ, n ( x )( ω ) q ( x ) d x    = 1 n    Z t ∈ [0 ,n ] Z u ∈ R d ψ ( Y t ( ω )) f ( X t ( ω ) K 3 ( u )( q ( X t + h 1 ,T u ) − q ( X t + h 1 ,n u )) dudt    + O ( 1 T ) ≤ M 7 n    Z [0 ,n ] Z R d K 3 ( u )( q ( X t ( ω ) + h 1 ,T u ) − q ( X t ( ω ) + h 1 ,n u )) d u dt    + O ( 1 T ) ≤ dM 7 n Z [0 ,n ] Z S 3 | K 3 ( u ) max 1 ≤ l ≤ d k ∂ q ∂ u l k ∞ k u k ( h 1 ,T − h 1 ,n ) | d u dt + O ( 1 T ) = O ( h 1 ,T − h 1 ,n ) + O ( 1 T ) (41) Whic h implies (38) b y (K.1). This ac hiev es the pro of o f Lemma 2. Set ǫ 2 ( T ) = C  log T T  2 k 2 k +1 , where C is a finite constant. There exists a finite num b er r ( T ) := (3 / M ′ h 2 1 ,T ε ( T )) d of balls B p of cen ter x p and radius h 2 1 ,T ε ( T ), suc h that C ⊂ ∪ r ( T ) p =1 B p , where M’ is a constant. F or eac h x ∈ B p w e denote t ( x ) = x p .W rite no w, sup x ∈C | b b m ψ, T ,add ( x ) − m ψ ( x ) | ≤ sup x ∈C | b b m ψ, T ,add ( x ) − b m ψ, T ,add ( x ) | + sup x ∈C | E b b m ψ, T ,add ( x ) − m ψ ( x ) | + sup x ∈C | b b m ψ, T ,add ( x ) − b b m ψ, T ,add ( t ( x )) | + sup x ∈C | E b b m ψ, T ,add ( x ) − E b b m ψ, T ,add ( t ( x )) | + sup x ∈C | b b m ψ, add ( t ( x )) − E b b m ψ, add ( t ( x )) | . Th us, to pro v e the Theorem 2, it suffices to establish the following Lemma. Lemma 3 Under the sam e hyp othesis as The or em 2, we h ave sup x ∈C | E b b m ψ, T ,add ( x ) − m ψ ( x ) | = O ( h k 1 ,T ) , (42) sup x ∈C | b b m ψ, T ,add ( x ) − b b m ψ, T ,add ( t ( x )) | = O  ε ( T )  (43) sup x ∈C | E b b m ψ, T ,add ( x ) − E b b m ψ, T ,add ( t ( x )) |O  ε ( T )  , (44) sup x ∈C | b b m ψ, T ,add ( t ( x )) − E b b m ψ, T ,add ( t ( x )) | = O   log T T  k / 2 k +1  a.s., (45) sup x ∈C | b b m ψ, T ,add ( x ) − m ψ ( x ) | = O   log T T  k / (2 k +1)  a.s.. (46) 13 Pr o of o f 42: W e hav e E b b m ψ, T ,add ( x ) − m ψ ( x ) = d X l =1 ( E b b η l ( x l ) − η l ( x l )) + E ( Z R d e e m ψ, T ( x ) q ( x ) d x ) − Z R d m ψ ( x ) q ( x ) d x . By F ubini’s theorem, w e obtain Bias( b b m ψ, T ,add ( x )) = d X l =1 Bias( b b η l ( x l )) + Z R d Bias( e e m ψ, T ( x )) q ( x ) d x . (47) W e can write, Bias( b b η 1 ( x 1 )) = E ( b b η 1 ( x 1 )) − η 1 ( x 1 ) = { E ( b α 1 ( x 1 )) − e m ψ, T ( x 1 ) } + E ( b C T − C T − C ) := ( I ) + ( I I ) . (48) First consider the term ( I ), w e ha v e e m ψ, T ( x 1 ) = 1 T Z T 0 E n E ( ψ ( Y t ) | X t ) G ( X t ) f ( X t, − 1 | X t, 1 ) d x − 1    X t, 1 = x 1 o dt = Z R d − 1 m ψ ( x 1 , u − 1 ) Z R d − 1 1 h d − 1 2 ,T K 2  x − 1 − u − 1 h 2 ,T  q − 1 ( x − 1 ) d x − 1 d u − 1 = Z R d − 1 m ψ ( x 1 , u − 1 ) G ( u − 1 ) d u − 1 . It follow s that, under the conditions ( C . 2), ( K . 2) and ( K . 3) E ( b α 1 ( x 1 )) − e m ψ, T ( x 1 ) = Z R 1 h 1 ,T e m ψ, T ( u 1 ) K 1  x 1 − u 1 h 1 ,T  du 1 − e m ψ, T ( x 1 ) = Z R [ e m ψ, T ( x 1 − v 1 h 1 ,T ) − e m ψ, T ( x 1 )] K 1 ( v 1 ) dv 1 = Z R Z R d − 1 h k − 1 X i =1 ( − h 1 ,T v 1 ) i i ! ∂ i m ψ ∂ x i 1 ( x 1 , u − 1 ) + ( − h 1 ,T v 1 ) k k ! ∂ k m ψ ∂ x k 1 ( x 1 − θ h 1 ,T v 1 , u − 1 ) i G ( u − 1 ) d u − 1 K 1 ( v 1 ) dv 1 = Z R Z R d − 1  ( − h 1 ,T v 1 ) k k ! ∂ k m ψ ∂ x k 1 ( x 1 − θ h 1 ,T v 1 , u − 1 )  G ( u − 1 ) d u − 1 K 1 ( v 1 ) dv 1 . Th us, w e obtain sup x 1 | E ( b α 1 ( x 1 )) − e m ψ, T ( x 1 ) | = O  h k 1 ,T  . (49) 14 Next, turning our atten tion to ( I I ), b y (21) w e hav e E ( b C T − C T ) − C = O ( h k 1 ,T ) . (50) Com bining (49) and (50), it follows that sup x 1 | E ( b b η 1 ( x 1 )) − η 1 ( x 1 ) | = O  h k 1 ,T  . (51) On the other ha nd, we hav e, for all 0 < θ < 1, Bias( e e m ψ, T ( x )) := E e e m ψ, T ( x ) − m ψ ( x ) (52) = Z R d [ m ψ ( x + h 1 ,T v ) − m ψ ( x )] K 3 ( v ) d v . = Z R d X i 1 + ... + i d = k h k 1 ,T k ! ∂ i 1 + ... + i d m ψ ∂ x i 1 1 ...∂ x i d d ( x + h 1 θ v ) v i 1 1 ...v i d d K 3 ( v ) d v (53) := O ( h k 1 ,T ) , Com bining the decomposition (47) and the statemen ts (5 1) and (54), w e deduce the result (42). Pr o of o f (43) Under the condition ( K. 5), there exists a constant M suc h that 1 T Z T 0 | Z t ( x ) − Z t ( t ( x )) | dt ≤ d X l =1 M h 2 1 ,T | x l − t ( x ) l | Consequen tly , using the expression of r ( T ), w e obtain sup x ∈C | b b m ψ, T ,add ( x ) − b b m ψ, T ,add ( t ( x )) | = O ( ǫ ( T )) . Pr o of o f (44): Similarly a s ab o v e, w e ma y deduce (4 4). Pr o of o f (45): In view o f Lemma 2, it is sufficien t to prov e discrete ve rsion of (45), that is sup x ∈C | b b m ψ, n,add ( t ( x )) − E b b m ψ, n,add ( t ( x )) | = O  ε ( n )  a.s. (54) Set n in N and,intro duce some notations. Set, b b m ψ, n,add ( x ) − E b b m ψ, n,add ( x ) =: 1 n Z n 0 ξ t ( x ) dt, (55) where ξ t = ξ t ( u ) := ( Z t ( u ) − E ( Z t ( u ))) 15 and Z t = Z t ( u ) = ψ ( Y t ) h 1 ,n h d − 1 2 ,n f ( X t ) d X l =1  Z R d − 1 K 1  u l − X t,l h 1 ,n  K 2  x − l − X t, − l h 2 ,n  q − l ( x − l ) d x − l − Z R d K 1  x l − X t,l h 1 ,n  K 2  x − l − X t, − l h 2 ,n  q ( x ) d x  + 1 h d 1 ,n ψ ( Y t ) f ( X t ) Z R d − 1 K 3  x − X t h 1 ,n  q ( x ) d x . Finally , w e use the notation V n i ( x ) := 1 n Z ip ( i − 1) p ξ t ( x ) dt i = 1 , ..., 2 q ′ where p := n 2 q ′ := ǫ ( n ) − 1 / 2 (56) So we can write b b m ψ, n,add ( x ) − E b b m ψ, n,add ( x ) = 2 q ′ X i =1 V n i ( x ) . (57) W e ha v e to sho w that the following quan tity is summable P (sup x ∈C | 2 q ′ X i =1 V i ( t ( x )) | ≥ ε ( n )) ≤ r ( n ) sup p =1 ,..., r ( n ) P ( | 2 q ′ X i =1 V i ( t p ) | ≥ ε ( n )) . (58) Let j b e fixed in [1 , r ( n )]. W e hav e P ( | 2 q ′ X i =1 V i ( t j ) | ≥ ε ( n )) ≤ P ( | q ′ X i =1 V 2 i ( t j ) | ≥ ε ( n ) / 2) + P ( | q ′ X i =1 V 2 i − 1 ( t j ) | ≥ ε ( n ) / 2 ) . Observing that for a giv en M ′′ , ξ t ( x )( ω ) < M ′′ h 1 ,n , ∀ ω ∈ Ω, w e can use recursiv ely Bradley’s lemma and define the indep enden t random v ariables W 2 ( t j ) , ..., W 2 q ′ ( t j ) such that, ∀ i ∈ [1 , q ′ ], W 2 i and V 2 i ha v e the same la w and ∀ ν > 0 P  | W 2 i ( t j ) − V 2 i ( t j ) | > ν  ≤ 11  k V 2 i ( t j ) k ∞ ν  1 2 α ( p ) ≤ 11  pM ′′ h 1 ,n ν  1 2 α ( p ) . (59) W e ha v e, for all 0 < λ < ε ( n ) 2 n | q ′ X i =1 V 2 i ( t j ) | > ǫ ( n ) 2 o ⊂ n {| q ′ X i =1 V 2 i ( t j ) | > ǫ ( n ) 2 ; | V i ( t j ) − W i ( t j ) | ≤ λ q ′ 1 ≤ i ≤ q ′ } o [ q ′ [ j =1 {| V i ( t j ) − W i ( t j ) | > λ q ′ } o . 16 The c hoice λ = ǫ ( n ) 4 giv es us P  | q ′ X i =1 V 2 i ( t j ) | > ǫ ( n ) 2  ≤ P  | q ′ X i =1 W 2 i ( t j ) | > ǫ ( n ) 4  + q ′ X i =1 P  | V 2 i ( t j ) − W 2 i ( t j ) | > ǫ ( n ) 4 q ′  . (60) W e treat separately the t w o terms of the last inequalit y . F o r the sec ond one, the application of ( 59)under the condition ( A. 1) driv es us to q ′ X i =1 P  | V 2 i ( t j ) − W 2 i ( t j ) | > ǫ ( n ) 4 q ′  ≤ 11 q ′  4 q ′ pM ′ h 1 ,n ǫ ( n )  1 / 2 α ( p ) = O ( r ( n ) − 1 n µ ) (61) w her e µ < − 1 . In order to dominate P  | P q ′ i =1 W 2 i ( t j ) | > ǫ ( n ) 4  , w e m ust b ound the v ariance o f W 2 i (whic h has the same law as V 2 i ) to use Bernstein’s inequalit y V ar( W 2 i ( t j )) = E ( V 2 i ( t j ) 2 ) ≤ 1 n 2 Z [(2 j − 1) p, 2 j p ] E ( ξ 2 t ) dt (62) The k ernels are b ounded, so w e can easily see, after a change of v ar ia bles, that there exists a constant M ′′′ suc h that E ( Z 2 t ) ≤ M ′′′ h 1 ,n , witc h implies E ( ξ 2 t ) ≤ M ′′′ h 1 ,n and V ar( W 2 i ( t j )) ≤ pM ′′′ n 2 h 1 ,n . Observ e that, for a giv en S in R ∗ + , ξ t ( ω ) < S h 1 ,n , ∀ ω ∈ Ω, w e readily hav e E | W i | k ≤ ( p M ′ nh 1 ,n ) k − 2 p ! E | W i | 2 , ∀ i. This allo ws us to apply Bernstein’s inequalit y P  | q ′ X i =1 W i ( t j ) | > ǫ ( n ) 4  ≤ 2 exp  − ǫ ( n ) 2 16( 4 q ′ pM n 2 h 1 ,n + M ′ pǫ ( n ) 2 nh 1 ,n )  = 2 exp  − ǫ ( n ) 2 nh 1 ,n 32 M + 8 M ′ pǫ ( n ))  . (63) 17 The expression of p and ε ( n ) giv es us pε ( n ) → 0 and the sequence P N n =1 r ( n ) P  | P q ′ i =1 W i ( t j ) | > ǫ ( n ) 4  con v erges as N grows to infinity if w e c ho ose a la r ge enough C in ǫ ( n ). In view of this last inequalit y and (61), w e obt a in (54 ) by Bor el- Can telli. Pr o of o f (46): By (4) , w e hav e sup x ∈C | e e m ψ, T ( x ) − e m ψ, T ( x ) | ≤ M sup x ∈C | f ( x ) − b f T ( x ) | inf x ∈C f 2 ( x ) + o (1 ) 1 T h d 1 ,T Z T 0    K 3  x − X t h 1 ,T     dt. Using the statemen ts ( 5 ) and (7), and the Theorem on a densit y estimator due to (2), we obtain sup x ∈C | b b m ψ, T ,add ( x ) − b m ψ, T ,add ( x ) | ≤ 2 d max 1 ≤ l ≤ d sup x ∈C | e e m ψ, T ,l ( x ) − e m ψ, T ,l ( x ) | + sup x ∈C | e e m ψ, T ( x ) − e m ψ, T ( x ) | = O   log T T  k / (2 k +1)  a.s. . References [1] Banon, G. (1978). Errata: “Nonparametric iden tification for diffusion pro cess es” (SIAM J. Con trol Optim. 16 (1978), no. 3, 380–3 95) [2] Bosq, D. (1993). Vitesses optimales et sup eroptimales des estimateurs fonctionnels p o ur les pro cessus ` a temps con tinu. C. R. A c a d . Sci. Paris S´ er. I Math. , 317 ( 10), 1075–107 8 . [3] D . Bosq. Nonp ar ametric statistics for sto chastic pr o c esse s , v olume 110 of L e ctur e Notes in Statistics . Springer-V erlag, New Y ork, 19 96. Estimation and prediction. [4] Camlong- Viot, C., Sar da , P ., and Vieu, P . (2000) . Additiv e time series: the k ernel in tegration metho d. MSpringe r-V erlag , New Y ork. [5] Jones, M., Da vies, S., a nd Park, B. (1994). V ersions of k ernel-t yp e regression estimators. J. A m. Statist. Asso c . , 89 , 825–83 2. [6] Linton, O. and Nielsen, J. (1995 ). A k ernel metho d of estimating structured nonpara - metric r egr ession based on marginal integration . B iometrika , 82 , 93 – 100. 18 [7] Nadaray a , E. and W a tson (1964). On estimating regression. T h e ory of pr ob a. and Appl. , 9 , 141–142 . [8] New ey , W. K. (1994). Kernel estimation of partial means and a general v ariance estima- tor. Ec onometric The ory , 10 (2), 233–2 53. [9] Parzen , E. (1962). On estimation of probabilit y densit y function and mo de. A nn.Math.Stat. , 33 , 1065–1 0 76. [10] R o sen blatt, M. (19 56). Remarks on some nonparametric estimates o f a densit y function. A nn. Math. Statist. , 27 , 832 – 837. [11] Sto ne, C. J. (1985). Additiv e regression and other no nparametric mo dels. Ann. Statist. , 13 (2), 689–705. 19