Improving Coverage Accuracy of Block Bootstrap Confidence Intervals

Impro ving Co v erage Accuracy of Blo c k Bo otstrap Confidence In terv als Stephen M.S. Lee e-mail: smsle e@hkusua.hku.hk P .Y. Lai e-mail: pylaipy@gr aduate.hku.hk Dep artment of Sta tistics and A ctuarial Scienc e, Th e University of Hong Kong Pokfulam R o ad, Hong Kong ABSTRACT The blo c k b o otstrap c onfidence in terv al based o n dependen t data can outp er- form the computationally more conv enien t normal a pproximation only with non-trivial Studen tization which, in the case of complicated statistics, calls for highly sp ecialist treatmen t. W e prop ose t wo different approac hes to im- pro ving the accuracy of the blo c k b o otstrap confidence in terv al under ve ry general conditio ns. The first calibrat es the co verage lev el by iterating the blo c k b o o t stra p. The second calculates Studentiz ing fa ctors directly from blo c k b o otstrap series and requires no non- trivial analytic treatmen t. Both approac hes in volv e tw o nes ted lev els of blo c k b o ot strap resampling and yield high-order accuracy with simple tuning of blo c k lengths at the t w o resam- pling lev els. A sim ulation study is repor t ed to pro vide empirical supp ort for our theory . Key w ords a nd phrases: bloc k b o otstrap; co v erage calibration; Studen ti- zation; we akly dependen t. 1 In tro ducti on The blo ck bo otstrap has b een dev elop ed as a completely mo del-free pro ce- dure for handling inference problems concerning dep enden t data. A ma jor criticism t hat imp edes widespread acceptance of the pro cedure in applica- tions is that it lacks second-order a ccuracy and that empirical selection of blo c k length is critical y et difficult. Although intensiv e w ork ha s b een done on the second issue, remedie s th us far pro p osed for the first dr awbac k are rather restrictiv e in the sense that they require either non-trivial, and sometimes algebraically formidable, Studen tization or assumptions of mor e stringen t mo del structures. Those well-es ta blished techniq ues, suc h as the iterative b o o t stra p and t he b o otstrap- t , designed for enhancing b o otstrap a ccuracy for indep enden t data a pp ear to hav e lost their a pp eal in the contex t of dep enden t data, b ecause the blo ck b o otstrap series typic ally exhibits undesirable a r te- facts as a consequence of pa sting ra ndo mly selected data blo c ks together. An imp ortant question is whether the blo c k b o otstrap can b e made more accurate, by an order asymptotically as w ell as for finite samples, without analytically cumbersome Studen tization nor ha ving to confine applications to dep enden t data generated b y sp ecific pro cesses. W e in ve stigate f ormally the applications of t w o general res ampling- based tec hniques, namely cov erage calibration and b o ot stra p Studentization, to 2 the blo c k b o otstrap confidence in terv als based on dep enden t dat a. A nov el double b o otstrap pro cedure is prop osed fo r either cov erage calibration or b o o t stra p Studen tization to improv e cov erage accuracy of the block bo otstrap b ey ond the first order. The pro cedure enables b oth tec hniques to retain the simplicit y and generalit y they hav e already enjoy ed when applied to indep enden t data. Hall ( 1985) and K ¨ unsc h (1989) intro duce the blo c k b o otstrap as a fully nonparametric extension of the b o otstrap to handle dep endent data. Its consistency fo r distributional estimation is v erified b y K ¨ unsc h (1989) and Liu and Singh (1992). Lahiri (199 2) prov es fo r m -dep enden t data that the block b o o t stra p distribution of an a djusted Studen tized sample mean is accurate to second order. Davis on and Hall (1993) achiev e similar results b y k ernel- based Studen tization. Hall, Horo witz and Jing (1995 ), G¨ otze and K ¨ unsc h (1996) and Z vingelis (20 0 3) sharp en the results b y giving explicit orders for the estimation error. V arian ts of the blo c k b o otstrap include circular blo c k resampling (Politis and R o mano, 19 9 2), the stationary b o otstrap (P olitis and Romano, 1993), the matched - blo c k b o otstrap (Carlstein, Do, Hall, Hesterb erg and K ¨ unsc h, 1998) a nd the tap ered b o ot stra p (P aparo ditis and P olitis, 2001 ). Lahiri (1999) compares the first t w o with the blo ck b o otstrap and confirms supe- riorit y of the la tter. Da vison and Hall (1 993), Choi and Hall (20 00) and 3 B ¨ uhlmann (2002) remark on the distortio n of dep endence structures in blo c k b o o t stra p s eries and, fo r that reason, express doubt o v er effectiv eness of co v- erage calibratio n b y b o o tstrap iterations. The subsampling metho d, as studied by P olitis and Romano (1994), is more generally applicable than the blo c k b o o t stra p, but has inferior asymp- totic prop erties: see Hall and Jing (1996) and Bertail (1 997). Nonpar a met- ric metho ds more accurate than the blo ck bo otstrap hav e b een found under more stringen t assumptions on the data generating pro cesses. Examples in- clude t he siev e b o otstrap (B ¨ uhlmann, 1997; Choi and Hall, 2000) f o r linear pro cesses, the Mark o v b o otstrap (Ra jarshi, 1990) and the lo cal b o otstrap (P aparo ditis and P olitis, 2002) for Mark o v pro cesses . W e in tro duce in Section 2 a double b o o tstrap pro cedure f o r either co v- erage calibration o r Studentiz at io n of the ov erlapping blo c k b o otstrap. Sec- tion 3 establishes asymptotic expansions for the co verage probabilities of b oth the iterated blo c k b o ot stra p a nd Studen tized blo c k b o otstrap confi- dence in terv als under sufficien tly general regularity conditions, deriv es the optimal second-lev el blo c k length in relation to the first-lev el blo c k length and prov es asymptotic sup eriorit y of our pro cedures. Section 4 rep orts a sim ulation study whic h compares our metho ds with the con v entional blo ck b o o t stra p and t w o alternativ e b o otstrap- t a ppro ac hes. Sec tion 5 concludes our findings. All tec hnical pro ofs a r e giv en in App endix 6.1. 4 2 Co v erage calib r ati on and Stude ntization 2.1 Blo ck b o otstrap confidence in terv al Let X = ( X 1 , . . . , X n ) b e a series of d -v aria te o bserv ations from the se- quence { X i : −∞ < i < ∞ } , whic h is a realization of a strictly stationary , discrete-time, sto c hastic pro cess with finite mean µ = E [ X 1 ]. D enote b y ¯ X = P n i =1 X i /n the sample mean. W e briefly review the blo c k b o otstrap construction of a lev el α upp er con- fidence b ound for a scalar parameter of inte rest θ = H ( µ ), fo r some smo oth function H : R d → R . A na t ural plug-in estimator of θ is ˆ θ = H ( ¯ X ). This smo oth function mo del setup encompasses a wide v ariety of estimators, or their high-order asymptotic approx imatio ns, pro viding a sufficien tly general platform for in ve stigating the blo c k b o o tstrap confidence pro cedure. F or a blo c k length ℓ (1 ≤ ℓ ≤ n ), let n ′ = n − ℓ + 1 a nd define ov erlapping blo c ks Y j,ℓ = ( X j , X j +1 , . . . , X j + ℓ − 1 ), j = 1 , . . . , n ′ . A generic first-lev el blo c k b o o t stra p series X ∗ = ( X ∗ 1 , . . . , X ∗ bℓ ), where b = h n/ℓ i and h x i denotes the in teger part o f x , is giv en b y sampling b blo cks randomly with replacemen t from { Y j,ℓ : 1 ≤ j ≤ n ′ } and pasting them end-to- end in the order sampled, so that ( X ∗ ( j − 1) ℓ +1 , . . . , X ∗ j ℓ ) denotes t he j th blo ck sampled, j = 1 , . . . , b . Let P ∗ and E ∗ denote the probability measure and exp ectation op erator induced b y blo c k b o otstrap sampling, conditional on X , respectiv ely . D efine 5 ¯ X ∗ = P bℓ i =1 X ∗ i / ( bℓ ) and t he blo c k b o otstrap distribution function G ∗ ( x ) = P ∗  ( bℓ ) 1 / 2 [ H ( ¯ X ∗ ) − H ( E ∗ ¯ X ∗ )] ≤ x  , x ∈ R . Then I ( α ) = ˆ θ − n − 1 / 2 G ∗− 1 (1 − α ) defines a lev el α blo c k bo otstrap upp er confiden ce b ound for θ . Note that sampling of ov erlapping blo cks incurs an edge effect whic h explains the use of H ( E ∗ ¯ X ∗ ), rather than the more conv en tional ˆ θ = H ( ¯ X ), for cen tering the b o o t stra p estimator in the definition of G ∗ . Under regularit y conditio ns to b e detailed in Section 3, the c hoice ℓ ∝ n 1 / 3 yields the smallest co v erage error, of order O ( n − 1 / 3 ), for I ( α ). 2.2 Second-lev el blo ck b o otstrap F or indep enden t and identic ally distributed data, co v erage calibration a nd Studen tization pro vide t wo w ell-kno wn tec hniques fo r improv ing co v erage accuracy of b o otstrap confidence in terv als. W e consider applications of the t w o tec hniques in the presen t con text o f dep enden t data . Both co v erage calibration and the v ersion of Studen tization prop osed herein call for a double b o o t stra p pro cedure as describ ed b elo w. Based on X ∗ , define blo ck s Y ∗ i,j,k = ( X ∗ ( i − 1) ℓ + j , X ∗ ( i − 1) ℓ + j +1 , . . . , X ∗ ( i − 1) ℓ + j + k − 1 ), eac h of length k (1 ≤ k ≤ ℓ ), for i = 1 , . . . , b and j = 1 , . . . , ℓ ′ , where ℓ ′ = ℓ − k + 1. Note that for eac h fixed i = 1 , . . . , b , Y ∗ i, 1 ,k , . . . , Y ∗ i,ℓ ′ ,k repre- sen t ov erlapping blo c ks within t he blo c k ( X ∗ ( i − 1) ℓ +1 , . . . , X ∗ iℓ ), whic h is itself 6 sampled randomly fro m the blo c ks { Y j,ℓ : 1 ≤ j ≤ n ′ } . T he second-lev el blo c k b o o tstrap series, denoted by X ∗∗ = ( X ∗∗ 1 , . . . , X ∗∗ ck ), f o r c = h n/k i , is sampled f rom the bℓ ′ blo c ks { Y ∗ i,j,k : 1 ≤ i ≤ b, 1 ≤ j ≤ ℓ ′ } in the same w a y as is X ∗ from { Y j,ℓ : 1 ≤ j ≤ n ′ } . That Y ∗ i,j,k is a subseries o f k consecu- tiv e observ a tions within X eliminates the p ossibilit y of dra wing second-lev el blo c ks t hat run across join ts of the first-lev el blo ck bo otstrap series , thereb y a v oiding the discon tin uit y problem whic h has aroused forejudged criticisms ab out the v ery usefulness of the do uble blo ck b o o t strap. Denote by P ∗∗ and E ∗∗ resp ectiv ely the proba bilit y measure and expecta- tion op erato r induced b y second-lev el blo ck b o ot stra p sampling, conditional on X ∗ . Define ¯ X ∗∗ = P ck i =1 X ∗∗ i / ( ck ) and G ∗∗ ( x ) = P ∗∗  ( ck ) 1 / 2 [ H ( ¯ X ∗∗ ) − H ( E ∗∗ ¯ X ∗∗ )] ≤ x  , x ∈ R . The second-lev el blo ck b o otstrap distribution G ∗∗ can b e used in t wo different w a ys, namely co v erage calibration and Studentization, to cor r ect I ( α ): 1. Cover age c a libr ation — The co v erage calibration metho d adjusts the nominal lev el α to ˆ α , ob- tained as solution to the equation P ∗  ( bℓ ) 1 / 2 [ H ( ¯ X ∗ ) − H ( E ∗ ¯ X ∗ )] ≤ G ∗∗− 1 (1 − ˆ α )  = 1 − α . The cov erag e- calibrated upp er confidence b ound is then I C ( α ) = I ( ˆ α ) = ˆ θ − n − 1 / 2 G ∗− 1 (1 − ˆ α ). 7 2. Studentization — Let ˆ τ b e the conditional standard deviation of ( bℓ ) 1 / 2 H ( ¯ X ∗ ) giv en X , and τ ∗ b e that of ( ck ) 1 / 2 H ( ¯ X ∗∗ ) giv en X ∗ . Define, for x ∈ R , J ∗ ( x ) = P ∗  ( bℓ ) 1 / 2 [ H ( ¯ X ∗ ) − H ( E ∗ ¯ X ∗ )] /τ ∗ ≤ x  . The lev el α Stude ntized upper con- fidence b ound is then give n b y I S ( α ) = ˆ θ − n − 1 / 2 ˆ τ J ∗− 1 (1 − α ). W e show in Section 3 that under regular ity conditions, I C ( α ) and I S ( α ) are asymptotically equiv alen t up to order O p  k − 2 n − 1 / 2 + ℓn − 3 / 2  . Both metho ds enjo y a reduced cov erag e error of order O ( n − 2 / 3 ) if w e set, for ex- ample, 2 k = ℓ ∝ n 1 / 3 . Our results rebut the criticisms expressed by , for ex- ample, D a vison and Hall (1993), Choi and Hall ( 2000) and B ¨ uhlmann (2002) o v er the effectiv eness of cov erage calibration. Indeed, I C ( α ) is the first ev er non-Studen tized blo c k b o o t strap in terv a l hav ing the same order of cov erage accuracy as has previously b een sho wn to b e p ossible only with Studen tiza- tion under the presen t regularity conditions. This has esp ecially imp ortan t implications f o r problems in which Studen tization is found t o be n umerically unstable and therefore results in highly v ariable interv al endp oin ts. On the other hand, construction of I S ( α ) mak es unnecess ary all those non-trivial, problem-sp ecific, algebraic manipulations whic h are instrumental to calcu- lation of the Studen tizing factors suggested b y Lahiri (1 9 92), D a vison and Hall (1993) and G ¨ otze and K ¨ unsc h (1996). Indeed, bot h ˆ τ and τ ∗ are readily obtained by direct Monte Carlo sim ulation fro m the b o ot stra p distributions 8 G ∗ and G ∗∗ resp ectiv ely , th us adhering mos t clos ely to the celebrated plug-in principle underlying the very b o otstrap metho dolog y . 3 Theory Higher-order asymptotic in ve stigat io n of cov erage accuracy of the blo c k b o ot- strap confidence b ounds is p ossible if w e assume regula r it y conditions that facilitate Edgew orth expansions of the distribution functions of n 1 / 2 ( ˆ θ − θ ) and n 1 / 2 ( ˆ θ − θ ) / ˆ τ . The set of conditions considered by G¨ otze and Hipp (1983) has generally b een accepted as t he standard assumptions underpin- ning a high-order asymptotic theory of t he blo c k b o otstrap. Importantly , previous studies hav e sho wn that the blo c k b o o tstrap can b e made accurate to sec ond o rder only with non-trivial Studen tization or subs tantial strength- ening of the G¨ otze and Hipp conditions. W e shall establish asymptotic r esults for our cov erage calibration and Studen tization approac hes under the G¨ otze and Hipp conditions, as mo dified by Lahiri (2003 , Section 6.5) b elo w, with k · k denoting the usual Euclidean norm: (A1) E k X 1 k 35+ δ < ∞ for some δ > 0. (A2) lim n →∞ Co v  n − 1 / 2 P n i =1 X i  exists a nd is nonsingular. 9 (A3) There exists a constan t C ∈ (0 , 1) suc h that f or i, j > 1 /C , inf ( s T Co v i + j X r = i +1 X r ! s : k s k = 1 ) > C j. (A4) There exist a constan t C > 0 and sub- σ -fields D 0 , D ± 1 , . . . of the σ - field underlying the pro babilit y space induced b y X 1 suc h that for i, j = 1 , 2 , . . . , (i) there exist D i + j i − j -measurable ra ndom v ectors ˜ X i,j satisfying E k X i − ˜ X i,j k ≤ C − 1 e − C j for j > 1 /C , where D s r denotes the sigma-field generated by {D t : r ≤ t ≤ s } ; (ii) | P ( A ∩ B ) − P ( A ) P ( B ) | ≤ C − 1 e − C j for a n y A ∈ D i −∞ and B ∈ D ∞ i + j ; (iii) E    E h exp( ιs T P i + j r = i − j X r ) | {D t : t 6 = i } i    ≤ e − C for i > j > 1 /C and s ∈ R d with k s k ≥ C , where ι 2 = − 1; (iv) E | P ( A | {D t : t 6 = i } ) − P ( A | {D t : 0 < | t − i | ≤ j + r } ) | ≤ C − 1 e − C j for r = 1 , 2 , . . . and A ∈ D i + r i − r . Note that (A4) intro duces an a uxiliary set of sub- σ -fields D t to bring a wide v ariety of w eakly dependen t pro cesses under a common fra mework. Sp ecial examples include linear pro cesses , m -dep enden t shifts, stationary homoge- neous Marko v chains and stationary Gaussian pro cesses . Bhattac harya and Ghosh’s (1978) smo oth function mo del supplies a rich class of estimators and has b een extensiv ely studied in the b o o t stra p liter- 10 ature: see, fo r example, Hall (19 92). In the dep enden t data con text, it en- compasses estimators suc h as sample auto co v ariances, sample a uto correlation co efficien ts, sample partial auto correlation coefficien ts and Y ule-W alk er esti- mators for auto r egressiv e pro cesses. Imp orta n tly , the mo del admits highly- structured asymptotic expansions to facilitate establishmen t of Edgew orth expansions and their blo ck b o otstrap v ersions. W e adopt the smo o th func- tion mo del as described by G¨ otze and K ¨ unsc h (1996) under the assumption (A5) H : R d → R is four times contin uously differen tiable with non-v anishing gradien t at µ and fourth-order deriv ativ es at x ∈ R d b ounded in mag- nitude by C (1 + k x k D ) for fixed constan ts C , D > 0. Next w e in tro duce some notation. W rite x = ( x (1) , . . . , x ( d ) ) for each x ∈ R d . Define, fo r r 1 , r 2 , . . . = 1 , . . . , d and i 1 , i 2 , . . . = 0 , 1 , 2 , . . . , γ r 1 ,r 2 ,... i 1 ,i 2 ,... = E  ( X 0 − µ ) ( r 1 ) ( X i 1 − µ ) ( r 2 ) ( X i 2 − µ ) ( r 3 ) · · ·  . F or r, s, . . . = 1 , . . . , d , define H r = ( ∂ /∂ x ( r ) ) H ( x )   x = µ , H r s = ( ∂ 2 /∂ x ( r ) ∂ x ( s ) ) H ( x )   x = µ , etc. Under con- ditions (A1 ) –(A4), we can expand the v aria nce-co v aria nce matrix of S n = n − 1 / 2 P n i =1 ( X i − µ ) suc h that Cov  S ( r ) n , S ( s ) n  = χ r,s 2 , 1 + n − 1 χ r,s 2 , 2 + O ( n − 2 ), r , s = 1 , . . . , d , for constants χ r,s 2 , 1 and χ r,s 2 , 2 not depending on n . In particular, w e hav e χ r,s 2 , 1 = P ∞ i = −∞ γ r,s i . D efine σ 2 = V ar  P d r =1 H r S ( r ) n  , whic h, under the ab o ve conditions, is p ositiv e and ha s o rder O (1). Let φ ( · ) and z ξ b e the standard normal densit y function and ξ th quan tile resp ectiv ely . 11 Our main theorem b elo w derive s expansions for the cov erage probabilities of the v arious blo c k b o otstrap upp er confidence b ounds. Theorem 1 L et { X i : −∞ < i < ∞} b e a strictly stationary, discr ete- time, sto chastic pr o c ess with finite me an µ = E [ X 1 ] . L et α ∈ (0 , 1) b e fixe d. Assume that c onditions (A1)–(A5) hold. Then, (i) for ℓ = O ( n 1 / 3 ) and ℓ/n ǫ → ∞ for some ǫ ∈ (0 , 1) , P ( θ ≤ I ( α )) = α + ℓ − 1 2 − 1 σ − 2 z α φ ( z α ) d X r,s =1 H r H s χ r,s 2 , 2 − n − 1 / 2 2 − 1 σ − 3 z 2 α φ ( z α ) × ( d X r,s,t =1 H r H s H t ∞ X i,j = −∞ γ r,s,t i,j + 2 d X r,s,t,u =1 H r H s H tu χ r,t 2 , 1 χ s,u 2 , 1 ) + O  ℓ − 2 + ℓn − 1  ; (1) (ii) for k ≤ ℓ = O ( n 1 / 3 ) and k /n ǫ → ∞ f o r some ǫ ∈ (0 , 1) , the c onfidenc e limits I C ( α ) and I S ( α ) differ by O p  k − 2 n − 1 / 2 + ℓn − 3 / 2  and have c o v- er age pr ob ability α + (2 ℓ − 1 − k − 1 )2 − 1 σ − 2 z α φ ( z α ) d X r,s =1 H r H s χ r,s 2 , 2 + O  k − 2 + ℓn − 1  . (2) It is clear from Theorem 1 that I ( α ) has co v erage error of order O ( ℓ − 1 + ℓn − 1 ), whic h can b e reduced b y either co v erage calibration or Studen tization to O ( ℓ − 2 + ℓn − 1 ) if w e set k = ℓ/ 2 . Heuristically , a c hief source of co v erage error of I ( α ) stems from the large bia s, of order 1 /ℓ , of the blo c k b o otstrap 12 v ariance estimator. The second-lev el blo c k b o o tstrap v ar ia nce estimator has leading bias of order 1 /k − 1 /ℓ when view ed as an es timato r of the first-leve l blo c k b o ot stra p v ariance estimate. Existence of suc h second-lev el bias term enables either the co ve ra g e calibration or Studen t izat io n strat egies to auto- matically offset the first-lev el bias of o r der 1 /ℓ , pro vided that k is set to ℓ/ 2. F urthermore, expansions (1) and (2) enable us to deriv e the optimal choice s of blo ck lengths ℓ a nd k fo r ac hieving the b est co ve rag e error rates. W e see from (1) that, in the absence of co ve ra ge calibration or Studen tization, the optimal blo c k length ℓ should hav e order n 1 / 3 in order to yield the smallest co v erage error, of order O ( n − 1 / 3 ), for I ( α ). With k = ℓ/ 2 and ℓ ∝ n 1 / 3 , the co ve rag e error of b o t h I C ( α ) and I S ( α ) has or der O ( n − 2 / 3 ), a signifi- can t impro v emen t ov er that of the unmo dified I ( α ). The fo llo wing corollary summarizes the ab ov e results. Corollary 1 Under the c onditions of T he or em 1, (i) I ( α ) has c over age err or of or der O ( n − 1 / 3 ) , ach i e ve d by setting ℓ ∝ n 1 / 3 ; (ii) I C ( α ) and I S ( α ) ar e asymptotic a l ly e quivalent up to o r der O p ( n − 7 / 6 ) and have c over age e rr or of or der O ( n − 2 / 3 ) , a chieve d by setting 2 k = ℓ ∝ n 1 / 3 . Corollary 1 confirms that second-order correction of the blo ck b o otstrap in- terv al can b e ac hiev ed by straightforw ar d application of either co v erage cal- 13 ibration or Studentization. Previous approaches prop osed in the literature to suc h second-order correction rely inv a r ia bly on explicit computation of a non-trivial expression of the Studen tizing factor, whic h mus t b e analyti- cally deriv ed for eac h smo oth function model under study . See, for example, H¨ ardle, Horowitz and Kreiss (2003) for a review of suc h approac hes. At the exp ense of computational efficiency incurred b y the double b o otstrap pro ce- dure, calculatio n of I C ( α ) or I S ( α ) inv olves no analytic formula and can b e carried out b y brute fo rce Monte Carlo sim ulation. P erhaps surprising is the extremely simple relat io nship ( k = ℓ/ 2) b etw een the optimal first-lev el and second-lev el blo c k lengths, whic h reliev es us of t he noto r io usly difficult task of determining the best blo c k length for the double block b o ot stra p, in so far as the selection of k is concerned. 4 Sim ulation study W e conducted a sim ulation study to in ve stigate the empirical p erformance of I C ( α ) and I S ( α ) in comparison with I ( α ). Tw o other Studen tized blo c k b o o t stra p confidence b o unds, based on constructions of D a vison and Hall (1993) and G¨ otze and K ¨ unsc h (1996 ) and denoted by I D H ( α ) and I GK ( α ) resp ectiv ely , we re also included in the study for reference: see App endix 6.2 for details of these tw o latter approac hes. Time series data w ere generated 14 under the follow ing three mo dels: (a) ARC H(1) pro cess: X i = e i (1 + 0 . 3 X 2 i − 1 ) 1 / 2 , (b) MA(1) pro cess: X i = e i + 0 . 3 e i − 1 , (c) AR(1) pro cess: X i = 0 . 3 X i − 1 + e i , where the e i are indep enden t N (0 , 1) v ariables. The parameter θ w as tak en to b e the mean, v ariance and lag 1 auto correlation, and the nominal lev el α w as set to b e 0 . 05 , 0.10 , 0 .9 0 and 0.95. F or eac h metho d, the co v erage probability of the lev el α upp er confidence b ound w a s approximated b y av erag ing ov er 1000 indep enden t time series o f length n = 500 and 1000. Construction of eac h confidence b o und was based on 1000 first-lev el blo c k b o ot stra p series using blo ck length ℓ = h n 1 / 3 i , in addition to whic h 1 000 second-lev el series based on block length k = h ℓ/ 2 i w ere generated f r om eac h firs t- lev el se ries to construct I C ( α ) and I S ( α ). Sp ecifically , w e hav e ( ℓ, k ) = (8 , 4) and (10 , 5) for n = 500 and 1 0 00 respectiv ely . The constant c was set to b e 0.5 in the calculation of the St udentizing factor f o r I GK ( α ): see App endix 6.2. The cov erag e results are giv en in T ables 1 – 3 for the mean, the v ariance and the lag 1 a uto correlation cases res p ectiv ely . In general, co verage calibra- tion and all three Studen tization metho ds succeed in reducing co v erage error of I ( α ) when the latter is noticeably inaccurate suc h as for θ = V ar( X 1 ). Our prop osed I C ( α ) and I S ( α ) either outp erform or are comparable to I D H ( α ) 15 and I GK ( α ) in the v ariance and lag 1 aut o correlation cases. Note that I GK ( α ) is exce ptio na lly p o or for small α in the auto correlation case. All five confi- dence b o unds hav e similar p erformance when θ = E [ X 1 ]. 5 Conclus ion W e ha v e prop o sed tw o double b o otstrap approac hes, one for calibrating the nominal co v erage and the other for calculating the Studen tizing factor, t o impro ving accuracy of the blo c k b o otstrap confidence in terv al. The main adv antage of the prop osed approac hes lies in the ease with whic h t he second- lev el blo c k length k can b e determined, namely half the first-lev el blo c k length, and the Studen tizing f actor can be computed, essen tially by a trivial application of the plug- in principle. Not in the literature has the same degree of impro ve men t b een achiev ed without ana lytic deriv a t ion of the Studen tiz- ing factor in a highly pro blem-sp ecific manner. The problem of empirical determination of the first-level blo ck length ℓ has b een dealt with b y v a r- ious a utho rs but metho ds whic h hav e prov en satisfactory p erfo rmance are not y et av aila ble. Both theoretical and empirical findings suggest that o ur prop osed co v erage calibration or Studentization approa c hes are effectiv e in reducing co v erage error eve n in the absence of a sophisticated data- based sc heme for selecting ℓ in the confidence pro cedure. While implemen tation 16 of the approaches is analytically effortless, the only price to pay is the extra computational cost induced by the second lev el of blo c k b o otstrapping. Although our fo cus is confined to the smo oth function mo del setting, it is b eliev ed that similar results extend also to von Mises-t yp e functionals as w ell a s to estimating functions, after appropriate mo difications of the pro of of our main theorem. Extension to dep endence structures outside the presen t framew ork, suc h as series exhibiting long- range dep endence, is less trivial and w orth in v estigating in future studies. 6 App endix 6.1 Pro of of Th eorem 1 W e first state a few lemmas concerning momen ts o f cen tred sums of stationary observ ations and t heir b o otstrap counte rpart s. Define S ∗ n ≡ ( bℓ ) 1 / 2  ¯ X ∗ − E ∗ ¯ X ∗  and S ∗∗ n ≡ ( ck ) 1 / 2  ¯ X ∗∗ − E ∗∗ ¯ X ∗∗  . De- fine, for i = 0 , ± 1 , . . . and r = 1 , 2 , . . . , Z i = X i − µ and V i,r = r − 1 / 2 P i + r − 1 s = i Z s . Lemma 1 Unde r the c onditions of The or e m 1, we have, for r = 1 , 2 , 3 , 4 and s 1 , s 2 , . . . , s r = 1 , . . . , d , V a r  P n ′ i =1 V ( s 1 ) i,ℓ · · · V ( s r ) i,ℓ /n ′  = O ( ℓ n − 1 ) . Lemma 1 follows immediately from Lemma 3.1 of La hiri (2003, Section 3.2.1). 17 Lemma 2 Unde r the c onditions o f The or em 1, we h ave, for r, s, t, u = 1 , . . . , d and as m → ∞ , E h V ( r ) 1 ,m V ( s ) 1 ,m i = χ r,s 2 , 1 + m − 1 χ r,s 2 , 2 + O ( m − 2 ) , E h V ( r ) 1 ,m V ( s ) 1 ,m V ( t ) 1 ,m i = m − 1 / 2 χ r,s,t 3 , 1 + O ( m − 3 / 2 ) , E h V ( r ) 1 ,m V ( s ) 1 ,m V ( t ) 1 ,m V ( u ) 1 ,m i = χ r,s,t,u 4 , 1 + O ( m − 1 ) , wher e χ r,s 2 , 1 , χ r,s 2 , 2 and χ r,s,t 3 , 1 ar e c onstants indep endent of m , and χ r,s,t,u 4 , 1 = χ r,s 2 , 1 χ t,u 2 , 1 + χ r,t 2 , 1 χ s,u 2 , 1 + χ r,u 2 , 1 χ s,t 2 , 1 . Lemma 2 can b e established using a rgumen ts similar to those for pro ving the univ ariate case: see G¨ otze and Hipp (1983). A generic first- lev el blo c k b o otstrap series X ∗ can b e r epresen ted as the ordered sequenc e o f observ a tions in ( Y N 1 ,ℓ , . . . , Y N b ,ℓ ), where N 1 , . . . , N b are indep enden t random v ariables uniformly distributed ov er { 1 , 2 , . . . , n ′ } . De- fine, fo r i = 0 , ± 1 , . . . , r = 1 , 2 , . . . and s 1 , s 2 , . . . , s r = 1 , . . . , d , Q s 1 ,...,s r i = ℓ ′ X j =1 V ( s 1 ) i + j − 1 ,k · · · V ( s r ) i + j − 1 ,k /ℓ ′ − E h V ( s 1 ) 1 ,k · · · V ( s r ) 1 ,k i , Q s 1 ,...,s r = ( n ′ ) − 1 n ′ X i =1 Q s 1 ,...,s r i , ˜ Q s 1 ,...,s r = b − 1 b X j =1 Q s 1 ,...,s r N j − Q s 1 ,...,s r and P s 1 ,...,s r = P n ′ i =1 V ( s 1 ) i,ℓ · · · V ( s r ) i,ℓ /n ′ − E h V ( s 1 ) 1 ,ℓ · · · V ( s r ) 1 ,ℓ i . W rite ˘ ℓ = min( ℓ ′ , k ) and ¯ ℓ = max( ℓ ′ , k ). 18 Lemma 3 Unde r the c onditions of T he or em 1, we have Q s 1 ,...,s r = O p  n − 1 / 2 k 1 / 2  and ˜ Q s 1 ,...,s r = O p  n − 1 / 2 ( ℓ ˘ ℓ/ℓ ′ ) 1 / 2  for r = 1 , 2 , 3 , 4 an d s 1 , s 2 , . . . , s r = 1 , . . . , d . Pr o of of L emma 3 . F or r, s, . . . = 1 , . . . , d and −∞ < i 1 , i 2 , . . . < ∞ , write ξ r,s,... i 1 ,i 2 ,... = Z ( r ) i 1 Z ( s ) i 2 · · · and ¯ ξ r,s,... i 1 ,i 2 ,... = ξ r,s,... i 1 ,i 2 ,... − E ξ r,s,... i 1 ,i 2 ,... . Consider first E ( Q s 1 ,...,s r 1 ) 2 = k − r ( ℓ ′ ) − 2 ℓ ′ X i,j =1 k − 1 X i 1 ,...,i r ,j 1 ,...,j r =0 E  ¯ ξ s 1 ,...,s r i + i 1 ,...,i + i r ¯ ξ s 1 ,...,s r j + j 1 ,...,j + j r  = O ( k − r +1 ( ℓ ′ ) − 1 ℓ ′ X j =1 k − 1 X i 2 ,...,i r ,j 1 ,...,j r =0 E  ¯ ξ s 1 ,...,s r 0 ,i 2 ,...,i r ¯ ξ s 1 ,...,s r j + j 1 ,...,j + j r  ) , whic h follo ws by statio na rit y of the series { Z j : −∞ < j < ∞} a nd a bac kw ard shift of i + i 1 units. Under the ass umed mixing conditions, the last exp ectatio n has order O ( n − K ) f or arbitrarily large K if the observ a t ions in ¯ ξ s 1 ,...,s r 0 ,i 2 ,...,i r and ¯ ξ s 1 ,...,s r j + j 1 ,...,j + j r are at least K log n units apart. W e can therefore restrict, up to O ( n − K ), the first sum to that ov er j = 1 , . . . , ˘ ℓ , so that E ( Q s 1 ,...,s r 1 ) 2 has order O ( k − r +1 ( ℓ ′ ) − 1 ˘ ℓ k − 1 X i 2 ,...,i r ,j 1 ,...,j r =0 E   ξ s 1 ,...,s r ,s 1 ,...,s r 0 ,i 2 ,...,i r ,j 1 ,...,j r   ) = O ( ( ℓ ′ ) − 1 ˘ ℓ max p,q ∈{ s 1 ,...,s r } ∞ X j = −∞ E   ξ p,q 0 ,j   ! r ) = O  ˘ ℓ/ℓ ′  . (3) Noting tha t V ar ∗ ( ˜ Q s 1 ,...,s r ) = ( bn ′ ) − 1 n ′ X i =1 ( Q s 1 ,...,s r i − Q s 1 ,...,s r ) 2 = O p  E ( Q s 1 ,...,s r 1 ) 2 /b  , 19 w e ha v e ˜ Q s 1 ,...,s r = O p  b − 1 / 2 ( ˘ ℓ/ℓ ′ ) 1 / 2  = O p  n − 1 / 2 ( ℓ ˘ ℓ/ℓ ′ ) 1 / 2  . Using similar argumen ts, w e see that E ( Q s 1 ,...,s r ) 2 = O ( k − r +1 ( n ′ ) − 1 ¯ ℓ − 1 X t =0 k − 1 X i 2 ,...,i r ,j 1 ,...,j r =0 E  ¯ ξ s 1 ,...,s r 0 ,i 2 ,...,i r ¯ ξ s 1 ,...,s r t + j 1 ,...,t + j r  ) = O ( k − r +2 ( n ′ ) − 1 k − 1 X i 2 ,...,i r ,j 1 ,...,j r =0 E   ξ s 1 ,...,s r ,s 1 ,...,s r 0 ,i 2 ,...,i r ,j 1 ,...,j r   ) = O ( k /n ′ ) , so that Q s 1 ,...,s r = O p  ( k /n ′ ) 1 / 2  . Lemma 4 Unde r the c onditions o f The or em 1, we h ave, for r, s, t, u = 1 , . . . , d , E ∗  S ∗ ( r ) n S ∗ ( s ) n  = E  S ( r ) n S ( s ) n  + P r,s + ℓ − 1 χ r,s 2 , 2 + O p ( ℓn − 1 + ℓ − 2 ) , E ∗  S ∗ ( r ) n S ∗ ( s ) n S ∗ ( t ) n  = E  S ( r ) n S ( s ) n S ( t ) n  + O p ( ℓn − 1 + ℓ − 1 n − 1 / 2 ) , E ∗  S ∗ ( r ) n S ∗ ( s ) n S ∗ ( t ) n S ∗ ( u ) n  = E  S ( r ) n S ( s ) n S ( t ) n S ( u ) n  + O p ( ℓ 1 / 2 n − 1 / 2 + ℓ − 1 ) . Pr o of of L emma 4 . Note first that, by Lemma 1, P r , P r,s , P r,s,t and P r,s,t,u ha v e order O p ( ℓ 1 / 2 n − 1 / 2 ) for r, s, t, u = 1 , . . . , d . Lemma 2 then implies that E ∗  S ∗ ( r ) n S ∗ ( s ) n  = P r,s + E h V ( r ) 1 ,ℓ V ( s ) 1 ,ℓ i − P r P s = P r,s + χ r,s 2 , 1 + ℓ − 1 χ r,s 2 , 2 + O p ( ℓn − 1 + ℓ − 2 ) . By Lemma 2 again, w e hav e E h S ( r ) n S ( s ) n i = χ r,s 2 , 1 + O ( n − 1 ) and the first result follo ws. 20 Similarly , t he second a nd third results follow by noting Lemma 2 and that E ∗  S ∗ ( r ) n S ∗ ( s ) n S ∗ ( t ) n  = b − 1 / 2 E h V ( r ) 1 ,ℓ V ( s ) 1 ,ℓ V ( t ) 1 ,ℓ i + O p ( b − 1 / 2 ℓ 1 / 2 n − 1 / 2 ) = n − 1 / 2 χ r,s,t 3 , 1 + O p ( ℓ − 1 n − 1 / 2 + ℓn − 1 ) and E ∗  S ∗ ( r ) n S ∗ ( s ) n S ∗ ( t ) n S ∗ ( u ) n  = b − 1  χ r,s,t,u 4 , 1 + O ( ℓ − 1 ) + O p ( ℓ 1 / 2 n − 1 / 2 )  + (1 − b − 1 )  χ r,s 2 , 1 χ t,u 2 , 1 + χ r,t 2 , 1 χ s,u 2 , 1 + χ r,u 2 , 1 χ s,t 2 , 1 + O p ( ℓ − 1 + ℓ 1 / 2 n − 1 / 2 )  = χ r,s,t,u 4 , 1 + O p ( ℓ − 1 + ℓ 1 / 2 n − 1 / 2 ) . A gene ric second-lev el blo ck b o o t strap series X ∗∗ can be iden t ified a s the ordered sequence of observ ations in ( Y ∗ I 1 ,J 1 ,k , . . . , Y ∗ I c ,J c ,k ) = ( Y N I 1 + J 1 ,k , . . . , Y N I c + J c ,k ), where the I j and J j are independen t random num b ers distributed uniformly o v er { 1 , 2 , . . . , b } and { 1 , 2 , . . . , ℓ ′ } resp ectiv ely , b oth indep enden tly of ( N 1 , . . . , N b ). Th us w e can write S ∗∗ n = c − 1 / 2 P c i =1 V N I i + J i − 1 ,k − c 1 / 2 ( bℓ ′ ) − 1 P b i =1 P ℓ ′ j =1 V N i + j − 1 ,k . Lemma 5 Unde r the c onditions o f The or em 1, we h ave, for r, s, t, u = 21 1 , . . . , d , E ∗∗  S ∗∗ ( r ) n S ∗∗ ( s ) n  = E ∗  S ∗ ( r ) n S ∗ ( s ) n  − P r,s + ˜ Q r,s + Q r,s + ( k − 1 − ℓ − 1 ) χ r,s 2 , 2 + O p ( ℓn − 1 + k − 2 ) , E ∗∗  S ∗∗ ( r ) n S ∗∗ ( s ) n S ∗∗ ( t ) n  = E ∗  S ∗ ( r ) n S ∗ ( s ) n S ∗ ( t ) n  + O p ( ℓn − 1 + k − 1 n − 1 / 2 ) , E ∗∗  S ∗∗ ( r ) n S ∗∗ ( s ) n S ∗∗ ( t ) n S ∗∗ ( u ) n  = E ∗  S ∗ ( r ) n S ∗ ( s ) n S ∗ ( t ) n S ∗ ( u ) n  + O p ( ℓ 1 / 2 n − 1 / 2 + k − 1 ) . Pr o of of L emma 5 . It follows from Lemmas 2 and 3 that E ∗∗  S ∗∗ ( r ) n S ∗∗ ( s ) n  = ˜ Q r,s + Q r,s + E h V ( r ) 1 ,k V ( s ) 1 ,k i −  ˜ Q r + Q r   ˜ Q s + Q s  = ˜ Q r,s + Q r,s + E  S ( r ) n S ( s ) n  + k − 1 χ r,s 2 , 2 + O p  k − 2 + n − 1 k + n − 1 ℓ ˘ ℓ/ℓ ′  . The first result then follows b y subtracting the expression for E ∗ h S ∗ ( r ) n S ∗ ( s ) n i stated in Lemma 4. Similar arg umen ts show that E ∗∗  S ∗∗ ( r ) n S ∗∗ ( s ) n S ∗∗ ( t ) n  = c − 1 / 2 n E h V ( r ) 1 ,k V ( s ) 1 ,k V ( t ) 1 ,k i + O p ( ˜ Q r,s,t + Q r,s,t ) o = E  S ( r ) n S ( s ) n S ( t ) n  + O p  k − 1 n − 1 / 2 + k n − 1 + n − 1 ( k ℓ ˘ ℓ/ℓ ′ ) 1 / 2  22 and E ∗∗  S ∗∗ ( r ) n S ∗∗ ( s ) n S ∗∗ ( t ) n S ∗∗ ( u ) n  = χ r,s 2 , 1 χ t,u 2 , 1 + χ r,t 2 , 1 χ s,u 2 , 1 + χ r,u 2 , 1 χ s,t 2 , 1 + O p  k − 1 + k 1 / 2 n − 1 / 2 + n − 1 / 2 ( ℓ ˘ ℓ/ℓ ′ ) 1 / 2  , whic h, on subtracting E ∗ h S ∗ ( r ) n S ∗ ( s ) n S ∗ ( t ) n i and E ∗ h S ∗ ( r ) n S ∗ ( s ) n S ∗ ( t ) n S ∗ ( u ) n i as ex- pressed in Lemma 4, yield the other tw o results. Set L n = K log n for some large K > 0. F or r, s, . . . = 1 , . . . , d a nd −∞ < p, q , i, i 1 , i 2 , . . . < ∞ , recall the definitions of ξ r,s,... i 1 ,i 2 ,... and ¯ ξ r,s,... i 1 ,i 2 ,... in the pro of of Lemma 3 , and split the sum S n = S i,p,q + ¯ S i,p,q = S i,p + ¯ S i,p suc h that S i,p,q = n − 1 / 2 X | j − ( i + p ) |∨| j − ( i + q ) |≤ L n Z j and S i,p = n − 1 / 2 X | j − ( i + p ) |≤ L n Z j . Lemma 6 Unde r the c onditions o f The or em 1, we h ave, for r, s, t, u = 1 , . . . , d , ( n ′ ℓ ) − 1 n ′ X i =1 ℓ − 1 X p =0 Z ( r ) i + p S ( s ) i,p = n − 1 / 2 χ r,s 2 , 1 + O p ( ℓn − 3 / 2 + n − 1 L 1 / 2 n ) , (4) ( n ′ ℓ ) − 1 n ′ X i =1 ℓ − 1 X p =0 Z ( r ) i + p S ( s ) i,p S ( t ) i,p = O p ( n − 1 ) , (5) ( n ′ ℓ ) − 1 n ′ X i =1 ℓ − 1 X p,q =0 ¯ ξ r,s i + p,i + q S ( t ) i,p,q = n − 1 / 2 ∞ X i,j = −∞ γ r,s,t i,j + O p ( ℓ − 1 n − 1 / 2 L n + ℓn − 1 ) , (6) 23 ( n ′ ℓ ) − 1 n ′ X i =1 ℓ − 1 X p,q =0 ¯ ξ r,s i + p,i + q S ( t ) i,p,q S ( u ) i,p,q = O p ( ℓn − 1 ) . (7) Pr o of of L emma 6 . W e outline the pro of of (6) and (7); that of (4) and (5) follo ws b y similar, alb eit simpler, arguments . Consider first Π r,s t ≡ P n ′ i =1 P ℓ − 1 p,q =0 P ( i + p,i + q ) j ¯ ξ r,s i + p,i + q Z ( t ) j , where P ( i 1 ,i 2 ) j denotes summation ov er j satisfying | j − i 1 | ∨ | j − i 2 | ≤ L n . Note that t he v ariance of Π r,s t has leading term n ′ ℓ X | q |≤ ℓ X j (0 ,q ) X | i ′ |≤ n ′ X | p ′ |∨| q ′ |≤ ℓ X j ′ ( i ′ + p ′ ,i ′ + q ′ ) E  ¯ ξ r,s 0 ,q Z ( t ) j − E [ ¯ ξ r,s 0 ,q Z ( t ) j ]   ¯ ξ r,s i ′ + p ′ ,i ′ + q ′ Z ( t ) j ′ − E [ ¯ ξ r,s i ′ + p ′ ,i ′ + q ′ Z ( t ) j ′ ]  ∼ n ′ ℓ 2 X | q |≤ ℓ X j (0 ,q ) X | p ′ |∨| q ′ |≤ ℓ X j ′ ( p ′ ,q ′ ) E  ¯ ξ r,s 0 ,q Z ( t ) j − E [ ¯ ξ r,s 0 ,q Z ( t ) j ]   ¯ ξ r,s p ′ ,q ′ Z ( t ) j ′ − E [ ¯ ξ r,s p ′ ,q ′ Z ( t ) j ′ ]  = O    n ′ ℓ 2 X | q |≤ ℓ X j (0 ,q ) X | p ′ |∨| q ′ |≤ ℓ X j ′ ( p ′ ,q ′ ) E   ξ r,s,t,r ,s ,t 0 ,q ,j,p ′ ,q ′ ,j ′      = O    n ′ ℓ 4 max u,v ∈{ r,s,t } ∞ X j = −∞ E   ξ u,v 0 ,j   ! 3    = O ( ℓ 4 n ) , using stationarit y prop erties and the fact that if bo th i ′ + p ′ and i ′ + q ′ differ b y at least 3 L n from 0 and q , then E  ¯ ξ r,s 0 ,q Z ( t ) j − E [ ¯ ξ r,s 0 ,q Z ( t ) j ]   ¯ ξ r,s i ′ + p ′ ,i ′ + q ′ Z ( t ) j ′ − E [ ¯ ξ r,s i ′ + p ′ ,i ′ + q ′ Z ( t ) j ′ ]  = O ( n − K ) for arbitrarily large K > 0 under the assumed mixing conditions. On the 24 other hand, Π r,s t has mean ( n + O ( ℓ )) ℓ − 1 X p =0 ℓ − 1 − p X q = − p X j (0 ,q ) γ r,s,t q ,j = ( n + O ( ℓ ) )( ℓ + O ( L n )) X | q |≤ L n X | j − q |≤ L n γ r,s,t q ,j . It follows that Π r,s t has expansion nℓ P ∞ i,j = −∞ γ r,s,t i,j + O ( nL n ) + O p ( ℓ 2 n 1 / 2 ), whic h yields (6) on m ultiplying it b y n − 1 / 2 ( n ′ ℓ ) − 1 . Consider next Π r,s t,u ≡ P n ′ i =1 P ℓ − 1 p,q =0 P ( i + p,i + q ) j 1 ,j 2 ¯ ξ r,s i + p,i + q ξ t,u j 1 ,j 2 , which has mean of order n ′ ℓ X | q |≤ ℓ X j 1 ,j 2 (0 ,q )   E  ¯ ξ r,s 0 ,q ξ t,u j 1 ,j 2    = O ( n ′ ℓ 2 ∞ X j = −∞ E   ξ r,t 0 ,j   ∞ X i = −∞ E   ξ s,u 0 ,i   ) = O ( ℓ 2 n ) , and v ariance of order n ′ ℓ 2 X | q |≤ ℓ X j 1 ,j 2 (0 ,q ) X | p ′ |∨| q ′ |≤ ℓ X j ′ 1 ,j ′ 2 ( p ′ ,q ′ ) E  ¯ ξ r,s 0 ,q ξ t,u j 1 ,j 2 − E [ ¯ ξ r,s 0 ,q ξ t,u j 1 ,j 2 ]   ¯ ξ r,s p ′ ,q ′ ξ t,u j ′ 1 ,j ′ 2 − E [ ¯ ξ r,s p ′ ,q ′ ξ t,u j ′ 1 ,j ′ 2 ]  = O    n ′ ℓ 2 X | q | , | p ′ | , | q ′ |≤ ℓ X j 1 ,j 2 (0 ,q ) X j ′ 1 ,j ′ 2 ( p ′ ,q ′ ) E    ξ r,s,t,u,r,s,t,u 0 ,q ,j 1 ,j 2 ,p ′ ,q ′ ,j ′ 1 ,j ′ 2       = O ( ℓ 5 n ) . Th us (7) fo llows b y multiplyin g Π r,s t,u b y ( n ′ ℓ ) − 1 n − 1 . Consider next the decomp osition S n = S i,j,p, q + ¯ S i,j,p, q = S i,j,p + ¯ S i,j,p suc h that S i,j,p, q = n − 1 / 2 P ( i + j − 1+ p,i + j − 1+ q ) t Z t and S i,j,p = n − 1 / 2 P | t − ( i + j − 1+ p ) |≤ L n Z t , for −∞ < p, q , i, j < ∞ . Argumen ts similar to those for pro ving Lemma 6 can b e used t o establish: Lemma 7 Unde r the c onditions o f The or em 1, we h ave, for r, s, t, u = 25 1 , . . . , d , ( n ′ ℓ ′ k ) − 1 n ′ X i =1 ℓ ′ X j =1 k − 1 X p =0 Z ( r ) i + j − 1+ p S ( s ) i,j,p = n − 1 / 2 χ r,s 2 , 1 + O p ( ℓn − 3 / 2 + n − 1 L 1 / 2 n ) , ( n ′ ℓ ′ k ) − 1 n ′ X i =1 ℓ ′ X j =1 k − 1 X p =0 Z ( r ) i + j − 1+ p S ( s ) i,j,p S ( t ) i,j,p = O p ( n − 1 ) , ( n ′ ℓ ′ k ) − 1 n ′ X i =1 ℓ ′ X j =1 k − 1 X p,q =0 ¯ ξ r,s i + j − 1+ p,i + j − 1+ q S ( t ) i,j,p, q = n − 1 / 2 ∞ X i,j = −∞ γ r,s,t i,j + O p ( k − 1 n − 1 / 2 L n + k n − 1 + ℓn − 3 / 2 ) , ( n ′ ℓ ′ k ) − 1 n ′ X i =1 ℓ ′ X j =1 k − 1 X p,q =0 ¯ ξ r,s i + j − 1+ p,i + j − 1+ q S ( t ) i,j,p, q S ( u ) i,j,p, q = O p ( k n − 1 ) . W e now pro ceed with the pro of of Theorem 1. Define H r ( x ) = ( ∂ /∂ x ( r ) ) H ( x ), H r s = ( ∂ 2 /∂ x ( r ) ∂ x ( s ) ) H ( x ), etc., for r , s, . . . = 1 , . . . , d . Recall that we write H r = H r ( µ ), H r s = H r s ( µ ), etc. f o r con v enience. Note that ˆ µ ( r ) ≡ E ∗ ¯ X ∗ ( r ) = ℓ − 1 / 2 P r + µ ( r ) . W rite ˆ H r = H r ( ˆ µ ), ˆ H r s = H r s ( ˆ µ ), etc. T aylor expansion sho ws that V ar( n 1 / 2 ˆ θ ) has leading term σ 2 = P d r,s =1 H r H s E h S ( r ) n S ( s ) n i . Define ˆ σ 2 = P d r,s =1 ˆ H r ˆ H s E ∗ h S ∗ ( r ) n S ∗ ( s ) n i , whic h can, by Lemmas 2 and 4, b e T aylor expanded to giv e ˆ σ 2 = σ 2 + d X r,s =1 H r H s  ℓ − 1 χ r,s 2 , 2 + P r,s  +2 ℓ − 1 / 2 d X r,s,t =1 H r H st χ r,s 2 , 1 P t + O p ( ℓn − 1 + ℓ − 2 ) . (8) Lahiri (2003, Section 6.4.3 ) provides an Edgew orth expansion for the distri- bution function G o f n 1 / 2 ( ˆ θ − θ ): G ( x ) = Φ( x/σ ) − n − 1 / 2  K 31 + K 32 ( x 2 /σ 2 − 1)  φ ( x/σ ) + O ( n − 1 ) , (9) 26 where K 31 and K 32 are smo oth functions, b oth o f order O (1), of the momen ts E h S ( s 1 ) n · · · S ( s r ) n i , for s 1 , . . . , s r = 1 , . . . , d and r = 2 , 3 , 4, a nd Φ denotes the standard no rmal distribution function. La hiri’s (2003) Theorem 6.7 derives a blo c k b o otstrap v ersion of (9) under the conditions of our Theorem 1: G ∗ ( x ) = Φ( x/ ˆ σ ) − n − 1 / 2 h ˆ K 31 + ˆ K 32 ( x 2 / ˆ σ 2 − 1 ) i φ ( x/ ˆ σ ) + O p ( ℓn − 1 ) , (10) where ˆ K 31 and ˆ K 32 ha v e the same expressions as K 31 and K 32 with the p opu- lation mo ments E h S ( s 1 ) n · · · S ( s r ) n i replaced by E ∗ h S ∗ ( s 1 ) n · · · S ∗ ( s r ) n i . With the aid of Lemma 4 and the expressions (8), (9) and (10), w e can expand the difference b et w een G ∗− 1 and G − 1 , so that, for ξ ∈ (0 , 1), P  n 1 / 2 ( ˆ θ − θ ) ≤ G ∗− 1 ( ξ )  = P ( T n ≤ y ) + O ( ℓn − 1 + ℓ − 2 ) , (11) where T n = n 1 / 2 ( ˆ θ − θ ) − z ξ (2 σ ) − 1  P d r,s =1 H r H s P r,s + 2 ℓ − 1 / 2 P d r,s,t =1 H r H st χ r,s 2 , 1 P t  and y = G − 1 ( ξ ) + ℓ − 1 z ξ (2 σ ) − 1 P d r,s =1 H r H s χ r,s 2 , 2 . Noting that P r and P r,s are O p ( ℓ 1 / 2 n − 1 / 2 ) b y Lemma 3, that n 1 / 2 ( ˆ θ − θ ) = P d u =1 H u S ( u ) n + O p ( n − 1 / 2 ) and expanding the c haracteristic function of T n ab out that of n 1 / 2 ( ˆ θ − θ ), w e get, for β ∈ R , E e ιβ T n − E e ιβ n 1 / 2 ( ˆ θ − θ ) = − ιβ z ξ (2 σ ) − 1 d X r,s =1 H r H s E " P r,s exp ιβ d X u =1 H u S ( u ) n !# − ιβ z ξ σ − 1 ℓ − 1 / 2 d X r,s,t =1 H r H st χ r,s 2 , 1 E " P t exp ιβ d X u =1 H u S ( u ) n !# + O ( ℓn − 1 ) . (12) 27 Note that for s 1 , s 2 , s 3 = 1 , . . . , d , ( n ′ ℓ ) − 1      n ′ X i =1 ℓ − 1 X p,q =0 E h ¯ ξ r,s i + p,i + q S ( s 1 ) i,p,q S ( s 2 ) i,p,q S ( s 3 ) i,p,q i      ≤ ( n ′ ℓ ) − 1 n − 3 / 2 n ′ X i =1 ℓ − 1 X p,q =0 X i 1 ,i 2 ,i 3 ( i + p,i + q ) E   ¯ ξ r,s i + p,i + q ξ s 1 ,s 2 ,s 3 i 1 ,i 2 ,i 3   = O    n − 3 / 2 X | q |≤ ℓ X i 1 ,i 2 ,i 3 (0 ,q ) E   ¯ ξ r,s 0 ,q ξ s 1 ,s 2 ,s 3 i 1 ,i 2 ,i 3      = O ( ℓ n − 3 / 2 L 3 n ) . (13) It follows b y expansion of the exp onen tial function, (6), (7 ) and (13 ) that E " P r,s exp ιβ d X u =1 H u S ( u ) n !# = ( n ′ ℓ ) − 1 n ′ X i =1 ℓ − 1 X p,q =0 E " ¯ ξ r,s i + p,i + q exp ιβ d X u =1 H u S ( u ) n ! ( ιβ d X u =1 H u S ( u ) i,p,q + 2 − 1 β 2 d X t,u =1 H t H u S ( t ) i,p,q S ( u ) i,p,q + exp − ιβ d X u =1 H u S ( u ) i,p,q !)# + O ( ℓn − 3 / 2 L 3 n ) = ( n ′ ℓ ) − 1 n ′ X i =1 ℓ − 1 X p,q =0 E " ¯ ξ r,s i + p,i + q exp ιβ d X u =1 H u ¯ S ( u ) i,p,q !# + ιβ n − 1 / 2 d X t =1 H t ∞ X i,j = −∞ γ r,s,t i,j E h e ιβ n 1 / 2 ( ˆ θ − θ ) i + O ( ℓ − 1 n − 1 / 2 L n + ℓn − 1 ) = ιβ n − 1 / 2 d X t =1 H t ∞ X i,j = −∞ γ r,s,t i,j E h e ιβ n 1 / 2 ( ˆ θ − θ ) i + O ( ℓ − 1 n − 1 / 2 L n + ℓn − 1 ) . (14) The last equalit y fo llo ws b y the assumed mixing prop erties and noting that observ ations defining ¯ S ( u ) i,p,q and ¯ ξ r,s i + p,i + q are at least L n units apart on the series and that E ¯ ξ r,s i + p,i + q = 0. Noting tha t ( n ′ ℓ ) − 1    P n ′ i =1 P ℓ − 1 p =0 E h Z ( r ) i + p S ( s 1 ) i,p S ( s 2 ) i,p S ( s 3 ) i,p i    = 28 O ( n − 3 / 2 L 3 n ) for s 1 , s 2 , s 3 = 1 , . . . , d , the same ar gumen ts sho w that ℓ − 1 / 2 E " P t exp ιβ d X u =1 H u S ( u ) n !# = ιβ n − 1 / 2 d X u =1 H u χ t,u 2 , 1 E h e ιβ n 1 / 2 ( ˆ θ − θ ) i + O ( ℓn − 3 / 2 + n − 1 L 1 / 2 n ) . (15) Substitution of (14) a nd (15) into (12) g iv es E e ιβ T n / E e ιβ n 1 / 2 ( ˆ θ − θ ) = 1 + n − 1 / 2 (2 σ ) − 1 β 2 z ξ d X r,s,t =1 H r H s H t ∞ X i,j = −∞ γ r,s,t i,j + n − 1 / 2 σ − 1 β 2 z ξ d X r,s,t,u =1 H r H u H st χ r,s 2 , 1 χ t,u 2 , 1 + O ( ℓn − 1 + ℓ − 1 n − 1 / 2 L n ) . (16) It follows b y in v erse F o urier- transforming E e ιβ T n that P ( T n ≤ x ) = G ( x ) + n − 1 / 2 2 − 1 σ − 4 z ξ xφ ( x/σ ) × ( d X r,s,t =1 H r H s H t ∞ X i,j = −∞ γ r,s,t i,j + 2 d X r,s,t,u =1 H r H u H st χ r,s 2 , 1 χ t,u 2 , 1 ) + O ( ℓn − 1 + ℓ − 1 n − 1 / 2 L n ) . (17) It then follow s by com bining (11) a nd (17), setting x = y and noting that y = σ z ξ + O ( n − 1 / 2 + ℓ − 1 ) that P  n 1 / 2 ( ˆ θ − θ ) ≤ G ∗− 1 ( ξ )  = ξ + ℓ − 1 2 − 1 σ − 2 z ξ φ ( z ξ ) d X r,s =1 H r H s χ r,s 2 , 2 + n − 1 / 2 2 − 1 σ − 3 z 2 ξ φ ( z ξ ) × ( d X r,s,t =1 H r H s H t ∞ X i,j = −∞ γ r,s,t i,j + 2 d X r,s,t,u =1 H r H u H st χ r,s 2 , 1 χ t,u 2 , 1 ) + O ( ℓn − 1 + ℓ − 2 ) , 29 whic h yields (1) on setting ξ = 1 − α and taking complemen t. F or pro ving (2), write µ ∗ = E ∗∗ ¯ X ∗∗ , H ∗ r = H r ( µ ∗ ), H ∗ r s = H r s ( µ ∗ ) etc. and define σ ∗ 2 = P d r,s =1 H ∗ r H ∗ s E ∗∗ h S ∗∗ ( r ) n S ∗∗ ( s ) n i . Note that, for r = 1 , . . . , d , µ ∗ ( r ) − ˆ µ ( r ) = k − 1 / 2  ˜ Q r + Q r  − ℓ − 1 / 2 P r = O p ( n − 1 / 2 ) b y Lemmas 1 and 3. It follows b y Lemma 5 and T ay lor expansion that σ ∗ 2 = ˆ σ 2 + d X r,s =1 H r H s h ˜ Q r,s + Q r,s − P r,s + ( k − 1 − ℓ − 1 ) χ r,s 2 , 2 i + 2 d X r,s,t =1 H r H st χ r,s 2 , 1 h k − 1 / 2  ˜ Q t + Q t  − ℓ − 1 / 2 P t i + O p ( n − 1 ℓ + k − 2 ) , (18) using the fact that ˆ µ = µ + O p ( n − 1 / 2 ). Denote by K ∗ 31 and K ∗ 32 the ve rsions of K 31 and K 32 with the mo ments E h S ( s 1 ) n · · · S ( s r ) n i replaced by E ∗∗ h S ∗∗ ( s 1 ) n · · · S ∗∗ ( s r ) n i in their definitions. Th us, by analogy with (10), we ha v e G ∗∗ ( x ) = Φ( x/σ ∗ ) − n − 1 / 2  K ∗ 31 + K ∗ 32 ( x 2 /σ ∗ 2 − 1)  φ ( x/σ ∗ )+ O p ( k n − 1 ) . (19) The expansions (10), (18), (19) and the results in Lemma 5 enable us to expand G ∗∗− 1 ( ξ ) a b out G ∗− 1 ( ξ ) a nd write P ∗  ( bℓ ) 1 / 2 [ H ( ¯ X ∗ ) − H ( E ∗ ¯ X ∗ )] ≤ G ∗∗− 1 ( ξ )  = P ∗  ( bℓ ) 1 / 2 [ H ( ¯ X ∗ ) − H ( E ∗ ¯ X ∗ )] − ∆ ∗ n ≤ ˆ y  + O p ( ℓn − 1 + k − 2 ) , ( 20) where ∆ ∗ n = b − 1 P b j =1 R ∗ j , R ∗ j = ( 2 ˆ σ ) − 1 z ξ ( d X r,s =1 H r H s ( Q r,s N j − Q r,s ) + 2 k − 1 / 2 d X r,s,t =1 H r H st χ r,s 2 , 1 ( Q t N j − Q t ) ) 30 and ˆ y = G ∗− 1 ( ξ ) + (2 ˆ σ ) − 1 z ξ ( d X r,s =1 H r H s  Q r,s − P r,s + ( k − 1 − ℓ − 1 ) χ r,s 2 , 2  + 2 d X r,s,t =1 H r H st χ r,s 2 , 1  k − 1 / 2 Q t − ℓ − 1 / 2 P t  ) . Define a lso Y ∗ j = P d r =1  V ( r ) N j ,ℓ − P r  ˆ H r for j = 1 , . . . , b , so that ( bℓ ) 1 / 2 [ H ( ¯ X ∗ ) − H ( E ∗ ¯ X ∗ )] = b − 1 / 2 P b j =1 Y ∗ j + O p ( n − 1 / 2 ) by T a ylor expansion. Note that the observ ations ( Y ∗ j , R ∗ j ) a re indep enden t, zero-mean and identically distributed with resp ect to first-lev el blo c k b o otstrap sampling, conditio na l on X . W e see by Lemma 3 that ∆ ∗ n = O p ( k 1 / 2 n − 1 / 2 ) a nd b y (3) tha t R ∗ j = O p (( ˘ ℓ/ℓ ′ ) 1 / 2 ), whereas Y ∗ j = O p (1) b y Lemmas 1 and 2. It follo ws that, conditional on X , ( bℓ ) 1 / 2 [ H ( ¯ X ∗ ) − H ( E ∗ ¯ X ∗ )] − ∆ ∗ n and ( bℓ ) 1 / 2 [ H ( ¯ X ∗ ) − H ( E ∗ ¯ X ∗ )] hav e iden- tical means, v ariances differing by − 2 b − 1 / 2 E ∗ [ Y ∗ 1 R ∗ 1 ] + O p ( k n − 1 ) and third cum ulan ts differing b y − 3 b − 1 E ∗ [ Y ∗ 2 1 R ∗ 1 ] + O p ( k n − 1 ) = O p ( ℓn − 1 ). Suc h cu- m ulan t differences can b e emplo ye d t o establish an Edgew orth expansion for ( bℓ ) 1 / 2 [ H ( ¯ X ∗ ) − H ( E ∗ ¯ X ∗ )] − ∆ ∗ n analogous to (10), bearing in mind t ha t ˆ K 31 and ˆ K 32 stem fro m the first and third cum ulan ts resp ectiv ely: P ∗  ( bℓ ) 1 / 2 [ H ( ¯ X ∗ ) − H ( E ∗ ¯ X ∗ )] − ∆ ∗ n ≤ x  = G ∗ ( x ) + b − 1 / 2 ˆ σ − 3 xφ ( x/ ˆ σ ) E ∗ [ Y ∗ 1 R ∗ 1 ] + O p ( ℓn − 1 ) . (21) 31 Note by Lemmas 1, 2 and 3 that for r , s, t = 1 , . . . , d , Co v ∗  V ( r ) N 1 ,ℓ , Q s,t N 1  = ( n ′ ℓ ′ ) − 1 n ′ X i =1 ℓ ′ X j =1 V ( r ) i,ℓ V ( s ) i + j − 1 ,k V ( t ) i + j − 1 ,k − P r E [ V ( s ) 1 ,k V ( t ) 1 ,k ] − P r Q s,t = ( n ′ ℓ ′ k ) − 1 ℓ − 1 / 2 n ′ X i =1 ℓ ′ X j =1 ℓ − 1 X a =0 k − 1 X p,q =0 ξ r,s,t i + a,i + j − 1+ p,i + j − 1+ q + O p ( ℓ 1 / 2 n − 1 / 2 ) . (22) Consider ℓ ′ X j =1 ℓ − 1 X a =0 k − 1 X p,q =0 E ξ r,s,t a,j − 1+ p,j − 1+ q = k − 1 X p,q =0 ℓ − 1 X a =1 − ℓ ′ { ( ℓ − a ) ∧ ℓ ′ − ( 1 − a ) ∨ 1 + 1 } E ξ r,s,t a,p,q = X | p | , | q |≤ L n   k + O ( L n ) X a = O ( L n ) { ( ℓ − a ) ∧ ℓ ′ − ( 1 − a ) ∨ 1 + 1 }   γ r,s,t p,q =  k ℓ ′ + O ( ℓ ′ L n + L 2 n )  ∞ X p,q = −∞ γ r,s,t p,q and V ar ( n ′ ) − 1 n ′ X i =1 ℓ ′ X j =1 ℓ − 1 X a =0 k − 1 X p,q =0 ξ r,s,t i + a,i + j − 1+ p,i + j − 1+ q ! ∼ ( n ′ ) − 1 k ℓ ′ X | i ′ |≤ n ′ X | j ′ |≤ ℓ ′ X | a | , | a ′ |≤ ℓ X | q | , | p ′ | , | q ′ |≤ k E  ¯ ξ r,s,t a, 0 ,q ¯ ξ r,s,t i ′ + a ′ ,i ′ + j ′ − 1+ p ′ ,i ′ + j ′ − 1+ q ′  = O    n − 1 k ℓ ′ ℓ X | j ′ |≤ ℓ ′ X | i ′ | , | a |≤ ℓ X | q | , | p ′ | , | q ′ |≤ k E   ξ r,s,t,r ,s ,t a, 0 ,q ,i ′ ,i ′ + j ′ − 1+ p ′ ,i ′ + j ′ − 1+ q ′      = O    n − 1 ( k ℓ ′ ℓ ) 2 X | q |≤ k E   ξ s,t 0 ,q   X | a |≤ ℓ E   ξ r,r a, 0   X | q ′ |≤ k E   ξ s,t 0 ,q ′      = O ( n − 1 ( k ℓ ′ ℓ ) 2 ) , 32 so that ( n ′ ) − 1 n ′ X i =1 ℓ ′ X j =1 ℓ − 1 X a =0 k − 1 X p,q =0 ξ r,s,t i + a,i + j − 1+ p,i + j − 1+ q = k ℓ ′ ∞ X i,j = −∞ γ r,s,t i,j + O p ( ℓ ′ L n + L 2 n + k ℓ ′ ℓn − 1 / 2 ) . (23) Similar arg umen ts sho w that Co v ∗  V ( r ) N 1 ,ℓ , Q t N 1  = ( k /ℓ ) 1 / 2 ∞ X a = −∞ γ r,t a + O p  ( k ℓ ) − 1 / 2 L n + ( ℓ ′ ) − 1 ( k ℓ ) − 1 / 2 L 2 n + k 1 / 2 n − 1 / 2  . (24) Com bining (22)–(24), we ha ve E ∗ [ Y ∗ 1 R ∗ 1 ] = ℓ − 1 / 2 (2 ˆ σ ) − 1 z ξ ( d X r,s,t =1 H r H s H t ∞ X i,j = −∞ γ r,s,t i,j + 2 d X r,s,t,u =1 H r H s H tu χ r,t 2 , 1 χ s,u 2 , 1 ) + O p ( ℓ − 1 / 2 k − 1 L n + ( k ℓ ′ ) − 1 ℓ − 1 / 2 L 2 n + ℓ 1 / 2 n − 1 / 2 ) . (25) Substitution of (25) into (21), setting x = ˆ y and not ing (20), w e hav e P ∗  ( bℓ ) 1 / 2 [ H ( ¯ X ∗ ) − H ( E ∗ ¯ X ∗ )] ≤ G ∗∗− 1 ( ξ )  = ξ + 2 − 1 σ − 2 z ξ φ ( z ξ ) ( d X r,s =1 H r H s  Q r,s − P r,s + ( k − 1 − ℓ − 1 ) χ r,s 2 , 2  + 2 d X r,s,t =1 H r H st χ r,s 2 , 1  k − 1 / 2 Q t − ℓ − 1 / 2 P t  ) + n − 1 / 2 2 − 1 σ − 3 z 2 ξ φ ( z ξ ) ( d X r,s,t =1 H r H s H t ∞ X i,j = −∞ γ r,s,t i,j + 2 d X r,s,t,u =1 H r H s H tu χ r,t 2 , 1 χ s,u 2 , 1 ) + O p ( ℓn − 1 + k − 2 ) , 33 in v ersion of whic h giv es ˆ α = α + δ n + B n + O p ( ℓn − 1 + k − 2 ), where δ n = − ( k − 1 − ℓ − 1 )2 − 1 σ − 2 z α φ ( z α ) d X r,s =1 H r H s χ r,s 2 , 2 + n − 1 / 2 2 − 1 σ − 3 z 2 α φ ( z α ) × d X r,s,t =1 H r H s H t ∞ X i,j = −∞ γ r,s,t i,j + 2 d X r,s,t,u =1 H r H s H tu χ r,t 2 , 1 χ s,u 2 , 1 ! , B n = − 2 − 1 σ − 2 z α φ ( z α ) × ( d X r,s =1 H r H s ( Q r,s − P r,s ) + 2 d X r,s,t =1 H r H st χ r,s 2 , 1  k − 1 / 2 Q t − ℓ − 1 / 2 P t  ) . It follows from (11) that the co v erage probability of I C ( α ) is 1 − P  n 1 / 2 ( ˆ θ − θ ) ≤ G ∗− 1 (1 − ˆ α )  = 1 − P ( ˜ T n ≤ ˜ y ) + O ( ℓn − 1 + k − 2 ) , (26) where ˜ T n = n 1 / 2 ( ˆ θ − θ ) + z α (2 σ ) − 1 d X r,s =1 H r H s P r,s + 2 ℓ − 1 / 2 d X r,s,t =1 H r H st χ r,s 2 , 1 P t ! + σ φ ( z α ) − 1 B n = n 1 / 2 ( ˆ θ − θ ) + z α (2 σ ) − 1 × ( d X r,s =1 H r H s (2 P r,s − Q r,s ) + 2 d X r,s,t =1 H r H st χ r,s 2 , 1 (2 ℓ − 1 / 2 P t − k − 1 / 2 Q t ) ) and ˜ y = G − 1 (1 − α − δ n ) − ℓ − 1 z α (2 σ ) − 1 P d r,s =1 H r H s χ r,s 2 , 2 . Similar to (14) and (15), E h Q r,s exp  ιβ P d u =1 H u S ( u ) n i and k − 1 / 2 E h Q t exp  ιβ P d u =1 H u S ( u ) n i can b e expanded b y in voking Lemma 7, so that the difference b et w een the c haracteristic functions of ˜ T n and n 1 / 2 ( ˆ θ − θ ) can b e established as in t he pro o f of (1 6). This enables us to derive an Edgew or t h expansion for ˜ T n analogous 34 to (1 7): P ( ˜ T n ≤ x ) = G ( x ) − n − 1 / 2 2 − 1 σ − 4 z α xφ ( x/σ ) × ( d X r,s,t =1 H r H s H t ∞ X i,j = −∞ γ r,s,t i,j + 2 d X r,s,t,u =1 H r H u H st χ r,s 2 , 1 χ t,u 2 , 1 ) + O ( ℓn − 1 + k − 1 n − 1 / 2 L n ) . (27) The co v erage expansion (2) for I C ( α ) then fo llo ws b y noting ( 26), setting x = ˜ y in (27) and T aylor expansion. It remains to prov e (2) for the Studen tized I S ( α ). W e see b y T a ylor expanding the smo oth function H ( · ) and the momen t relatio ns asserted in Lemmas 4 and 5 that ˆ τ 2 = ˆ σ 2 + O p ( n − 1 + ℓn − 3 / 2 ) and τ ∗ 2 = σ ∗ 2 + O p ( n − 1 + ℓn − 3 / 2 ). Expanding τ ∗ ab out ˆ σ based on (18), we ha v e, for ξ ∈ (0 , 1), J ∗ ( z ξ ) = P ∗  ( bℓ ) 1 / 2 [ H ( ¯ X ∗ ) − H ( E ∗ ¯ X ∗ )] − ∆ ∗ n ≤ ˆ w  + O p ( ℓn − 1 + k − 2 ) , where ∆ ∗ n is defined as in (20) and ˆ w = ˆ σ z ξ + (2 ˆ σ ) − 1 z ξ ( d X r,s =1 H r H s  Q r,s − P r,s + ( k − 1 − ℓ − 1 ) χ r,s 2 , 2  + 2 d X r,s,t =1 H r H st χ r,s 2 , 1  k − 1 / 2 Q t − ℓ − 1 / 2 P t  ) . Noting (2 5) and (21 ) , w e ha ve J ∗ ( z ξ ) = G ∗ ( ˆ w ) + n − 1 / 2 2 − 1 ˆ σ − 3 z 2 ξ φ ( z ξ ) × ( d X r,s,t =1 H r H s H t ∞ X i,j = −∞ γ r,s,t i,j + 2 d X r,s,t,u =1 H r H s H tu χ r,t 2 , 1 χ s,u 2 , 1 ) + O p ( ℓn − 1 + n − 1 / 2 k − 1 L n + ( k ℓ ′ ) − 1 n − 1 / 2 L 2 n ) . (28) 35 Recall the expression for ˆ α = α + δ n + B n + O p ( ℓn − 1 + k − 2 ). Putting z ξ = G ∗− 1 (1 − ˆ α ) / ˆ σ in (28), w e ve rify that J ∗ ( G ∗− 1 (1 − ˆ α ) / ˆ σ ) = 1 − α + O p ( ℓn − 1 + k − 2 ), so that ˆ τ J ∗− 1 (1 − α ) = G ∗− 1 (1 − ˆ α ) + O p ( ℓn − 1 + k − 2 ). Th us I S ( α ) is equiv alen t asymptotically t o I C ( α ) up to O p  n − 1 / 2 ( ℓn − 1 + k − 2 )  , yielding for its co v erage pro ba bilit y the same express ion as giv en by (26) up t o order O ( ℓn − 1 + k − 2 ). This completes the pr o of o f part (ii). 6.2 Other Studen tizing approac h es Under the smo oth function mo del setting, Davison and Hall (199 3) a nd G ¨ otze and K ¨ unsc h (1 996) suggest Studen tizing the blo ck b o o t stra p based on closed- form expressions. Their constructions are similar to that of our I S ( α ), ex- cept that ˆ τ and τ ∗ are replaced by closed-form expres sions dep ending on partial deriv ativ es { H r } of H . Sp ecifically , D avison and Hall (199 3) define ˆ τ 2 = P d r,s =1 H r ( ¯ X ) H s ( ¯ X ) ˆ Σ r s , where ˆ Σ r s = n − 1 P n i =1 ( X i − ¯ X ) ( r ) ( X i − ¯ X ) ( s ) + n − 1 P ℓ − 1 j =1 P n − j i =1 ( X i − ¯ X ) ( r ) ( X i + j − ¯ X ) ( s ) , and τ ∗ analogously with X replaced b y the blo ck b o o tstrap series X ∗ in t he ab o ve definition of ˆ τ . G¨ otze and K ¨ unsc h’s (1996) Studen tizing fa cto r s hav e similar expressions except that they define ˆ Σ r s = P ℓ − 1 j =0 w j n − 1 P n − ℓ i =1 ( X i − ¯ X ) ( r ) ( X i + j − ¯ X ) ( s ) , where w 0 = 1 and w j = 2 { 1 − c ( j /ℓ ) 2 } for 1 ≤ j ≤ ℓ − 1 and some c > 0, and its bo otstrap v ersion by b − 1 P b j =1 ℓ − 1 n P ℓ i =1 ( X ∗ ( j − 1) ℓ + i − ¯ X ∗ ) ( r ) o n P ℓ i =1 ( X ∗ ( j − 1) ℓ + i − ¯ X ∗ ) ( s ) o . 36 References [1] Bertail, P . (1 997). Second order prop erties of an extrapo lated b o otstrap without replacemen t: the i.i.d. and the strong mixing cases. Bernoul li , 3 , 149 -179. [2] Bhat t a c hary a, R.N. and Ghosh, J.K. (1978). On t he v alidit y of the for- mal Edgew orth expansion. Annals of S tatistics , 7 , 43 4-451. [3] B ¨ uhlmann, P . (1997). Siev e b o otstrap f o r time series. Bernoul li , 3 , 123- 148. [4] B ¨ uhlmann, P . (2002 ). Bo otstraps for time series. Statistic al Scienc e , 17 , 52-72. [5] Carlstein, E., Do, K.-A., Hall, P ., Hesterb erg , T. and K ¨ unsc h, H.R. (1998). Matched-block b o otstrap fo r dep enden t data. Bernoul li , 4 , 305- 328. [6] Choi, E. and Hall, P . (20 00). Bo otstrap confidence regions computed from autoregressions of arbitrary order. Journal of the R oyal Statistic al So cie ty, Series B , 62 , 461 -477. 37 [7] D a vison, A.C. and Hall, P . (1993). On Studentiz ing and blo c king meth- o ds for implemen ting the b o otstrap with dep enden t data. Austr alia n Journal of Statistics , 35 , 215 - 224. [8] G ¨ otze, F. and Hipp, C. (1983 ) . Asymptotic expansions fo r sums of w eakly dependent random vectors. Zeitschrift f¨ ur Wahrscheinlichkeit- sthe orie und V erwandte Gebiete , 64 , 21 1 -239. [9] G ¨ otze, F. and K ¨ unsc h, H.R. ( 1996). Blo ckw ise b o otstrap for dep en- den t observ at io ns: higher order appro ximations f or Studen t ized statis- tics. Annals of Statistics , 24 , 1914-19 3 3. [10] Hall, P . (1985). Resampling a co ve rag e pro cess. Sto chastic Pr o c es s Ap- plic ation s , 19 , 259-26 9 . [11] Hall, P . (1992 ) . The Bo otstr ap and Edgeworth Exp ans i o n . Springer- V erlag: New Y ork. [12] Hall, P ., Horo witz, J.L. and Jing, B.-Y. (1 995). On blo ck ing rules f o r the b o otstrap with dep enden t data. Biometrika , 82 , 5 61-574. [13] Hall, P . and Jing, B.-Y. (1996). On sample reuse metho ds for dep endent data. Journal of the R oyal Statistic al So ciety, Serie s B , 58 , 727-7 3 7. [14] H¨ ardle, W., Horo witz, J. and Kr eiss, J.-P . (2003). Bo ot stra p metho ds for time series. International Statistic al R eview , 71 , 435-459. 38 [15] K ¨ unsc h, H.R. (1989). The jac kknife and the b o o tstrap for general sta- tionary observ ations. Annals of Statistics , 17 , 1217 -1241. [16] Lahiri, S.N. (1992). Edgew or th correction b y ’mo ving blo c k’ b o otstrap for stationary and nonstationary data. In Exploring the Limits of Bo ot- str ap , Eds. R. LeP ag e and L. Billard, Wiley: New Y ork, pp. 183-214 . [17] Lahiri, S.N. (1999 ) . Theoretical comparisons of blo ck bo otstrap meth- o ds. Annals of Statistics , 27 , 38 6-404. [18] Lahiri, S.N. (2 0 03). R esa m pling Metho ds for Dep endent Data . Springer: New Y ork. [19] Liu, R.Y. and Singh, K. (1992). Mo ving blo c ks j ac kknife and b o o tstrap capture w eak dependence. In Explorin g the Limits of Bo otstr ap , E ds. R. LeP age a nd L. Billard, Wiley: New Y or k, pp. 225-2 48. [20] P aparo ditis, E. and P olitis, D.N. (2001 ). T ap ered blo c k b o otstrap. Biometrika , 88 , 1 105-111 9. [21] P aparo ditis, E. and P olitis, D.N. (2002 ). The lo cal b o o tstrap for Mark ov pro cesses. Journal of Statistic a l Planning and I nfer e nc e , 108 , 301-328 . [22] P olitis, D.N. and R omano, J.P . (1992). A general resampling sc heme for triangular arra ys of alpha-mixing random v ariables with application 39 to the problem of sp ectral density estimation. Annals of S tatistics , 20 , 1985-20 07. [23] P olitis, D.N. and R o mano, J.P . (1993). The stationary bo otstrap. Jour- nal of the A meric an Statistic al Asso ciation , 89 , 1303- 1313. [24] P olitis, D.N. and Romano, J.P . (1994). Larg e sample confiden ce regions based on subsamples under minimal assumptions. A nnal s of Statistics , 22 , 203 1-2050. [25] Ra jarshi, M.B. (1990). Bo ot stra p in Mark o v-sequences based on esti- mates of transition dens ity . A nnals of the Institute o f Statistic al Mathe- matics , 42 , 253- 2 68. [26] Zvingelis, J. (2 003). On b o otstrap cov erage probabilit y with dep endent data. In Computer-Aide d Ec onometrics , Ed. D. Giles, Marcel Dekk er: New Y ork, pp. 69-9 0. 40 n = 500 n = 100 0 nominal level α 0.05 0.1 0 0.90 0.95 0.05 0.1 0 0.90 0.95 (a) ARCH(1) series I ( α ) 0.053 0.099 0.89 7 0.943 0.03 4 0.110 0.903 0.939 I C ( α ) 0.056 0.096 0.89 7 0.942 0.03 7 0.113 0.903 0.936 I S ( α ) 0.052 0.097 0.89 8 0.944 0.03 4 0.109 0.901 0.937 I DH ( α ) 0.053 0.100 0.89 9 0.944 0.03 3 0.106 0.902 0.939 I GK ( α ) 0.052 0.102 0.89 9 0.941 0.03 6 0.107 0.902 0.935 (b) MA(1) series I ( α ) 0.059 0.088 0.90 4 0.948 0.05 0 0.104 0.899 0.952 I C ( α ) 0.056 0.086 0.91 2 0.951 0.04 8 0.096 0.902 0.952 I S ( α ) 0.053 0.085 0.91 4 0.954 0.04 4 0.098 0.904 0.952 I DH ( α ) 0.052 0.087 0.91 2 0.955 0.04 8 0.097 0.902 0.951 I GK ( α ) 0.053 0.087 0.90 8 0.953 0.04 6 0.098 0.900 0.954 (c) AR(1) series I ( α ) 0.059 0.104 0.89 4 0.937 0.04 9 0.108 0.891 0.934 I C ( α ) 0.045 0.096 0.90 2 0.941 0.04 3 0.104 0.899 0.939 I S ( α ) 0.045 0.095 0.90 2 0.942 0.04 1 0.103 0.899 0.942 I DH ( α ) 0.046 0.100 0.90 2 0.941 0.04 5 0.105 0.896 0.938 I GK ( α ) 0.046 0.101 0.90 2 0.940 0.04 3 0.104 0.899 0.939 T able 1: Mean example — cov erage probabilities of nominal lev el α upp er confidence b ounds for mean, approximated from 1,000 indep endent series of length n . n = 500 n = 100 0 nominal level α 0.05 0.1 0 0.90 0.95 0.05 0.1 0 0.90 0.95 (a) ARCH(1) series I ( α ) 0.025 0.090 0.84 0 0.902 0.02 8 0.089 0.828 0.889 I C ( α ) 0.053 0.111 0.88 7 0.943 0.05 4 0.109 0.869 0.926 I S ( α ) 0.052 0.112 0.88 8 0.944 0.05 2 0.106 0.871 0.926 I DH ( α ) 0.054 0.114 0.88 1 0.940 0.05 5 0.108 0.864 0.924 I GK ( α ) 0.058 0.113 0.88 3 0.942 0.05 3 0.110 0.867 0.927 (b) MA(1) series I ( α ) 0.046 0.090 0.88 3 0.930 0.05 6 0.097 0.872 0.921 I C ( α ) 0.059 0.099 0.90 6 0.946 0.06 4 0.105 0.884 0.936 I S ( α ) 0.058 0.100 0.90 9 0.948 0.06 5 0.105 0.883 0.935 I DH ( α ) 0.059 0.101 0.90 5 0.944 0.06 4 0.105 0.881 0.938 I GK ( α ) 0.061 0.103 0.90 7 0.944 0.06 4 0.105 0.877 0.937 (c) AR(1) series I ( α ) 0.042 0.091 0.88 5 0.928 0.04 7 0.097 0.863 0.916 I C ( α ) 0.053 0.106 0.90 3 0.950 0.05 9 0.107 0.881 0.936 I S ( α ) 0.052 0.104 0.90 2 0.953 0.05 8 0.110 0.880 0.932 I DH ( α ) 0.054 0.104 0.89 9 0.952 0.05 7 0.109 0.883 0.930 I GK ( α ) 0.055 0.107 0.90 1 0.949 0.05 7 0.108 0.883 0.927 T able 2: V ariance example — co ve ra g e probabilities of nominal lev el α upp er confidence b ounds for v aria nce, appro ximated from 1,000 independen t series of length n . n = 500 n = 100 0 nominal level α 0.05 0.1 0 0.90 0.95 0.05 0.1 0 0.90 0.95 (a) ARCH(1) series I ( α ) 0.063 0.103 0.87 8 0.930 0.05 6 0.098 0.888 0.930 I C ( α ) 0.054 0.096 0.88 5 0.937 0.04 9 0.098 0.891 0.934 I S ( α ) 0.054 0.099 0.88 4 0.936 0.05 2 0.098 0.890 0.933 I DH ( α ) 0.057 0.100 0.88 3 0.934 0.05 8 0.100 0.888 0.930 I GK ( α ) 0.049 0.095 0.87 8 0.934 0.05 4 0.094 0.887 0.935 (b) MA(1) series I ( α ) 0.056 0.095 0.90 5 0.952 0.04 1 0.084 0.888 0.947 I C ( α ) 0.052 0.087 0.90 1 0.953 0.03 9 0.078 0.886 0.944 I S ( α ) 0.052 0.087 0.89 9 0.949 0.03 7 0.075 0.884 0.947 I DH ( α ) 0.051 0.090 0.90 3 0.952 0.04 1 0.079 0.886 0.947 I GK ( α ) 0.022 0.058 0.92 0 0.966 0.02 6 0.051 0.914 0.962 (c) AR(1) series I ( α ) 0.067 0.110 0.88 2 0.941 0.04 5 0.101 0.873 0.936 I C ( α ) 0.057 0.100 0.88 0 0.941 0.04 2 0.093 0.877 0.935 I S ( α ) 0.055 0.101 0.88 1 0.937 0.03 9 0.091 0.878 0.937 I DH ( α ) 0.060 0.103 0.88 2 0.946 0.04 4 0.095 0.873 0.941 I GK ( α ) 0.025 0.065 0.89 7 0.959 0.02 3 0.054 0.899 0.955 T able 3: Auto correlation example — cov erage probabilities o f nominal lev el α upp er confidence b ounds for lag 1 auto correlation, appro ximated from 1,000 indep enden t series of length n .