Joint Detection and Identification of an Unobservable Change in the Distribution of a Random Sequence

Joint Detection and Identification of an Unobserv able Change in the Distrib ution of a Random Sequence Sav as Dayanik and Christian Goulding H. V incent Poor Dept. of Operati ons Research and Financial Engineering School of Engine ering and A pplied Scienc e Princeton Uni versity , Princeton , N J 085 44 Princeton Univ ersity , Princeton , N J 08544 Email: { sday anik, cgouldin } @princeton.edu Email: poor@pr inceton.edu Abstract– This paper examines the joint problem of detec- tion and identification of a sudden and unobservable change in the pr oba bility distrib ution function (pdf) of a sequence of independent and identically distrib uted (i.i.d.) ra ndom variables to one of finitely many alternative pdf ’ s. The ob- jective is quick detectio n of the chang e and ac curate infer - ence of the ensuing pdf. Following a Bayesian approach, a new sequential decision strateg y f or this pr oblem is revealed and is proven optimal. Geometrical properties of this strat- egy are demonstrated via numerical examples. I. I N T R O D U C T I O N Consider a sequence of i.i.d. random variables X 1 , X 2 , . . . , taking values in some measura ble space ( E , E ) . The common probab ility distribution of the X ’ s is initially some known pr ob- ability measur e P 0 on ( E , E ) , and then, at some un observable disorder time θ , the co mmon pro bability distribution chang es suddenly to another probability measure P µ for some unobserv- able index µ ∈ M , { 1 , . . . , M } . The objective is to d etect the change as quickly as possible, and, at the same time, to iden- tify the ne w pro bability distribution as accurately as po ssible, so that the most suitable actions can be taken with the least delay . This prob lem can be viewed as the fusion of two fund amen- tal areas of sequential ana lysis: change detection and multi- hypoth esis testing. In traditio nal change detection pro blems, there is only one change d istribution, P 1 ; therefor e, the focus is e x clusiv ely on detec ting th e change time. Whereas, in tr adi- tional s equ ential multi-hypothesis testing p roblems, there is no change time to consider . Instead, every o bservation has com- mon distribution P µ for some unknown µ , and the fo cus is ex- clusiv ely on the inference of µ . Both of these subpr oblems hav e been studied extensi vely . For r ecent r evie ws o f th ese areas, we refer the reader to [1] and [2] and the referen ces therein. Howe ver , th e joint prob lem inv olves key trade- off decisions not taken into accoun t by separately applying techniq ues for these subp roblem s. While raising an alarm as soo n as the change occurs is ad vantageous for the change detection task, it is undesirable for th e id entification task be cause waiting longer provides more observations for inferrin g the ch ange dis- tribution. Like wise, the un known cha nge time complicates the identificatio n task, an d, as a result, adaptation of existing sequential multi-hypo thesis testing algorithms is p roblematic. The research of Sa vas Dayanik was supported by the Air Force Office of Sci- entific Research , under grant AFOSR-F A9550-06-1-0496. T he researc h of H. V incent Poor was supported in part by the U.S. Army Panthe on Projec t. Decision strategies for the joint p roblem have a wide array of applications, such as fault detection and isolation in indus- trial processes, target detection and id entification in national de- fense, pattern r ecognitio n and ma chine learning, radar and son ar signal p rocessing, seismolog y , sp eech and image p rocessing, biomedica l signal processing , finance , and insurance. Ho wever , the theory has not been broad ly de veloped . Nikif orov [ 3] pro- vides th e first results for this problem, s howing asymptotic op- timality for a certain non- Bayesian ap proach , and Lai [4] gen- eralizes these results throu gh the d ev elo pment of infor mation- theoretic bounds a nd the application of likelihood methods. In this paper, we fo llow a Bayesian appr oach to r ev eal a new o p- timal strategy for this p roblem and we d escribe an accu rate nu- merical scheme for its implementation . In Sec. II we form ulate prec isely the problem in a Bayesian framework, and in Sec. III we show that it can b e reduced to an optimal stopp ing of a Mar kov pro cess whose state sp ace is the standard probability simplex. In addition, we establish a simple recursive fo rmula that captur es the dynamics of the process and yields a sufficient statistic fit fo r online tracking. In Sec. IV we use optimal stoppin g theo ry to substantiate the optima lity equation for the value fu nction of the op timal stopping problem . More over , we prove that this value func tion is bou nded, con cav e, and continu ous on the stan dard pr obabil- ity simplex an d th at the optimal stoppin g regio n consists of M non-em pty , conv ex, closed, and bounded su bsets. Also, we con- sider a trun cated version o f the prob lem th at allows at m ost N observations from the seq uence of rando m m easuremen ts. W e establish an explicit bound (in versely proportion al to N ) for the approx imation erro r associated with this truncated problem. In Sec. V we show that the separ ate problems of change detection an d seq uential mu lti-hypo thesis testing are solved as special cases of the overall joint solution. W e illustrate some geometrica l proper ties of the o ptimal m ethod an d d emonstrate its impleme ntation by numerical examples for the special cases M = 2 and M = 3 . Specifically , we show instances in which the M convex sub sets comp rising the optimal stopping region are connected and instances in which they are not. Likewise, we sho w that the continu ation region ( i.e., the c omplemen t of the stopping re gio n) need not be connected . W e refer the reader to [5] for complete proo fs of the resu lts. II. P R O B L E M S TA T E M E N T Let (Ω , F , P ) be a prob ability space hosting random v ar i- ables θ : Ω 7→ { 0 , 1 , . . . } and µ : Ω 7→ M , { 1 , . . . , M } and a process X = ( X n ) n ≥ 1 taking values in some measurab le space ( E , E ) . Sup pose that for every t ≥ 1 , i ∈ M , n ≥ 1 , and ( E k ) n k =1 ⊆ E we h av e P { θ = t, µ = i, X 1 ∈ E 1 , . . . , X n ∈ E n } = (1 − p 0 )(1 − p ) t − 1 pν i ( t − 1) ∧ n Y k =1 P 0 ( E k ) n Y ℓ = t ∨ 1 P i ( E ℓ ) for some gi ven prob ability m easures P 0 , P 1 , . . . , P M on ( E , E ) , known con stants p 0 ∈ [0 , 1] , p ∈ (0 , 1) , an d ν i > 0 , i ∈ M such that ν 1 + · · · + ν M = 1 , where x ∧ y , min { x, y } and x ∨ y , max { x, y } . Namely , θ is in depend ent of µ ; it has a zero-mo dified geo metric distribution with par ameters p 0 and p in the terminolog y of [6 , Sec. 3.6], which redu ces to the stan- dard geometric distribution when p 0 = 0 . Conditionally on θ and µ , the rand om variables X n , n ≥ 1 are inde penden t; X 1 , . . . , X θ − 1 and X θ , X θ +1 , . . . are ide nti- cally distributed with common distributions P 0 and P µ , respec- ti vely . The probab ility measure s P 0 , P 1 , . . . , P M always ad- mit densities with respect to som e sigma -finite measure m on ( E , E ) ; for example, we can take m = P 0 + P 1 · · · + P M . So, we fix m and den ote the co rrespon ding densities by f 0 , f 1 , . . . , f M , respectively . Suppose now that we obser ve sequentially the ran dom vari- ables X n , n ≥ 1 . Their co mmon pdf f 0 changes a t stage θ to some o ther pdf f µ , µ ∈ M . Our objective is to detect the change time θ as quic kly as possible and to ide ntify the change index µ as accur ately as possible. More p recisely , given costs associated with detec tion d elay , fals e alar m, a nd false identifi- cation of the change index, we seek a strategy tha t m inimizes the expected total change detection and iden tification cost. Let F = ( F n ) n ≥ 0 denote th e natu ral filtration of the obser- vation pro cess X , wher e F 0 = { ∅ , Ω } and F n = σ ( X 1 , . . . , X n ) , n ≥ 1 . A strate gy δ = ( τ , d ) is a pair consisting of a stopping time τ of the filtration F and a terminal d ecision rule d : Ω 7→ M measurable w ith respect to th e history F τ = σ ( X n ∧ τ ; n ≥ 1) of observation process X throu gh stage τ . Ap plying a strategy δ = ( τ , d ) consists o f anno uncing at th e en d of stage τ that th e common pdf has changed fr om f 0 to f d at or befo re stage τ . Let ∆ , { ( τ , d ) | τ ∈ F , an d d ∈ F τ is an M -valued r . v . } denote the collection of all such sequential decision strategies. For every strategy δ = ( τ , d ) ∈ ∆ , we define a Bayes risk function R ( δ ) = c E [( τ − θ ) + ] + E [ a 0 d 1 { τ <θ } + a µd 1 { θ ≤ τ < ∞} ] (1) as th e expected diagn osis cost: the su m of the expected detec- tion delay cost and the exp ected term inal decision cost upon alarm, wh ere c > 0 and a ij ≥ 0 , i ∈ { 0 } ∪ M , j ∈ M are known constants satisfying a ii = 0 , i ∈ M (i.e., n o co st for a correct terminal decision), and ( x ) + , max { x, 0 } . The problem is to find a sequential decision strategy δ = ( τ , d ) ∈ ∆ (if it exists) with the minimum Bayes risk R ∗ , inf δ ∈ ∆ R ( δ ) . (2) III. P O S T E R I O R A N A LY S I S A N D F O R M U L A T I O N A S A N O P T I M A L S T O P P I N G P RO B L E M In this section we show that the Bayes r isk function in (1) can b e written as the expected value of the r unning an d term i- nal co sts d riv en by a cer tain Markov pro cess. W e use this fact to recast the minimum Baye s risk in (2 ) as a Mar kov o ptimal stopping proble m. Let us introdu ce the posterior prob ability processes Π (0) n , P { θ > n | F n } and Π ( i ) n , P { θ ≤ n, µ = i | F n } for i ∈ M , n ≥ 0 . Having observed the first n ob servations, Π (0) n is the posterior probability that th e change has not yet occurre d at or b efore stag e n , while Π ( i ) n is th e posterior joint probab ility that the change has occur red by stage n and that the hypoth esis µ = i is cor rect. The con nection o f th ese po sterior probab ilities t o the loss structu re for our pro blem is established in the next proposition. Proposition 1 . F or e very seq uential de cision strate gy δ ∈ ∆ , the Bayes risk function (1) can be expr essed in terms of the pr o- cess Π , { Π n = (Π (0) n , . . . , Π ( M ) n ) } n ≥ 0 as R ( δ ) = E   τ − 1 X n =0 c (1 − Π (0) n ) + 1 { τ < ∞} M X j =1 1 { d = j } M X i =0 a ij Π ( i ) τ   . While our original f ormulatio n of the Bayes risk function (1) was in terms of the values of the un observable random vari- ables θ a nd µ , Pro position 1 gi ves us an eq uiv alent version of the Bayes risk function in ter ms of th e posterio r distrib utio ns for θ an d µ . This is particularly effecti ve i n light of Proposition 2, wh ich w e state with the aid of so me additio nal n otation th at is referred to throug hout the paper . Let S M , n π = ( π 0 , π 1 , . . . , π M ) ∈ [0 , 1] M +1   P M i =0 π i = 1 o denote the standard M - dimension al probability simplex. De- fine the mapping s D i : S M × E 7→ [0 , 1] , i ∈ M a nd D : S M × E 7→ [0 , 1] by D i ( π , x ) , ( (1 − p ) π 0 f 0 ( x ) , i = 0 ( π i + π 0 pν i ) f i ( x ) , i ∈ M ) and D ( π , x ) , P M i =0 D i ( π , x ) , and the o perator T on the c ol- lection of boun ded func tions f : S M 7→ R by ( T f )( π ) , Z E m ( dx ) D ( π , x ) f  D 0 ( π ,x ) D ( π , x ) , . . . , D M ( π ,x ) D ( π , x )  (3) for every π ∈ S M . Proposition 2. (a) The pr ocess Π (0) , { Π (0) n , F n } n ≥ 0 is a su- permartingale, and E Π (0) n ≤ (1 − p ) n for every n ≥ 0 . (b) The pr ocess Π ( i ) , { Π ( i ) n , F n } n ≥ 0 is a submartinga le for every i ∈ M . (c) The pr ocess Π = { (Π (0) n , . . . , Π ( M ) n ) } n ≥ 0 is a Markov pr ocess, and Π ( i ) n +1 = D i (Π n , X n +1 ) D (Π n , X n +1 ) , i ∈ { 0 } ∪ M , n ≥ 0 , (4) with initial state Π (0) 0 = 1 − p 0 and Π ( i ) 0 = p 0 ν i , i ∈ M . Mor e- over , for every b ounde d functio n f : S M 7→ R a nd n ≥ 0 , we have E [ f (Π n +1 ) | Π n ] = ( T f )(Π n ) . Remark 3. Since Π is unifo rmly bo unded , the limit lim n →∞ Π n exis ts b y th e ma rtingale co n verg en ce theor em. Mor eover , lim n →∞ Π (0) n = 0 a.s. by Pr oposition 2(a) since p ∈ (0 , 1) . Now , let the functio ns h, h 1 , . . . , h M from S M into R + be defined by h ( π ) , min j ∈M h j ( π ) and h j ( π ) , M X i =0 π i a ij , j ∈ M , respectively . Then, we note th at for every δ = ( τ , d ) ∈ ∆ , w e have R ( τ , d ) = E   τ − 1 X n =0 c (1 − Π (0) n ) + 1 { τ < ∞} M X j =1 1 { d = j } h j (Π τ )   ≥ E " τ − 1 X n =0 c (1 − Π (0) n ) + 1 { τ < ∞} h (Π τ ) # = R ( τ , ˜ d ) where we define on the event { τ < ∞} the termin al deci- sion rule ˜ d to b e any index satisfying h ˜ d (Π τ ) = h (Π τ ) . In other words, an optimal terminal decision d epends only upo n the value of the Π pr ocess at the stage in which we sto p. Note also that the functio ns h an d h 1 , . . . , h M are bou nded on S M . Therefo re, we have the follo wing: Lemma 4. The min imum B ayes risk (2 ) r edu ces to the follo w- ing optimal stopping of the Markov pr ocess Π : R ∗ = inf ( τ ,d ) ∈ ∆ R ( τ , d ) = inf ( τ , ˜ d ) ∈ ∆ R ( τ , ˜ d ) = inf τ ∈ F E " τ − 1 X n =0 c (1 − Π (0) n ) + 1 { τ < ∞} h (Π τ ) # . W e simplify this f ormulatio n fur ther by showing that it is enoug h to take the infimum over C , { τ ∈ F | τ < ∞ a.s. an d E Y − τ < ∞} , (5) where we define − Y n , n − 1 X k =0 c (1 − Π (0) k ) + h (Π n ) , n ≥ 0 as the minim um partial risk ob tained by making the best termi- nal decision on { τ = n } . Since h ( · ) is bo unded on S M , th e process { Y n , F n ; n ≥ 0 } consists of integrable rando m vari- ables. So th e expectatio n E Y τ exists for every τ ∈ F , and our problem becomes − R ∗ = sup τ ∈ F E Y τ . (6) Observe tha t E τ < ∞ for every τ ∈ C be cause ∞ > (1 /c ) E Y − τ ≥ E ( τ − θ ) + ≥ E ( τ − θ ) ≥ E τ − E θ ≥ E τ − (1 / p ) . In fact, we h av e E Y τ > −∞ ⇔ E Y − τ < ∞ ⇔ E τ < ∞ for ev ery τ ∈ F . Since sup τ ∈ F E Y τ ≥ E Y 0 > − h (Π 0 ) > −∞ , it is enou gh to consider τ ∈ F such th at E τ < ∞ . Name ly , (6 ) reduces to − R ∗ = sup τ ∈ C E Y τ . (7) IV. S O L U T I O N V I A O P T I M A L S T O P P I N G T H E O RY In this sectio n we deriv e an optimal solution for the prob- lem in (2 ) by building on the fo rmulation of (7 ) via the too ls of optimal stopping theory , which are detailed in [7]. A . The optimality equa tion . W e begin by ap plying the metho d of truncation with a view of passing to the limit to arrive at th e final result. Define fo r ev ery pair of in tegers n, N satisfying 0 ≤ n ≤ N the sub- collections C n , { τ ∨ n | τ ∈ C } and C N n , { τ ∧ N | τ ∈ C n } of stopping times in C of (5) an d the families of (tr uncated) optimal stopping problems − V n , sup τ ∈ C n E Y τ and − V N n , sup τ ∈ C N n E Y τ (8) correspo nding to ( C n ) n ≥ 0 and ( C N n ) 0 ≤ n ≤ N , respectively . No te that C ≡ C 0 and R ∗ ≡ V 0 . T o in vestigate these o ptimal stopping prob lems, we intro- duce versions o f the Snell envelope of ( Y n ) n ≥ 0 (i.e., the small- est regular sup ermarting ale do minating ( Y n ) n ≥ 0 ) corr espond- ing to ( C n ) n ≥ 0 and ( C N n ) 0 ≤ n ≤ N , respectiv ely , defined by γ n , ess sup τ ∈ C n E [ Y τ | F n ] and γ N n , ess sup τ ∈ C N n E [ Y τ | F n ] . Then thr ough the following series of lemmas we poin t out sev- eral u seful p roperties of these Snell en velop es. Finally , we extend these results to an arbitrary initial state vector and es- tablish the optimality equation. Note that e ach o f the ensuing (in)equ alities between random variables are in the P -almost sur e sense. First, these Snell en velopes provide the following alternative expressions for the optimal stopping problems introduc ed in (8) above. Lemma 5. F or every N ≥ 0 a nd 0 ≤ n ≤ N , we have − V n = E γ n and − V N n = E γ N n . Second, we have the fo llowing backward- induction eq ua- tions. Lemma 6. W e have γ n = max { Y n , E [ γ n +1 | F n ] } for every n ≥ 0 . F or every N ≥ 1 and 0 ≤ n ≤ N − 1 , we ha ve γ N N = Y N and γ N n = max { Y n , E [ γ N n +1 | F n ] } . W e also ha ve that these versions of the Snell en velopes coin- cide in the limit as N → ∞ . Th at is, Lemma 7. F or every n ≥ 0 , we have γ n = lim N →∞ γ N n . Next, recall from (3) and Proposition 2(c) the operator T and let us introd uce the operator M on the collectio n of boun ded function s f : S M 7→ R + defined by ( M f )( π ) , min { h ( π ) , c (1 − π 0 ) + ( T f )( π ) } , π ∈ S M . Observe tha t 0 ≤ M f ≤ h . That is, π 7→ ( M f )( π ) is a nonnegative boun ded fu nction. Th erefor e, M 2 f ≡ M ( M f ) is well-defined. If f is non negative and b ound ed, then M n f ≡ M ( M n − 1 f ) is d efined for every n ≥ 1 , with M 0 f ≡ f by defi- nition. Using operator M , we can express ( γ N n ) 0 ≤ n ≤ N in terms of the process Π as stated in the following lemma. Lemma 8. F or every N ≥ 0 , and 0 ≤ n ≤ N , we ha ve γ N n = − c P n − 1 k =0 (1 − Π (0) k ) − ( M N − n h )(Π n ) . The next lemma shows h ow the optimal stopp ing problem s can be rewritten in terms of the operato r M . It also conveys the conn ection between the trun cated optimal stopping pro b- lems and the initial state Π 0 of the Π process. Lemma 9. W e have (a) V N 0 = ( M N h )(Π 0 ) for every N ≥ 0 , and (b) V 0 = lim N →∞ ( M N h )(Π 0 ) . Observe that since Π 0 ∈ F 0 = { ∅ , Ω } , we have P { Π 0 = π } = 1 for some π ∈ S M . On the other hand, for e very π ∈ S M we can con struct a pr obability space (Ω , F , P π ) h ost- ing a Markov pr ocess Π with the same d ynamics as in (4) a nd P π { Π 0 = π } = 1 . M oreover , on such a pr obability space, the preced ing resu lts r emain valid. So, let us d enote b y E π the expectation with respect to P π and rewrite (8) as − V n ( π ) , s up τ ∈ C n E π Y τ and − V N n ( π ) , sup τ ∈ C N n E π Y τ for every π ∈ S M . Then Lemma 9 implies that V N 0 ( π ) = ( M N h )( π ) and V 0 ( π ) = lim N →∞ ( M N h )( π ) (9) for every π ∈ S M . T aking limits as N → ∞ of bo th sides in ( M N + 1 h )( π ) = M ( M N h )( π ) an d apply ing the mono tone conv ergenc e the orem o n the rig ht-hand side yields V 0 ( π ) = ( M V 0 )( π ) . Hence, we ha ve shown the following result. Proposition 10 (Optimality equation) . F or e very π ∈ S M , V 0 ( π ) = ( M V 0 )( π ) ≡ min { h ( π ) , c (1 − π 0 ) + ( T V 0 )( π ) } . (10 ) Remark 11. By solving V 0 ( π ) for an y initial state π ∈ S M , we ca pture the solution to the original pr oblem since pr op - erty (c) o f Pr op osition 2 a nd (7) imply th at R ∗ = V 0 (1 − p 0 , p 0 ν 1 , . . . , p 0 ν M ) . B . Some pr operties of the value function . Now , we reveal some important proper ties of the v alue func- tion V 0 ( · ) of (9). T hese results help us to estab lish an o ptimal solution fo r V 0 ( · ) , a nd hen ce an o ptimal solution fo r R ∗ , in the next subsection. Lemma 12. I f g : S M 7→ R is a bound ed con cave fu nction, then so is T g . Proposition 1 3. The mappin gs π 7→ V N 0 ( π ) , N ≥ 0 and π 7→ V 0 ( π ) are concave. Proposition 14. F or every N ≥ 1 an d π ∈ S M , we have V 0 ( π ) ≤ V N 0 ( π ) ≤ V 0 ( π ) +  k h k 2 c + k h k p  1 N . Since k h k , sup π ∈ S M | h ( π ) | < ∞ , lim N →∞ ↓ V N 0 ( π ) = V 0 ( π ) uniformly in π ∈ S M . Proposition 15. F or every N ≥ 0 , th e fun ction V N 0 : S M 7→ R + is continuo us. Corollary 16. The function V 0 : S M 7→ R + is continuo us. Note that S M is a com pact subset o f R M +1 , so wh ile conti- nuity of V 0 ( · ) on the interior of S M follows from the con cavity of V 0 ( · ) by Pro position 1 2, Coro llary 16 establishes co ntinuity on all of S M , including its bound ary . C . A n optimal sequential decision strate g y . Finally , we describe th e optimal stop ping r egion in S M im- plied by the value fun ction V 0 ( · ) , a nd we presen t an optimal sequential de cision strategy for our p roblem. Let us d efine for ev ery N ≥ 0 , Γ N , { π ∈ S M | V N 0 ( π ) = h ( π ) } , Γ ( j ) N , Γ N ∩ { π ∈ S M | h ( π ) = h j ( π ) } , j ∈ M , Γ , { π ∈ S M | V 0 ( π ) = h ( π ) } , Γ ( j ) , Γ ∩ { π ∈ S M | h ( π ) = h j ( π ) } , j ∈ M . For each j ∈ { 0 } ∪ M , let e j ∈ S M denote the u nit vec- tor con sisting of zero in e very comp onent except for the j th compon ent, which is eq ual to one. No te that e 0 , . . . , e M are the extreme po ints of the closed con vex set S M , and a ny vec- tor π = ( π 0 , . . . , π M ) ∈ S M can be expr essed in terms of e 0 , . . . , e M as π = P M j =0 π j e j . Theorem 17. F or every j ∈ M , (Γ ( j ) N ) N ≥ 0 is a d ecr easing se- quence of non-emp ty , closed, con vex subsets of S M . Mor eover , Γ ( j ) 0 ⊇ Γ ( j ) 1 ⊇ · · · ⊇ Γ ( j ) , Γ ( j ) ⊇  π ∈ S M | h j ( π ) ≤ min { h ( π ) , c (1 − π 0 ) }  ∋ e j , Γ = ∞ \ N = 1 Γ N = M [ j =1 Γ ( j ) , and Γ ( j ) = ∞ \ N = 1 Γ ( j ) N , j ∈ M . Furthermore , S M = Γ 0 ⊇ Γ 1 ⊇ · · · ⊇ Γ % { e 1 , . . . , e M } . Lemma 18. F or every n ≥ 0 , we have γ n = − c P n − 1 k =0 (1 − Π (0) k ) − V 0 (Π n ) . Theorem 19 . Let σ , inf { n ≥ 0 | Π n ∈ Γ } . (a) The stopp ed pr ocess { γ n ∧ σ , F n ; n ≥ 0 } is a martinga le. (b) The random variable σ is an optimal stopping time for V 0 , and (c) E σ < ∞ . Therefo re, the pair ( σ, d ∗ ) is an op timal sequen tial decision strategy fo r (2 ), where the op timal stop ping ru le σ is given b y Theorem 19, and, as in th e proo f of Lemma 4, the op timal ter- minal decision rule d ∗ is giv en by d ∗ = j on the event { σ = n, Π n ∈ Γ ( j ) } for e very n ≥ 0 . According ly , the set Γ is called the stopp ing r egion implied by V 0 ( · ) , an d Theo rem 17 re veals its basic structure. W e dem on- strate the use of these resu lts in the numer ical e xam ples of Sec. V. Note that we can take a similar appro ach to p rove that th e stopping rules σ N , inf { n ≥ 0 | Π n ∈ Γ N − n } , N ≥ 0 are optimal for the tru ncated problem s V N 0 ( · ) , N ≥ 0 in (9) . Th us, for each N ≥ 0 , the set Γ N is called the stopping region for V N 0 ( · ) : it is optimal to termin ate the experiments in Γ N if N stages are left before truncation . V. S P E C I A L C A S E S A N D E X A M P L E S A . A. N. Shiryaev’s seq uential change detection pr oblem . Set a 0 j = 1 f or j ∈ M and a ij = 0 for i, j ∈ M , then the Bayes risk f unction (1) becomes R ( δ ) = P { τ < θ } + c E [( τ − θ ) + ] . Th is is th e Bayes r isk stud ied by Sh iryaev [8, 9] to solve the sequential change detection problem. B . Sequ ential multi-hypothesis testing . Set p 0 = 1 , then θ = 0 a.s. and thus the Bayes risk func tion (1) becomes R ( δ ) = E [ cτ + a µd 1 { τ < ∞} ] . This gi ves the se- quential m ulti-hypo thesis testing prob lem studied b y W ald an d W olfowitz [10] and Arrow , Blackwell, and Girshick [11]; see also [12]. C . T wo alternatives after the change . In this sub section we consider the special case M = 2 in which we hav e o nly two possible change distributions, f 1 ( · ) and f 2 ( · ) . W e describe a graphical repr esentation of the stop- ping an d con tinuation regions for an arbitrary instan ce of the special case M = 2 . Then we use th is repre sentation to illus- trate g eometrical prop erties o f the optimal m ethod (Sec. IV. C ) via model instances for certain choices of the model parameters p 0 , p , ν 1 , ν 2 , f 0 ( · ) , f 1 ( · ) , f 2 ( · ) , a 01 , a 02 , a 12 , a 21 , and c . Let the linear map ping L : R 3 7→ R 2 be define d by L ( π 0 , π 1 , π 2 ) , ( 2 √ 3 π 1 + 1 √ 3 π 2 , π 2 ) . Since π 0 = 1 − π 1 − π 2 for every π = ( π 0 , π 1 , π 2 ) ∈ S 2 ⊂ R 3 , we can recover the preimage π o f a ny point L ( π ) ∈ L ( S 2 ) ⊂ R 2 . For every point π = ( π 0 , π 1 , π 2 ) ∈ S 2 , the coordinate π i is g iv en by the Eu- clidean distanc e from the image point L ( π ) to the edg e of the image triangle L ( S 2 ) that is opposite the image point L ( e i ) , for each i = 0 , 1 , 2 . For examp le, th e distanc e fro m the ima ge point L ( π ) to the edge of the image triangle opposite the lower - left-hand corner L (1 , 0 , 0) = (0 , 0) is the v alue of the preim age coordin ate π 0 . See Fig. ?? . Therefo re, we can work with the mappings L (Γ) an d L ( S 2 \ Γ) of the stopping region Γ and the con tinuation region S 2 \ Γ , respectively . Accordingly , we dep ict the decision region for each instance in this subsectio n using the two-dimensional rep- resentation as in the rig ht-hand -side of Fig. ?? and we d rop the L ( · ) notation when labeling various parts o f each figur e to em- phasize their source in S 2 . Each of the examp les in this sectio n h av e the following model parameters in commo n: p 0 = 1 50 , p = 1 20 , ν 1 = ν 2 = 1 2 , f 0 =  1 4 , 1 4 , 1 4 , 1 4  , f 1 =  4 10 , 3 10 , 2 10 , 1 10  , f 2 =  1 10 , 2 10 , 3 10 , 4 10  . W e vary the delay cost and false alarm/identification costs to illustrate cer tain geometrical pro perties of the con tinuation and stopping regions. See Figs. ? ? , ?? , and ?? . These figures ha ve certain features in comm on. On each sub- figure there is a dashed line represen ting those states π ∈ S 2 at which h 1 ( π ) = h 2 ( π ) . Also, each subfigu re sho ws a sample path of (Π n ) σ n =0 and the realizations of θ and µ for the sam- ple. The sh aded area, including its solid bound ary , rep resents the optimal stopping region, while the unshaded area repr esents the continuatio n region. Specifically , these figures show instances in which the M = 2 conve x subsets comprising the optimal stopping region are connected (Fig . ?? ) an d instances in which they are no t (Figs. ?? and ?? (a)). Fig. ?? (b) shows a n instan ce in wh ich the co n- tinuation region is disconnected. An implem entation of the optimal strategy as described in Sec. IV. C is as follows: In itialize the statistic Π = (Π n ) n ≥ 0 by setting Π 0 = (1 − p 0 , p 0 ν 1 , p 0 ν 2 ) as in p art (c) of Pro po- sition 2. Use the dy namics of ( 4) to u pdate the statistic Π n as each observation X n is realized . Stop taking o bservations when the statistic Π n enters the stopping region Γ = Γ (1) ∪ Γ (2) for the first time, p ossibly before the first obser vation is taken (i.e., n = 0 ). The o ptimal te rminal decision is based upo n wh ether the statistic Π n is in Γ (1) or Γ (2) upon stopping. Each of the sample paths in Figs. ?? , ?? , a nd ?? were generated via this al- gorithm. As Fig. ?? shows, the sets Γ (1) and Γ (2) can in tersect on their boundaries an d so it is possible to stop in their inter- section. In this case, either o f the decision s d = 1 or d = 2 is optimal. W e use value iteration of the optimality equation (10) over a fine discretization of S 2 to comp ute V 0 ( · ) and g enerate th e de- cision region for each subfigure. The resulting discretized deci- sion r egion is ma pped into the pla ne v ia L . See [ 13, Ch. 3 ] f or technique s of computing th e v alue function via th e optimality equation such as value iteration . D . Thr ee alternatives after the change . In this sub section we consider the special case M = 3 in which we hav e three po ssible change distributions, f 1 ( · ) , f 2 ( · ) , and f 3 ( · ) . Here, the continu ation and stopping regions are sub - sets of S 3 ⊂ R 4 . Similar to the two-altern ativ es case, we intro- duce the mappin g of S 3 ⊂ R 4 into R 3 via ( π 0 , π 1 , π 2 , π 3 ) 7→  q 3 2 π 1 + 1 2 q 3 2 π 2 + 1 2 q 3 2 π 3 , 3 2 q 1 2 π 2 + 1 2 q 1 2 π 3 , π 3  . Then we use this representation —actually a rotatio n of it—to illustrate in Fig. ?? an in stance with th e following model pa- rameters: p 0 = 1 50 , p = 1 20 , ν 1 = ν 2 = ν 3 = 1 3 , f 0 =  1 4 , 1 4 , 1 4 , 1 4  , f 1 =  4 10 , 3 10 , 2 10 , 1 10  , f 2 =  1 10 , 2 10 , 3 10 , 4 10  , f 3 =  3 10 , 2 10 , 2 10 , 3 10  , c = 1 , a 0 j = 40 , a ij = 20 , i, j = 1 , 2 , 3 . Fig. ? ? can be inter preted in a m anner similar to the fig - ures of the previous subsection. In th is case, for ev ery p oint π = ( π 0 , π 1 , π 2 , π 3 ) ∈ S 3 , the coordinate π i is given by the (Euclidean ) distance fr om the im age poin t L ( π ) to the face of the image tetrahed ron L ( S 3 ) that is opp osite the imag e cor ner L ( e i ) , for each i = 0 , 1 , 2 , 3 . R E F E R E N C E S [1] M. Bassevi lle and I. V . Nikiforov . D etect ion of Abrupt Chang es: Theory and Applicat ion . Prenti ce Hall Inc., Engle wood Clif fs, NJ, 1993. [2] T . L. L ai. Sequential analy sis: some classical problems and ne w cha l- lenges. Statist. Sinica , 11(2): 303–408, 2001. [3] I. V . Nikiforov . A genera lized change detection problem. IEE E T rans. In- form. Theory , 41(1):171– 187, 1995. [4] T . L. Lai. Sequential multipl e hypothesis testing and ef ficient fault detec tion-isolati on in stochast ic s ystems. IEEE Tr ans. Inform. Theory , 46(2):595– 608, 2000. [5] S. Dayani k, C. Goulding, and H. V . Poor . Bayesian sequential change di- agnosis. http: //www .princeton.ed u/˜sdayanik/diagnosis.pdf. [6] S. A. Klugman, H. H. P anjer , and G. E . W illmot. Loss Models . Joh n W iley & Sons Inc., Ne w Y ork, 1998. [7] Y . S. Chow , H. Robbins, and D. Siegmund. Great Expe ctations: The The- ory of Optimal Stopping . Houghton Mif flin Co., Boston, Mass., 1971. [8] A. N. Shiryae v . Optimal m ethods in quickest detectio n problems. T eor . V eroj atnost. i P rimenen. , 8:26–51 , 1963. [9] A. N. Shiryae v . Optimal Stoppin g Rules . Springer -V erlag, Ne w Y ork, 1978. [10] A. W ald and J. W olfowit z. Bayes solutions of sequential decision prob- lems. Ann. Math. Stati stics , 21:82– 99, 1950. [11] K. J. Arrow , D. Blackwell, and M. A. Girshick. Bayes and m inimax solu- tions of sequenti al dec ision problems. Econometrica , 17:213–244, 1949. [12] D. Blackwell and M. A. Girshick. Theory of Games and Statistical De- cisions . D ov er Publications Inc., New Y ork, 1979. Reprint of the 1954 editi on. [13] D. P . Bertse kas. Dynamic Pr ogramming and Opti mal Cont rol. Vol. II . Athena Scient ific, Bel mont, MA, second edition, 2001.