Anglers fishing problem

Anglers’ fishing pro blem Anna Karpowicz and Krzysztof Szajowski Abstract The considered mo del will be for mulated as relate d to ”the fishing prob - lem” ev en if the other applicatio ns of it are much more obvious. The an gler goes fishing. He uses various techn iques and he h as at mo st two fishing rods. He buys a fishing tick et for a fixed time. Th e fishes are caught with the use of different methods accordin g to the renewal processes. The fishes’ value and the inter arri val times are giv en by the sequen ces of indep endent, identically distributed (i.i.d.) random vari- ables with the known distribution functions. It for ms the marked renewal–rew ard process. The angler’ s measu re of satisf action is gi ven by the dif ference between the utility fu nction, dependin g o n the value of the fishes caugh t, and the cost fu nction connected with the time of fishing. In this w ay , the angler’ s relati ve opinion about the methods o f fishing is modelled . The angler ’ s aim is to have as muc h satisfaction as possible and ad ditionally he has to leave the lake befo re a fixed moment. Therefor e his go al is to fin d two optimal stop ping times in order to max imize his satisfaction. At the first moment, he changes the tec hnique of fishing e.g. by excluding on e rod and intensify ing on the rest. Next, he decides when he sh ould stop the e xpedition . These stopping times ha ve to be shorter than the fixed time of fishing. The dynamic progr amming me thods ha ve been used to find these tw o optimal stopping times and to specify the expected satisf action of the angler at these times. Key words: fishing p roblem, optim al stopping, dynam ic progra mming, semi-Markov process, marked rene wal process, renew al– rew ar d pr ocess, infinitesimal generator AMS 2010 Subject Classifications: 60G4 0, 60K99, 90A4 6 Anna Karpo wicz Bank Zachodni WBK, Rynek 9/11, 50-950 Wrocław , Poland e-mail: a.m.karp owicz@ gmail.com Krzysztof Szajo wski Institute of Mathematics and Computer Sci., W ybrze ˙ ze W yspia ´ nskieg o 27, 50-37 0 Wrocła w , Poland e-mail: Krzyszto f.Szaj owski@pwr.wroc.pl 1 2 A. Karpo wicz, K. Szajowsk i 1 Introd uction Before we start the analy sis o f th e double optimal stoppin g pro blem (cf. id ea of multiple sto pping for sto chastic sequences in Hag gstrom [8], Nikolaev [1 6]) for the marked re new a l p rocess related to the angler behavior, let us pr esent the so called ” fishing pro blem”. One of the first authors wh o consider ed the b asic version of th is pr oblem was Starr [19] and further gen eralizations were d one by Starr and W oodroof e [21], Starr, W ardrop and W ood roofe [ 20], Kr amer , Starr [14] et al. The detailed re view of the papers related to the ”fishing problem” was presented by Fer - guson [7]. The s imple formulation o f the fishing problem , wher e th e angler chan ges the fishin g place or tech nique before leaving the fishing place, h as been done by Karpowicz [12]. W e e xtend the p r oblem to a mor e ad vanced model by taking into accoun t the various techniques of fishing used the same time (the parallel r en ewal– r ewar d pr ocesses or the multivariate rene wal–r ewar d pr o cess). It is motivated by the natural, mo r e pr ecise models o f th e k nown, r ea l a pplication s of the fi shing p r ob- lem. The typica l pr o cess of softwar e testing consists of checking subr outines. At the be ginning many kind s of bugs are being sear ched. The co nsecutive stopping times ar e mo ments when the expert stops general testing of mod ules and starts checking the most important, dan ger o us type of err or . Similarly , in pr oo f reading, it is natural to look fo r typographic and grammar err ors a t the same time. Ne xt, we ar e loo king for langu age mistakes. As various works are do ne by different gr oups of experts, it is natural that we would com pete with e ach oth er . If in the first per iod work is meant for one g roup and the secon d period needs oth er experts, th en they can be p layers of a ga me be- tween t hem. In t his case the proposed solu tion is to fin d the Nash equilib rium where strategies of players are the stopping times. The app lied techniques of modeling and find ing the o ptimal solutio n are similar to tho se u sed in the formulation and so lution of the optimal stop ping prob lem fo r the risk pr ocess. Both models are b ased on the method ology explicated by Boshuiz en and G ouweleeuw [1]. The back groun d mathem atics for f urther re ading are mono- graphs by Br ´ emaud [3 ], Davis [4] a nd Shir yaev [18]. The optima l stoppin g pr oblems for the r isk p rocess are subject of consideration in pape rs b y Jensen [10], Feren stein and Sieroci ´ nski [6], Muciek [ 15]. A similar prob lem for the risk p rocess ha v ing d is- ruption ( i.e. when the p robability structure of the considered process is changed at one mo ment θ ) h as been analyzed by Ferenstein and Pasternak– W iniar ski [5]. The model of th e last paper b rings to mind the chan ge of fishing metho ds considered here, however it should be made by a decision maker , not the type of the environ- ment. The following two sections usher details of the model. It is pro per to emph asize that th e slight m odification of the backg round assumption by adopting multivariate tools (two rods) and the possible contr ol of their n umbers in use extort a d ifferent structure of the base mod el (the un derlining proc ess, sets of strategies – admissible filtrations and stopping times). Th is mod ified structure allows the introdu ction of a new kind of k nowledge selection which con sequently lead s to a game model o f the anglers’ expedition problem in the section 1. 2 an d 2.2. After a qu ite gen eral Anglers’ fishing problem 3 formu lation a version of the pr oblem for a de tailed solutio n will be ch osen. However , the solution is p resented as the scalab le procedure dependent on param eters wh ich depend s on various circum stances. It is not difficult to adop t a s olution to wid e range of natural cases. 1.1 Single Angler’ s expedition The angler goes fi shing. He buys a fishing tick et for a fi xed time t 0 which gi ves him the right to u se at most two rods. T he total cost of fishing depends on real time of each eq uipment u sage and the numbe r of r ods used simultaneously . He starts fishing with two ro ds up to the moment s . The e ffect o n each rod can be mo delled by the renewal processes { N i ( t ) , t ≥ 0 } , wh ere N i ( t ) is the n umber of fishes caug ht on th e rod i , i ∈ A : = { 1 , 2 } durin g the time t . Let u s combine them together to the marked renewal process. The usage of the i -th rod b y the time t generates cost c i : [ 0 , t 0 ] → ℜ (when the r od is u sed simultaneously with oth er rods it will be denoted by th e index depen dent on the set of rods, e.g. a , c a i ) and the rew ard repr esented by i.i.d. random variables X { i } 1 , X { i } 2 , . . . ( the value of the fi shes caught on the i-th r o d ) with cumulative distrib ution fu nction H i 1 . The streams of two kinds of fishes are mutu ally indepen dent and th ey are in depend ent of the sequence o f rando m mo ments whe n the fishes ha ve been caug ht. The 2-vector process − → N ( t ) = ( N 1 ( t ) , N 2 ( t )) , t ≥ 0, can b e represented also by a sequence of rand om v ar iables T n taking v alues in [ 0 , ∞ ] such that T 0 = 0 , T n < ∞ ⇒ T n < T n + 1 , (1) for n ∈ N , and a sequence of A -valued random variables z n for n ∈ N ∪ { 0 } (see Br ´ emaud [3] Ch. II, Jaco bsen [9]). The rand om variable T n denotes the m oment of catching the n -th fish ( T 0 = 0 ) of any kind an d the rand om variable z n indicates to which kind the n -th fish belongs. The processes N i ( t ) can b e d efined by th e sequen ce { ( T n , z n ) } ∞ n = 0 as: N i ( t ) = ∞ ∑ n = 1 I { T n ≤ t } I { z n = i } . (2) Both the 2-variate process − → N ( t ) and th e double sequ ence { ( T n , z n ) } ∞ n = 0 are called 2 -variate r enewal pr ocess . The optimal stopping proble ms for the compo und risk process based on 2 -variate r enewal pr ocess was considered by Szajowski [22]. Let us define, for i ∈ A and k ∈ N , the sequence n { i } 0 = 0 , n { i } k + 1 = inf { n > n { i } k : z n = i } (3) 1 The following con vention is used in all the paper: − → x = ( x 1 , x 2 , . . . , x s ) for the ordered collection of the elements { x i } s i = 1 4 A. Karpo wicz, K. Szajowsk i and put T { i } k = T n { i } k . Let us define rand om variables S { i } n = T { i } n − T { i } n − 1 and as- sume that they a re i.i.d. with con tinuous, cumulative distribution functio n F i ( t ) = P ( S { i } n ≤ t ) and the conditional distribution function F s i ( t ) = P ( S { i } n ≤ t | S { i } n ≥ s ) . In the section 2.1 the alternati ve represen tation of the 2-variate renewal pro cess will be propo sed. There is also m ild extension of th e model in which the stream o f e ven ts after some moment change s to another stream of ev e nts. Remark 1. In various procedures it is needed to localize the e vents in a group of the renewal pro cesses. Let C be the set of indices related to such a group. T he sequence { n C k } ∞ k = 0 such that n C 0 = 0, n C k + 1 : = inf { n > n C k : z n ∈ C } ha s an obvious meaning. Analogou sly , n C ( t ) : = in f { n : T n > t , z n ∈ C } . Let i , j ∈ A . Th e an gler’ s satisfaction me asure ( the net r ewar d ) at th e period a from th e rod i is the difference between the utility functio n g a i : [ 0 , ∞ ) 2 × A × ℜ + → [ 0 , G a i ] which ca n be in terpreted as the rew ar d from th e i -th ro d when the last success was o n rod j and, additionally , it is dependent on th e v alue of t he fishes caught, the moment o f results’ evaluation, and the cost function c a i : [ 0 , t 0 ] → [ 0 , C a i ] reflecting the cost of d uration of th e angler’ s exped ition. W e assume that g a i and c a i are continuo us and bounded, ad ditionally c a i are differentiable. Each fishing m ethod ev aluatio n is based on different utility functions and cost fun ctions. In this way , the angler’ s relative op inion about them is modelled. The angler can change his method o f fishing at the m oment s and decide to u se only o ne ro d. It could be one o f th e ro ds used up to the mo ment s or th e oth er one. Event tho ugh th e ro d used after s is the on e cho sen fr om the ones used before s its effecti veness could be d ifferent before and af ter s . Following th ese argumen ts, the mathematical mo del of catching fishes, and th eir value after s , could (and in practice should) be d ifferent from those for the rod s used before s . Th e reason for red uction of the numb er of ro ds could be their better effectiveness. Th e value of the fishes which h av e been caught up to time t , if the change o f the fishing techn ology took place at the time s , is giv en by M s t = ∑ i ∈ A N i ( s ∧ t ) ∑ n = 1 X { i } n + N 3 (( t − s ) + ) ∑ n = 1 X { 3 } n = M s ∧ t + N 3 (( t − s ) + ) ∑ n = 1 X { 3 } n , where M { i } t = ∑ N i ( t ) n = 1 X { i } n , and M t = ∑ 2 i = 1 M { i } t W e denote − → M t = ( M { 1 } t , M { 2 } t ) . Let Z ( s , t ) deno te the a ngler’ s pay- off for stopping at time t (the end of the expe dition) if the change of the fishing meth od took place at time s . If the effect of extend ing the expedition after s is d escribed by g b j : ℜ + 2 × A × [ 0 , t 0 ] × ℜ × [ 0 , t 0 ] → [ 0 , G b j ] , j ∈ B , minus the add itional cost of time c b j ( · ) , wher e c b j : [ 0 , t 0 ] → [ 0 , C b j ] (when card ( B ) = 1 then in dex j will be abando ned, also c b = ∑ j ∈ B c b j will b e u sed, whic h will be adequate) . T he payoff can be e x pressed as: Anglers’ fishing problem 5 Z ( s , t ) =          g a ( − → M t , z N ( t ) , t ) − c a ( t ) if t < s ≤ t 0 , g a ( − → M s , z N ( s ) , s ) − c a ( s ) + g b ( − → M s , z N ( s ) , s , M s t , t ) − c b ( t − s ) if s ≤ t ≤ t 0 , − C if t 0 < t . (4) where the f unction c a ( t ) , g a ( − → m , i , t ) and the co nstant C can b e taken as follows: c a ( t ) = ∑ 2 i = 1 c a i ( t ) , g a ( − → M s , j , t ) = ∑ 2 i = 1 g a i ( − → M t , j , t ) , C = C a 1 + C a 2 + C b . After mo - ment s the modelling p rocess is the renewal–rew ard one with the stream of i.i.d. random v aria bles X { 3 } n at the m oments T { 3 } n (i.e. appearing accord ing to the rene wal process N 3 ( t ) ). W ith the no tation w b ( − → m , i , s , e m , t ) = w a ( − → m , i , s ) + g b ( − → m , i , s , e m , t ) − c b ( t − s ) and w a ( − → m , i , t ) = g a ( − → m , i , t ) − c a ( t ) , formula (4) is reduced to: Z ( s , t ) = Z { z N ( t ) } ( s , t ) I { t < s ≤ t 0 } + Z { z N ( s ) } ( s , t ) I { s ≤ t } , where Z { i } ( s , t ) = I { t < s ≤ t 0 } w a ( − → M t , i , t ) + I { s ≤ t ≤ t 0 } w b ( − → M s , i , s , M s t , t ) − I { t 0 < t } C . 1.2 The competiti ve fishing When the methods of fishing ar e o perating by sepa rated ang lers then the stopp ing random field can be b u ilt based on the structure of the marked rene wal–reward p ro- cess as a mod el of the competitive expedition results. One possible defin ition of pay-off is based on the assumption that each p layer has his o wn account related to the exploration of the fishery . Th e states of the accoun ts depen d on who forces the first stop for changing tech nique, unde r which circumstances an d what tec hniques they c hoose. The first stopping moment, the minimum o f stopping moments cho- sen by the p layers, is af ter the momen t o f the event (catching fish) T n by the ro d z n and the rew ard function s depen d on the type of fishing which g i ves recen t fish (i.e. j , where j = z n ). Th e playe r’ s pay- off w a i ( − → m , j , t ) = g a i ( − → m , j , t ) − c a i ( t ) . The part o f the pay-off which depends o n the second chosen m oment, which stops the expedition, i s different for the player who forces the change of fi shing methods (the leader) by h imself and the othe r the opp onent. The leader is th e responsible ang ler for determin ing the expedition deadline. Lets assume for a wh ile that the i -th player, i = 1 , 2, will take the r od of the oppon ent and g i ves his rod to him. It is not a crucial assumption a nyway a nd the method of fishin g after the chan ge can be differ ent fr om both available b efor e the considered moment. T he method of treatment of the case without this assum ption will be explained later (see page 9), when the behavior of the player in the second part of the expedition will be formulated. Define the function ˜ w b i ( − → m , j , s , k , e m , t ) = ˜ w a i ( − → m , j , s ) + ˜ g b i ( − → m , j , s , k , e m , t ) − c b ( t − s ) 6 A. Karpo wicz, K. Szajowsk i for j ∈ A , k ∈ B , w here j is th e rod by which the fish had been caught just bef ore the moment of the first stop an d k is the techniqu e used by i -th player after the chan ge (the d enotation − k is u sed for a com plimentary rod or play er who has decided , which is appr opriate). It describes the case when the play er d eciding to change the method chooses the perspective techniqu e of fishing as the first one. Presumably he will e x plore the best me thods with improvements and the second angler will use the rod which is not used b y the leader . The pay- off of the p layers, when i -th is the on e who forces the first stop, has the following form: Z i ( j , s , t ) = I { t ≤ s ≤ t 0 } ˜ g a i ( − → M t , j , t ) + I { s < t ≤ t 0 } ˜ w b i ( − → M s , i , s , − i , M s t , t ) − I { t 0 < t } C ( 5) Z − i ( j , s , t ) = I { t ≤ s ≤ t 0 } ˜ g a − i ( − → M t , j , t ) + I { s < t ≤ t 0 } ˜ w b − i ( − → M s , i , s , i , M s t , t ) − I { t 0 < t } C . (6) In the above pay– offs it is assumed that the fin al sto p can be decla red at any mome nt. The chang e of techn iques de claration each p layer makes just af ter an event at his rod (the catching fish at his rod) as long as on th e oppo nent’ s rod there is no event. The details o f the strategy sets an d the solution concep t are formulate d in the fur ther parts of the paper . The extension consider ed here is mo ti vated by the natural, mor e precise m od- els of th e known real application s of the fishing prob lem. The typical pro cess of software testing consists of checking subrou tines. V arious types of bugs can be dis- covered. Each problem with sub routines generates the co st of a bug removal and increases the value o f the software. It d epends on the types o f the b ug f ound. The preliminar y testing requires various ty pes of experts. The stable version of sub rou- tines can be kept b y less edu cated com puter scientists. The consecutive stopping times are moments wh en the expert of the defined class stops testin g one module and the another tester starts checking . Similar ly as in the proo f r eading. 2 The optimization problem and a tw o person game 2.1 F iltra tions and Markov mom ents Let the sequenc es o f p airs { ( T n , z n ) } ∞ n = 0 be 2-variate r enew al pr ocess ( A -marked renewal pro cess) d efined on ( Ω , F , P ) . According to the d enotation of the p revious section there are three r enew a l p rocesses { T { i } n } ∞ n = 0 , i = 1 , 2 , 3, and denoted b y T n = T { z n } N z n ( T n +) . There are also three ren ew al– rew arded pr ocesses { ( T { i } n , X { i } n ) } ∞ n = 0 , i = 1 , 2 , 3 . By con vention let us d enote X n = X { z n } N z n ( T n ) . Th e fo llowing σ -field gen erated by history of the A -marked rene wal processes are defined F t = F A t = σ ( X 0 , T 0 , z 0 . . . , X N ( t ) , T N ( t ) , z N ( t ) ) , (7) for t ≥ 0 . This σ -field can be defined as Anglers’ fishing problem 7 F A t = σ { ( − → N ( s ) , X N ( s ) , z N ( s ) ) , 0 ≤ s ≤ t , i ∈ A } . Definition 1. Let T be a set o f stopping ti mes with respect to σ -fields { F t } , t ≥ 0, defined by (7). The restricted sets of stopping times are T n , K = { τ ∈ T : τ ≥ 0, T n ≤ τ ≤ T K } (8) for n ∈ N , n < K a re subsets of T . The elements of T n , K are denoted τ n , K . The stoppin g times τ ∈ T ha ve n ice r epresentation which will be help ful in the solu- tion of the o ptimal stopping prob lems for the renew al processes (see Br ´ em aud [3]). The crucial role in o ur subsequ ent co nsideration s p lays such a repr esentation. The following lemma is for the unrestricted stopping times. Lemma 1. If τ ∈ T then ther e exist R n ∈ Mes ( F n ) such tha t the cond ition τ ∧ T n + 1 = ( T n + R n ) ∧ T n + 1 on { τ ≥ T n } a.s. is fulfilled. V arious restrictions in the class of admissible stopping times will ch ange this repre- sentation. Some examples of subclasses of T are formulated here (see Le mma 1). Only a few of them are used in optimiza tion problem s investigated in the paper (see page 9, Corollary 1). Let F s , t = σ ( F A s , X { 3 } 0 , T { 3 } 0 , . . . , X { 3 } N 3 (( t − s )) + , T { 3 } N 3 (( t − s ) + ) ) be the σ -field gener- ated by all ev e nts u p to time t if the switch at time s from 2-variate renewal process to ano ther renew a l process took place. For simplicity of no tation we set 2 F { i } n : = F T { i } n , F n : = F T n , F s n : = F s , T { 3 } n . Let Mes ( F n ) ( Mes ( F { i } n ) ) denote th e set of non -negativ e and F n ( F { i } n )-measurab le rando m variables. From now on, T and T s stands f or the sets of sto pping times with respect to σ -fields F s and { F s , t , 0 ≤ s ≤ t } , respectively . Furthermore, we can define for n ∈ N an d n ≤ K the sets 1. T { i } n , K = { τ ∈ T : τ ≥ 0 , T { i } n ≤ τ ≤ T K } ; 2. T { i } n = { τ ∈ T : τ ≥ T { i } n } ; 3. ¯ T { i , A {− i } } n , K = { τ ∈ T : τ ≥ 0 , T { i } n ≤ τ ≤ T K , ∀ k τ / ∈ [ T A − i k , T A − i k + 1 ∨ T { i } n { i } ( T A − i k ) ] } where A {− i } : = A \ { i } , T A − i k : = min { j ∈ A {− i } } { T { j } n { j } ( T { i } k ) } ; 4. ¯ T { i } n = { τ ∈ T : τ ≥ T { i } n , ∀ k τ / ∈ [ T A − i k , T A − i k + 1 ∨ T { i } n { i } ( T A − i k ) ] } ; 5. T s n , K = { τ ∈ T s : 0 ≤ s ≤ τ , T { 3 } n ≤ τ ≤ T K } . The stoppin g times τ ∈ T { i } and τ ∈ ¯ T { i } can also be re presented in th e way shown in Lemma 1. 2 For the optimizati on problem there are two epochs: before the first stop, where there are some pay-of fs, the model of stream of ev ents, and after the first stop, when t here are other pay-of fs and dif ferent streams of even ts. In s ection 3 this wil l be e mphasized, by adopting adequate denotations. 8 A. Karpo wicz, K. Szajowsk i Lemma 2. Let the index i ∈ A b e chosen and fixed. 1. F o r e very τ ∈ T { i } and n ∈ N there exis t R { i } n ∈ Mes ( F { i } n ) such that τ ∧ T { i } n + 1 = ( T { i } n + R { i } n ) ∧ T { i } n + 1 on { τ { i } ≥ T { i } n } a .s. is fulfilled. 2. If τ ∈ ¯ T { i } and n ∈ N ther e exist R { i } n ∈ Mes ( F { i } n ) such th at the c ondition τ ∧ T { i } n + 1 = ( T { i } n + R { i } n ) ∧ T { i } n + 1 on { τ ≥ T { i } n } a .s. is fulfilled. Obviously the angler wants to hav e as m uch satisfaction as possible and he has to lea ve t he lake before the fixed m oment. Therefore, his goal is to find two optimal stopping times τ a ∗ and τ b ∗ so that the expected gain is maximized E Z ( τ a ∗ , τ b ∗ ) = su p τ a ∈ T sup τ b ∈ T τ a E Z ( τ a , τ b ) , (9) where τ a ∗ correspo nds to the moment, when he e ventua lly should chang e the two rods to the more effecti ve one and τ b ∗ , when he should stop fishing. The se stop- ping mo ments should ap pear befo re the fixed time of fishing t 0 . Th e process Z ( s , t ) is piecewise-deterministic and belongs to the class of semi-Markov processes. The optimal stopping of similar semi-Markov processes was studied by Bo shuizen and Gouweleeuw [1] and the multivariate po int pro cess by Boshuize n [2]. Here the structure of multiv ariate processes is discovered and their importan ce f or the model is shown. W e use the dynam ic progr amming method s to find these two op timal stopping times and to s pecify the expected satisfaction of the angler . The way of the solution is similar to th e method s used by Karp owicz an d Szajowski [13], Karpow- icz [ 12] and Szajowski [2 2]. L et u s first observe that b y the pro perties o f con ditional expectation we ha ve E Z ( τ a ∗ , τ b ∗ ) = su p τ a ∈ T E { E h Z ( τ a , τ b ∗ ) | F τ a i } = sup τ a ∈ T E J ( τ a ) , where J ( s ) = E h Z ( s , τ b ∗ ) | F s i = ess su p τ b ∈ T s E h Z ( s , τ b ) | F s i . (10) Therefo re, in or der to find τ a ∗ and τ b ∗ , we have to calculate J ( s ) first. The process J ( s ) correspo nds to the v alue of t he re venu e fu nction in one stopping problem if the observation starts at the moment s . 2.2 Anglers’ games Based on th e consideration of the section 1 .2 a version of comp etiti ve fishing is formu lated here. Ther e are two anglers, each using o ne m ethod of fishing at the beginning of an expedition and an additional fishing p eriod after a certain momen t by ano ther method u p to the mom ent chosen by a certain rule. Th e rando m field which is th e m odel of p ayoffs in such a case is g iv en by (5) and (6). The final Anglers’ fishing problem 9 segment starts at th e moment when one of the angle rs wants it. Let τ i ∈ ¯ T { i } , i = A , be the strategies of the players to stop individual fishing per iod an d switch to th e time segment which is stopp ed at moment σ determined by on e angler (let us call them a leader ). The payoffs of the players are ψ i ( τ 1 , τ 2 ) = Z i ( z N ( τ 1 ∧ τ 2 ) , τ 1 ∧ τ 2 , σ τ 1 ∧ τ 2 ) I { τ 1 6 = τ 2 } (11) + Z i ( z N ( τ 1 ∧ τ 2 ) ∧ z N ( τ 1 ∧ τ 2 ) , τ 1 ∧ τ 2 , σ τ 1 ∧ τ 2 ) I { τ 1 = τ 2 } . The aim is to find a pair ( τ ⋆ 1 , τ ⋆ 2 ) of stopping times such that for i ∈ { 1 , 2 } we have E ψ i ( τ ⋆ i , τ ⋆ − i ) ≥ E ψ i ( τ i , τ ⋆ − i ) . (12) The optimization problem of the angler and the game between two anglers will in volve the construction of the optimal second stopping moment. 3 Construction of the optimal second stopping time In th is section, we will find the solution o f one stopping problem defined by (10). W e will first solve the proble m fo r the fixed num ber of fishe s caught, next we will consider the case with the infinite stream of fishes caug ht. In this section we fix s - the m oment when the chan ge took place and m = M s - the mass o f the fishes at the time s . T aking into accoun t various models o f fishing after the first stop it is needed to ad mit v ario us mo dels o f stream of events. Assume that the m oments of successiv e fishes catching after the first stop are T { 3 } n and the times between the e ven ts are i.i.d. with continuou s, cumulative distribution fun ction F ( t ) with the density function f ( t ) and the fishes value represented by i.i. d. random variables with distribution function H ( t ) (f or conveniences this part of e xpedition is mod elled by the renewal p rocess denoted ( T { 3 } n , X { 3 } n ) ). 3.1 F ixed number of fishes caught In this subsection we are lookin g for optimal stopping time τ b ∗ 0 , K : = τ b K ∗ E h Z ( s , τ b K ∗ ) | F s i = ess sup τ b K ∈ T s 0 , K E h Z ( s , τ b K ) | F s i , (13) where s ≥ 0 is a fixed time wh en th e position was chang ed and K is the maximum number of fishes which can be caught. Let us define Γ s n , K = ess su p τ b n , K ∈ T s n , K E h Z ( s , τ b n , K ) | F s n i = E h Z ( s , τ b ∗ n , K ) | F s n i , n = K , . . . , 1 , 0 (14) 10 A. Karpo wicz, K. Szajowsk i and observe that Γ s K , K = Z ( s , T { 3 } K ) . In the subsequent co nsiderations we will use the representatio n of stopping time formulated in Lemm a 1 and 2. The exact form of the stopping strategies are gi ven in the following corollary . Corollary 1. Let i ∈ A . If τ a ∈ T { i } , τ b ∈ T s , then ther e exist R a n ∈ Mes ( F { i } n ) and R b n ∈ Mes ( F s n ) r espective ly , such that for co nditions τ a ∧ T { i } n + 1 = ( T { i } n + R a n ) ∧ T { i } n + 1 on { τ a ≥ T { i } n } a.s. and τ b ∧ T { 3 } n + 1 = ( T { 3 } n + R a n ) ∧ T { 3 } n + 1 on { τ a ≥ s ∧ T { 3 } n } a.s. ar e valid. Now we can d eriv e the dyn amic p rogram ming equations satisfied by Γ s n , K . T o simplify the notation we can write M t = M s t for t ≤ s , b M { 1 } n = M T 1 n , M s n = M s T { 3 } n and ¯ F i = 1 − F i . The pay off fun ctions ar e simp lified he re to ˆ g a ( m ) = g a ( m 1 , m 2 , i , t ) I { m 1 + m 2 = m } ( m 1 , m 2 ) , ˆ g b ( m ) = g b ( m 1 , m 2 , i , s , e m , t ) I { e m − m 1 − m 2 = m } Lemma 3. Let s ≥ 0 be the moment of changing fi shery . F or n = K − 1 , K − 2 , . . . , 0 Γ s K , K = Z ( s , T { 3 } K ) , Γ s n , K = ess sup R b n ∈ Mes ( F s n ) ϑ n , K ( M s , s , M s n , T { 3 } n , R b n ) a.s., (15) wher e ϑ n , K ( m , s , e m , t , r ) = I { t ≤ t 0 }  ¯ F ( r )[ I { r ≤ t 0 − t } ˆ w b ( m , s , e m , t + r ) − C I { r > t 0 − t } ] + E  I { S { 3 } n + 1 ≤ r } Γ s n + 1 , K | F s n   − C I { t > t 0 } and ther e exists R b n ⋆ ∈ Mes ( F s n ) such that Γ s n , K = ϑ n , K ( M s , s , M s n , T { 3 } n , R b n ⋆ ) a.s., (16) τ b ∗ n , K = ( τ b ∗ n + 1 , K if R b n ∗ ≥ S { 3 } n + 1 , T { 3 } n + R b n ∗ if R b n ∗ < S { 3 } n + 1 , (17) τ b ∗ K , K = T { 3 } K and ˆ w b ( m , s , e m , t ) = ˆ w a ( m , s ) + ˆ g b ( e m − m ) − c b ( t − s ) where ˆ w a ( m , t ) = ˆ g a ( m ) − c a ( t ) . Remark 2. Let { R b ∗ n } K n = 1 , R b ∗ K = 0 , be a sequence of F s n –measurab le random vari- ables, n = 1 , 2 , . . . , K , an d η ⋆ s n , K = K ∧ inf { i ≥ n : R b i ⋆ < S { 3 } i + 1 } . Th en Γ s n , K = E h Z ( s , τ b ∗ n , K ) | F s n i for n ≤ K − 1, where τ b ∗ n , K = T η ⋆ s n , K + R b ⋆ η ⋆ s n , K . P R O O F O F R E M A R K . 2. It is a consequen ce o f an optimal choice R b ⋆ n in (15).  Anglers’ fishing problem 11 P R O O F O F L E M M A . 3 First observe that th e f orm of the Γ s n , K for the case T { 3 } n > t 0 is obvious f rom (4) and (14). Let us assume (15) and (16) for n + 1 , n + 2 , . . . , K . For any τ ∈ T s n , K (i.e. τ ≥ T { 3 } n we have { τ < T { 3 } n + 1 } = { τ ∧ T { 3 } n + 1 < T { 3 } n + 1 } = { T { 3 } n + R b n < T { 3 } n + 1 } . It implies { τ < T { 3 } n + 1 } = { S { 3 } n + 1 > R b n } , { τ ≥ T { 3 } n + 1 } = { S { 3 } n + 1 ≤ R b n } . (18) Suppose tha t T { 3 } K − 1 ≤ t 0 and ta ke any τ b K − 1 , K ∈ T s K − 1 , K . According to (18) and the proper ties of conditional expectation E [ Z ( s , τ ) | F s n ] = E  I { S { 3 } n + 1 ≤ R b n } E [ Z ( s , τ ∨ T { 3 } n + 1 ) | F s n + 1 ] | F n  + E  I { S { 3 } n + 1 > R b n } Z ( s , τ ∧ T { 3 } n + 1 ) | F s n  = I { R b n ≤ t 0 − T n } ¯ F ( R n ) ˆ w b ( M s , s , M s n , T { 3 } n + R b n ) + E  I { S { 3 } n + 1 ≤ R b n } E [ Z ( s , τ ∨ T { 3 } n + 1 ) | F s n + 1 | F s n  . Let σ ∈ T b n + 1 . For e very τ ∈ T n we have τ = ( σ if R b n ≥ S { 3 } n + 1 , T { 3 } n + R b n if R b n < S { 3 } n + 1 . W e ha ve E [ Z ( s , τ ) | F s n ] = E  I { S { 3 } n + 1 ≤ R b n } E [ Z ( s , σ ) | F s n + 1 ] | F n  + I { R b n ≤ t 0 − T n } ¯ F ( R b n ) ˆ w b ( M s , s , M s n , T { 3 } n + R b n ) ≤ sup R ∈ Mes ( F s n ) { E  I { S { 3 } n + 1 ≤ R } Γ s n + 1 , K | F n  + I { R ≤ t 0 − T n } ¯ F ( R ) ˆ w b ( M s , s , M s n , T { 3 } n + R ) } = E [ Z ( s , τ ⋆ n , K | F s n ] It follows sup τ ∈ T s n E [ Z ( s , τ ) | F s n ] ≤ E [ Z ( s , τ ⋆ n , K | F s n ] ≤ sup τ ∈ T b n E [ Z ( s , τ ) | F s n ] wh ere the last ineq uality is because τ ⋆ n , K ∈ T s n , K . W e apply th e induction hyp othesis, which completes the proo f. z Lemma 4. Γ s n , K = γ s , M s K − n ( M s n , T { 3 } n ) for n = K , . . . , 0 , wher e the sequence of functions γ s , m j is given r ecu rsively as follows: 12 A. Karpo wicz, K. Szajowsk i γ s , m 0 ( e m , t ) = I { t ≤ t 0 } ˆ w b ( m , s , e m , t ) − C I { t > t 0 } , γ s , m j ( e m , t ) = I { t ≤ t 0 } sup r ≥ 0 κ b γ s , m j − 1 ( m , s , e m , t , r ) − C I { t > t 0 } , (19) wher e κ b δ ( m , s , e m , t , r ) = ¯ F ( r )[ I { r ≤ t 0 − t } ˆ w b ( m , s , e m , t + r ) − C I { r > t 0 − t } ] + Z r 0 d F ( z ) Z ∞ 0 δ ( e m + x , t + z ) d H ( x ) . P R O O F O F L E M M A . 4. Sinc e the case for t > t 0 is obviou s let us assume that T { 3 } n ≤ t 0 for n ∈ { 0 , . . . , K − 1 } . Let us notice that accordin g to Lemma 3 we obtain Γ s K , K = γ s , M s 0 ( M s K , T { 3 } K ) , th us the pro position is satisfied for n = K . Let n = K − 1 then Lemma 3 and the induction hypo thesis lead s to Γ s K − 1 , K = ess su p R b K − 1 ∈ Mes ( F s , K − 1 )  ¯ F ( R b K − 1 )[ I { R b K − 1 ≤ t 0 − T { 3 } K − 1 } ˆ w b ( M s , s , M s K − 1 , T { 3 } K − 1 + R b K − 1 ) − C I { R b K − 1 > t 0 − T { 3 } K − 1 } ] + E  I { S { 3 } K ≤ R b K − 1 } γ s , M s 0 ( M s K , T { 3 } K ) | F s , K − 1   a . s ., where M s K = M s K − 1 + X { 3 } K , T { 3 } K = T { 3 } K − 1 + S { 3 } K and the r andom variables X { 3 } K and S { 3 } K are independ ent o f F s , K − 1 . Moreover R b K − 1 , M s K − 1 and T { 3 } K − 1 are F s , K − 1 - measurable. It follows Γ s K − 1 , K = ess su p R b K − 1 ∈ Mes ( F s , K − 1 )  ¯ F ( R b K − 1 )[ I { R b K − 1 ≤ t 0 − T { 3 } K − 1 } ˆ w b ( M s , s , M s K − 1 , T { 3 } K − 1 + R b K − 1 ) − C I { R b K − 1 > t 0 − T { 3 } K − 1 } ] + Z R b K − 1 0 d F ( z ) Z ∞ 0 γ s , M s 0 ( M s K − 1 + x , T { 3 } K − 1 + z ) d H ( x )  = γ s , M s 1 ( M s K − 1 , T { 3 } K − 1 ) a . s . Let n ∈ { 1 , . . . , K − 1 } and suppose that Γ s n , K = γ s , M s K − n ( M s n , T { 3 } n ) . Sim ilarly like b efore, we conclu de by Lemma 3 and induction hypo thesis th at Γ s n − 1 , K = ess su p R b n − 1 ∈ Mes ( F s n − 1 )  ¯ F ( R b n − 1 )[ I { R b n − 1 ≤ t 0 − T { 3 } n − 1 } ˆ w b ( M s , s , M s n − 1 , T { 3 } n − 1 + R b n − 1 ) − C I { R b n − 1 > t 0 − T { 3 } n − 1 } ] + Z R b n − 1 0 d F ( s ) Z ∞ 0 γ s , M s K − n ( M s n − 1 + x , T { 3 } n − 1 + s ) d H ( x )  a . s . therefor e Γ s n − 1 , K = γ s , M s K − ( n − 1 ) ( M s n − 1 , T { 3 } n − 1 ) . z Anglers’ fishing problem 13 From now on we will use α i to denote the hazard rate of th e distribution F i (i.e. α i = f i / ¯ F i ) and to shorten notation we set ∆ · ( a ) = E  ˆ g · ( a + X { i } ) − ˆ g · ( a )  , where · can be a o r b . Remark 3. The sequen ce of functions γ s , m j can be expressed as: γ s , m 0 ( e m , t ) = I { t ≤ t 0 } ˆ w b ( m , s , e m , t ) − C I { t > t 0 } , γ s , m j ( e m , t ) = I { t ≤ t 0 }  ˆ w b ( m , s , e m , t ) + y b j ( e m − m , t − s , t 0 − t )  − C I { t > t 0 } and y b j ( a , b , c ) is given recursi vely as follows y b 0 ( a , b , c ) = 0 y b j ( a , b , c ) = max 0 ≤ r ≤ c φ b y b j − 1 ( a , b , c , r ) , where φ b δ ( a , b , c , r ) = R r 0 ¯ F ( z ) { α 2 ( z )  ∆ b ( a ) + E δ ( a + X { 3 } , b + z , c − z )  − c b ′ ( b + z ) } d z . P R O O F O F R E M A R K . 3 Clearly Z r 0 d F ( s ) Z ∞ 0 γ s , m j − 1 ( e m + x , t + s ) d H ( x ) = E h I { S { 3 } ≤ r } γ s , m j − 1 ( e m + X { 3 } , t + S { 3 } ) i , where S { 3 } has c.d .f. F a nd X { 3 } has c.d .f. H. Since F is co ntinuou s and κ b γ s , m j − 1 ( m , s , e m , t , r ) is bou nded and continu ous for t ∈ R + \ { t 0 } , the sup remum in (19) can be change d into maximum . Let r > t 0 − t the n κ b γ s , m j − 1 ( m , s , e m , t , r ) = E h I { S { 3 } ≤ t 0 − t } γ s , m j − 1 ( e m + X { 3 } , t + S { 3 } ) i − C ¯ F ( t 0 − t ) ≤ E h I { S { 3 } ≤ t 0 − t } γ s , m j − 1 ( e m + X { 3 } , t + S { 3 } ) i + ¯ F ( t 0 − t ) ˆ w b ( m , s , e m , t 0 ) = κ b γ s , m j − 1 ( m , s , e m , t , t 0 − t ) . The above calculations cau se that γ s , m j ( e m , t ) = I { t ≤ t 0 } max 0 ≤ r ≤ t 0 − t ϕ j ( m , s , e m , t , r ) − C I { t > t 0 } , wher e ϕ j ( m , s , e m , t , r ) = ¯ F ( r ) ˆ w b ( m , s , e m , t + r ) + E h I { S { 3 } ≤ r } γ s , m j − 1 ( e m + X { 3 } , t + S { 3 } ) i . Obviously f or S { 3 } ≤ r and r ≤ t 0 − t we h av e S { 3 } ≤ t 0 therefor e we can con sider the cases t ≤ t 0 and t > t 0 separately . Let t ≤ t 0 then γ s , m 0 ( e m , t ) = ˆ w b ( m , s , e m , t ) and the hypoth esis is true fo r j = 0. The task is now to calculate γ s , m j + 1 ( e m , t ) given γ s , m j ( · , · ) . The induction hypoth esis implies that for t ≤ t 0 14 A. Karpo wicz, K. Szajowsk i ϕ j + 1 ( m , s , e m , t , r ) = ¯ F ( r ) ˆ w b ( m , s , e m , t + r ) + E h I { S { 3 } ≤ r } γ s , m j ( e m + X { 3 } , t + S { 3 } ) i = ˆ g a ( m ) − c a ( s ) + ¯ F ( r ) h ˆ g b ( e m − m ) − c b ( t − s + r ) i + Z r 0 f ( z ) { E ˆ g b ( e m − m + X { 3 } ) − c b ( t − s + z ) + E y b j ( e m − m + X { 3 } , t − s + z , t 0 − t − z ) } d z . It is clear that for any a and b ¯ F ( r ) h ˆ g b ( a ) − c b ( b + r ) i = ˆ g b ( a ) − c b ( b ) − Z r 0 { f ( z ) h ˆ g b ( a ) − c b ( b + z ) i + ¯ F ( z ) c b ′ ( b + z ) } d z , therefor e ϕ j + 1 ( m , s , e m , t , r ) = ˆ w b ( m , s , e m , t ) + Z r 0 ¯ F ( z ) { α 2 ( z )[ ∆ b ( e m − m ) + E y b j ( e m − m + X { 3 } , t − s + z , t 0 − t − z )] − c b ′ ( t − s + z ) } d z , which proves th e theorem . The case for t > t 0 is trivial.  Follo wing the methods of Ferenstein and Sieroci ´ nski [6], we find the second op- timal stopping time. Let B = B ([ 0 , ∞ ) × [ 0 , t 0 ] × [ 0 , t 0 ]) be the space of all bounded, continuo us functions with the norm k δ k = sup a , b , c | δ ( a , b , c ) | . I t is easy to check that B with the n orm sup remum is co mplete space . Th e op erator Φ b : B → B is defined by ( Φ b δ )( a , b , c ) = max 0 ≤ r ≤ c φ b δ ( a , b , c , r ) . (2 0) Let us observe that y b j ( a , b , c ) = ( Φ b y b j − 1 )( a , b , c ) . Rem ark 3 now implies that there exists a function r b ∗ j ( a , b , c ) su ch that y b j ( a , b , c ) = φ b y b j − 1 ( a , b , c , r b ∗ j ( a , b , c )) and this giv es γ s , m j ( e m , t ) = I { t ≤ t 0 }  ˆ w b ( m , s , e m , t ) + φ b y b j − 1 ( e m − m , t − s , t 0 − t , r b ∗ j ( e m − m , t − s , t 0 − t ))  − C I { t > t 0 } . The con sequence of th e fo regoing considerations is the theorem, which determines optimal stopping times τ b ∗ n , K in the following manner: Theorem 1. Let R b i ∗ = r b ∗ K − i ( M s i − M s , T { 3 } i − s , t 0 − T { 3 } i ) for i = 0 , 1 , . . . , K m or e- over η s n , K = K ∧ inf { i ≥ n : R b i ∗ < S { 3 } i + 1 } , then the s topping time τ b ∗ n , K = T { 3 } η s n , K + R b ∗ η s n , K is optimal in the class T s n , K and Γ s n , K = E h Z ( s , τ b ∗ n , K ) | F s n i . Anglers’ fishing problem 15 3.2 Infinite number of fishes caught The task is now to find the function J ( s ) and stoppin g time τ b ∗ , which is optimal in class T s . In o rder to get the solution of on e stopping pro blem for infinite nu mber of fishes caught it is necessary to put the restriction F ( t 0 ) < 1. Lemma 5. If F ( t 0 ) < 1 then the operator Φ b : B → B defi ned by ( 20) is a con trac- tion. P R O O F O F L E M M A . 5 . Let δ i ∈ B assuming that i ∈ { 1 , 2 } . T here e x ists ρ i such that ( Φ b δ i )( a , b , c ) = φ b δ i ( a , b , c , ρ i ) . W e thus get ( Φ b δ 1 )( a , b , c ) − ( Φ b δ 2 )( a , b , c ) = φ b δ 1 ( a , b , c , ρ 1 ) − φ b δ 2 ( a , b , c , ρ 2 ) ≤ φ b δ 1 ( a , b , c , ρ 1 ) − φ b δ 2 ( a , b , c , ρ 1 ) = Z ρ 1 0 d F ( z ) Z ∞ 0 [ δ 1 − δ 2 ]( a + x , b + z , c − s ) d H ( x ) ≤ Z ρ 1 0 d F ( z ) Z ∞ 0 sup a , b , c | [ δ 1 − δ 2 ]( a , b , c ) | d H ( x ) ≤ F ( c ) k δ 1 − δ 2 k ≤ F ( t 0 ) k δ 1 − δ 2 k ≤ C k δ 1 − δ 2 k , where 0 ≤ C < 1. Similarly , like as bef ore, ( Φ b δ 2 )( a , b , c ) − ( Φ b δ 1 )( a , b , c ) ≤ C k δ 2 − δ 1 k . Finally we conclud e tha t   Φ b δ 1 − Φ b δ 2   ≤ C k δ 1 − δ 2 k which com- pletes the proo f. z Applying Remark 3, Lemma 5 and the fixed point theorem we conclude Remark 4. There exists y b ∈ B such that y b = Φ b y b and lim K → ∞ k y b K − y b k = 0. According to the above remark , y b is the un iform limit of y b K , w hen K tends to infinity , which implies that y b is measurable and γ s , m = lim K → ∞ γ s , m K is giv en by γ s , m ( e m , t ) = I { t ≤ t 0 } h ˆ w b ( m , s , e m , t ) + y b ( e m − m , t − s , t 0 − t ) i − C I { t > t 0 } . (21) W e can now calcu late the optimal strategy and the expected gain after changing t he place. Theorem 2. If F ( t 0 ) < 1 an d has the density function f, then (i) for n ∈ N the limit τ b ⋆ n = lim K → ∞ τ b ∗ n , K a . s . exists a nd τ b ⋆ n ≤ t 0 is an op timal stopping rule in the set T s ∩ { τ ≥ T { 3 } n } , (ii) E  Z ( s , τ b ⋆ n ) | F s n  = γ s , m ( M s n , T { 3 } n ) a.s. P R O O F . (i) Let us first prove the existence of τ b ⋆ n . By definition of Γ s n , K + 1 we have Γ s n , K + 1 = ess sup τ ∈ T s n , K + 1 E [ Z ( s , τ ) | F s n ] = ess sup τ ∈ T s n , K E [ Z ( s , τ ) | F s n ] ∨ ess sup τ ∈ T s K , K + 1 E [ Z ( s , τ ) | F s n ] = E h Z ( s , τ b ∗ n , K ) | F s n i ∨ E [ Z ( s , σ ∗ ) | F s n ] 16 A. Karpo wicz, K. Szajowsk i thus we ob serve that τ b ∗ n , K + 1 is equal to τ b ∗ n , K or σ ∗ , wher e τ b ∗ n , K ∈ T s n , K and σ ∗ ∈ T s K , K + 1 respectively . It follows th at τ b ∗ n , K + 1 ≥ τ b ∗ n , K which implies that the sequen ce τ b ∗ n , K is no ndecreasing with respect to K . Moreover R b i ∗ ≤ t 0 − T { 3 } i for all i ∈ { 0 , . . . , K } thus τ b ∗ n , K ≤ t 0 and therefo re τ b ⋆ n ≤ t 0 exists. Let us now look at the pro cess ξ s ( t ) = ( t , M s t , V ( t )) , where s is fixed and V ( t ) = t − T { 3 } N 3 ( t ) . ξ s ( t ) is Markov pro cess with the state space [ s , t 0 ] × [ m , ∞ ) × [ 0 , ∞ ) . In a general case the infinitesimal operato r f or ξ s is giv en by A p s , m ( t , e m , v ) = ∂ ∂ t p s , m ( t , e m , v ) + ∂ ∂ v p s , m ( t , e m , v ) + α 2 ( v )  Z R + p s , m ( t , x , 0 ) d H ( x ) − p s , m ( t , e m , v )  , where p s , m ( t , e m , v ) : [ 0 , ∞ ) × [ 0 , ∞ ) × [ 0 , ∞ ) → R is c ontinuo us, bo unded, measur able with bounded left-ha nd derivati ves with r espect to t and v (see [1] and [17]). Let us notice th at for t ≥ s the process Z ( s , t ) can be expressed as Z ( s , t ) = p s , m ( ξ s ( t )) , where p s , m ( ξ s ( t )) =  ˆ g a ( M s ) − c a ( s ) + ˆ g b ( M s t − M s ) − c b ( t − s ) if s ≤ t ≤ t 0 , − C if t 0 < t . It follows easily that in our case A p s , m ( t , e m , v ) = 0 for t 0 < t and A p s , m ( t , e m , v ) = α 2 ( v )[ E ˆ g b ( e m + X { 3 } − m ) − ˆ g b ( e m − m )] − c b ′ ( t − s ) (22) for s ≤ t ≤ t 0 . The process p s , m ( ξ s ( t )) − p s , m ( ξ s ( s )) − R t s ( A p s , m )( ξ s ( z )) d z is a mar- tingale with r espect to σ ( ξ s ( z ) , z ≤ t ) which is the same as F s , t . Th is can be fo und in [4]. Since τ b ∗ n , K ≤ t 0 , applying the Dynkin’ s formula we obtain E h p s , m ( ξ s ( τ b ∗ n , K )) | F s n i − p s , m ( ξ s ( T { 3 } n )) = E " Z τ b ∗ n , K T { 3 } n ( A p s , m )( ξ s ( z )) d z | F s n # a . s . (23) From (22) we conclude that Z τ b ∗ n , K T { 3 } n ( A p s , m )( ξ s ( z )) d z = [ E ˆ g b ( M s n + X { 3 } − m ) − ˆ g b ( M s n − m )] Z τ b ∗ n , K T { 3 } n α 2 ( z − T { 3 } n ) d z − Z τ b ∗ n , K T { 3 } n c b ′ ( z − s ) d z . Moreover let us check that Anglers’ fishing problem 17      Z τ b ∗ n , K T { 3 } n α 2 ( z − T { 3 } n ) d z      ≤ 1 ¯ F ( t 0 ) Z τ b ∗ n , K T { 3 } n f ( z − T { 3 } n ) d z ≤ 1 ¯ F ( t 0 ) < ∞ ,      Z τ b ∗ n , K T { 3 } n c b ′ ( z − s ) d z      =    c b ( τ b ∗ n , K − s ) − c b ( T { 3 } n − s )    < ∞ ,    E ˆ g b ( M s n + X { 3 } − m ) − ˆ g b ( M s n − m )    < ∞ , where the two last inequalities result from the fact that the functions ˆ g b and c b are bound ed. On account of th e above observation we can use the d ominated co n ver- gence theorem and lim K → ∞ E " Z τ b ∗ n , K T { 3 } n ( A p s , m )( ξ s ( z )) d z | F s n # = E " Z τ b ⋆ n T { 3 } n ( A p s , m )( ξ s ( z )) d z | F s n # . (24) Since τ b ⋆ n ≤ t 0 applying the Dy nkin’ s form ula to the left side of (24) we con clude that E " Z τ b ⋆ n T { 3 } n ( A p s , m )( ξ s ( z )) d z | F s n # = E h p s , m ( ξ s ( τ b ⋆ n )) | F s n i − p s , m ( ξ s ( T { 3 } n )) a . s . (25) Combining (23), (24) and (25) we can see that lim K → ∞ E h p s , m ( ξ s ( τ b ∗ n , K )) | F s n i = E h p s , m ( ξ s ( τ b ∗ n )) | F s n i , (26) hence that lim K → ∞ E h Z ( s , τ b ∗ n , K ) | F s n i = E  Z ( s , τ b ∗ n ) | F s n  . W e n ext pr ove the opti- mality of τ b n ∗ in the class T s ∩ { τ b n ≥ T { 3 } n } . Let τ ∈ T s ∩ { τ b n ≥ T { 3 } n } and it is clear that τ ∧ T { 3 } K ∈ T s n , K . As τ b ∗ n , K is optim al in the class T s n , K we hav e lim K → ∞ E h p s , m ( ξ s ( τ b ∗ n , K )) | F s n i ≥ lim K → ∞ E h p s , m ( ξ s ( τ ∧ T { 3 } K )) | F s n i . (27) From (26) and (27) we conclude th at E  p s , m ( ξ s ( τ b ∗ n )) | F s n  ≥ E [ p s , m ( ξ s ( τ )) | F s n ] for a ny stoppin g time τ ∈ T s ∩ { τ ≥ T { 3 } n } , wh ich implies that τ b ∗ n is optimal in th is class. (ii) Lemma 4 and (26) lead to E h Z ( s , τ b n ∗ ) | F s n i = γ s , M s ( M s n , T { 3 } n ) .  The remainder of this section will b e dev oted to the proof of the left- hand differ - entiability of the fun ction γ s , m ( m , s ) with respect to s . This pr operty is n ecessary to construct the first optimal stopp ing tim e. Fir st, l et us briefly denote δ ( 0 , 0 , c ) ∈ B by ¯ δ ( c ) . Lemma 6. Let ¯ ν ( c ) = Φ b ¯ δ ( c ) , ¯ δ ( c ) ∈ B and   ¯ δ ′ + ( c )   ≤ A 1 for c ∈ [ 0 , t 0 ) then   ¯ ν ′ + ( c )   ≤ A 2 . 18 A. Karpo wicz, K. Szajowsk i P R O O F O F L E M M A . 6. First o bserve th at the d eriv ative ¯ ν ′ + ( c ) exists becau se ¯ ν ( c ) = m ax 0 ≤ r ≤ c ¯ φ b ( c , r ) , where ¯ φ b ( c , r ) is d ifferentiable with respe ct to c and r . Fix h ∈ ( 0 , t 0 − c ) an d defin e ¯ δ 1 ( c ) = ¯ δ ( c + h ) ∈ B and ¯ δ 2 ( c ) = ¯ δ ( c ) ∈ B . Obvio usly , k Φ b ¯ δ 1 − Φ b ¯ δ 2 k ≥ | Φ b ¯ δ 1 ( c ) − Φ b ¯ δ 2 ( c ) | = | Φ b ¯ δ ( c + h ) − Φ b ¯ δ ( c ) | and on the other side using T aylor’ s formula for right-han d deriv atives we ob tain   ¯ δ 1 − ¯ δ 2   = sup c   ¯ δ ( c + h ) − ¯ δ ( c )   ≤ h sup c   ¯ δ ′ + ( c )   + | o ( h ) | . From the above an d Remark 8 it follows that − C  sup c   ¯ δ ′ + ( c )   + | o ( h ) | h  ≤ ¯ ν ( c + h ) − ¯ ν ( c ) h ≤ C  sup c   ¯ δ ′ + ( c )   + | o ( h ) | h  and letting h → 0 + giv es   ¯ ν ′ + ( c )   ≤ C A 1 = A 2 . z The significance o f L emma 6 is such that the fu nction ¯ y ( t 0 − s ) h as bounded left- hand de riv ative with respec t to s fo r s ∈ ( 0 , t 0 ] . The im portant conseq uence of this fact is the following Remark 5. The functio n γ s , m can b e expressed as γ s , m ( m , s ) = I { s ≤ t 0 } u ( m , s ) − C I { s > t 0 } , where u ( m , s ) = ˆ g a ( m ) − c a ( s ) + ˆ g b ( 0 ) − c b ( 0 ) + ¯ y b ( t 0 − s ) is continuou s, b ound ed, measurable with the bounde d left-hand deriv ati ves with r espect to s . At th e e nd o f this section, we determine the conditional value fu nction of the second optimal stopping problem. According to (10), Theorem 2 and Remark 5 we have J ( s ) = E h Z ( s , τ b ∗ ) | F s i = γ s , M s ( M s , s ) a.s. (28) 4 Construction of the optimal first stopping time In this section, we formu late the solution of the doub le stopping problem. On the first epoch of the expedition th e admissible strategies (stopping tim es) depend on the formu lation of the prob lem. For the optimiz ation problem the most natur al are the stopping times from T (see the relev an t pro blem co nsidered in Szajows ki [22]). Howe ver, when the bilateral problem is considered the natural class of ad missible strategies dep ends o n who u ses the strategy . It shou ld be T { i } for the i - th player . Here the optimization pr oblem with restriction to the strategies from the T { 1 } at th e first epoch is in vestigated . Let us first n otice that the function u ( m , s ) has a similar properties to the f unction ˆ w b ( m , s , e m , t ) an d the process J ( s ) h as similar structure to the p rocess Z ( s , t ) . By this o bservation one can fo llow the calculations of Section 3 to g et J ( s ) . Let us define again Γ n , K = ess sup τ a ∈ T n , K E [ J ( τ a ) | F n ] , n = K , . . . , 1 , 0 , which fulfills th e following representation Anglers’ fishing problem 19 Lemma 7. Γ n , K = γ K − n ( b M { 1 } n , T { 1 } n ) for n = K , . . . , 0 , wher e the seq uence of fu nc- tions γ j can be expr essed as: γ 0 ( m , s ) = I { s ≤ t 0 } u ( m , s ) − C I { s > t 0 } , γ j ( m , s ) = I { s ≤ t 0 }  u ( m , s ) + y a j ( m , s , t 0 − s )  − C I { s > t 0 } and y a j ( a , b , c ) is give n r ec ursively as follows: y a 0 ( a , b , c ) = 0 y a j ( a , b , c ) = max 0 ≤ r ≤ c φ a y a j − 1 ( a , b , c , r ) wher e φ a δ ( a , b , c , r ) = Z r 0 ¯ F 1 ( z ) n α 1 ( z ) h ∆ a ( a ) + E δ ( a + x { 1 } , b + z , c − z ) i − ( ¯ y b ′ − ( c − z ) + c a ′ ( b + z )) o d z . Lemma 7 correspo nds to the combination of Lemma 4 and Remark 3 fr om Subsec- tion 3.1. Let the operato r Φ a : B → B be defined by ( Φ a δ )( a , b , c ) = max 0 ≤ r ≤ c φ a δ ( a , b , c , r ) . (29) Lemma 7 implies that there exists a function r ∗ 1 , j ( a , b , c ) such that γ j ( m , s ) = I { s ≤ t 0 }  u ( m , s ) + φ a y a j − 1 ( m , s , t 0 − s , r ∗ 1 , j ( m , s , t 0 − s ))  − C I { s > t 0 } . W e can no w state the analogue of Theorem 1. Theorem 3. Let R a ∗ i = r a ∗ K − i ( M i , T { 1 } i , t 0 − T { 1 } i ) a nd η n , K = K ∧ inf { i ≥ n : R a i ∗ < S { 1 } i + 1 } , th en τ a ∗ n , K = T { 1 } η n , K + R a ∗ η n , K is op timal in the class T n , K and Γ n , K = E h J ( τ a ∗ n , K ) | F n i . The following results may be proved in much the same way as in Section 3. Lemma 8. If F 1 ( t 0 ) < 1 then the operator Φ a : B → B defi ned by (29) is a contrac- tion. Remark 6. There exists y a ∈ B such that y a = Φ a y a and lim K → ∞ k y a K − y a k = 0. The above r emark implies that γ = lim K → ∞ γ K is giv en by γ ( m , s ) = I { s ≤ t 0 } [ u ( m , s ) + y a ( m , s , t 0 − s )] − C I { s > t 0 } . (30) W e can no w formulate our main results. Theorem 4. If F 1 ( t 0 ) < 1 and has the density function f 1 , then 20 A. Karpo wicz, K. Szajowsk i (i) for n ∈ N the limit τ a ∗ n = lim K → ∞ τ a ∗ n , K a . s . exists and τ a ∗ n ≤ t 0 is an optimal stop- ping rule in the set T ∩ { τ ≥ T { 1 } n } , (ii) E  J ( τ a ∗ n ) | F n  = γ ( M n , T { 1 } n ) a . s . P R O O F . The proo f follows the sam e metho d as in Th eorem 2. T he difference lies in the form of the in finitesimal op erator . Define the processes ξ ( s ) = ( s , M s , V ( s )) where V ( s ) = s − T { 1 } N 1 ( s ) . Like before ξ ( s ) is the Markov p rocess with the state space [ 0 , ∞ ) × [ 0 , ∞ ) × [ 0 , ∞ ) . N otice that J ( s ) = p ( ξ ( s )) and p ( s , m , v ) : [ 0 , t 0 ] × [ 0 , ∞ ) × [ 0 , ∞ ) → R c ontinuo us, boun ded, measurable with the bo unded left-ha nd d eriv atives with respect to s and v . It is easily seen that A p ( s , m , v ) = α 1 ( v )[ E ˆ g a ( m + x { 1 } ) − ˆ g a ( m )] − h ¯ y b ′ − ( t 0 − s ) + c a ′ ( s ) i for s ≤ t 0 . Th e rest of the pr oof remains the sam e as in the proo f of Theorem 2.  Summarizin g, the solution of a double stopping prob lem is giv e n by E Z ( τ a ∗ , τ b ∗ ) = E J ( τ a ∗ ) = γ ( M 0 , T { 1 } 0 ) = γ ( 0 , 0 ) , where τ a ∗ and τ b ∗ are defined according to T heorem 2 and Theorem 4 respecti vely . 5 Examples The form of the solution results in the fact that it is dif ficu lt to calcu late th e solution in an analytic way . In this sectio n we will present examples of the conditions for which the solution can be calculated exactly . Remark 7. If the pro cess ζ 2 ( t ) = A p s , m ( ξ s ( t )) ha s dec reasing p aths, then th e second optimal stopping time is given b y τ b ∗ n = in f { t ∈ h T { 3 } n , t 0 i : A p s , m ( ξ s ( t )) ≤ 0 } on the oth er side if ζ 2 ( t ) has n on-de creasing paths, then the seco nd optim al stoppin g time is equal to t 0 . Similarly , if the proce ss ζ 1 ( s ) = A p ( ξ ( s )) has de creasing paths, then the first opti- mal stopping time is gi ven b y τ a ∗ n = inf { s ∈ h T { 1 } n , t 0 i : A p ( ξ ( s )) ≤ 0 } on the other side if ζ 1 ( s ) has non -decreasing paths, then th e first optimal stopping time is eq ual to t 0 . P R O O F . Fr om (25) we obtain E  Z ( s , τ b ∗ n ) | F s n  = Z ( s , T { 3 } n ) + E  R τ b ∗ n T { 3 } n ( A p s , m )( ξ s ( z )) d z  a.s. and the application results of Jensen and Hsu [11] completes the proof.  Corollary 2. If S { 3 } has e xponen tial distribution with constant hazard rate α 2 , the function ˆ g b is increasing a nd c oncave, the cost function c b is co n vex and t 2 , n = T { 3 } n , m s n = M s n then Anglers’ fishing problem 21 τ b ∗ n = inf { t ∈ [ t 2 , n , t 0 ] : α 2 [ E ˆ g b ( m s n + x { 3 } − m ) − ˆ g b ( m s n − m )] ≤ c b ′ ( t − s ) } , (31) wher e s is the moment o f changin g th e p lace. Mor eover , if S { 1 } has exponentia l distribution with constant ha zar d rate α 1 , ˆ g a is increasing and concave, c a is conve x and t 1 , n = T { 1 } n , m n = b M { 1 } n then τ a ∗ n = inf { s ∈ [ t 1 , n , t 0 ] : α 1 h E ˆ g a ( m n + x { 1 } ) − ˆ g a ( m n ) i ≤ c a ′ ( s ) } P R O O F . The form of τ a ∗ n and τ b n ∗ is a consequen ce of Remark 7. Let us observe that by o ur assum ptions ζ 2 ( t ) = α 2 ∆ b ( M s t − m ) − c b ′ ( t − s ) has decreasing paths for t ∈ [ T { 3 } n , T { 3 } n + 1 ) . It suf fices to prove that ζ 2 ( T { 3 } n ) − ζ 2 ( T 2 − n ) = α 2 [ ∆ b ( M s n − m ) − ∆ b ( M s n − 1 − m )] < 0 for all n ∈ N . It remains to ch eck tha t ¯ y b ′ − ( t 0 − s ) = 0 . W e c an see that τ b ∗ = τ b ∗ ( s ) is d eterministic, which is clea r from ( 31). Let us no tice that if s ≤ t 0 then combinin g (25), (26) and (28) gives γ s , m ( m , s ) = E  Z ( s , τ b ∗ ) | F s  = Z ( s , s ) + E h R τ b ∗ s ( A p s , m )( ξ s ( z )) d z | F s i . By Remark 5 it follows that ¯ y b ( t 0 − s ) = E " Z τ b ∗ ( s ) s ( A p s , m )( ξ s ( z )) d z # = Z τ b ∗ ( s ) s h α 2 ∆ b ( 0 ) − c ′ 2 ( z − s ) i d z and this yields ¯ y b ′ − ( t 0 − s ) = Z τ b ∗ ( s ) s c ′′ 2 ( z − s ) d z + τ b ∗ ′ ( s ) h α 2 ∆ b ( 0 ) − c ′ 2 ( τ b ∗ 2 ( s ) − s ) i (32) − h α 2 ∆ b ( 0 ) − c ′ 2 ( 0 ) i = c ′ 2 ( τ b ∗ ( s ) − s ) − c ′ 2 ( 0 ) − h α 2 ∆ b ( 0 ) − c ′ 2 ( 0 ) i = 0 . The rest of proof runs as befor e.  Corollary 3. If for i = 1 a nd i = 2 the fu nctions g i ar e incr easin g and con vex, c i ar e con cave and S { i } have the exponential distr ibution w ith constan t hazar d r ate α i then τ a ∗ n = τ b ∗ n = t 0 for n ∈ N . P R O O F . It is also the straightfor ward con sequence of Remark 7. It suf fices to check that ¯ y b ′ − ( t 0 − s ) is non -increasing with respect to s . First observe that τ b ∗ ( s ) = t 0 . Considering (32) it is obvious that ¯ y b ′ − ( t 0 − s ) = α 2 ∆ b ( 0 ) − c ′ 2 ( t 0 − s ) and this com- pletes the proo f.  22 A. Karpo wicz, K. Szajowsk i 6 Conclusions This article presents th e solution of the doub le stopping prob lem in the ”fishing model” f or the finite horizon . The analytical proper ties of the reward function in one stopping pro blem played the crucial rule in our con siderations and allowed u s to get the solution for the extended problem of a double stopping. Let us notice that by repeating con siderations from Section 4 it is easy to generalize o ur model an d the solu tion to the multiple stopping problem but th e no tation can be in conv e nient. The co nstruction of t he equilibrium in the two person non-zero sum problem form u- lated in the section 2 can be redu ced to the two d ouble optimal stopping problems in the case when the payoff structure is given b y (5), (6) and (11). The key assump - tions were related to the pr operties of the distribution fun ctions. Assuming general distributions an d the infinite horizon one can get the e xtensions of the above mo del. Refer ences 1. Boshuizen, F ., Gouweleeuw , J.: General optimal stopping theorems for semi-mark ov pro- cesses . Adv . in Appl. Probab . 4 , 825–846 (1993) 2. Boshuizen, F .A.: A general frame work for optimal stopping pro blems associated w ith mult i- v ariate point proces ses, and applications. Sequen tial Anal. 13 (4), 351–365 (1994) 3. Br ´ emau d, P .: Point Process es and Queues. Martingale Dynamics. Springer -V erlag, Ne w Y ork (1981) 4. Davis, M.H.A.: Mark ov Models and Optimization. Chapman and Hall, New Y ork (1993) 5. Ferenstein, E., Pasternak-W iniarski, A.: Opti mal stopping of a r isk process with disruption and interest rates. In: M. Br ` eton, K. Szajowski (eds.) Advance s i n Dynamic Games: Differ ential and Stoch astic games: theory , application and nume rical methods, Annals of the International Society of Dynamic Games , vo l. 11, p. 18pages. Birkh ¨ ause r , Boston (2010) 6. Ferenstein, E., Sieroci ´ nski, A. : Optimal stopping of a risk proc ess. Appli cationes Mathemati- cae 24(3) , 335–342 (1997) 7. Ferguson , T .: A Poiss on Fishing Model. In Fe stschrift for Lucien Le Cam: Resear ch Papers in Probability and Statistics (D. Pollard, E. T orgersen and G. Y ang, eds.). Springer , New Y ork (1997) 8. Haggstrom, G.: Optimal sequential procedures when more then one stop is required. Ann. Math. Statist. 38 , 1618–162 6 (1967) 9. Jacobsen, M.: Point process theory an d applications. Mark ed point and pice wise deterministic proces ses., Prob ability and Its Applications , vol. 7. Birkh ¨ auser , Boston (2006) 10. Jensen, U.: An optimal stopping pr oblem in r isk the ory . Scan d. Actuarial J. 2 , 149–159 (1997) 11. Jensen, U. , Hsu, G.: Optimal stopping by means of point process obs erv ations with applica- tions in reliability . Mathematics of Operations Resear ch 18(3) , 645–657 (1993) 12. Karpo wicz, A.: Double optimal stopping in t he fishing problem. J. Appl. Probab . 46 (2), 415– 428 (2009 ). DOI 10.1239/jap/124 5676097 13. Karpo wicz, A., Szajowski, K.: Double optimal stopping of a risk process. GSSR Stochastics: An International Journa l of Probability and Stochastic Processe s 79 , 155–167 (2007) 14. Kramer , M., Star r, N.: O ptimal stopping in a size dependent sear ch. Sequential Anal. 9 , 59–8 0 (1990) 15. Muciek, B. K., Szajowski, K.: Optimal stopping of a risk process when claims are co ver ed immediately . In: Mathematical Economics, RIMS K ˆ oky ˆ ur oku , vol. 155 7, pp. 132–139 (2007) 16. Nikolae v , M.: Obobshchenn ye pos ledov atel ′ nye procedur y . Litovski ˘ i Mat ematicheski ˘ i Sbornik 19 , 35–44 (1979) Referen ces 23 17. Rolski, T . , Schmidli, H., Schimdt, V ., T eugels, J.: Stochastic Processes for Insurance and Fi- nance. John Wile y & Sons, Chichester (1998) 18. Shiryae v , A. : Optimal Stopping Rules. Springe r-V erlag, Ne w Y ork, Heidelber g, Berlin (1978) 19. Starr , N.: Optimal and adapti ve stopping based on capture times. J. Appl. Prob . 11 , 294 – 301 (1974) 20. Starr , N., W ardrop, R., W oodroofe, M. : Estimating a mean fr om delayed observ ations. Z. f ¨ ur W ahr . 35 , 103–113 (1976) 21. Starr , N., W oodroofe, M.: Gone fishin’: Opti mal stopping based on catch times. U. Mich. Report., Dept. of Statistics No. 33 (1974) 22. Szajowsk i, K.: Optimal s t opping of a 2-vector risk process. In: Stab ility in Probability , B anac h Center Publications , vol. 90, pp. 179–191. PWN, W arsza wa (2010) Index filtrations F { i } n –the short denotation of F T { i } n , 7 F t , F A t –the filt ration generated by the A -mark ed renew al–rewa rded process to the moment t , 6 F t –the filtration generated by the A -marke d rene wal–re warded process to the moment t , 6 pay-of f functions C b j –the bounds of the costs, 4 Z i ( j , s , t ) –the pay–of f process of the anglers, when the first stop has been forced by i -th one, 6 ˜ w a i ( − → m , j , s , k , e m , t ) –the pay-of f of the angler i -th at moment t , when his change of fishing method to k ∈ B has been forced by the angler j at s ( ≤ t ) and the state of the rene wal-re ward process − → m , 6 c b j ( t ) –the cumulativ e costs of fishing after the change of fishing method using method j , 4 c i , c a i , c a –the cumulative cost of usage the i -th rod at the period a , 4 g a ( − → m , j , t ) , ( g a i ( − → m , j , t ) )–the utilit y o f fishes gotten t o t he moment t (at the i -th rod) when the last catch was at the j -th rod and the state of the ren e wal-re ward proces s i s − → m , 5 g b j ( − → m , i , s , e m , t ) –the re ward function after the change of the fishing methods when the state of the rene wal-re ward processes at s has been − → m and the final state of the rene wal-re ward process at t ( ≥ s ) has been e m , 4 w a i ( − → m , j , t ) –the i -th player’ s pay-of f at moment t when the stop has been made by the j -th and t he state of t he rene wal-re ward process − → m , 5 rene wal–re ward processe s ( T { i } n , X { i } n ) –the rene wal–re warded proces ses, 6 F i ( t ) –the distrib ution function of the holding times of the i -th type, 4 M { i } t –the rene wal-re ward process at moment t related t o the rod i -th, 4 M t ( M s t )–the rene wal-re ward process at moment t (with change of a structure at moment s ), 4 N i ( t ) –the number of fishes caugh t on the rod i to the moment t , 3 S { i } n – n -th holding time of the i -th type, 4 T { i } k – k -th jump time of the i -th type, 4 T n – n -th jump moment, 3 X { i } k –the value of the k -th fish cached on the i -th rod, 3 F ( t ) , f ( t ) –the distribution and density functions of the holding times after the change of fishing method, 9 H ( t ) –the distribu tion function of the re wards after the chang e of fishing method, 9 z n –the index of n -th jump, 3 − → N ( t ) –the 2-dimensional rene wal proces s, 3 n { i } k –the index of k -th jump of i -th type, 4 stopping times T , T n , K –sets of stopping times with respect to σ -fields { F t } , 7 T { i } n , K –the stopping times bounded by T { i } n and T K , 7 T { i } n –the stopping times bounde d by T { i } n , 7 25 26 Inde x T n , K –the subset of stopping times τ ∈ T with respect to t he filt ration { F t } such that T n ≤ τ ≤ T K , 7 τ a ∗ –the optimal moment of the fir st decision, 8 τ b ∗ –the optimal moment of the second decision, 8 τ n , K –the element of the set T n , K , 7 τ b 0 , K ∗ , τ b K ∗ –the second optimal stopping time in a restricted prob lem, 9