How Not to Win a Million Dollars: A Counterexample to a Conjecture of L. Breiman

Ho w Not to Win a Million Dollars: A Coun terexample to a Conjecture of L. Breiman Thomas P . Ha yes ∗ Abstract Consider a gam bling game in whic h we are allo w ed to repeatedly bet a p ortion of our bankroll at fa v orable odds. W e in vestigate the question of how to minimize the exp ected num b er of rounds needed to increase our bankroll to a giv en target amount. Sp ecifically , we dispro ve a 50-y ear old conjecture of L. Breiman [1], that there exists a thr eshold str ate gy that optimizes the exp ected n um b er of rounds; that is, a strategy that alwa ys b ets to try to win in one round whenever the bankroll is at least a certain threshold, and that mak es Kelly b ets (a simple prop ortional betting sc heme) whenev er the bankroll is b elow the threshold. Keyw ords: Optimal b etting, Kelly b etting, Algorithms, Coun terexample, Optional stopping, Computer assistance. 1 The Conjecture Consider a fav orable gambling game, such as b etting at 3:1 o dds on the outcome of a fair coin toss. If we are allo w ed to pla y this as man y times as w e like (decided adaptively), we can even tually increase our winnings to an y desired target amount, with certaint y . F or instance, prop ortional b etting strategies such as the Kelly criterion (see Kelly [2] or Breiman [1]), hav e long b een kno wn to accomplish this. Breiman [1] “hop efully conjectured” the following. Supp ose our goal is to ac hieve a set target bankroll, sa y $1M, starting with a fraction ξ of that amoun t. Let T ( ξ ) b e the exp ected num b er of rounds we hav e to play b efore we attain our goal. Then there exists a threshold 0 < ξ 0 < 1, and an optimal strategy of the following form: • When the current bankroll is less than ξ 0 , b et to optimize E log(bankroll). This is ac hieved b y b etting a particular fraction of the current bankroll, whic h is only dep endent on the prop osition b eing offered. This is sometimes known as “Kelly b etting.” • When the curren t bankroll is at least ξ 0 , b et to reac h the target bankroll in the curren t round. W e will call such strategies “threshold strategies.” Breiman describ es this conjecture as “expressing a mo derate faith in the simplicit y of things.” Indeed, his prop osed strategy seems quite plausible. How ev er, our main result is a pro of that this strategy is not optimal. Theorem 1.1 Ther e exists a favor able gambling game and initial b ankr ol l ξ , for which the optimal T ( ξ ) is at most 13 / 14 times that of any thr eshold str ate gy. ∗ Departmen t of Computer Science, Univ ersity of New Mexico, Albuquerque, NM 87108, U.S.A. Email: ha yes@cs.unm.edu. 1 W e hav e not attempted here to optimize the constant 13 / 14, but it would be nice to know what its b est p ossible v alue is. 2 Preliminaries W e will use the following version of Do ob’s Optional Stopping Theorem (see [3, Theorem 10.10]). Theorem 2.1 (Do ob’s Optional Stopping Theorem) L et ( X t ) t ≥ 0 b e a sup ermartingale. L et T b e a stopping time with E T < ∞ , and supp ose ther e is a c onstant C such that, for al l t ≥ 1 , | X t +1 − X t | ≤ C . Then X T is inte gr able and E X T ≤ E X 0 . 3 Coun terexample F or our gam bling game, consider a biased coin whic h comes up heads with probabilit y 2 / 3. Supp ose w e are allo w ed to b et on heads, and be paid off at 2 : 1 o dds. That is, for each unit bet, the net c hange in our bankroll is − 1 with probability 1 / 3, and +2 with probabilit y 2 / 3. In this case, it is easily c heck ed that the Kelly criterion says to b et 1 / 2 of the current bankroll at each timestep, so that the new bankroll will b e: half the curren t bankroll with probability 1 / 3, and t wice the curren t bankroll with probability 2 / 3. F or an initial bankroll, x , let T ( x ) be the exp ected num b er of pla ys of this game un til a bankroll of at least 1 is achiev ed, under the strategy minimizing this quan tity Lemma 3.1 F or our example game, when x = 1 / 2 k , for k a p ositive inte ger, the (unique) optimal str ate gy is to b et 1 / 2 of the curr ent b ankr ol l every r ound until a b ankr ol l of 1 is r e ache d. Ther efor e, T (1 / 2 k ) = 3 k . Pro of: First w e observ e that the giv en strategy results in an exp ectation of 3 k rounds un til the bankroll reaches 1. This is b ecause the base-2 logarithm of the bankroll is a biased random walk on the negativ e integers, that mo ves one unit tow ards 0 with probabilit y 2 / 3 at each timestep, and one unit aw ay with probabilit y 1 / 3. The analysis of the hitting time to 0 for this random walk is standard. T o see that this is the b est p ossible result, consider any p ossible strategy , and le t x ( t ) denote the bankroll after t steps. Let τ denote the hitting time to the target bankroll: τ := min { t : x ( t ) ≥ 1 } . Define Y t := 3 log 2 ( x ( t )) − t . Observ e that, regardless of the strategy chosen, ( Y t ) is a sup ermartingale, since Kelly b etting maximizes the conditional exp ectation of Y t +1 giv en Y t , and, for Kelly b etting, Y t w ould b e a martingale. F urther, note that Y 0 = − 3 k , and Y τ = − τ . No w, we ma y assume E τ < ∞ , since otherwise this strategy is clearly w orse than Kelly b etting. Supp ose for now that | Y t +1 − Y t | is b ounded almost surely . Then applying the Do ob’s Optional Stopping Theorem 2.1, we hav e E τ = − E Y τ ≥ − Y 0 = 3 k , whic h prov es that, again, the Kelly strategy is sup erior. Finally , w e will show that an y strategy with | Y t +1 − Y t | un b ounded can b e strictly improv ed up on b y another strategy with | Y t +1 − Y t | ≤ 25, and therefore our previous assumption that the steps tak en by Y are b ounded was made without loss of generalit y . First note that Y t +1 ≤ Y t + log 2 (3) − 1 absolutely , since ev en if we b et the en tire bankroll, we cannot more than triple our stake. Suppose, for some v alue of x ( t ), this strategy bets more than 1 − 3 − 15 of the bankroll. Let’s lo ok at the exp ected n umber of rounds needed un til x ( t 0 ) ≥ 2 x ( t ). With probability 2 / 3, t 0 = t + 1, but with probability 1 / 3, the bankroll initially drops by a factor of 3 15 , and hence, no matter what, it will take more than 15 rounds to return to its initial v alue 2 of x ( t ). Thus, in exp ectation, it takes more than 6 rounds to exceed x ( t ) for the first time. And, moreo ver, the v alue of x ( t 0 ) is in the interv al ( x ( t ) , 3 x ( t )). No w, note that Kelly b etting quadruples the stake in an exp ected 6 rounds, whic h clearly dominates the ab o ve strategy . So, it would b e strictly sup erior to “b et to double” until the stak e reac hes 4 x ( t ), and then pro ceed optimally from that point onw ard. Th us, an y strategy that ev er b ets more than 1 − 3 − 15 of its bankroll can b e improv ed up on by one that do es not, which gives us b ounds of − 15 log 2 (3) − 1 ≤ Y t +1 − Y t ≤ log 2 (3) − 1 on the revised strategy , whic h completes the pro of. Lemma 3.1 is interesting b ecause it shows that Kelly b etting is in fact optimal for infinitely man y starting bankrolls. F urthermore, when coupled with our next tw o (easy) lemmas, this actually implies that there exist threshold strategies for which T ( ξ ) is within a constan t factor of optimal. These results are not sp ecific to the example game chosen for this pap er; precise statemen ts and pro ofs are left as an exercise to the reader. On the flip side, as we will see, Lemma 3.1 is also the key to our proof that every threshold strategy is actually sub optimal. Lemma 3.2 In gener al, T ( x ) is a de cr e asing function of x . F urthermor e, for our example game, T ( x ) > 3 / 2 for al l x < 1 . Pro of: Clearly T ( x ) is non-increasing, since extra money can alwa ys b e ignored with no p enalty . W e omit a detailed pro of that T is strictly decreasing, noting only if there is extra money b eing ignored, then after a s ufficien tly long sequence of consecutiv e losses under, say , a prop ortional b etting scheme, this extra money will b ecome the v ast ma jorit y of the bankroll, at whic h p oin t w e can app eal to Lemma 3.1 to see that the exp ected hitting time to 1 is strictly b etter than without the extra money . T o see that T ( x ) > 3 / 2, note that we cannot ac hieve a bankroll of 1 without winning at least one coin toss. But the exp ected num b er of coin tosses until the first heads is 1 / (2 / 3) = 3 / 2. So this is clearly a low er b ound on the hitting time to 1. The inequalit y is strict, since if we keep losing, ev entually our b ets must b ecome to o small to guarantee reaching 1 on the first heads. Lemma 3.3 F or our example game, any optimal str ate gy always b ets to 1 when the b ankr ol l is x ≥ 1 / 2 . A dditional ly, T (2 / 3) = 2 and T (7 / 9) = 5 / 3 . Pro of: Supp ose for a bankroll x ∈ [1 / 2 , 1), and our strategy b ets to some v alue y < 1. Then, since the first coin flip either results in a bankroll of y , or a bankroll < x , we hav e T ( x ) = 1 + 2 3 T ( y ) + 1 3 T ( x − ( y − x ) / 2) > 1 + 2 3 T ( y ) + 1 3 T ( x ) > 2 + 1 3 T ( x ) By Lemma 3.2 This implies T ( x ) > 3. But, by Lemma 3.1, we kno w that T (1 / 2) = 3, so we ha ve a contradiction to the fact that T is a decreasing function (Lemma 3.2). Thus the correct strategy must b e to b et to 1. Since, from x = 2 / 3 or x = 7 / 9, the strategy of b etting to 1 either results in winning directly , or reaching a bankroll of 1 / 2, and since Lemma 3.1 tells us that T (1 / 2) = 3, an easy calculation yields the v alues of T (2 / 3) and T (7 / 9), for which we shall ha ve a use later. 3 Lemma 3.4 F or our example game, if Br eiman ’s c onje ctur e wer e true, then the critic al thr eshold ξ 0 would ne c essarily b e in (1 / 3 , 1 / 2] . Pro of: Since winning b ets are paid off at 2 : 1 o dds, it is imp ossible to “b et to 1” with a bankroll of less than 1 / 3. Moreov er, unless the bankroll is strictly greater than 1 / 3, we cannot b et to 1 without risking the entire bankroll, in whic h case E τ = + ∞ . So ξ 0 > 1 / 3. On the other hand, if x ( t ) > 1 / 2, then b etting the Kelly criterion (risking half the bankroll) is too m uch, as winning results in a bankroll exceeding 1, whic h has no added utility . So ξ 0 ≤ 1 / 2. (Lemma 3.3 also implies ξ 0 ≤ 1 / 2.) Lemma 3.5 F or any ξ 0 ∈ (1 / 3 , 1 / 2] , playing Br eiman ’s str ate gy r esults in T (7 / 18) = 14 / 3 . Pro of: W e consider t wo cases. Case A: ξ 0 ≤ 7 / 18. In this case, we first b et to 1, winning the game with probability 2 / 3, and otherwise losing 11 / 36, for a new bankroll of 1 / 12. Assuming we lost, we will now risk half the bankroll at eac h ste p, un til our bankroll has doubled up to a v alue of 2 / 3. (Note that, b y Lemma 3.4, ξ 0 > 1 / 3.) Note that, by Lemma 3.1, doubling up thrice takes 9 rounds in exp ectation. Since Lemma 3.3 tells us T (2 / 3) = 2, we can no w compute T (7 / 18) = 1 + 1 3 (9 + T (2 / 3)) = 14 3 . Case B: ξ 0 > 7 / 18. In this case, we first bet to double up to 7 / 9, which, by Lemma 3.1 tak es 3 rounds in exp ectation. Since Lemma 3.3 tells us T (7 / 9) = 5 / 3, we therefore hav e T (7 / 18) = 3 + T (7 / 9) = 14 3 , just as in Case A, completing the pro of. Com bining Lemmas 3.4 and 3.5 shows us that, if Breiman’s conjecture w ere true, then T (7 / 18) = 14 / 3 for our example game. Our next result con tradicts this. Lemma 3.6 F or our example game, T (7 / 18) ≤ 13 / 3 . Pro of: On the first bet, s uppose we b et 5 / 36. If w e win, our bankroll go es up to 2 / 3. If w e lose, our bankroll go es down to 1 / 4. Since Lemmas 3.1 and 3.3 tell us that with optimal pla y , T (1 / 4) = 6 and T (2 / 3) = 2, we therefore ha ve T (7 / 18) ≤ 1 + 2 3 T (2 / 3) + 1 3 T (1 / 4) = 1 + 2 3 2 + 1 3 6 = 13 3 . Com bining the ab ov e Lemmas sho ws us that Breiman’s conjecture is false, and with a tin y bit more w ork, that an y strategy meeting Breiman’s form has an expected cost at least 14 / 13 of optimal starting from the initial bankroll of 7 / 18, th us proving Theorem 1.1 It w ould b e nice to know the “price” for playing a threshold strategy . W e hav e seen that this is at least 14 / 13 for our example game. Although w e ha ve made no serious attempt to improv e this v alue, it seems likely that it is not far off the mark. Figure 1 shows a graph of upp er and low er bounds on the optimal T ( x ). It was obtained b y a computer program, whic h, b eginning with the upp er and low er b ounds that follow immediately from Lemma 3.1 and 3.2, recursiv ely deriv es better and better upp er and lo wer b ounds in terms of the previous ones. All of the bounds the program works with are step functions, which allow 4 Figure 1: Computer-generated upp er and low er b ounds on the optimal exp ected num b er of rounds needed to reach a bankroll of 1 from an initial bankroll of x , in our example game. for exact calculations, up to the precision of the mac hine arithmetic. F or efficiency reasons, the program frequently approximates these step functions with more conserv ative ones, so as to a void an exp onen tial growth in the n umber of distinct function v alues it has to k eep track of. Note that the upp er b ound is v ery close to all the v alues we know exactly from our lemmas, suggesting that the low er b ound remains to o conserv ative. Also note that the “bump” visible in the graph at x ≈ 0 . 41 is not an artifact. F or comparison, 7 / 18 = 0 . 3888. W e ha ve no go o d explanation for this feature of the graph, but it seems to suggest that optimal b etting has some interesting structure whic h remains to b e understo o d. Ac kno wledgmen ts I would lik e to thank Y uv al P eres and Ev angelos Georgiadis for in tro ducing me to this problem and for helpful comments. 5 References [1] L. Breiman. Optimal Gambling Systems for F av orable Games. Source: Pro c. F ourth Berkeley Symp. on Math. Statist. and Prob., V ol. 1 (Univ. of Calif. Press, 1961), 65–78. [2] J. B. Kelly . A new in terpretation of information rate. Bell System T ec hnical J., 35 (1956) 917–926. [3] D. Williams. Pr ob ability with Martingales. Cambridge Universit y Press, 1991. 6