Fairly Random: The Impact of Winning the Toss on the Probability of Winning

Fairly Random: The Impact of Winning the T oss on the Probability of Winning ∗ Gaurav Sood † Derek Willis ‡ June 4, 2022 Abstr act In a competitive sport, every little thing matters. Y et, many sports leave some large levers out of the reach of the teams, and in the hands of fate. In cricket, world’s second most popular sport by some measures, 1 one such lever—the toss—has been subject to much recent attention. Using a large nov el dataset of 44,224 cricket matches, w e estimate the impact of winning the toss on the probability of winning. The data sugg est that winning the toss increases the chance of winning by a small ( ∼ 2.8%) but signi  cant margin. The advantage varies heftily and systematically , b y how closely matched the competing teams are, and by playing conditions—tautologically , winning the toss in conditions where the toss grants a greater advantag e, for e .g., in day and night matches, has a larger impact on the probability of winning. ∗ The paper bene  tted from comments by Andrew Gelman, Donald Green, and Daniel Stone. Scripts use d to scrape the data, the  nal data set, and scripts used for analysis can be found at: https://github.com/dwillis/toss-up . † Gaurav is an independent researcher . He can b e reached at: gsood07@gmail.com ‡ Derek is a news applications developer at ProPublica. He can b e reached at: dwillis@gmail.com 1 See B. R. ( 2011 ) in The Economist. 1 In nearly all cricket matches, it is claimed that there is a clear advantage to either b owling or batting  rst. The advantage is p ointe d to by commentators, by the te am captains in the pre-toss interview , and by the captain of the losing team in the post-match inter view . And to wrest the said advantage , the captain merely ne eds to pick the side of the coin that will be left facing the sky after the toss. And while this method of granting advantage is fair on averag e, the sy stem isn’t fair in any one game. At  rst glance, this imbalance se ems inevitable. After all, someone has to bat  r st. One can, howe ver , devise a baseball like system, with a series of short innings wov en together . If that violates the nature of the game to o much, one can e asily create pitches that don’t deteriorate appreciably over the course of a match. Or , one can come up with an estimate of the advantage, and adjust the scores accordingly , akin to an adjustment that is issue d when the matches are shortened due to rain. None of this to say that there is actually an advantage in winning the toss, or that teams are able to successfully exploit any such advantage. For it may be impossible to predict well in advance the advantage of bowling or batting  rst. ( If pre-match assessments of the pitch by media commentators are anything to go by , error in assessment of conditions is likely large.) Or , it may be that teams squander the potential advantage by using bad heuristics to choose what they do. For instance, teams may weigh outcomes from recent matches ‘too’ heavily; e.g., a team that has, of late, won chasing may choose to chase even though pitch conditions favor batting  r st. T o assess the net observed advantage of winning the toss, we exploit data from a novel dataset of over 43,000  rst-class men’s cricket matches— to our knowledge , the largest ever dataset assembled for the question, and nearly 50–100 times larger than used in prominent previous attempts (see, Dawson et al. , 2009 ; D e Silva and Swartz , 1998 ). In analy zing these data, we avoid a common but important pitfall that some other studies on the topic fall into. T o avoid post-treatment bias (see Acharya, Blackwell and Sen , 2015 ), unlike Dawson et al. ( 2009 ), Shafqat ( 2015 ), etc., we do not condition on p ost coin-toss decisions. W e  nd that winning the toss grants a small but signi  cant advantage , but that advantage varies considerably and systematically . W e next assess whether the advantage of winning the toss varies, broadly speaking, by how closely matched the teams are, and by how large an advantage winning the toss grants—the advantage of winning the toss is greater in certain playing conditions than others. W e  nd that the advantage of winning the toss varies widely and systematically , in expected ways. Data Data are from 44,224  rst-class cricket matches. It is a near census of the relevant population. 2 W e have data on all types of matches: domestic and international T wenty20s—T20s and T20Is respectively , domestic and international one-dayers—List A and One-D ay Internationals (ODI) respectively , and domestic and international multi-day matches — First Class (FC) and T ests respectively . 3 Of the 44,224 matches, 1,019 matches were abandoned without play . W e exclude these 2 Data excludes scheduled matches that were abandoned without the toss being conducted. 3 There is a rich variety of  rst-class matches. In English county cricket,  rst-class matches last four day s. Some  rst class matches last just a day . Others two day s. Y et others three days. And till a particular p oint in histor y , a test match lasted as long as it was ne eded to  nish a game. W e elide over such di  erences. 2 matches. In another 1,376 matches, we do not have information on whether the team chose to bat or bowl after the toss. Informal inspection suggests that data are missing b ecause no match was played. W e excludes these matches as well. In limited overs cricket, a minimum number of overs must be bowled to establish a result. In a one-day match, for instance, each side must bat at least 20 overs for a result to be de clared. In 769 matches, or roughly 1.7% of the total matches, not enough overs were bowled to get a result. W e ex clude these matches from our analysis. This leaves us with data from 41,060 matches. W e analyze these data. Analyses and Results W e assume that the outcome of a toss is random. Conditional on the outcome of a toss b eing random, the e  ect of winning the toss can be attributed to the toss itself. Any decision made after the outcome of the toss is known, howev er , is ‘post-treatment. ’ In particular , the decision to bat or bowl  rst is made after accounting for the relative strengths and weaknesses vis-à-vis the competing team at that particular instance, and thus not independent of team attributes. Hence, conditioning on decision to bowl or bat  rst can bias estimates of advantage of winning a toss. Thus, unlike Dawson et al. ( 2009 ), Shafqat ( 2015 ), we solely rely on the assumption that the outcome of a toss is random. But before we exploit the design, we shed some light on the validity of the assumption. In particular , we assess whether the coin toss is somehow rigged, with the home side enjoying the rub of the green more often. For this analy sis, we only get to exploit international matches as establishing which of the teams is the home team in local matches is somewhat arduous. Of the 5,684 international matches for which we can match the country of the ground to the country of one of the teams, the home team won the toss in 2,892 matches, or about 50.87%. The chance of getting as many wins by  uke after tossing 5,684 coins is ab out 18.91%. The chance is low , but not eyebr ow raisingly so. Howe ver , rather than consider all home matches, we may instead want to only consider matches that are o  ciate d only by home umpires—the norm till 1992. 4 In matches featuring umpires from only the home countr y , the te am with ‘home umpire ’ advantage won toss nearly 51.9% of the times. And the chance of getting a greater p ercentage of wins than 51.9% in 2,965 is a shade less than 4%. Thus, there is some reason to worry that the tosses are rigged. Any such rigging would bias estimates of advantage of winning the coin toss to the extent that it is correlated with ability . More plainly , if stronger te ams win more tosses, estimates of the advantage of the coin toss would be in  ated upwards. And vice versa, if other wise. W e, howev er , do not have good reasons to think that there is a correlation. So for now , we procee d as if the tosses are random. Another caveat about interpretation before we present the results. As we discuss in the introduction, we cannot estimate the actual advantage of winning a toss. W e can only estimate the net observed advantage, which is the extent to which the teams capitalize on the potential advantage . With that, the results. The team that wins the toss wins the match 2.8% more often than the team that loses the toss. This is a reasonable sized advantage in a comp etitive sp ort — though likely much smaller than the 4 For more information on move to neutral umpires, se e Neutral Umpires by S. Rajesh on ESP Ncricinfo. 3 number that most commentators car ry in their he ads. This advantage, however , varies by format, by conditions, by whether or not a particular formula was used to adjust scores when it rains, and how much b etter the team that won the toss is vis-à-vis the competing team. Much of the variability follows expected patterns. The conventional wisdom among lay cricket followers is that toss grants the greatest advantage in multi-day a  airs like tests and  rst class matches, followed by day long a  airs, and T wenty20s. And there is good dose of common sense b ehind the conventional wisdom. Pitches invariably deteriorate over multiple days and batting last in a test match is often the most challenging time to bat. The pitch deteriorates far less over the course of the day , or in case of T wenty20s, a few hours. And indeed unlimite d over matches provide the greatest advantage— the averag e advantage over FC and test matches is north of 2.6% (see Figure 1 ). Looked in relative terms, the advantage of winning the toss is also close to the greatest in multi-day a  airs. Only about 60% of test matches end in a clear decision, the rest end in a draw . Thus, the advantage is closer to 4.5%. The heftiest raw advantage , howe ver , is in one-day matches (List A and ODIs), approximately 3.3%. In T20s and T20Is, the advantage is considerably smaller , just about 1.27%. 5 And unlike the estimate of advantage for multi-day and one-day a  airs, we cannot statistically reject that the possibility that there is no advantage . T ype of matches are but one source of variation and theorizing ab out the advantage granted by the toss. It is often claimed that the toss is more cr ucial in day and night matches. Due to dew—it is thought to make bowling hard, and the visibility of the white ball is thought to be lower under lights, which makes catching hard—the team that  elds second is thought to b e at a disadvantage . The conventional wisdom is largely vindicated for one day a  airs (se e Figure 2 ). In one-day matches, the advantage of winning the toss in a day and night match is 5.92%, whereas the advantage of winning the toss in a one-dayer played during the day is less than half—2.89%. In T wenty20s—domestic and international—, howev er , we cannot distinguish b etween the advantage granted by the toss in day and night matches and day time a  airs. W eather has a large impact on the playing conditions in cricket. For instance, cool overcast weather is thought to aid swing bowling, especially on certain pitches. More generally , the advantage of winning the toss likely varies by weather . However , we do not have data on weather . But, we can proxy it with seasons. In particular , students of the game susp e ct that the advantage of winning the toss in early English season is esp e cially great. W e next assess whether that is so. There is some evidence of a seasonal p attern, with advantage of winning the toss somewhat greater in spring and early summer (May and June) than in mid and late summer and early fall ( July to September) 3 . However , the thing that catches attention is the large disadvantage of winning the toss in April. W e don’t have a good explanation for the p attern, except for teams choosing badly . Aside from a  e cting the playing conditions, weather a  ects cricket matches in other , more forceful way s—inter rupting, and sometimes ending matches. When a limited over match that is already underway is interrupte d by bad weather , and more than a certain amount of time is lost 5 Splitting data by whether the match was domestic or international yields some additional insights. Like De Silva and Swartz ( 1998 ), who based on analysis of data from 427 international one-day matches conclude that ‘winning the toss at the outset of a match provides no comp etitive advantage ’ in one-day international matches, we  nd that in ODIs teams that win the toss win games at about the same rate as those that lose the toss. For  rst-class and test matches, the advantage to winning the toss is roughly the same. Meanwhile in T wenty20s, the advantage of winning the toss is greater in international than domestic matches. 4 Figure 1: Di  erence in Winning Percentages of T eams that W on the T oss and T eams that Lost the T oss by Type of Match. ● ● ● 1.27% (n = 5,352) 3.3% (n =18,102) 2.64% (n =17,606) T20/T20I LIST A/ODI FC/TEST −2% −1% 0% 1% 2% 3% 4% 5% 6% 7% Note: Means and 95% con  dence intervals. n refers to the number of matches. 5 Figure 2: Di  erence in Winning Percentages of T eams that W on the T oss and T eams that Lost the T oss by Day or D ay and Night ● ● ● ● 2.89% (n =15,654) 5.92% (n = 2,448) 1.47% (n = 4,210) 0.53% (n = 1,142) Day Day/Night Day Day/Night LIST A/ODI T20/T20I −2% 0% 2% 4% 6% 8% 10% Note: Means and 95% con  dence intervals. n refers to the number of matches. 6 Figure 3: Di  erence in Winning Percentages of T eams that W on the T oss and T eams that Lost the T oss by Month in England ● ● ● ● ● ● −4.82% (n = 933) 4.01% (n =3,091) 3.93% (n =2,722) 3.46% (n =2,543) 2.14% (n =2,293) 2.86% (n = 943) Apr May Jun Jul Aug Sep −12% −10% −8% −6% −4% −2% 0% 2% 4% 6% 8% 10% 12% Note: Means and 95% con  dence intervals. n refers to the number of matches. 7 due to the interruption, the match is curtailed and the total that the team batting se cond must achieve to win is adjusted using a method invented by Duckworth and Lewis (see Duckworth and Lewis , 1998 ). 6 W e can use the random nature of who wins the toss to see if winning percentages of the teams that win the toss are strongly conditione d by whether or not Duckworth-Lewis is used. If the advantage of winning the toss in matches using Duckworth-Lewis is di  erent from matches that don’t use it, it suggests that the Duckworth-Lewis method is biased. (For a precise estimate of the bias, ideally , we would want to compare matches using Duckworth-Lewis method with matches held in similar conditions.) In both one-day and T wenty20 matches, the advantage of winning the toss in a match where the target is adjusted using Duckworth-Lewis, is considerably greater (see Figure 4 ). In one-day matches, the advantage is 5.35% in matched adjudicate d by Duckworth-Lewis and 3.17% in matches that don’t use it. Statistically , the chances that the two numbers are the same is less than 10%. (And if you make the plausible assumption that winning a toss only improves the chances of winning, the chance that the two numb ers are the same is half that.) In T wenty 20s, the advantage of winning the toss shrinks from 3.9% in matches using Duckworth-Lewis to 1.17% in matches without it. Once again, the chance that the two number s are the same is about 10%. Winning the toss ought to matter the most when the di  erence b etween the quality of the teams that are playing is the least. Similarly , it is unlikely that winning the toss would change who wins the game when two ill-matched te ams are playing. T o study the issue, we collected data on team quality . The ICC publishes team ratings for international test and one-day teams each month. 7 Ratings of the men’s ODI teams have been published since 1981, and of the test teams, since 1952. Of the entire ranking dataset, that spans 1981–today and 1952–today for ODI and test teams respectively , we only have we have data till 2013. 8 T e am ratings range from 0 to 143 in our data. For instance, Bangladesh had a rating of 0 in tests for most of 2002 and 2003. And A ustralia in 2007 twice held a rating of 143. W e measure how closely matche d the teams by di  erencing the ranking p oints of one team from the other . Commercial considerations me an that a majority of the games are played among highly ranked and closely matched teams. Thus, the precision of our estimates is greatest for matches b etween closely matched teams. The results are expecte d, but new . As Figure 5 — which plots percentage of matches won or drawn by the team that won the toss— illustrates, there is a sharp cur ve around 0. When closely matched teams win, winning the toss has a large impact on the probability of winning. Lastly , we investigate how the advantage varies by countr y winning the toss in international matches. Are some countries better than others at capitalizing on a toss win? W e investigated the question by tallying the advantag e by team that wins the toss. As 6 illustrates, all of our estimates are imprecise enough that we cannot say with con  dence that any of the teams capitalizes on winning the toss. Neither can we discount the p ossibility that the actual di  erences across the teams are zero . The most puzzling result is from New Zealand. Like Shafqat ( 2015 ), data suggest 6 Before the Duckworth and Lewis metho d was adopted, weather a  ected matches sometimes continue d on the next day; an extra day was delib erately left in the sche dule for dealing with such eventualities. In cases where the spare day proved inadequate, the match was de clared a draw . 7 For details ab out how the ICC produces these ratings, see ICC Rating F AQs . 8 The format in which the ICC publishes the ratings changed in 2013. And scraping the latter data p osed additional hurdles. W e decided that the additional e  ort wasn’t worth the small amount of additional data. 8 Figure 4: Di  erence in Winning Percentages of T eams that W on the T oss and T eams that Lost the T oss by Whether or not Duckworth-Lewis was invok ed ● ● ● ● 3.17% (n =16,980) 5.35% (n = 1,122) 1.17% (n = 5,147) 3.9% (n = 205) No D/L D/L No D/L D/L LIST A/ODI T20/T20I −2% −1% 0% 1% 2% 3% 4% 5% 6% 7% Note: Means and 95% con  dence intervals. n refers to the number of matches. 9 Figure 5: Percentag e of Matches W on Minus Matches Lost After Winning the T oss by Di  erence in Ranks ODI TEST 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% −120 −90 −60 −30 0 30 60 90 120 −120 −90 −60 −30 0 30 60 90 120 Difference in Ranking P oints P ercentage Won/Dr awn Note: Smoothed relationship between di  erence in ranks and winning probability by whether or not the team won the toss. 10 that New Zealand does more poorly in matches where it wins tosses than where it loses it. On the other hand, Sri Lanka and India appe ar to do especially well, winning 3.85% and 3.09% additional matches, respectively , when they win the toss. A ustralia, Pakistan, and W est Indies hover around 1–1.5%, and England is at 2.29%. Figure 6: Di  erence in Winning Percentages of T eams that W on the T oss and T eams that Lost the T oss by Countr y ● ● ● ● ● ● ● 1.33% (n =1,702) 2.29% (n =1,733) 3.09% (n =1,433) −2.38% (n =1,189) 1.01% (n =1,350) 3.85% (n =1,072) 1.5% (n =1,302) Austr alia England India New Zealand P akistan Sri Lanka West Indies −10% −8% −6% −4% −2% 0% 2% 4% 6% 8% 10% Note: Means and 95% con  dence intervals. n refers to the number of matches. Till now , we have focused on assessing the impact of winning the toss on the probability of winning, and how the imp act is conditioned by playing conditions, by the typ e of match, and by the teams involved. Winning the toss, however , likely not only a  ects the probability of winning, but also the margin of victory . But b efore we assess the impact of winning the toss on the margin of victory , a short primer . In limited over matches, when the te am chasing the total falls short, the margin of victory is given in di  erence in runs. When the team is able to successfully chase the total, the margin of 11 victory is the given by two numb er s: number of balls remaining, and the number of wickets in hand. In unlimited overs matches, the metrics for margin of victor y di  er in two small ways. W e don’t tally the number of balls remaining when the winning team achieves the target (principally we could). Inste ad, we note whether or not the winning te am had to bat twice—whether or not the team won by an ‘innings’ and additional r uns. T e ams that win the toss and the match in  rst-class and test matches win with more wickets in hand (Mean = 6 . 91 ) than winning teams that lose the toss (Mean = 6 . 64 ). In one-dayers and T wenty20s, the teams that win the toss and the match have about the same numb er of wickets in hand as the teams that lose the toss but win the match (One-Day: Mean Lose T oss = 5 . 57 , Mean Win T oss = 5 . 65 ; T wenty20: Mean Lose T oss = 6 . 37 ; Mean Win T oss = 6 . 21 ). In  rst-class and test matches teams that win the toss and the match also win by few more runs on average (Me an = 136 . 48 ; Median = 124 ) than teams that lose the toss but win the match (Mean = 133 . 71 ; Median = 122 ). Similarly , in T wenty20s, the team that wins the toss wins by a few more runs (Mean = 37 . 60 ; Median = 28 ) than team that loses it (Me an = 34 . 50 ; Median = 26 ). In one-dayers, howe ver , the margin of victor y is largely indistinguishable across cases where the winning te am wins the toss and where it loses it (Me an Lose T oss = 63 . 95 , Median Lose T oss = 51 ; Mean Win T oss = 63 . 15 , Median Win T oss = 51 ). Similar patterns hold for balls remaining—teams that win the toss generally win with a few more balls remaining than teams that lose the toss. In one-day matches, Me an Lose T oss = 49 . 87 , Median Lose T oss = 28 , Mean Win T oss = 53 . 27 , and Median Win T oss = 30 . And in T wenty20s, Mean Lose T oss = 18 . 69 , Median Lose T oss = 12 , Mean Win T oss = 16 . 41 , and Median Win T oss = 10 . And lastly , on number of innings, the teams that win the toss win by an innings about as often as teams that lose the toss (Mean Lose T oss = 22 . 26% , Mean Win T oss = 22 . 43% ). Discussion The data suggest that winning the toss has a sizable impact on the probability of winning, especially in closely contested games. The data also suggest that the advantage varies considerably and systematically —in expected ways—, with advantage greater in day and night matches, matches in which Duckworth-Lewis is used to adjust scores, and where the match is played between closely matched teams. In showing so, the data lend credence to, and quantify , the suspicion that many of the cricket fans have long had—that tosses matter . Besides that, the data also help quantify the bias in Duckworth-Lewis method. More generally , the analysis we do here could be replicated elsewhere to assess bias in competing methods, and use d to prov e that a particular method is better or worse than the Duckworth-Lewis. 12 References Acharya, A vidit, Matthew Blackwell and Maya Sen. 2015. “Explaining Causal Findings Without Bias: Detecting and Assessing Direct E  ects. ” . B. R., (The Economist). 2011. “ And the silver goes to... ” http://www.economist.com/blogs/ gametheory/2011/09/ranking- sports%E2%80%99- popularity . Dawson, Peter , Bruce Morley , David Paton and Dennis Thomas. 2009. “T o bat or not to bat: An examination of match outcomes in day-night limited overs cricket. ” Journal of the Op erational Research So ciety 60(12):1786–1793. De Silva, Basil M and Tim B Swartz. 1998. “Winning the coin toss and the home team advantage in one-day international cricket matches. ” . Duckworth, Frank C and Anthony J Lewis. 1998. “ A fair method for resetting the target in interrupted one-day cricket matches. ” Journal of the Operational Research Society 49(3):220–227. Shafqat, Saad. 2015. “ Analysis: How many ODIs are decide d by the toss of a coin?” http://www.telegraph.co.uk/sport/cricket/9163738/ Analysis- How- many- ODIs- are- decided- by- the- toss- of- a- coin.html . 13