A Sharper discrepancy measure for post-election audits

The Annals of Applie d Statistics 2008, V ol. 2, No. 3, 982–985 DOI: 10.1214 /08-A OAS171 c  Institute of Mathematical Statistics , 2 008 A SHARPER DISCRE P ANCY MEASURE F OR POST-ELECTION AUDITS By Philip B. St ark University o f Califor nia—Berkeley P ost-election audits use the discrepancy b etw een machine counts and a hand tally of votes i n a random sample of precincts to infer whether error affected the electoral outcome. The maximum relative o vers tatement of pairwise margins (MRO) q uantifies that discrep- ancy . The electoral outcome a full hand tally sho ws must agree with the apparent o utcome if the MRO i s less than 1. This condition is sharp er than previous ones when there are more than t w o candidates or when voters may vote for more than one cand idate. F or the 2006 U.S. Sen ate race in Minnesota, a t est using MRO giv es a P -v alue of 4.05% for t he h yp othesis th at a full hand tally wo uld find a different winner, less t han half the v alue Stark [ Ann. Appl. Statist. 2 (2008) 550–581 ] fin ds. 1. Maxim um relativ e o ve rstatemen t of pairwise margins. F or a candi- date other than a n apparent winn er to b e a real winn er of an election, error that h urts that candidate or helps an apparen t winn er m u st exceed that apparent winn er’s margin of victory o v er that candidate. The max- im u m relativ e o verstate m en t o f pairwise margins (M R O) tak es that in to accoun t; p revious measures compare errors with the margin of victory o ve r the ru nner-up alone. Consider a con test with K candidates, 1 , . . . , K , and N precincts, 1 , . . . , N . Eac h v oter ma y v ote f or up to f c andidates. Th e f candidates who app ar- en tly w on are those in K w . T hose wh o apparentl y lost are in K ℓ . The appar- en t vote for candidate k in precinct p is v k p . Th e app aren t vo te for candid ate k is V k ≡ P N p =1 v k p . The apparent m argin of candid ate w o ve r candidate ℓ is V w ℓ ≡ V w − V ℓ . F or w ∈ K w and ℓ ∈ K ℓ , V w ℓ > 0: the apparent win ners are the f cand idates with strictly p ositiv e apparent margins o ve r the other K − f . A ctual , as a mo difier of vote , margin, winner or electoral outcome, means what a f u ll hand tally w ould show. The actual v ote for candidate k in Received March 2008; rev ised April 2008. Key wor ds and phr ases. Elections, auditing, error b ounds in au d iting. This is an elec tr onic reprint o f the original article published by the Institute of Ma thema tical Statistics in The Annals of Applie d St atistics , 2008, V ol. 2, No. 3, 9 82–98 5 . T his reprint differ s from the or ig inal in paginatio n and typogra phic detail. 1 2 P . B. ST ARK precinct p is a k p . The actual vote for candid ate k is A k ≡ P N p =1 a k p . The actual margin of cand idate w ov er candid ate ℓ is A w ℓ ≡ A w − A ℓ . The ap- paren t winn ers are the actual winn ers if min w ∈K w ,ℓ ∈K ℓ A w ℓ > 0 . (1) Define e pw ℓ ≡ ( v w p − v ℓp ) − ( a w p − a ℓp ) V w ℓ . (2) F or the apparent and actual electoral outcomes to differ, th er e must exist w ∈ K w and ℓ ∈ K ℓ for which P N p =1 e pw ℓ ≥ 1. Th e maximum r elative over- statement of p airwise mar gins ( M R O ) in pr e cinct p is e p ≡ max w ∈K w ,ℓ ∈K ℓ e pw ℓ . (3) No w max w ∈K w ,ℓ ∈K ℓ N X p =1 e pw ℓ ≤ N X p =1 max w ∈K w ,ℓ ∈K ℓ e pw ℓ = N X p =1 e p . (4) The sum on the right is the maximum r elative o verstatement of p airwise mar gins ( MRO ). If the apparent and actual electoral outcomes differ, P N p =1 e p ≥ 1. When K = 2 an d f = 1, this is equiv alen t to the condition S tark ( 2008 ) tests. But for K > 2 or f > 1, this condition can b e muc h sharp er. Supp ose the n umber of v alid ballots cast in precinct p is at most b p . Clearly , a w p ≥ 0 and a ℓp ≤ b p . Hence, e pw ℓ ≤ ( v w p − v ℓp + b p ) /V w ℓ , and so e p ≤ max w ∈K w ,ℓ ∈K ℓ v w p − v ℓp + b p V w ℓ ≡ u p . (5) Let { w p ( · ) } N p =1 b e monotonic functions. Stark’s ( 2008 ) metho d can test the h y p othesis that P N p =1 e p ≥ 1 giv en the constrain t e p ≤ u p using the m axim um observ ed v alue o f w p ( e p ) as the test statistic: substitute M = 1 and the definitions of u and e p giv en h er e. 2. The 2006 U.S. Sen ate race in Minn esota. T able 1 lists the v ote totals for the 2006 U.S. S enate race in Minnesota. Th e app aren t winner was Amy Klobuc h ar. 1 Minnesota elections la w S.F. 2743 (2006) requires that countie s with few er than 50,000 r egistered v oters audit at least tw o p recincts c hosen at rand om; that coun ties w ith b etw een 50,000 and 100,000 registered v oters 1 See www.sos.st ate.mn.us /docs/200 6_General_Results.XLS , electionresults.sos .state.mn.us/200 6 1107/Ele cRslts.asp?M=S&Races=0102 , and www.sos.st ate.mn.us /home/index.asp?page=544 . DISCREP ANCY MEAS URE FOR POST-ELECTION AUDITS 3 T able 1 Summary of 2006 U.S. Senate r ac e in Minnesota V oters Undervotes Fitzgerald Ke nnedy Klobuchar Ca vlan Po wers W ri te-ins & inv alid (Indep) (Rep ub) (Democ/F arm/ (Green) (Constit) ballots Lab or) 2,217,8 18 15,099 71,194 835,65 3 1,278,8 49 10,714 5,408 901 V wℓ N/A 1,207,655 443,196 N/A 1,268,1 35 1,273,4 41 1,277,948 audit at least thr ee; and that countie s w ith more than 100,000 registered v oters audit at least four. A t least one precinct audited in eac h coun ty m u st ha v e 150 or more vo tes. Minnesota has 4,123 pr ecincts in 87 coun ties. 202 precincts were au d ited after the 2006 election. Sev eral counti es audited more than the legal minimum. F ollo wing Stark ( 2008 ), w e p o ol Ca vlan, P ow ers, and the write-in can- didates to form a p seudo-candidate wh o app aren tly lost to Klobuchar b y 1,261, 773 v otes. Thus, K = 4, f = 1 and N = 4,123. The maxim um v alue of u p is 0.0097. The maximum observ ed v alue of e p is 4 . 5 × 10 − 6 . If Klobuc h ar actually lost, th e MR O in at least 166 precincts must b e larger than an y in the sample. In con trast, for the measure of margin o v erstatemen t Stark ( 2008 ) uses, only ab out 130 pr ecincts would need to hav e v alues exceeding an y in the sample. 2 Th us, it is easier to confi rm that the apparent and actual out- comes agree using the MR O. W e calculate a conserv ativ e P -v alue for the h yp othesis that K lobuc h ar ac- tually lost by pr etending that the sample w as dra wn w ith replacemen t fr om all 4,123 p recincts, but th at only 78 precincts w ere dra w n, as if the p opula- tion w ere sampled u s ing the m inim um sampling fraction among counties. 3 F or we ight functions w p ( x ) = x , the P -v alue is the maximum c h ance that 78 p recincts chose n at r andom with replacemen t would hav e e p ≤ 0 . 0097 if , among all 4,123 precincts, there w er e at least 166 with e p > 0 . 0097. Th at v alue is ( 4123 − 166 4123 ) 78 = 4 . 05%, roughly half the conserv ativ e P -v alue of 8.22% Stark ( 2008 ) find s . If 202 precincts were dra wn as a simp le random samp le and the same discrepancies w ere observ ed, the P -v alue w ould b e ab out 0.02% u sing th e MR O. I n con trast, Stark ( 2008 ) fin ds a corresp onding P -v alue of about 0.13%. 3. Conclusion. The MR O yields a sh arp er necessary condition for the apparen t elect oral outcome to differ from th e outcome a full hand ta lly 2 See T able 5 of Stark ( 2008 ). 3 See Section 4.2.1 of Stark ( 2008 ). 4 P . B. ST ARK w ould sho w than previous measures of the d iscrepancy b et w een mac hine and hand coun ts do. An a priori b ound for the MR O in a precinct can b e deriv ed from a b ound on the num b er of v alid ballots in that precinct. The testing framewo rk Stark ( 2008 ) d ev elops works with MRO if the definitions of M , u and e are r evised, and yields a more p o w erful test. Ac kn o wledgment s. I thank Kathy Dopp, Mark Lin deman and Luk e Mi- ratrix for helpful con versati ons. REFERENCE St ark, P. B. (2008). Conserv ative statistical p ost-election aud its. Ann. Appl. Statist. 2 550–581 . Dep ar tment of St a tistics, Code 386 0 University of California—Berkeley Berkeley, California 94 720-3860 USA E-mail: stark@stat.berkeley .edu