Auditing Elections
Contents
Auditing Elections¶
Risks to Elections¶
https://www.nytimes.com/interactive/2018/07/16/us/elections/russian-interference-statements-comments.html
https://www.usatoday.com/story/news/2018/08/03/russian-us-election-interference-donald-trump/878910002/
https://www.nbcnews.com/politics/elections/u-s-intel-russia-compromised-seven-states-prior-2016-election-n850296
https://www.newyorker.com/news/news-desk/america-continues-to-ignore-the-risks-of-election-hacking
https://www.nytimes.com/2018/09/26/magazine/election-security-crisis-midterms.html
https://freedom-to-tinker.com/2018/10/16/design-flaw-in-dominion-imagecast-evolution-voting-machine/
https://freedom-to-tinker.com/2018/10/22/an-unverifiability-principle-for-voting-machines/
https://www.apnews.com/f6876669cb6b4e4c9850844f8e015b4c
http://www8.nationalacademies.org/onpinews/newsitem.aspx?RecordID=25120
http://time.com/5340060/donald-trump-vladimir-putin-summit-russia-meddling/
Potential goals: FUD (fear, uncertainty, doubt)¶
Alter election results through social manipulation, propaganda, etc. “social hacking”
“fake news” propagated through social media, etc.
disinformation regarding how to vote, polling-place location, etc. http://fortune.com/2016/11/03/text-vote-hillary-clinton/
Alter election results through election equipment, tabulation, reporting. “computer hacking”
Alter election results by removing voters from voter registration lists. “computer hacking”
Cast doubt on election results and on the electoral process: clumsy hacks are effective
Election technology, including vote tabulation, is vulnerable. What can we do?¶
If there’s a trustworthy, voter-verified paper trail, that can be used to check the results.
For instance, could hand count the votes on all the ballots.
Can we use Statistics to do less work than a full hand count?
Audit goals¶
The basic goal of auditing is to check the electronic results, using the paper trail, which is assumed to be trustworthy (because it passed a compliance audit).
Precisely what do we want to check? What is the right standard for accuracy?
How can Statistics help?
State laws https://www.verifiedvoting.org/verifier/
some states have no voter-verifiable paper trail
of those that do, rules governing protecting the paper generally are inadequate
many states with paper many do not audit results
of those that audit results,
only Colorado currently audits in a rigorous way–but it only audits a few contests
Rhode Island will soon conduct good audits
New Mexico’s audits have some good features
California’s audits got worse in 2017; has had good pilot programs
The Detection Paradigm¶
Before 2006, audits generally looked at some fixed percentage of votes or precincts selected “at random” and compared electronic results to manual tabulation
Some even worse, e.g., select 500 ballots and tabulate them by machine, retabulate by hand
Academic work focused on detection:
Suppose the reported outcome is wrong. How many precincts/ballots need to be checked by hand to have a high probability of finding at least one error?
Not the best question to ask: audits of hand-marked ballots likely to find at least one misinterpreted ballot. So what?
The evidence paradigm¶
An audit should give strong evidence that the outcome is right (if it is right), or should correct the outcome if it is wrong.
To correct an outcome requires a full manual tally: can’t overturn the result using statistical evidence alone.
A risk-limiting audit (RLA) is any post-election procedure that offers the following statistical guarantee:
If a full manual tally of the complete voter-verifiable paper trail would show a different electoral outcome, there is a known, pre-determined minimum chance that the procedure will lead to a full manual tally.
If the procedure does lead to a full manual tally, the result of that tally replaces the reported result, thereby correcting it.
The maximum chance that the procedure will not lead to a full manual tally if that tally would show a different outcome is called the risk limit.
The risk limit is the largest chance that the audit will fail to correct an outcome that is incorrect, where incorrect means that a full manual tally of the voter-verifiable paper trail would find different winner(s).
A RLA with a risk limit of 5% has at least a 95% chance of requiring a full manual tally, if that tally would show a different outcome.
For an overview of risk-limiting audits, see https://www.stat.berkeley.edu/~stark/Preprints/lhc18.pdf
For a more technical discussion and example methods, see Lindeman and Stark (2012).
Aside: Compliance Audits¶
An audit that relies on an untrustworthy paper trail, or any audit that purports to ascertain voter intent from an electronic record or from an artifact that the voter did not have the opportunity to check, is “security theater.”
There is little reason to believe that a full manual tally of such records would reveal the true winner(s).
It’s crucial to:
base audits on voter-verifiable paper records
ensure that those records include every validly cast vote exactly once, and no others (checking the determination of eligibility, in particular)
ensure that those records remain complete and intact from the moment they are cast through the audit
assess the evidence that they are trustworthy.
Without evidence that the paper trail is a trustworthy record of voter intent–that it accurately reflects the intent of every voter who legitimately cast a ballot in the contests under audit, and no others–the audit might simply confirm the incorrect outcome.
The process of assessing the trustworthiness of the paper trail is called a compliance audit.
Compliance audits should include the following steps, among others:
Ballot accounting.
Check that the number of ballots sent to polling places equals the number returned voted plus the number returned spoiled, plus the number returned unvoted. (For systems that print ballots on demand, check that the paper stock–sheets cast, spoiled, and still blank–adds up to the number of sheets sent to the polling place or vote center.)
Check that the number of ballots returned from each polling place does not exceed the number of voters registered at that polling place or the number of pollbook signatures at the polling place
Check that the number of ballots of each style corresponds to the number of ballots of each style reported by the voting system. Ballot counts for this purpose should be based on the physical paper, not on the voting system: the audit needs external touchstones to check the voting system.
Eligibility.
Check signature verification on vote-by-mail ballots, especially if signature verification was automated.
Check the disposition of provisional ballots to ensure that all that were validly cast (and no others) were included in the results.
Check that each voter received the correct ballot style based on her eligibility. For vote-by-mail ballots, there should be a record of the ballot style mailed to the voter; for in-person voting, this might require recording (e.g., in pollbooks) the ballot style given to the voter. For provisionally cast ballots, this might be more complicated.
Physical chain of custody.
Record seal numbers whenever a batch of ballots is sealed.
Check physical seals for signs of tampering whenever a batch is unsealed. (Good procedure is to photograph each seal after it is applied, check each seal against its photograph before breaking the seal, perhaps taking a second photograph to document the state of the seal, and record any discrepancies.)
Use numbered seals that are hard to forge or bypass; check seal numbers against the numbers recorded when the boxes or bags were sealed; and log the result.
Review custody logs. Check that at least two staff accompanied the ballots whenever ballots were not locked securely and under surveillance.
Review surveillance video of secure ballot storage facility to ensure there was no unauthorized access to ballots.
Due diligence regarding processes, equipment, etc.
Review voting equipment event logs.
Review any complaints made by voters or anomalies or problems noted by pollworkers.
Obstacles¶
Local election officials
State laws / legislators
Perceived costs
Sampling Strategies¶
Comparison audits: compare machine tabulation to hand tabulation.
3 stages:
export subtotals from voting system and “commit”
check that exported subtotals reproduce reported contest results
check random sample of subtotals against hand counts of the paper
batch-level (precincts, machines, etc.) Requires data export / munging
ballot-level. Requires modern equipment
Polling audits: use paper directly
one stage: look at ballots, not at electronic results
batches
individual ballots
Combinations of comparison and polling
Sample design: with or without replacement? stratified or unstratified?
Comparison audits and polling audits generally require a ballot manifest as a sampling frame (c.f. Bernoulli ballot polling). Ballot manifest describes how the ballots are stored.
Auditing as a statistical hypothesis test¶
Null hypothesis: reported outcome is incorrect
outcome means winner(s), not exact counts
incorrect means an accurate, full handcount would find other winner(s)
Type I error: certifying an incorrect outcome (seating the wrong winner)
Transforming “incorrect outcome” into math¶
Depends on social choice function. Concentrate on plurality-type functions.
\(\mathcal{W}\): the set of reported winners of the contest
\(\mathcal{L}\): the set of reported losers of the contest
\(N_s\) ballots were cast in stratum \(s\). (The contest might not appear on all \(N_s\) ballots.)
\(P\) “batches” of ballots are in stratum \(s\). A batch contains one or more ballots. Every ballot in stratum \(s\) is in exactly one batch.
\(n_p\): number of ballots in batch \(p\). \(N_s = \sum_{p=1}^P n_p\).
\(v_{pi} \in \{0, 1\}\): reported votes for candidate \(i\) in batch \(p\)
\(a_{pi} \in \{0, 1\}\): actual votes for candidate \(i\) in batch \(p\). If the contest does not appear on any ballot in batch \(p\), then \(a_{pi} = 0\).
\(V_{w\ell,s} \equiv \sum_{p=1}^P (v_{pw} - v_{p\ell})\): Reported margin in stratum \(s\) of reported winner \(w \in \mathcal{W}\) over reported loser \(\ell \in \mathcal{L}\), in votes.
\(V_{w\ell}\): overall reported margin in votes of reported winner \(w \in \mathcal{W}\) over reported loser \(\ell \in \mathcal{L}\) for the entire contest (not just stratum \(s\))
\(V \equiv \min_{w \in \mathcal{W}, \ell \in \mathcal{L}} V_{w \ell}\): smallest reported overall margin in votes between any reported winner and reported loser
\(A_{w\ell,s} \equiv \sum_{p=1}^P (a_{pw} - a_{p\ell})\): actual margin in votes in the stratum of reported winner \(w \in \mathcal{W}\) over reported loser \(\ell \in \mathcal{L}\)
\(A_{w\ell}\): actual margin in votes of reported winner \(w \in \mathcal{W}\) over reported loser \(\ell \in \mathcal{L}\) for the entire contest (not just in stratum \(s\))
Reported winner(s) really won if \(A_{w\ell} > 0\) for all \(w \in \mathcal{W}\), \(\ell \in \mathcal{L}\).
For unstratified polling-based audits, natural to work with proportions: \(w\) really beat \(\ell\) if \(\pi_{w\ell} > 1/2\), where
Can test using Wald’s Sequential Probability Ratio Test (also see SPRT without replacement).
Social choice functions¶
Most common in the US are variants of plurality:
Majority (common for ballot measures)
Plurality (“first past the post”)
\(k\)-plurality
Also in the US: Instant Runoff Voting (IRV), a form of Ranked-Choice Voting (RCV)
Elsewhere and non-political:
proportional representation (e.g., D’Hondt for parliamentary elections)
single transferrable vote (STV)
approval, Borda, many others