Hack SIV • DEF CON 2024 Report

This submission was reported by Dr. Chris Jackett (@cjackett), a computer scientist and machine learning researcher at Australia's National Science Agency, CSIRO.

It highlighted a security implementation flaw in the Voter Authentication Token generation process. The Authentication Tokens are one-time codes sent by election administrators to voters to enable them to identify themselves and cast their vote.

The issue identified was that the SIV backend was using Math.random() to generate these, which is not a cryptographically secure random number generator. SIV was already aware of this issue, documented here.

Exploiting this attack would require repeatedly hitting the production API, at a minimum hundreds of times. SIV had other logging and alerting mechanisms to immediately warn if such an attempt took place, and the team is confident that it never has. Even if it did ever take place, auth tokens can be invalidated and re-issued to remediate the issue.

Dr. Jackett also provided sample code to attempt a proof-of-concept brute force attack against the auth tokens, which was tested locally for 24-hours. Even after 24-hours of tries, this approach did not find any valid tokens.

However, highlighting this weak RNG problem and helping SIV prioritize it was valuable.

Especially admirable is that Dr. Jackett not only identified the vulnerability, but also provided a strong solution. He offered simple and straightforward code to upgrade the RNG to use cryptographically secure randomness, completely mitigating the issue, which SIV has already validated and now implemented. This also greatly validated the decision to make the full SIV codebase public as part of the prep for this contest.

In total, this submission earned a combined $1,133.36.

For the full submission details, see its public issue.

This submission, reported by Dr. Michael Specter (@mspecter), a Computer Science professor at Georgia Tech and computer security researcher, highlighted the increased risks of vote buying and selling in a system where voters can easily verify if their vote was cast correctly.

The issue was written to demonstrate an attack against the Public Vote, where DEF CON attendees were allocating $5,000 in prize money. Because this vote was allocating money, it's particularly straightforward to offer money to buy votes. This strategy could be directly profitable, enabling such an attack to quickly scale.

SIV was aware of the risks of vote selling, and had planned from the beginning to use the public Prize Awarding vote as an opportunity to test a new proposal to prevent vote buying, called the "Vote Seller's Dilemma". It aims to disrupt trust between would-be buyers and sellers, by publicly incentivizing honeypots.

Dr. Specter's submission explicitly considered the Voter Seller's Dilemma mechanism. He evaluated that while it may provide benefits against:

1. anonymous buyers — because of distrust created by incentivized honeypots
2. prosecutable non-anonymous buyers — vote buying is a felony carrying jail-time if caught

It may not work against a Vote Buyer who is both outside of law-enforcement jurisdiction, and willing to publicly identify themselves in a reliable way. For example, this could include the leader of a hostile foreign nation. While we had previously examined this issue, Dr. Specter's submission was particularly valuable as it brought the possible threat to life.

Dr. Specter's submission had another novel element: it used the HACK SIV contest's public submissions venue of GitHub Issues to advertise the vote buying offer. While we could have closed or deleted the issue, that would not have been in the spirit of the contest — a point Dr. Specter preemptively highlighted. For future contests, we lean towards incorporating an additional rule that we will remove any explicit offers of vote buying, which would more closely match real-world conditions. However, since we had not established this ahead-of-time for this contest's rules, we did want to leave the submission in place.

For these helpful contributions, we awarded the issue $113 of our $5,000 allotment.

A key question is whether the attack actually worked or not.

6 of the 7 independent judges voted to award the issue even less than us. 1 of the 7 voted to award it $4999 of the $5000 available. On one hand, this looks rather suspicious. But it's hard to be sure, as voting for it could be genuinely and sincerely wanting to call attention to the issue of vote selling. Rather than being motivated to collect $1.

From the Public Vote results page

This contrast — between 6 of the 7 judges awarding the issue a relatively low amount, and 1 giving it 99.9% of their allotment — reflects a broader trend we've observed beyond this contest.

The vast majority of people we've spoken with — including citizens, candidates, and election officials — do not prioritize risks of vote buying & selling as reasons to reject secure internet voting. In contrast, some people view it almost axiomatically as a show-stopping problem, despite vote-by-mail ballots already being trivially easy to coerce (in addition to other problems) and in widespread use.

We don't mean to dismiss the issue out of hand, and believe those most vocal about it are genuine in their concern. We too don't want to see our governments compromised by corruption, of course. But we do hope to see more thoughtful analysis weighing the issue on both sides, recognizing the inherent compromises and trade-offs with verifiability, accessibility, and other key considerations that currently weaken our voices as citizens.

SIV has been thinking about threats of vote buying since its inception in 2020. We have attempted to provide more detailed analysis examining the risks more holistically, especially in context compared the current status quo.

One aspect of this issue that makes it particularly challenging is the difficulty to objectively quantify the threat, as opposed to merely theorizing about it.

In the end, in our best evaluation, we don't feel the Public Vote was compromised by this particular $1 offer.

This issue ended up earning $744 of the $5000 public allotment— approximately 15%. We had already awarded it $113, or about 2% of our allotment. So the public vote awarded it 13% more than our judgment. If this difference had been overwhelming, we could not deny its influence. But as it stands, the impact is not clear.

As of now, we would like to continue using a similar Public Vote format to evaluate future HACK SIV submissions.

Improving the Attack

While we don't want to see SIV compromised, in the spirit of open adversarial challenge to uncover potential threats as early as possible, we wished this particular vote-buying offer had actually been higher than just $1. This could have provided clearer data showing definitive compromise.

We offer that an optimal amount might look more like "Whatever your vote awards me, minus $1". For instance, the one judge's $4999 vote divided by 7 (averaged among all the votes), would earn that seller $713.

Improving Anti-Vote Selling and Coercion Resistance

With all of the above said, SIV is taking the threat seriously and aims to continue testing and refining its game-theoretic solution, the "Vote Seller's Dilemma".

Additionally, since the end of the HACK SIV contest, our team and several expert collaborators have started working on an additional solution, built on Zero-Knowledge Proofs. This enables a would-be seller to secretly trick any would-be compromisers. This technique works against both financial offers to buy votes and physical threats of coercion, which some other contest participants had also asked about. Importantly, the external compromiser can't be certain whether a seemingly cooperative voter has actually used this capability or not.

This would allow SIV votes to achieve a similar level of coercion resistance as in-person paper voting. Even at an official polling center, voters can already photograph or video record themselves filling out a ballot a certain way as evidence for a coercer. But such photographic or video evidence can also be faked, by a particularly crafty voter. This possibility deters would-be vote buyers. Similarly, this new SIV capability can enable voters to secretly trick 3rd-party coercers or would-be vote buyers.

This design also includes strong protection against abuse from malicious election administrators or malware on voters' devices. They cannot use the technique to secretly trick voters themselves.

This solution is not perfect, but in combination with the Vote Seller's Dilemma bounty-reward approach, we believe a high level of defense can be achieved, disrupting voter's trust of would-be buyers or coercers, and the reverse as well.

More details forthcoming, but do feel free to reach out if you are interested in seeing an early draft.

In total, Dr. Specter's submission earned a total of $857.67, which he has chosen to donate to the Electronic Frontier Foundation.

For the full submission details see its public issue.

This submission was reported by Dr. Drew Springall (@aaspring), professor at Auburn University and an election security researcher.

It identifies a vulnerability in one of SIV's verification layers — specifically, the second-device check. This feature is designed to enable voters to confirm their own device didn't tamper with their vote selections maliciously. As an additional verification method, voters can scan a QR code with a second device to double check their selections were submitted as intended.

Dr. Springall highlighted a potential attack in which the QR code used for the second-device verification could be redirected by the compromised device to a malicious site or a fake verification website.

During in-person discussions with Dr. Springall, two possible solutions were identified:

1. In civic elections where vote invitations are sent via postal mail, include a verification QR code in the mailed invitation, instead of getting it from the compromised device. This QR code would link directly to the second-device verification page, and that page would capture the necessary private vote data from the first device.

2. In lower stakes non-civic elections, voters can be reminded to confirm that the second device loads the correct domain (e.g., siv.org). This requires a reliable broadcast channel to communicate with voters, which is common.

While this is one out of a few verification layers, enhancing this particular one is important. It offers voters a straightforward means to identify potential malware before preliminary results are disclosed to the public.

For those unfamiliar with SIV's election design, "made public" means that everyone's vote contents (how people voted) is published as a list, without voters' identities, accessible to all. Each vote content is linked to each voters' secret Verification Number, that only the voter knows. This Verification Number then allows voters to personally find & check their vote was accurately recorded in the tally.

Example:

While other verification methods, such as SIV's Risk Limiting Audits, would also catch this type of attack, addressing the issue at the second-device-check level offers earlier detection.

In the end, the two proposed solutions appear to effectively address the attack, and SIV plans to focus on implementing them.

This submission earned a total of $851.82.

For full details, see the public issue.

While planning the contest, we discussed the pros and cons of offering financial rewards for identifying vulnerabilities. We knew we wanted to reward people for their time and insight.

One concern raised was that similar contests offering financial rewards can often backfire. Too often, organizers receive numerous submissions, but in the end, claim that few or none deserve payout. This can lead to anger, disappointment, and a sense of betrayal.

To avoid this, the SIV team committed from the outset to awarding the full $10,000 amount, regardless of submission quality. This ensured there would be no debate over whether the money would be awarded or not — only a question of who would receive it.

With that settled, the next question became: Who decides who receives awards?

To ensure fairness, we initially planned to divide the decision among three groups: a panel of expert judges, DEF CON attendees, and the SIV team.

Some people suggested we enlist third-party commercial providers, like HackerOne or similar, as independent judges. But we were skeptical they would have the bandwidth and specific product-knowledge to closely and effectively evaluate the different submissions we hoped for. Coincidentally, since our contest has ended, this was proven out with an embarrassing viral episode of low-quality triage: 1 bug, $50,000+ in bounties, how Zendesk intentionally left a backdoor in hundreds of Fortune 500 companies.

Due to time constraints, we ultimately decided to split the decision between two groups: DEF CON attendees and the SIV team, each deciding half of the award money.

We published daily End-of-Day Reports from the contest's start to its conclusion, sharing them widely with the community:

• Day 1, Launch Day, August 6th: github.com/siv-org/siv/issues/179

• Day 2, August 7th: github.com/siv-org/siv/issues/180

• Day 3, August 8th: github.com/siv-org/siv/issues/192

• Day 4, August 9th: github.com/siv-org/siv/issues/200

• Day 5, August 10th: github.com/siv-org/siv/issues/222

• Day 6, August 11th: github.com/siv-org/siv/issues/233

• Day 7, August 12th: github.com/siv-org/siv/issues/234

The SIV team first reviewed each submission, documenting what we found interesting and detracting from each.

This assessment is available in this public spreadsheet under the "SIV Scoring" tab.

Before finalizing the SIV awards, we shared our preliminary rankings with the contest submitters and the HACK SIV updates-subscribers community via Signal & email, inviting them to provide rebuttals. No one raised any issues, publicly nor privately, with our allocations.

For the second $5000 of prize awards, we wanted anyone attending DEF CON to be able to weigh in. But we needed to protect against a few unique threats, detailed below.

Voter Registration (0.5 min read)

To enforce 'One Person, One Vote' — including an auditable voter roll — we registered voters — the public judges, in this case — in person. We did not want to ask for legal names or government IDs, as the DEF CON culture highly values privacy and pseudonymity.

Instead, we wanted a unique anonymous identifier for each voter. We luckily discovered the DEF CON badges could serve this role, as they had a unique serial number in the form of a QR code, printed on the inside of each badge.

So we took a photo of the badge's QR code, noted down the unique serial number, and recorded an email address for each voter to send them their custom vote invitation.

Everyone can view this information in this public spreadsheet, with emails hidden for privacy.

Anti-Vote-Selling (1.5 min read)

We have actively been working on solutions to mitigate the potential threat of vote selling, and one such proposal is "The Vote Seller's Dilemma", a bounty reward system. This system creates an incentive for honey-potting vote buyers to defect on would-be vote sellers, and earn even more money. This is meant to destroy the trust necessary between would-be voter buyers and vote sellers.

We were excited to get the chance to partially test it during DEF CON, as the public vote provided a great opportunity to gather data on its effectiveness in a real election with $5,000 on the line.

After voter registration, we sent all 19 voters the agreement below. Seven agreed, one declined, and the rest did not respond.

In a civic election, law enforcement can also often restrict and fine the 3rd-party vote buyers. Vote buying is a federal felony, with existing laws authorized to fine $10k for a single purchased vote. This contest, on the other hand, could only provide contractual enforcement against the sellers (voters), not criminal enforcement against both sellers and buyers.

In other words, in a government election, the offered bounty reward (point #4 in the email above) can go in both directions, but for this HACK SIV contest, it can only reward defecting buyers, not defecting sellers.

Nonetheless, in the end we were satisfied, as the vote itself did not appear to be compromised.

For future contests, we are also exploring options to strengthen these protections even further. See the detailed discussion above in the section Top 3 Submissions Examined > 2. "I will pay $1 for your vote" for more.

Privacy (1.5 min read)

SIV's privacy design is based on knowing who is voting, but keeping individual vote selections private.

This is meant to enable a truly "Free & Fair Election", where voters can make their choices freely, without anyone — including election officials or technology providers — learning how they voted.

Strong security comes from independently verifiable privacy design, not just unverifiable promises to delete data without looking. It is not simply a question of ethics or ill-intent, but also resistance from vendors' systems being compromised.

This is achieved through what SIV calls "Privacy Protectors". These are individuals who each contribute an additional anonymizing shuffle of the encrypted votes. To complete the election, Privacy Protectors must collaborate to decrypt the anonymized votes for final tallying. This is a type of cryptographic "mixnet".

In addition to anonymizing votes, each Privacy Protector's device also creates and publishes cryptographic proofs that the list of votes entering the shuffle are the same as the one exiting it. Or in other words, that no votes are lost or tampered with.

The power of this Privacy Protectors design is that all Privacy Protectors need to be compromised in order for voter's individual selections to be de-anonymized. Only a single Privacy Protector needs to remain honest and not be compromised to ensure the election's vote privacy.

For the HACK SIV awarding process, all voters were invited to participate as additional Privacy Protectors. In the end, the election's decryption key was split into 4 parts, with one quarter each held by the 3 Privacy Protectors and SIV server.

Before voting began, the Privacy Protectors published individual statements about their participation in the pre-election multi-part key-generation ceremony. After voting closed, the votes were shuffled and unlocked. Again, all privacy protectors published another statement acknowledging their participation in this process, with no issues detected.

All of this was documented publicly on Github, and shared with participants in real-time as the vote took place.

Public Vote Ballot (0.5 min read)

Each voter saw this ballot design, which included the title of each submission, a link to the original submission, and if the voter chose to view them: SIV's evaluations and reward amounts. The names of the submitters were not displayed on this page, in an attempt to minimize bias. Although voters could discover them by following the links to the original Github submission.

The order in which the items were displayed was randomized for each voter, again to promote fairness.

This ballot used a new Participatory Budgeting format (type=budget) that we added to SIV specifically for this contest. It allows the administrator to set a total budget amount — $5000 in this case — which is then tracked and subtracted from as voters enter their individual allocations.

This new budget format is now available for everyone to use in any SIV election.

Voters Verify Election Results (1.5 min read)

SIV is designed to enable a "Zero Trust" election process. Voters can confirm for themselves that their vote was cast and counted as intended. They do not need to trust election administrators or the many intermediate providers involved in the process.

Not only must such a process be available — without it, the entire election can be secretly corrupted. It must also be quick and easy to use, so that voters actually do it.

For this HACK SIV contest, 7 of 7 judges confirmed vote verification. In other elections we ran, we have similarly seen very high numbers, nearing 100%.

For this election, the process took place as follows — After preliminary results were published, voters were each invited to check if their vote was accurately recorded in the final tally.

They were sent a Voter Verification Guide. This guide included instructions on how to use their secret Verification Number. It also detailed more advanced checks. The secret Verification Number is a unique 12-digit number. It is randomly generated on the voter's own device. This number is a very quick and simple tool available to voters.

Example of a general Verification Number check:

After a 48-hour period, each voter confirmed that they saw their votes correctly recorded in the tally.

All voter confirmations are publicly documented on Github.

There are more advanced checks possible — against on-device malware, paper-only to achieve software independence, risk limiting audits, and auditing the voter roll, detailed in the Voter Verification Guide.

The entire HACK SIV process was documented on Github, making it accessible for public inspection.

For the 2024 HACK SIV challenge, we decided to define availability attacks as out-of-scope. One reason was that any successful ones would make it significantly harder for other participants to engage with the system during the contest period, which would not be fun.

Our priorities for this contest were Integrity — altered votes, lost votes, changed results, false voters — and Confidentiality Attacks.

Nonetheless, many people proposed all sorts of creative ways to take SIV's infrastructure offline.

Though SIV has not yet faced any significant Availability Attacks, we acknowledge they are crucial to defend against— especially because in an election setting such attacks can act as a form of voter suppression.

We have begun documenting defenses against Denial of Service Attacks at docs.siv.org/mitigating-attacks/dos, and have a number of other potentially promising ideas to make the system even more resilient in the face of these types of threats.

In the future, we would like to hold other challenges, explicitly focused on taking SIV offline.

One critique is that if a person can break into a ballot center, they can switch the ballots only for that location. But if one can break into SIV, they might be able to change ballot selections for millions of people. This is about the increased blast radius.

Another concern is that SIV has many complex parts, which in turn have many of their own sub-components. And every last one of them needs to work well and not be compromised by attacks. This is about the total surface area through which attacks can potentially occur.

SIV addresses these concerns by providing mathematical evidence of correct results, which a paper-only system does not offer. In a paper system, someone can simply switch out ballots, and no one finds out. With SIV, this is not possible.

For a proper comparison, it would be incorrect to assume these problems do not exist in the current status quo. For example, over 90% of voting equipment used in the US comes from just three companies, producing a huge blast area if any of them are compromised.

Current US elections also suffer from a massive geographic surface area. There are tens of thousands of polling centers, and it is simply not possible for all of them to be observed simultaneously for correctness.

In contrast, SIV empowers vastly more people to verify correctness against compromise. And specifically to be able to verify after the election has taken place, not just during the election itself, which paper processes are limited to.

While the SIV Protocol is designed to defend against on-device malware tampering with voters' votes, it had not yet been designed to defend against on-device spyware (a particular type of malware) learning people's private vote choices. We had documented this issue before the HACK SIV contest and labeled spyware as out-of-scope, as we do not claim to solve it.

Very simply if you're tapping your vote selections into a touch-screen, it is quite challenging to prevent that device itself from learning the selections you are making. There have been some strategies developed to be able to defend against this risk, using a technique known as Code Voting. This involves voters receiving a randomized set of numeric codes, each corresponding to a choice on the ballot, which they enter instead of the choice itself. Unfortunately, this makes the voting process significantly more difficult and prone to errors. For this reason, defending vote confidentiality against on-device spyware has been out-of-scope for the SIV Protocol.

Nonetheless, one person at DEF CON this year demonstrated impressive in-person spyware attacks to show they could learn how people vote.

They had multiple ways to almost instantaneously install spyware, while holding a target's unlocked phone for just a few seconds. This allowed the attacker to view the phone's screen remotely, including the vote content.

This type of attack extends beyond the SIV Protocol to compromise the phone's entire operating system. This allows the attacker to see anything and everything a victim does on their phone, including other private data such as financial info, messages, and photos.

From Day 1, we have built SIV to be an additional voting option, so that people who do not trust or want to use their devices can continue to vote in-person or via postal mail.

To compare the levels of privacy of existing systems, consider the vulnerabilities of current methods. Mail-in voting is susceptible to both minor and major privacy breaches: postal workers can open ballots, or workers at the polling centers that receive the ballots have some ability to look at how people voted. Overseas voters currently have to agree to abdicate their right to a secret ballot altogether. And even in-person voting could also be compromised by hidden cameras at polling places. Both in-person and postal votes privacy could be secretly compromised for all voters at once if there are unique IDs linking marked ballots back to individuals. These are present on many ballot designs, and required for certain types of Post-Election Audits.

We are not claiming these privacy attacks are currently taking place on any sort of large scale, but they are relevant threats if we are fairly evaluating different voting options.

Click the specific scores from the Comparing Voting Methods page for more details

We have already added support to SIV for preparing encrypted votes on separate air-gapped devices. We are also open to adding support for voters to be able to opt-in to a Code Voting mode, which can defend against this on-device spyware threat. There are a number of details to be worked out, to ensure using such a feature would not sacrifice SIV's many other privacy and verifiability defenses. We would appreciate if interested people would write to us at hack@siv.org, so we can prioritize it sooner.

SIV focuses on protecting voters from servers or election administrators learning how people voted. For this, SIV uses strong cryptographic privacy, which can be investigated by all at their own pace.

If the spyware can gain remote control as well, SIV does claim to defend voters from attempts to alter their votes. SIV provides various methods to check for tampering, allows vote-by-vote remediation when necessary, and enables rigorous post-election auditing tools to ensure that sufficient verifications are conducted.

Yes, paper printouts of cast ballot selections can be made available in public places like the Department of Elections, local libraries, or delivered to voters by auditors conducting Risk Limiting Audits. Voters can use their private Verification Numbers to personally check if their selections were tallied correctly.

Although doing checks with multiple devices or using paper printouts may seem more time-consuming, only a small number of people verifying this way can be combined with powerful statistical methods to calibrate the far more accessible digital verification methods.

A key difference between paper-only and SIV is that with the paper-only, privacy requirements force key verification steps (e.g. against ballot stuffing or tampering) to be done during the voting process itself. In contrast, SIV allows people to verify each step after the fact.

On Election Day, it is impossible to be everywhere at once to ensure legitimacy. This is exacerbated even further by longer voting period. But with SIV, anyone can revisit the entire process afterward and verify it top-to-bottom.

Anyone dissatisfied with results has a strong incentive to do this, especially after preliminary results are announced.

In contrast, with paper, by the time preliminary results are announced, it is already too late.

Once preliminary results are published, official auditors and independent parties can conduct post-election audits. For example, they can call voters from the voter roll to guide them through verifying their votes without compromising the privacy of their selections. In addition, the voter roll can be displayed in public spaces, such as libraries, allowing individuals to confirm their votes against paper records. Auditors may also visit voters in person for verification.

The SIV Post-Election Audit aims to determine if voters' devices are cheating and whether ballots were received correctly.

Significantly, such paper-based checks provide "software independence", preventing potential tampering by malware.

We can employ all three methods outlined in the table above, and specifically use the more expensive ones to calibrate the cheaper ones.

SIV's RLAs are also efficient in their execution. For example, in Georgia's 2020 election where the margin of error was only 0.12%, a SIV RLA would have required checking just 7,500 votes out of 5 million to get 99.98% confidence of correct results. This is a stark contrast to the actual scenario, which involved a labor-intensive hand recount of all 5 million ballots.