A few days before the presidential election, Mitt Romney’s campaign announced what it hoped would be its secret weapon at the polls. Dubbed Project Orca, it let volunteers use a web app to search for and mark off voters as they left the polling location, then collected the data to use in projections or check which Romney supporters might need a call. "When the exit polls come out, we won't pay attention to that," Communications Director Gail Gitcho told PBS. "We will have had much more scientific information just based on the political operation we have set up."

Project Orca was designed to replace the standard phone- and paper-based get-out-the-vote effort, creating a central system that could track voters and allocate volunteers in real time. In a statement to the Huffington Post, the campaign promised that "by knowing the current results of a state, we can continue to adjust and micro target our get-out-the-vote efforts to ensure a Romney victory." It was also supposed to compete with Narwhal, the vote-tracking tool Obama would be using.

By the time election night was over, though, Orca was an embarrassment. Reporter Erin McPike tweeted that the app had crashed, rendering it unusable. A source in Boston said the tool had been hacked, and an anonymous aide corroborated the reports of crashes, telling The Washington Examiner that "somebody said Orca is lying on the beach with a harpoon in it." According to others, the problems didn’t stop there: even before the crash, users had trouble just finding the site or logging in. Few blamed Romney’s loss on Orca’s failure, but it was a final blow to his attempt to outmatch the notoriously tech-savvy Obama campaign. So what went wrong?

If reports written since the election are correct, just about everything. Orca seemed to suffer from a combination of technical errors and a confusing training program that made it easy to think something had gone wrong even at the best of times. John Ekdahl, a Romney volunteer and author of a detailed Orca postmortem, wrote that training materials were vague and inaccurate, even with nightly information sessions. Volunteers were told to look for a native mobile app when Orca was a web app. The URL was for an HTTPS site, but the equivalent HTTP address didn’t forward users to the right page, so anyone who mistyped or simply started with "www" would find nothing.

Offline, Ekdahl writes that problems persisted. "I was immediately turned away [from the polls] because I didn't have my poll watcher certificate... apparently, I was supposed to go on my own to a Victory Center to pick it up." The Orca checklist did not mention the certificate but contained duplicate reminders to bring a chair, suggesting that it had been accidentally left out. Training documents on the Huffington Post show similar problems, with duplicate checklist items and FAQ responses that appear to answer the wrong questions. Volunteers who called the help line, including Ekdahl, report being unable to get through.

If these had been the only issues, Orca would have been poorly implemented but fundamentally sound technology. Unfortunately for the campaign, technical problems apparently kept more people from using it. "I worked on the Colorado team," an anonymous campaign source told Breitbart News, "and we were called by hundreds (or more) volunteers who couldn't use the app or the backup phone system. The usernames and passwords were wrong, but the reset password tool didn't work, and we couldn't change phone PINs." Eventually, they were told that every PIN issued to Colorado had been wrong; the source says a North Carolina campaign worker reported that the same thing had happened there. "The system went down for a half hour during peak voting, but for hundreds or more, it never worked all day... Many members of our phone bank got up and left."

By all accounts, Orca finally crashed hard the afternoon of November 6th, though there’s little to support the allegation that it was hacked. PIN and password problems could have been the result of either technical issues or user error, and given how confusing the system seems to have been for volunteers and organizers alike, it’s hard to tell which is more likely. Romney campaign digital director Zac Moffatt admitted to CNET that the app hadn’t been tested adequately. "The primary issue was we beta-tested in a different environment than [headquarters Boston Garden]," he said. "There was so much data coming in — 1200 records or more per minute — it shut down the system for a time. Users were frustrated by lag, and some people dropped off and we experienced attrition as a result."

Moffatt insisted that Orca did its job. "We had 91 percent of all counties report into the system, 14.3 million voters were accounted for as having voted, and we received 5,397 reports on voting issues, such as instances where they ran out of ballots. The information came in, so you can't say it didn't work. You run into issues because it's so massive in scale." Even with these numbers, it remains a cautionary tale. Data can make or break a campaign, but gathering it successfully requires a focus on actual humans and a lot of stress testing.