Amidst the post mortems about the presidential race, much has been written about how the Romney campaign's get-out-the-vote effort sunk under the weight of its bug-ridden ORCA data tracking system. Over the last few days, more techie publications have dug into the technical details of how the Obama and Romney campaigns ran their respective data mining and vote tracking operations. In the end, the campaigns' respective approaches to how they used technology in conjunction with the ground game spokes volumes about the candidates themselves.
As I read these articles, two major themes strike me:
TOP DOWN VS. BOTTOM UP
Technology informed the Obama GOTV effort, but all of the technological prowess was deployed to drive the decidedly lower tech face-to-face and personal contacts. Even though the microtargeting identified the voters, the GOTV effort was ultimately conducted on the ground with volunteers using personal contacts that they had been building for months prior. Bottom up.
With the Romney campaign, the technology seemed to be the end game in itself. Using their vaunted ORCA system, the Romney campaign deployed their GOTV efforts to the polls with 30,000+ volunteers checking in. They would using a web app to feed the strike lists over to a central server, which would then redeploy resources and make robocalls. 30,000+ volunteers were deployed just to feed information to ORCA, so that the system could further saturate swing state households with more robocalls. Top down.
INCOMPETENT VS. COMPETENT
Even though the Obama campaign has been written up as having a state-of-the-art microtargeting setup, the actual deployment was decidedly conservative. The technology team opted to go with off-the-shelf services, with an emphasis on proven performance and reliability. They did not sprint to the shiny object. They remained focused, and did their due diligence. The system was simulated and tested under all sorts of worst case scenarios, and it performed as intended on Election Day. Competent.
The Romney campaign went for the shiny object. ORCA was supposed to be their equalizer against the Obama ground game. The system crashed on Election Day under the heavy load. ORCA had never been tested in any sort of real world simulation. Volunteers were sent inoperative passwords. The instructions to volunteers had key reminders missing (i.e., where to pick up a poll observer credential). Election Day was their beta test. Incompetent.
ROMNEY: Faith Based Technology Deployment
For all of Romney's supposed prowess as a data-driven manager, I still can't get over how faith-based a campaign he ultimately ran.
They had faith in their "unskewed" polls.
They had faith in their strategy of wildly veering from position to position, and lying with impunity.
They had faith that all of their campaign gimmicks and distractions would tip the electorate.
And they had faith that their GOTV system would ultimately triumph over the Obama campaign's boots on the ground.
Here's how the system was supposed to work—though it probably wasn't a great idea for Romney's communications director, Gail Gitcho, to tell PBS how amazing Orca was going to be before it had been used. "The Obama campaign likes to brag about their ground operation," Gitcho said, "but it's nothing compared to this."
Ultimately, the system was switched on at 6am on Election Day. This would be the first time that users actually use the system. The beta test would double as the final product.
But Orca turned out to be toothless, thanks to a series of deployment blunders and network and system failures. While the system was stress-tested using automated testing tools, users received little or no advance training on the system. Crucially, there was no dry run to test how Orca would perform over the public Internet.
Part of the issue was Orca's architecture. While 11 backend database servers had been provisioned for the system—probably running on virtual machines—the "mobile" piece of Orca was a Web application supported by a single Web server and a single application server. Rather than a set of servers in the cloud, "I believe all the servers were in Boston at the Garden or a data center nearby," wrote Hans Dittuobo, a Romney volunteer at Boston Garden, to Ars by e-mail.
http://arstechnica.com/...
ORCA had never been tested with volunteers doing a dry run. Consequently, volunteers did not know whether their passwords would work, whether their instructions were clear, or if their mobile devices would connect properly to the servers.
It had never been tested using the internet connections that it would rely on during Election Day. Consequently, the incoming data traffic was interpreted by the server as denial-of-service attack, and shut down.
The mobile device servers were all in one location with no redundant backup in the event of failure. Consequently, the system was unreliable and at times unavailable.
I'm no computer expert, but I would think that a dry run using live volunteers in the days prior would have ferreted out some of these issues.
The lack of "beta testing" was not just an Election Day failing, it ran endemic throughout the Romney campaign. Throw stuff on the wall and see what sticks. Have faith that everything will work itself out in the end. What astounds me is that due diligence is a supposed given with MBA types, yet in striking short supply with the Romney campaign.
OBAMA: K.I.S.S.
In contrast to the Romney campaign's build-a-bigger-beast approach, the Obama campaign had a marked aversion to going with novel or cutting edge technologies. They primarily relied on proven technologies from outside service providers. If something had never been used on a large scale, the Obama campaign avoided it.
Rather than focusing on creating something significantly new, Reed said, the team focused on taking what they already knew worked and fitting the pieces together.
"We aggressively stood on the shoulders of giants like Amazon, and used technology that was built by other people," he said. "We had a pretty good culture of using not-invented-here technologies. And we weren't scared about it."
In some cases, he added, it was just the opposite—if something came up as a solution and had never been used at that scale by someone else, it was "super scary" and usually avoided. The Democrats were going to play it conservative.
http://arstechnica.com/...
Ars Technica's description of how the Obama campaign's Narwahl system worked shows it more as an organizing tool. It processes a tremendous amount of information, but ultimately, it informs the people working on the ground -- people who have been working local neighborhoods and getting to know the voters they need to reach months ahead of Election Day. Narwahl was already working well before Election Day. The Obama campaign's technology team had not only been optimizing and deploying the system for months, they had also anticipated any and all potential disaster scenarios.
The Obama campaign's technologists were tense and tired. It was game day and everything was going wrong.
Josh Thayer, the lead engineer of Narwhal, had just been informed that they'd lost another one of the services powering their software. That was bad: Narwhal was the code name for the data platform that underpinned the campaign and let it track voters and volunteers. If it broke, so would everything else.
They'd been working 14-hour days, six or seven days a week, trying to reelect the president, and now everything had been broken at just the wrong time. It was like someone had written a Murphy's Law algorithm and deployed it at scale.
And that was the point. "Game day" was October 21. The election was still 17 days away, and this was a live action role playing (LARPing!) exercise that the campaign's chief technology officer, Harper Reed, was inflicting on his team. "We worked through every possible disaster situation," Reed said. "We did three actual all-day sessions of destroying everything we had built."
http://www.theatlantic.com/...
So, basically you have the Obama campaign hacking their own system in order to prepare the team for potential disaster. Over the entire campaign, Obama's team had less than 30 minutes of total downtime.
But, for all of the technology, the simplicity of the Obama campaign remained in its reliance on organization and getting people focused around the tasks that needed to be done to find the voters and make sure that they vote. It's not something that the Obama campaign thought they could just flip the switch and activate. It was a long process that they had in play for months before Election Day.
Messina says the campaign has some 5,100 GOTV staging locations, which he described as “hyper-local Obama hubs” that would continue to push voters to the polls. Jeremy Bird, the campaign’s national field director, says the Obama camp has made 125 million personal contacts with voters to date—either phone calls or door-knocks. It’s a number Bird says doesn’t include automated calls.
“Unlike campaigns of the past, our volunteers are not driving to some large office miles from their homes and handed a phone and a call sheet,” reads an Obama campaign memo released late Saturday afternoon that makes the case for a sizable Obama edge when it comes to early voting and GOTV. “Instead, Canvass Captains, Phone Bank Captains and scores of local volunteers will be knocking on the doors of the very voters they registered, have been talking to for months and know personally.”
http://www.campaignsandelections.com/...
And that's a theme that runs throughout the Obama campaign. Take the long view. Anticipate multiple steps ahead. Rely on hundreds of thousands of volunteers and hundreds of thousands of donors, rather than a single automated system and a handful of billionaires writing big checks. Have the technology serve the volunteers on the ground, rather than have volunteers in service of the technology.
No matter how the contrasts are described, it all boils down to the simple fact that one approach worked, the other one didn't. And the candidate whose approach worked also won.