What does it take to build a massive, never-before-seen behemoth of a campaign contribution and voter mobilization management platform? Millions of dollars spent on outside consultants to build from the top down or considerably less hiring an army of senior technologists to build from the bottom up?
Anyone who’s read any of the post-election analysis of the respective systems used by the Romney campaign (Orca) and the Obama campaign (Narwhal) know what worked (Narwhal). As members of the team detailed in the Coding Forward 3 city tour sponsored by New Relic, the Obama campaign spent 18 months and built a system around the fundamentals of good design, coding a simple, easy-to-use API with small, easily-managed components that could be turned on or off as necessary.
The core of the system, namely the contribution processing system, was made robust by following one simple principle: If it didn’t impact the user experience, throw it out. This meant a database that could be taken off-line, as it was for a minute during the second Presidential debate, and still receive approximately 100,000 donations without anyone noticing because they were queued for later processing. The developers and engineers were able to code in whatever language best suited them, in a supportive environment with the tools they needed to do what they were expected to do best, ship code.
As they pointed out repeatedly, they didn’t do anything revolutionary, they just proved that the fundamentals of good software design, testing (“we got away from HQ and out into the field”), analysis (“we measured everything”), and Agile development (“The best way to overcome resistance to iterative deployment is to show that it works”) can lead to something truly massive. Extensive “Game Day” testing meant the election was run repeatedly on their systems before the actual day, making them confident when the election came around that they were confident (one even said “cocky”) the system would work or if it broke, they would know how to handle it. They would even have real-world problems crop up while they were in a Game Day test and they continued the testing while dealing with the real-world issue.
As someone who comes at the field of computer science as something of an outsider, without the skills and experience to be a rockstar out of the box, it was great to see a number of highly experienced technology professionals preaching the value of the tried and true principles of good design rather than the gimmicks or tricks of the latest framework or language as the path to success in high-stakes environments and critical development situations. Best of all, their advice to other programmers was to get involved with whatever project is out there and “don’t stop.”
Thank you to New Relic for making this open forum happen. As you can tell, I had a great time. If you’re in Chicago or San Francisco, I highly recommend attending. You won’t regret it, assuming you can get in the door.