Cue socially awkward victory lap
So the votes are in and President Obama has won a second term. I’ll leave the prognostication about the future of America to others. I want to talk about poll averages!
The media assessment of the US campaign has had a very clear split. Almost all the TV talking heads and newspaper columnists said the race was “too close to call” and “impossible to pick in advance,” and that you needed to “stay tuned to see the drama.” The main exceptions were the Friends-of-Fox-and-Friends, who have been confidently predicting a solid Romney victory for weeks. I suspect both sets of journalists were, consciously or not, talking up the kind of campaign that maximized their ratings.
On the other hand, all of the statisticians, political scientists, and other nerds who pore over the polling numbers have said something different: they said the race was close, but not too close to call. They said Obama would win the electoral college decisively, with 300 to 330 votes, and win narrowly win the popular vote, too. All the analysts said the same thing, leading some TV talking heads to say the analysts were crazy or dangerous.
And then the analysts were bang on the money.
The accuracy of their forecasts was uncanny. All the final state-by-state forecasts based on polling averages called all fifty states correctly and, of course, Washington DC as well. When Nate Silver at 538 debuted his poll averaging in the 2008 election, he called 49 of the fifty states right. DC, too. And the states where their predictions expressed the most uncertainty were precisely the states where the vote was closest. One of the poll aggregator sites, Drew Linzer’s Votamatic, has made the correct forecast consistently for over a month, through all the supposed twists and turns of the campaign end-game.
To be sure, predicting many of the state-by-state results isn't hard: everyone knew California would vote for Obama, and Texas for Romney, before the campaign even began. But in the ten or so truly competitive states in the election, the poll aggregators went 10-for-10, and Silver went 9-for-10 in 2008 as well. Those are truly excellent success rates in close electoral contests, rates almost all the TV pundits would fail to match.
Silver also predicted the popular vote margin, by the way. He predicted Obama +2.5%; as at this writing, and with almost all the votes in, the popular vote margin is Obama +2.2%. Uncanny.
There are a couple of us in New Zealand who have tried to import this poll aggregation idea into our own politics. I run an ongoing poll of polls at this site, David Farrar does one at his curiablog site, and Danyl McLaughlin hosts one as well. The New Zealand polls-of-polls have performed creditably in the last two elections, as I document in the VUW series of post-election books, though we are not the oracles that our American counterparts appear to be. I think there are at least two reasons for that:
- Speaking only for myself, I do not think my statistical intuition or technical skills rival those of Silver, Linzer, Stanford’s Simon Jackman, or Princeton’s Sam Wang in the US. I am not even close. I know this because Linzer is a friend of mine, and know some of Jackman’s work, too. Both are wizards. I’ll let David or Danyl puff themselves up on this front if they wish.
- The volume of raw information coming into poll aggregators in New Zealand is tiny compared the US. In the month prior to the 2011 New Zealand election, there were fourteen published polls. In the final part of this US campaign, there were often twenty state- and national-level polls coming in each day. So there is 30-50 times as much information going into their aggregators as go into ours. More information going in to an aggregating system generally means more accurate information coming out.
Nonetheless I think there is a lot that the New Zealand punditocracy can learn from this election in the USA. The most important lesson is that the data really does matter, and it has a neutral quality that a journalistic conversation with a partisan official usually lacks. Numbers can cut through the spin and reveal the truth much better than multiple spins from different angles can. (This lesson extends beyond just horse-race coverage. Any time I see the all-too-common journalistic frame “He said X. She said Y. So who knows?” I feel cheated. Why did I watch all those ads to pay the salary of someone who says that is journalism?)
The second lesson is that while individual polls can bounce around all over the place, the average of many polls from different sources (often collected with slightly different methods) is fairly stable, and a reasonably reliable guide.
Third, the common practice among political journalists of generating a view or “narrative” about a campaign based on one or two polls can often produce dangerously misleading coverage of the democratic process. CNN’s selective airing of polls showing a popular vote tie may have served to get more people to watch CNN, but it also misinformed American about the true state of the election. It was cynical manipulation of the public for commercial ends.
And the practice among some conservative pundits to note the polls showing the race tightening after the first debate to declare that Romney had momentum, but then pointedly ignore the later polls showing that momentum ceased at the second debate, was equally bad. Declaring Romney’s momentum alive-and-well long after it was actually dead-and-buried, was worse than providing a partisan “spin”; the pundits that did it wore partisan “blinkers.”
The US is not New Zealand. There are important differences. But there are enough similarities that we can learn from mistakes made over there. Our pundits should grasp that opportunity today.