Poll of Polls

Pundit Poll of Polls: How we do it

The NZ Pundit Poll of Polls is updated weekly using a elements of a poll aggregation method originally developed by Nate Silver of fivethirtyeight.com. This poll of polls was adapted for the New Zealand context by Rob Salmond of the University of Michigan. The details of our poll of polling method are:

  1. We collect information from four polling sources: One News/Colmar Brunton, TV3/TNS, Roy Morgan, and NZ Herald/Digipoll.
  2. We adjust total sample sizes to take account of the “don’t know” respondents. So a poll of 1,000 people with 8% “don’t know” is treated as a poll of 920 people. This is because all the percentages the polling firms report are only out of those 920 people.
  3. Each poll is weighted according to its sample size, so an 800 person poll is twice as influential as a 400 person poll.
  4. Each poll is also weighted according to how recent it is (measured from the middle day of the poll’s period in the field). We treat a poll as having a half-life of thirty days, meaning that a thirty-day old poll has a recency weight of 0.5, a 60 day old poll has a recency weight of 0.25,a 90 day old poll is weighted at 0.125, and so on. Once a poll has a recency weight of less than 0.05, we drop it from the poll of polls entirely. One important benefit on this result is that in most circumstances our estimated vote shares for parties will not change without new polling information. In some other poll of polling methods, there are day-to-day shifts in estimated support as an artifact of the weighting process. We have minimized this problem.
  5. This procedure gives us one poll of polls each week from which we can draw estimates of parties’ levels of popular support, complete with a time-corrected pseudo sample size. (Note that we do not use this sample size to generate standard errors and test hypotheses because the “sample” we are using doesn’t meet the statistical standards required to generate standard errors.)
  6. We also do seat projections on the basis of the polling results. The projections are for an election held today, not for an election held in November 2011 (we don’t have the crystal balls to make that kind of forecast). In making these projections, we are explicit about our assumptions about electorate seat victories for the smaller parties.

Further details, including research underlying the choice of time-weighting algorithm, can be found at fivethirtyeight’s FAQ page.