Mike Hosking’s statistical ignorance is so extraordinary that one’s immediate reaction is that a description of it was satire; nobody could be as ill-informed as that – and certainly not a major public commentator.
The details are amusingly told on the Spinoff website, but briefly the NZ Herald reported an Auckland Transport survey of 1459 people which stated that ‘support for cycling overall is at 57 per cent, with 34 per cent saying they are “very supportive”. Those not supportive of cycling are at 11 per cent, with 8 per cent "very unsupportive"’
Hosking exploded, rejecting the survey results in part because, he said, so few people were involved. He went on ‘the Herald, who should know better, pass this crap off as news nowadays. And it’s wrong. It’s fraudulent.‘ His partner, Kate Hawkesby, added ‘I refuse to believe anyone in Auckland wants more cycle lanes,’ ignoring that in the survey at least 1000 Aucklanders had said they did.
So the pair opened a Facebook page inviting respondents to answer the question, ‘are cycle lanes good for the country?’. Forty-five minutes later Hosking announced that his own poll has received the same number of votes as the AT survey but that only 36 percent were in favour of cycleways. (More than zero, Kate.) Hosking adds that ‘under the [AT survey] rules, our results are as legitimate as theirs.’
Before explaining the ‘huh’, I report that eventually some 15,000 votes were cast with support for cycleways reaching 79 percent. Attempting a cowshot for a six, Hosking snicked the ball onto his middle stump.
But whatever the result would have been, the ignorance Hosking displayed is remarkable. Remember he said that under the statistician’s ‘rules’ his results were as good as theirs. Statisticians certainly would not have thought so.
Their conclusion was derived from a random sample in which every Aucklander had an equal chance of being questioned. (In practice it is a little more complicated – a stratified sample – but the underlying theory holds.) If that happens then the average from the sample would be near the answer you would get if you asked every Aucklander.
In fact we can even tell how near the estimate is likely to be to the true figure – it is reported as the ‘margin of error’. The probability that the true figure is less than 50 percent is negligible.
The rules are quite clear, Mr Hosking, If a survey is not based on a random sample, any estimate it produces is, to use a firm you appear fond of, crap, That is as true for the later Hosking poll estimate of 79 percent supporting cycleways as for the earlier 36 percent.
(The size of the survey does not make a difference to the validity of the estimate. If the sample is not random, the estimate is still biased; to treat it otherwise is still nonsensical.)
Even more extraordinarily, Hosking interviewed the statisticians who ran the survey. If he did not understand the ‘rules’, he did not understand what he was discussing. It was like a toddler asking an adult about – er – sex. And that must apply every time Hosking discusses any survey, including political opinion polls. He has no credibility at all. Any opinion he offers on a survey is crap.
Of course statisticians would welcome the opportunity to teach him some elementary statistics. But that would require Hosking to start being humble enough to know how ignorant he was.
PS. Had Hosking attained the numeracy of a seven-year-old, he would have noticed that the sum of the numbers the Herald appear to exceed 100 percent. (57 + 34 + 11 + 8 = 110) The report conflated strong support for cycleways with those who just supported; it did the same for the unsupportive. The actual figures were 34% strongly supported, 23% supported, 3% were unsupportive and 8% strongly unsupportive with 32% not expressing an opinion. That the Hosking biased poll does not mention the latter group shows it was self-selected and invalid. As commonsense would tell you, even if you knew no statistical theory.