Treasury made a egregious error in its calculation of the impact of Labour’s tax package on poverty. Can we learn from it?
I told my econometrics students I would not penalise them for a minor calculation error in any of the statistical problems I gave them, but would halve the marks if their conclusion was obviously absurd. I had already explained to them Moser’s Law: that if a statistic looked interesting it was probably wrong (i.e. the result of an entry or computational mistake). You could say that I was rewarding them for commonsense, a necessary attribute in statisticians (as it is among all professions).
I was surprised when the Treasury announced that the Government’s tax package would take 88,000 children out of poverty for it seemed remarkably effective given the modesty of the outlay. I trusted that the Treasury had got it right and put the issue of checking it aside while I dealt with other things.
(For the cognoscenti, the numbers of poor below a poverty line that a package will impact upon depends on its size and the slope of a particular statistical curve. The Treasury estimate suggested the curve was much steeper that I had ever come across and I thought I must investigate why.)
It has turned out that commonsense won, at least for me, for the Treasury made a dreadful error for which they have just apologised. (Full marks for the promptness and openness.) Apparently they forgot to apply the commonsense test of Moser’s Law.
I am disappointed. All statisticians make mistakes. Sometimes deep in the computer code there is an error; sometimes two analysts working on the same problem make conflicting assumptions; sometimes (often, in my case) you enter a number wrongly (so I have to double check each entry); sometimes definitions do not match what you think; and so on. But the commonsense test should identify the worst examples. If it fails, the statistician is sloppy or incompetent and not understanding what he or she is doing.
The Treasury said that such mistakes were extremely rare. If it meant an error of such egregious proportions, I am sure they are right. In regards to small errors, which commonsense cannot pick up (and which do not matter that much), such a claim would be boasting (or a very inaccurate estimate).
Yet, there is a sense that the poverty and inequality debate has been riddled with poor-quality quantitative analysis. I skip over a book or two with statistical charts which are badly labelled and give a couple of examples.
Consider the proposal to eliminate school fees. Commonsense tells you that reducing them will reduce financial pressures on parents. By how much? If you look at the poverty research you will find that there is no indication whatsoever so the government is not going to be able to pat itself quantitatively on the back. Moreover, in the past the fees were increasing, adding to poverty. Nobody mentioned this creep or that it was increasing hardship in families.
The current method for calculating poverty on an income basis is flawed because it ignores such impositions on the family. We know how to adjust for it – commonsense says we should – but far too many poverty commentators have not enough grasp of the statistics they are using to be able to think this through; explanations to them go down the black hole of their incomprehension.
Any income based poverty line is not absolute in itself but an indicator of a level of income at which most people face unacceptable hardship. It really needs to be calibrated against the way people live. Often it seems pulled out of thin air – very often by people who have had no personal experience of real hardship.
Second, did you realise that the way we set the poverty line means that if we took money from those in the middle of the income distribution and gave it to the rich, measured poverty would be reduced? Yes, you had better read the last sentence again. Apparently by making the rich richer the measure reports fewer poor. (This is not trickle-down, which is a second order effect, if it exists at all.),
The paradox arises because we use a poverty measure based on the (median) income of the middle person in the distribution. If that median income is reduced by those in the middle paying more taxes (while cutting taxes on top incomes), the level of what constitutes the poverty threshold falls and, ergo, there is less measured poverty, despite there being no change to the hardship the poor are experiencing. The apparent fall is an artefact of the statistics.
This is not just an exercise in statistical understanding for students. It actually happened in the early 1990s. The government tax packages were redistributing income from those in the middle to the rich. Consequently the measured poverty line fell; so did the measured number of poor. The neoliberal Business Roundtable and the Treasury of the time (a somewhat more challenged one than today’s) seized on the result as evidence that the government policies were working. Had they been in touch with reality, commonsense would have told them that since the voluntary sector was struggling with unprecedented requests for food aid and other assistance, hardship was really rising.
Curiously, those who calculated the statistic that the BRT and Treasury quoted denied that their data was deceptive saying that they would never do anything that would misrepresent the state of the poor. (One was reminded of Ronald Reagan who said he would never do anything to increase the US budget deficit as he signed approval to increase military expenditure which did exactly that.)
Yet, we have persisted with using the median income as the reference income for calculating the poverty line. The government has said it will be reviewing the definition of the line. Compared to a quarter of a century ago we have more much information about living standards, which will help us calibrate a more authoritative poverty line. (As it happens, the living standards method was used in a special 1974 study of the incomes of the elderly, but it was never followed up.)
It will be interesting to see how seriously the government undertakes the task. Will it use independent advisers with the technical skills and commonsense or will it rely on people of goodwill but with little competence or practical shrewdness?
Whatever, it is to be hoped that the Government does not use the excuse of the Treasury error to renege on its election promise to substantially reduce poverty among children and their guardians. Since the Opposition made the same promise, it will be holding them to account. So should we.