International comparisons suggest that New Zealand secondary students are not doing well. It may even be that recent policy measures have worsened their performance.

The 2015 results for the  triennial OECD PISA (Programme for International Student Assessment) evaluation were reported just before Christmas so they did not get much coverage. We need to think about them. Many will jump to a conclusion that the current government’s education policy is failing. Certainly the international evidence does not suggest it is succeeding.  

Every three years, the OECD surveys the science, mathematics and reading skills of 15-year-olds from across the world – some 540,000 pupils in 72 countries or regions of a country.

New Zealand is, and has been, among the top countries in the world. In 2015 it was 21th in mathematics, 10th in reading and 12th in science knowledge – 16th across the three. Rankings tend to jump around from year to year; it is the levels which are the concern They seem to suggest that our absolute attainment has been falling.

The worst fall has been in mathematics. In 2006 the average score for New Zealand was 522. In 2015 it was 495. Not only does that mean New Zealand is falling towards the ‘world’ average of about 490, but a 27 point fall is almost equivalent to having lost a year of schooling; an increase in score of 30 points is roughly akin to completing an extra year of schooling.

The falls for reading of 12 points (say 5 months) and for science knowledge of 17 points (say 7 months) are not as great but certainly perceptible. Across all three subjects the fall averages 18 points so that, I suppose, the 15 year-olds could have left school in 2006 six months earlier and yet attained the levels of those 15 year-olds in schools in 2015. Alternatively the data suggests that in 2015 they had to stay on an extra six months or so to attain the levels of their peers born nine years earlier.

I would have thought that indicates a big fall in the productivity of the schooling system. (This assumes that the only purpose of education is those three subjects; sometimes the rhetoric of those making the changes seems to suggest they believe that.)

Even more disturbing, the falls have not been even over time. To simplify, one might conclude that while there was no significant fall between the 2006 and 2009 surveys, it has been downhill since then.

Many will jump to the conclusion that this is the impact of the Key-English Government’s education policies. I am cautious about such a conclusion, only being willing to conclude that the PISA data suggests that the policy changes the government has made have not resulted in any obvious gains.

Warwick Elley, a retired professor of education with experience in leading international surveys of school achievement (and far more qualified in the area than this social statistician could ever be), takes a stronger line, attributing the decline to NCEA. He points out that Australia, Britain, Sweden and the US, the only countries in the PISA survey with a clear system of standards-based assessment, have also experienced declines. In contrast the attainment levels of ‘countries such as Norway, Portugal, Spain, and Poland that rejected this  system and league tables have risen steadily in PISA.’

Elley suggests that assessment systems like NCEA distort teacher behaviour, that they are so busy aiming for the official targets that they fail on the wider goals implicit in the international assessments. PISA does not measure knowledge so much as the application of knowledge and critical thinking for solving novel problems. It cannot be so easily ‘coached’ for.

The International Association for the Evaluation of Educational Achievement runs a four yearly survey – Trends in International Mathematics and Science Study. (TIMSS is more about testing curriculum knowledge.) It covers the mathematics and science of students at Year 4 (mostly 10-year-olds) and Year 8 students. Of the 19 countries which were in both the 1995 and 2015 survey of Year 4, New Zealand was in places 15 to 16. In the case of Year 8 there were 31 countries and we were in the 15 to 21 range. (Most of the 12 additional countries would probably rank below us at Year 4). There is little evidence of a wholesale improvement over the years – but that is generally true for many other countries.

There is a strange story behind this apparent deterioration. The general view is that the quality of teachers is critical to our educational performance. It is hard to believe that their quality has fallen that quickly. But if targets are changed, good quality teachers will adapt to the targets. (Gilling’s Law – the game is shaped by the scoring system.) Apparently there is a lot of systematic evidence to explain how responses to the new standards are related to the deterioration.

The government appears to be driven here – and elsewhere – by over-simplistic notions and ideologies which do not connect with reality rather than by the evidence. It introduced charter schools because they seemed to be a good idea, fitting its ideological predisposition. There has been hardly any subsequent systematic monitoring; ideologues have no difficulty picking a few irrelevant facts to buttress their prejudices.

What is puzzling is that the Government seems relatively complacent, describing the PISA outcomes as ‘pleasing’. Yet even a Tertiary Education Commission study concluded that 40 percent of Year 12 students who met official NCEA literacy and numeracy requirements fell below the OECD minimum for a knowledge economy.

Later in the year we are getting a new Minister of Education, Nikki Kaye, when the current one, Hekia Parata,who has presided over these not very satisfactory outcomes, retires. One wishes Kaye well, and hopes she will begin her tenure with a thorough evidence-based review of the sector.

Parata’s achievement has been to get education stories off the front page. I’d like to see more there – evidence about our doing better.