The triennial jamboree that is the publication of the OECD PISA tests kicked off with the usual round of blaming and name calling among politicians. Both sides of the political debate naturally found ammunition in the results, with Michael Gove saying they proved the “urgent need for our reforms” while for the opposition, Tristram Hunt claimed they were a “big wake-up call” for the government.
Behind the noise lay the perhaps surprising finding that British 15 year olds had actually done marginally better than last time, although in comparison to other countries there has been a slight fall in overall standing. So does this mean we should be damning our education system or feel marginally pleased that we don’t appear to be getting any worse?
The answer is neither. When the first ever PISA results came out in 2001, Germany was fully expecting to do extremely well, the fact that it did pretty badly caused such astonishment that a new term was born: PISA Shock. I think it’s time for a new phrase to join the lexicon: PISA Yawn.
This is not to say that the UK education system is not in need of dramatic improvement. In fact, I’d argue it needs rebuilding from the ground up. However, we don’t need PISA to tell us this.
The results released by the OECD would be more useful if they helped us to identify how to improve, but the countries ahead of us have such radically different approaches that anybody can pick and choose almost any educational approach and say there is “evidence from PISA” that it would work.
Do you favour long hours and rote-learning? Well apparently they work, since countries in the Far East do very well in PISA. Perhaps you prefer shorter hours, very highly qualified teachers and light-touch accountability. Well PISA tells us they work too, as can be seen in the success of Finland, which, despite a dip in the 2012 results, has historically always been at or near the top. The data you cherry pick from the PISA results is likely to tell you far more about your own political inclinations than something meaningful about educational answers for the UK.
Sketchy methodology
Looking for solutions in PISA may be more valid if the methodology behind the tests was sound. The idea is that pupils across 65 countries take a two-hour test to measure their aptitude in a range of areas. However, it turns out that pupils aren’t always answering the same questions. Children in each country only take a small sample of each of the three tests, with their total score being estimated using a mathematical model that David Spiegelharter, Professor of Statistics at Cambridge University, has declared “demonstrably inadequate”.
Other studies, have highlighted numerous other methodological flaws ranging from problems with sampling, to language and to choice of question approaches. PISA focuses on problem solving, for example, which gives some countries the advantage over others.
It’s clear that as a comparative tool the PISA results are incredibly limited. However, the methodological flaws become less significant when they are used for in-country analysis. Analysis of the 2009 results suggested that after advantage of socio-economic status has been stripped out, state schools did much better than private schools; and disadvantaged pupils are strongly over-represented in schools serving economically challenged areas. Findings like these help us gain a better understanding of the national picture.
However, this more useful side to the results does not justify the central flaw with the whole PISA approach, which is the fact that the scores are used as a tool to reconfigure education according to economic goals.
Governments in the UK, of whatever colour, have long given up on the idea of education having an intrinsic self-worth; instead its main, perhaps only, function is to prepare children for the world of work and “the global race”. PISA helps this argument by neatly ranking each county as if the race started at 15 and, in the case of the UK, by making debateable claims that we would be US$6 trillion richer over a generation if we raised our PISA rankings by 25 points over the next 20 years. Free market, economic approaches to education rely on measurable outcomes and in this, PISA is manna from heaven.
PISA has almost become a victim of its own success, as its growing significance since 2000 has led to a high degree of scrutiny, little of which has been complimentary. For this reason large numbers of educationalists are rightly sceptical of its usefulness and PISA Yawn could well become a more common reaction than PISA Shock. It’s time we stopped looking at simplistic, non-transferable answers to our educational problems and started looking a little closer to home for the solutions.