Evaluate education reforms today to avoid mistakes being repeated for our grandchildren

Wiping the slate clean won’t always help. Chalkboard via Lonni/Shutterstock

For most governments, it’s their platform of education reforms that is politically one of the hardest programmes to push through. Yet push it through they do, in what has become a constant effort by politicians to keep reforming the structure and content of education systems to keep up with a fast and unpredictably changing world.

Now a new report by the Organisation for Economic Co-operation and Development, looking at the global trends in education reform, has come to an important conclusion:

Once new policies are adopted, there is little follow-up. Only 10% of the policies considered in this dataset have been evaluated for their impact. Measuring policy impact more rigorously and consistently will not only be cost-effective in the long run, it is also essential for developing the most useful, practicable and successful education policy options.

Politically inconvenient

There may be three reasons for this. First, rigorous evaluation of reforms is very difficult and expensive. Complex education reforms take a long time to implement. National curriculum reforms, for example, often take a decade before they are fully implemented in schools and adopted in classrooms. Fair and reliable impact evaluation can therefore be made only when implementation is completed or progressed so that impact can be measured. The OECD reports that preparing students for the future is one common policy trend in many countries. How to evaluate the success of this policy is very difficult given the unpredictably changing future landscape.

Second, too many education reforms end in failure or the end results are something very different than initially expected. It is understandable that there may be reluctance among politicians to spend significant amounts of money to evaluate something that was a failure. Moreover, national education reforms are often influenced by changing social or economic conditions that significantly affect their implementation.

And third, more often than not, education reforms are highly political constructions – and therefore are high-stakes gambits when it comes to national politics. Many education reforms today are seen as part of more complex and comprehensive social policies that are interdependent and sometimes difficult to be evaluated separately. Only a few politicians would take a risk and evaluate their, or their colleagues’, reforms knowing that the chances of getting a poor rating are abnormally high.

Other ways of measuring

The OECD has been very courageous in taking on some very challenging works of analysis, with its Programme for International Student Assessment, its 2014 report Measuring Innovation in Education and now the Education Policy Outlook 2015. These reports all should have clear notes upfront of the reliability and validity of these exercises and, perhaps most importantly, what national politicians and pundits should not do with the findings.

Similar notes are needed in this new report on education policies around OECD countries. It is useful in indicating some trends and challenges in global education policy and reform. But at the same time it is based on one chosen way of analysing international education policies that relies primarily on self-reported information and data from member countries.

There are various alternative ways to systematically analyse and understand global education reform movements to that presented by the OECD. I would argue that the international academic community has been rather active during the past ten years in researching education policies and reforms around the world.

But the OECD has been sparing in its use of this existing research literature and other current policy analysis. For example, it remains silent about the negative effects of some common global education policy trends, such as the increased competition between schools, growing use of external standardised testing, misuses of technology and market-like school choice around the world, often referred to as GERM – the global educational reform movement.

Together with “what-to-do” types of recommendations it would have been useful to tell those who make education policy “what doesn’t work” in education policy reform. The important question of why national education policies are following the trends the OECD has identified in this report is not adequately answered. I think it would have been helpful for readers to hear about why national education policies and reforms look so much alike.

Body of research growing

But I actually do believe that educational researchers today are doing more to understand the nature and impact of education policies and reforms than the OECD gives them credit for. Take Chile, England, the US, Sweden, Australia, Germany, just to mention a few, and you’ll find more research and policy analysis on their education systems than ever before by both their own and international scholars.

Chileans have repeatedly protested against education reforms. Mario Ruiz/EPA

Much of this research is directly addressing questions of the impact of reforms. There are good examples of how difficult it is to get reliable, up-to-date knowledge about the real impact of major national education reforms to be found in the Chilean market-based education reforms in the 1980s, Swedish and English school governance reforms in the 1990s and the new US education policy and legislation in the 2000s.

Measuring the impact of these case studies have been targets of massive studies, but the results often take too long for the purpose of making policy. Now researchers are looking at their impact, it’s becoming clearer what does and doesn’t work. Politicians should ensure they look at both sides of that picture when planning their next education reform promises.

Nevertheless, the OECD should be applauded because it has bravely brought new, important policy themes to national and global education reform discussions, including equity in education, early childhood education and the need for more student-centred pedagogy – things that have previously been put on the back seat of education policy vehicles.