Only a tenth of education reforms carried out around the world since 2008 have been analysed by governments for the impact they have on children’s education.
A new report by the Organisation for Economic Co-operation and Development (OECD) think-tank looked at 450 education reforms carried out by its 34 member countries between 2008 and 2014. It found that only one in ten of these reforms were scrutinised for impact.
Andreas Schleicher, the OECD’s director of education and skills, said it was “more the exception than the rule” for education policies to be evaluated by governments. “If we want to improve education outcomes we need to have more systematic and evidence-based approach to reforms”, he said.
Schleicher said the reluctance to evaluate was due to the fact that the trajectory of education reforms is very long and their educational outcomes for children are difficult to entangle. “Parents are a very conservative force,” said Schleicher. “Everybody wants the education system to improve, but not with my child,” he said.
Pasi Sahlberg, Finnish education expert and visiting professor of practice at Harvard University’s Graduate School of Education, said the findings showed a reluctance among those making education policy “to spend significant amounts of money to evaluate something that was a failure”. But he pointed to the fact that much research into education reforms is going on outside of government.
Drivers of reform
The Education Policy Outlook 2015 looked at a wide range of education reforms, but as it was the first report of its kind it did not list which reforms had been most successful. As the graph below shows, the OECD found that those reforms aimed at preparing students for the future were the most prevalent, accounting for 29% of all reforms. These included policies aimed at improving the quality of vocational education, carried in recent years by Portugal, Denmark and Sweden.
The next most popular set of reforms have been around school improvement, including policies to target the teaching profession and school leadership. In Australia, the Australian Professional Standards for Teachers and for Principals were introduced in 2013, while in Germany there has been a “quality offensive” in teacher training.
Reforms targeting governance and funding policies were also included in the analysis, such as the increase in the number of academies in England as part of moves to gives schools greater autonomy. Other examples include Denmark’s 2013 Folkeskole reform, brought in to raise the standards of public schools, and the Race to the Top programme introduced in the US in 2009 that aims to give more funding to reforming states.
The OECD found that those reforms that had been implemented most successfully (even if they had not been evaluated for impact) were those where students and learning were placed at the centre and where teachers and parents were also engaged in the process. Schleicher said success was more likely when governments found a good balance between the policy aspiration and capacity to reform.
The costs of reforms
Peter Dolton, professor of economics at the University of Sussex, who has recently carried out research on the cost-effectiveness of education systems, said the OECD was “right to suggest that there needs to be more rigorous evaluation of new education policy initiatives”.
But Dolton said the report did not address two key questions: “What is the role of, and how do we measure, the differences between teacher input – in terms of quality and status – in different countries. And second, how do we focus on the trade-offs between extra expenditure on education and the reality of better performance – such as efficiency in education systems.”
Evaluation takes time
Daniel Muijs, director of research at University of Southampton, said that in England, the Department of Education has “shown a commitment to evaluating the outcomes of interventions, and has over time increased the rigour of its approaches”.
But he said the problem of short-termism the OECD has identified was still present: “The political cycle often requires shorter time frames than effective educational interventions do,” he said. “This is especially an issue with complex structural changes to education systems, such as the move to a school-led system, be it in the form of the academies programme, teaching schools or free schools. Such changes usually entail a set-up period, during which new structures are developed and formed, which may take several years.
Muijs said that when looking at pupil outcomes – "the ultimate arbiter of successful education reform” – it’s important to take into account of the time it takes for a whole cohort to go through a reformed school before taking national examinations. In most cases, he said, rigorous evaluation of education reforms requires a control or comparison group, which has consequences for roll-out.
The OECD will continue to monitor the education reforms of its members with an aim to start measuring the effectiveness of particular reforms over time.