There has been much discussion in recent years about why East Asian children perform so well on international education tests. I’ve argued before that there is no one reason for these countries’ stellar results, but that home background and culture plays an important role. In the UK, moves to introduce teaching methods popular in countries such as Singapore into the classroom have been heralded by politicians eager to replicate some of the successes of East Asian education systems.
We are beginning to see whether these borrowed methods are working in the classroom. My new study, which looked at a method called “Mathematics Mastery” that was introduced in primary and secondary schools in England, has shown a small impact on children’s progress in maths after one year. In the programme, fewer topics are covered than in a standard maths lesson and in greater depth. All the children are expected to master the material before the rest of the class moves on.
Over the last two and a half years I have been evaluating the “Mathematics Mastery” programme along with Anna Vignoles from the University of Cambridge. The study involved more than 10,000 pupils in Year 1 (5-6 years old) at 90 primary schools and Year 7 (11-12 years old) at 50 secondary schools. Half of the schools were taught using the new method, after training and resources from the education charity and academy chain sponsor ARK, and half were taught with standard maths lessons.
We evaluated the impact of the approach via two randomised controlled trials – one for the primary schools and one for the secondary schools – funded by the Education Endowment Foundation. We have since written an academic paper on the findings.
Early signs of success
The two trials both pointed towards a small positive effect of the maths mastery programme, though neither reached statistical significance independently. When combining the evidence across the two trials, we found children exposed to the programme made around a month more progress in mathematics than those who did not. To put this another way, in a school with a 100 children, the child would move from being ranked 50th in maths to being ranked 47th.
There is of course quite a bit of uncertainty surrounding this result. For instance, it is not clear how far one can extrapolate results from this trial to the wider population. Also, the fact that the trial has been based on only a sample of schools means that the “true” effect size could be a lot bigger (double) or smaller (essentially zero) than we report.
There is no escaping that the effect size we found was small. This suggests that introducing such methods across the education system would be unlikely to springboard England to the top of the Programme for International Student Assessment rankings. As I have noted previously, there are likely to be a lot of other factors at play in high-performing Asian countries.
Yet, at the same time, effects of this magnitude are also not trivial, particularly given the low cost per pupil. It costs around £130 per pupil in the first year, dropping to below £50 per pupil in subsequent years once teachers are trained in the programme. For instance, effects of a similar magnitude were reported for The Literacy Hour – a daily hour set aside for literacy in primary schools – which many consider to be a good example of a low-cost intervention that was a success.
Not enough to build national policy on
Our trials only considered the impact after just one year, the first year such methods were used in these schools. But programmes like Maths Mastery are meant to develop children’s skills over several years, and so may result in bigger gains in the long-run. However, there is currently no empirical evidence available for us to judge whether this is indeed the case or not.
Given the above, our advice is that we need to proceed with investigations into the impact of East Asian teaching methods, while also exercising caution. The empirical evidence currently available does not have sufficient scope or depth to base national policy upon, despite showing some positive signs. What is now needed is further research establishing the long-run impact of such methods after they have been implemented within schools for several years, and after teachers have more experience with this different approach.