Free schools are having an important effect on collaboration and raising standards in nearby schools … We can see free schools are both popular but they’re also effective.
Nicky Morgan, secretary of state for education
The statement by Nicky Morgan on the BBC Radio 4 Today programme on March 9 came on the day that the prime minister, David Cameron, pledged to open at least 500 new free schools by 2020 if the Conservatives win the May general election.
Morgan’s statement was comprised of three main points:
- Free schools are raising standards in nearby schools
- Free schools are popular
- Free schools are effective
Raising standards in nearby schools
The background for this claim is a new report published by the think tank Policy Exchange. Its authors claim:
The data suggests, for the first time, evidence of the wider effect which is taking place at the time that new Free schools are opening in local communities. Free schools are helping to raise standards not just for the pupils who attend them but for other pupils across the local community – especially for those in lower performing schools.
It is very difficult to see support for this conclusion in the report itself. The overall analysis compares the performance of those schools nearest to the free school with the national average. The results for both primary schools and secondary schools show that these are essentially identical – that is to say, the nearest schools to free schools perform no differently to schools overall, so there is no evidence there of a spill-over from competitive pressure.
While there are no differences overall, a split by school performance shows that the lowest performers among the closest schools do better than the national average, and the high and middle performers do worse than average.
This evidence is far from robust. The sample sizes are extremely small. There are no tests for statistical significance for the differences for low performers and they would almost certainly not be statistically significant given the tiny sample and the large natural variation in school outcomes.
There is no presumption at all that this is a causal relationship, given that free schools have not been randomly located. In fact the different split between those low performers that have improved and the high performers that have declined seems much more likely to be reversion to the mean. Even taken at face value, the results show that while low performers improved, the rest were worse off.
Free schools are popular
“Popularity” is about parental preferences for schools, and this data is simply not available. The basis for the claim comes from a Department for Education (DfE) survey of free schools in March 2014. The survey collected data on applications per place for September 2014 and 109 of 142 mainstream free schools responded. The result is that on average, for every available place, a school received 2.7 total preferences.
Is that a lot? Crucially, the report states: “We are unable to provide a breakdown of the number of applications by order of preference due to variation in the way schools responded to this part of the survey.” So this number is all preferences, not just first preferences.
What would an estimate be of the number of total preferences a “neutral” school would receive? For simplicity, consider a neighbourhood with the same number of pupils as school places. If parents typically state x number of preferences, then a “neutrally” popular school should have x times more total preferences than places. Evidence suggests that, today, parents make roughly four preferences, though since parents do not always use all their preferences, perhaps three times is a closer estimate.
So this rough calculation suggests that three times as many preferences as places is about neutral in an average setting. But since free schools are disproportionately in areas with very low-performing schools and a lack of places, then a neutrally popular school might expect a lot more preferences than the average.
Other concerns with the underlying data are that this survey is based on self-reported answers by schools, and only three-quarters of all free schools answered.
So, the data available so far does not show whether free schools are more or less popular than equivalent regular new Local Authority schools. While some individual free schools appear to be popular, some are not and a rough estimate suggests that on average they are less popular than a neutrally popular school.
Free schools are effective
The Education Select Committee recently concluded that there is no basis for telling whether free schools are more or less effective than other similar schools. For example, the Policy Exchange report states that “so far there have been nine free schools with GCSE results”.
The Committee stated: “We agree with Ofsted that it is too early to draw conclusions on the quality of education provided by free schools or their broader system impact”.
More broadly: “Current evidence does not allow us to draw conclusions on whether academies in themselves are a positive force for change.”
Verdict
There is no empirical basis for Morgan’s statements. It is neither definitely untrue nor definitely true. There are reasons for scepticism, but there is simply insufficient evidence to establish any of the three claims for wider local spill-overs, for popularity nor for effectiveness.
Review
The fact check usefully divides Morgan’s claims into three main aspects: standards, popularity and effectiveness. The section on standards rightly points to the overclaiming evident in the interpretations of the Policy Exchange report, which as the author states does not really show any effect, either positive or negative, on standards.
The section on popularity defines popularity in terms of parental preference. This is of course only one possible way of defining the concept, but it is certainly a legitimate one, and the author is correct in stating there is no real evidence here either.
The final section on effectiveness defines this in terms of GCSE results. Usually, we would define effectiveness in value-added terms, but in either case it is accurate to state that at present there is not enough evidence to make any statement on this. One issue is the second citation of the select committee report which refers to academies rather than free schools – though the conclusion of a lack of evidence is even more true for free schools.
The overall conclusion that there is not enough evidence for either positive or negative conclusions about free schools is entirely correct at this stage.
Click here to request a check. Please include the statement you would like us to check, the date it was made, and a link if possible. You can also email factcheck@theconversation.com