In 2013 the former secretary of state for education, Michael Gove, and his minister of state for schools, David Laws, decided to change the A Level qualifications taken by English and Welsh students with academic aspirations. But their reasons for doing so were based on flawed analysis, and the reforms could make it harder for students who flourish in the sixth form to get into university.
Most students currently take GCSE examinations in eight or more subjects at age 16. In the first year of post-compulsory education they are examined in four subjects leading to the award of AS Level grades. These are followed a year later by exams in three (or all four) of them for A2 qualifications. The AS and A2 marks are combined to form an A Level grade.
While A Level results are the most important basis for university entrance, most students apply before they are known: most offers of places are conditional on their getting specified grades. To decide who should receive offers, university admissions officers use the GCSE and AS Level results to assess students’ potential alongside personal statements, school references and teachers’ assessments of likely grades.
Reforms based on shaky evidence
This system will end in 2015. A Level courses will then comprise two-year modules, examined at the end of that period only. AS Levels are being redesigned for other purposes, not to be taken by students in their chosen A level subjects. University admissions officers will no longer have quantitative, comparable evidence of students’ post-GCSE performance in subjects central to their university applications.
Admissions officers and others argued that these changes would make identifying those best-fitted to study for their chosen degree courses more difficult. To counter that, Laws commissioned in-house research, on the issue. The results claimed to show that, since degree performance could be as accurately predicted from GCSE as from AS Level results in one student cohort (those graduating in 2011 on three-year degrees), the future absence of AS Levels would not harm the admissions process.
We have challenged the quality of that research elsewhere. Now our further research has used the department for education’s data to show how its results have been misinterpreted.
Who gets an offer?
We have placed UK universities into five types according to their average entry requirements: the Russell Group; all other universities founded pre-1992; universities (the ex-polytechnics) founded in 1992; post-1992 universities; and other degree-awarding institutions.
To identify which institutions the “best qualified” students were admitted to, we divided the 80,420 students for whom we had data on GCSE scores in 2006 and AS Level scores in 2007 into two sets of five groups (quintiles) according to their performance.
The table below shows that only 4% of students in the lowest GCSE quintile (the weakest performers) went to Russell Group universities, compared to 75% from the highest quintile. There was a very similar difference between the lowest and highest quintiles according to AS Level results. The common pattern across the table’s two parts apparently sustains the Gove/Laws argument: better-qualified students at either GCSE or AS Level disproportionately gained places in more prestigious institutions.
But note that we said or, not and. Implicit in Laws’s conclusion is a strong correlation between GCSE and AS Level performance. Perhaps surprisingly, there isn’t one. Many students performed relatively badly at GCSE but much better at AS Level (the “late developers”). Others did well at GCSE but not at AS Levels a year later (the “drifters”).
So if admissions officers had only GCSE grades available they may not have offered places to many students whose AS Level results indicated much greater potential to succeed on a degree course than their GCSEs did. Many late developers might have missed out on degree places at prestigious universities.
AS Levels do matter
Our data analysis has gone on to elaborate this conclusion, showing the difference in students’ destinations by GCSE and AS Level qualifications combined.
For example, just 2% in the lowest quintile for both GCSE and AS Level went to a Russell Group university. But of those who got the lowest GCSE grades but the highest for AS Level (the most emphatic group of “late developers”) 30% did. In general, students who performed much better at AS Level relative to GCSE were more likely to obtain a place at an “elite university”. Conversely, those students who didn’t do as well at AS Level relative to GCSE had a greater probability of studying at other institutions.
The department for education analysts and ministers were wrong in interpreting their (flawed) statistical analysis to conclude that university admissions officers gained no additional value in assessing students’ potential from AS Level results. This re-analysis suggests the opposite.
Many students performed relatively badly at GCSE but their improved performance at AS Level a year later may have encouraged them both to apply to more prestigious universities and admissions officers to recognise their potential. Without their AS Level results, many late developers may not have been given that opportunity. Many of them may have been from disadvantaged backgrounds and benefited from universities’ efforts to expand the number of places offered to those students.
Whatever the political, ideological, or educational arguments for restructuring A Levels, the claim that abolishing the intermediate exams at AS Level would not harm the university admissions process is not sustained by this analysis.
Richard Harris, Tony Hoare, Kelvyn Jones and David Manley also contributed to this article.
This article was adpated from a post on the University of Bristol’s Policy Bristol blog.