A brief analysis published by the Universities and Colleges Admissions Service (UCAS) appears to explain the stark differences in offer rates for different ethnic groups that apply to highly selective universities. UCAS says these differences in offer rates – which are up to 15 percentage points lower on average for ethnic minority applicants than for white applicants – can be attributed almost entirely to differences in applicants’ predicted A-level grades and the popularity of their chosen degree course and institution.
UCAS says that the actual offer rates to ethnic minority applicants “are close to what would be expected (from the predicted grades they hold and the courses they apply to), ranging from two percentage points higher than expected, to two percentage points lower than expected”, across different degree subject areas.
These findings would seem to rebuff evidence from a number of academic studies, including my own work, which have found that ethnic differences in offer rates remain substantial, even after controlling for applicants’ grades at A-level and a range of student and course characteristics.
But the UCAS findings are not quite what they appear. In fact, they are far from conclusive and serve to highlight the need for comprehensive data sharing so that researchers outside of UCAS can carry out more detailed and nuanced analyses to explain the gap.
Oxbridge missing – and medicine
“Offer rates to different ethnic groups close to expected values,” was the headline of the UCAS analysis. But the devil may be in the detail. As the technical notes reveal, some applicants were excluded from the analysis, with potentially huge consequences for the results.
One exclusion likely to be important is of those applicants who applied before October 15. This means that the UCAS analysis excludes all applicants who included Oxford or Cambridge among their five application choices as October 15 is the cut-off date for applying to these universities. This is despite the fact that these universities are undoubtedly “high-tariff” – the highly academically selective institutions that are the focus of this UCAS analysis.
It also means that the UCAS analysis excludes everyone who applied to courses in medicine and dentistry (for which the application deadline is also October 15), despite the fact that these are also clearly “high-tariff” courses, and ones that attract many ethnic minority applicants.
A second potentially important exclusion is of applicants whose best three predicted A-level grades were all A*s. UCAS says this “avoid[s] predicted ability being censored by the measure”. In other words, UCAS thinks that applicants predicted A*A*A* are a heterogenous group in terms of predicted achievement, and so including them in the analysis might give a misleading impression of the importance of predicted grades for offer rates. This may be so, but it seems perverse to exclude applicants predicted to get the highest set of grades in an analysis that claims to be about admission to highly selective universities, not least when there are other ways of modelling the data which don’t require such exclusions.
Not detailed enough
The UCAS analysis is not only based on a selected subgroup of applicants, it is also not nearly detailed enough. It aggregates all ethnic minorities into a single category, problematic because previous research has shown that some ethnic minority groups fare much worse than others with respect to offer rates. Aggregating ethnic minorities in this way is likely to obscure important differences between ethnic minority groups.
The fact that the UCAS analysis reports “average offer rates” for broad degree subject areas – for example computer sciences, or law – is also problematic. My own research suggests that ethnic differences in offer rates are relatively small when ethnic minorities make up a small proportion of applicants to a course, but are much larger for particular courses at particular institutions that attract high proportions of ethnic minority applicants. The tyranny of averages means that large ethnic inequalities in offer rates for particular courses at particular institutions could be easily overlooked.
UCAS has published some of the underlying data on which their analysis is based – but this does not contain enough detail for anyone else to replicate their analysis. Nor is it detailed enough to enable others to undertake a deeper and more nuanced analysis, starting with one that distinguishes between ethnic minority groups.
More data should be shared
UCAS recently agreed to share more detailed de-identified data – which doesn’t include details of names or date of birth – with external researchers using the safe and secure infrastructure provided by the Administrative Data Research Network. They had been encouraged to do so by the minister in charge of higher education, Jo Johnson.
This is a welcome development. However, UCAS insists that it will only share de-identified data for those applicants who explicitly consent to this. This seems an honourable intention but may result in a dataset that is very unrepresentative of all students if a significant number of applicants opt-out of data sharing. This would of course make it very difficult to draw reliable conclusions about the fairness or otherwise of university admissions.
A likely consequence of UCAS’s policy on data sharing is the provision of an unnecessarily uncomprehensive and unrepresentative dataset from which few if any reliable conclusions about fair admissions can be drawn.