What do world university rankings actually mean?

In third place, Oxford University is the top UK institution in the World University Rankings 2014-15. Andrew Matthews/PA Archive

From the “best beaches” to the “best slice of pizza” to the best hospital to have cardiac surgery in, we are inundated with a seemingly never-ending series of reports ranking everything that can be ranked and even things that probably shouldn’t be.

Over the last few decades, schools and universities in many parts of the world have become targets of this ranking mania. Some of these are official, state sponsored rankings, such as those used in various performance-based funding formulas or, in the US, President Barack Obama’s yet undefined plan to rank American universities. Some are unofficial, such as the World University Rankings just released by the Times Higher Education (THE) magazine, or the Shanghai Rankings and US News & World Report ranking of American universities also recently released.

A real measure of quality?

But do all these rankings actually tell us anything substantive about the relative quality of universities? Should US or UK universities be concerned with their slippage in the latest THE rankings? There are 74 US universities on the new list – the most from any country – but this was down from 77 last year, and 60% of American institutions lost ground in the rankings. The UK, in second place, now has 29 universities in the top 200, two fewer than last year.

Do such rankings separate the chaff from the wheat, or the pure from the dangerous, or are they distractions and distortions to achieving a real equality of quality? Are they incentivising universities to perform better or actually inviting cynical “gaming” of the system?

Natural pecking order

Despite the various methodological caveats contained in the small print, in their more positivist moments, the agencies and magazines who produce university rankings contend that rankings do in fact unearth and quantify a naturally occurring pecking order. There is never much difference at the very top. In this year’s 2014-15 ranking, California Institute of Technology retained its top spot, followed by Harvard University in second and Oxford University in third.

Times Higher Education. Created with Datawrapper

University rankings are merely rational social science in action and, when done properly following sound methodological procedures, they constitute a valid and value free means for quantifying amorphous things called “excellence” or “world class.”

Some proponents extend this further by arguing that rankings also serve moral and political purposes, particularly in these neoliberal times of austerity and tight governmental budgets. They shame the slackers and arouse the complacent into “upping their game”.

In the end, ranking universities creates a “consumer sovereignty” where finicky students and in-demand professors use rankings to carefully determine which university they will attend or work at. All this creates quasi-market conditions for universities, fuelling competition and creative destruction. At the same time, and in terms that would have made British sociologist Herbert Spencer and the social Darwinists proud, it sheds the weaklings and hones the fittest.

Politics behind the metrics

But the metrics used in these rankings tend to hide all the “value work” that takes place underneath them and the political purposes they often serve. They also tend to greatly overvalue the “haves” at the expense of the “have nots”.

Rankings often conceal a purposeful politics hiding underneath the veneer of objectivity. Just as GDP uses selective measures to determine the underlying health of an economy and becomes factual as it is continually used, university rankings use items selected from a sea of possibilities to establish a higher education reality. The THE rankings use 13 indicators, including research income, citation impact and percentage of international students. Deliberately or not, they steer organisations toward particular political outcomes.

Rankings do have real effects on what universities value and how they operate. Their increased popularity also signals a profound historical change in our attitude toward equality and quality in public institutions. No longer can public institutions be “good enough” or “roughly equal”. Under the ethos of competition, some now must necessarily be better than others. There must always be winners who go up the rankings and losers who go down in order to stoke the fires of competition.

Pursuit of excellence

Certainly the case can be made that all our public institutions should be of the highest quality, and, dare I say, “excellent”. So the question must be asked: does the generation of competition in public enterprises actually make them better?

Rather than encouraging excellence they may actually extinguish it as universities turn from doing their job to “gaming”, and “juicing the results”. An ongoing testing scandal in Atlanta is a good example of how this can happen lower down the educational chain. In higher education, reports and research have examined what lengths colleges go to in order to boost their ranking.

So as administrators around the world now anxiously pore over the THE rankings to see if their university went up or down, it is perhaps important to keep in mind that rankings may not tell us much about actual quality, except in blunt and often obvious ways: the haves will always have. But they do say a great deal about the political direction of current global higher education reform.