The Asian century white paper – released this week – presents a clear vision for the role of Australian universities in building links with Asia.

To underscore this, the government announced a new target to have ten Australian universities among the top 100 in the world by 2025. But instead of using a composite or average of all the university ranking measures, the paper suggests just one – the Shanghai Jiao Tong rankings.

But the Jiao Tong, also known as the Academic Ranking of World Universities (ARWU), has flaws and should not be relied upon as the sole measure for Australian universities.

Reliable rankings?

Ranking universities is notoriously difficult and many use different criteria to assess research output and quality. Around 70% of the evaluation in the ARWU rankings is robust, but the remaining 30% is based solely on the number of Fields and Nobel Prize winners.

On this criteria, younger universities can’t compete – they tend to have fewer prize-winners and are put at a real disadvantage. They are effectively judged on only .7 of the total score.

With so many young institutions in Australia, the government’s target for Australian universities using just ARWU is almost impossible.

Why younger universities miss out

There are several reasons why younger universities are less likely to have Nobel Prize winners.

First, the work that leads to these awards is often done decades before the prize is received. This is because it usually takes that long for the significance of the work to become clear. For example, one of the recipients of this year’s Nobel Prize in Medicine, John Gurdon, was recognised for a discovery he made in 1962.

Second, credence is only given to the graduating institution and the current institution. The connection between the site of one’s original graduation and the work that earns a Nobel Prize is often tenuous – the work is usually done quite some time afterwards, typically after much post-doctoral experience and a series of academic positions in a range of institutions.

Third, a maximum of three scientists can receive the awards, but often many more than three people are involved in the discovery. In great discoveries one typically stands on the shoulders of other people – some giants and some not. An ad hoc committee’s determination of who the three most significant contributors were may be somewhat arbitrary.

Ultimately the committee’s decision is also subjective. When a committee distinguishes one achievement as standing head and shoulders above others, it necessarily calls upon personal opinion as much as scientific data. This is obviously true for the literature prize but just as true in the sciences.

It’s also true that the Nobel Laureate may have a relatively weak link with the university to which they are attributed. Often the work is done in an affiliated institution, such as a medical research institution. The scientist may have little contact across the university or with teaching or students.

Finally, the inclusion of Nobel Prizes and Fields Medals is to some extent double dipping. Scientists’ output is already recognised in rankings scales through high-impact, high-citation papers and that indeed is how they should be judged.

Research favouritism

The use of measures such as a Nobel Prize favours older, better-established institutions that go back decades. It is very hard for younger institutions, even those that are breaking through very rapidly to great excellence, to have the same opportunities for Nobels.

The ARWU rankings put far too much emphasis on this single award. There are a large number of awards in academia and science, some of which have great prestige in themselves.

Nobel Prizes and Fields Medals are lagging indicators. At a time of such rapid change in human knowledge and science it is a major flaw to use a university ranking system that places such weight from measures from the past.

Of course, those who get Nobel Prizes fully deserve them but there are often many other candidates who could have been considered with equal validity.

We are a young country with young institutions – getting ten Australian Universities into the top 100 in the ARWU will be difficult. The government needs to consider a broader based assessment of university excellence, not just one problematic measure.