Sections

Services

Information

UK United Kingdom

Strength in numbers: do ERA rankings add up for universities?

The Excellence in Research for Australia Initiative (ERA) is the federal government’s latest attempt to quantify the “excellence” (or otherwise) of Australian researchers. And just a few short weeks ago…

How best to quantify the performance of Australian researchers? Storyvillegirl

The Excellence in Research for Australia Initiative (ERA) is the federal government’s latest attempt to quantify the “excellence” (or otherwise) of Australian researchers.

And just a few short weeks ago submissions closed for ERA 2012, to the great relief of university research offices around Australia.

Unlike the dreaded Higher Education Research Data Collection (HERDC) exercise – which rewards universities for pumping out as many papers as possible, even if they’re of low impact - many of the aims of the ERA process are to be welcomed. ERA combines an assessment of quality and quantity.

And that’s a good thing, because bad research isn’t worth doing (or funding).

Leo Reynolds

In my experience, Australian academics used to be uniformly of the opinion their research was of world class; or that, if it wasn’t, it was rapidly getting better; or that, if it wasn’t for all their teaching and administrative duties, they’d be awesome.

And then ERA 2010 came along.

In ERA 2010 the Group of Eight (GO8) universities, which get the bulk of competitive grant funding, did pretty well. Lots of ERA 4s and 5s with a few blemishes but, on the whole, the results were reassuring for the government (and the taxpayer).

The emerging universities didn’t do so well. The odd 5 and 4, some 3s, many 2s, and even (gulp) 1s.

In case you haven’t worked it out yet, an ERA 5 rating means well above world standard; ERA 1 means the opposite.

Universities brag about high ERA scores in the same way first-year undergraduates have their Australian Tertiary Admissions Rank (ATAR) scores tattooed to their foreheads.

A change for the better

ERA 2010 measured the period 2003-2008 inclusive, and the newer research institutions might argue their staff and outputs, as of 2012, are better than from the middle of last decade. To some extent they are probably right.

Leo Reynolds

The only problem is that when the next change of government occurs we can’t possibly expect the Coalition to want to keep the current system, because that would be admitting the Labor government did something right. Which, as improbable as it sounds, has many of us wondering whether perhaps this really is the end of (an) ERA?

Fortunately, ERA can be greatly improved. So apart from coming up with a different acronym, how could the Coalition change ERA for the better?

Senator Mason take note!

Hidden codes

When the ERA 2012 assessment is completed later this year, we’ll all know what the Australian Research Council (ARC) thinks of our universities, succinctly distilled into a single number, between 1 and 5 (whole numbers only) for each research grouping.

And therein lies the problem: one whole number (or integer) for each field of research. So although there may be 100 researchers in a given discipline area at any given university, their ERA ranking will be represented by a single digit, regardless of each individual’s own score.

Furthermore, it’s not hard to imagine that a lot of time and effort has been spent by university administrators cleverly “hiding” their poorer researchers and outputs in “ballast” four-digit codes.

Leo Reynolds

Some research outputs span areas, and can be submitted under different research codes. So if a university takes all of the low-quality outputs and places them into a sacrificial code, and writes off the relevant authors, it can strengthen other areas that it aims for a high score in.

Since ERA doesn’t report on the dimension of the research grouping, it becomes tempting to maximise the number of highly-ranked disciplines, even if they are tiny.

On paper, four 5s and a 1 looks a lot better than two 4s and two 2s, which might be achieved via some relabelling of some research outputs.

Of course the silliness here is that the gross output of the university doesn’t change just because you’ve managed to hide your poorly-performing researchers in a few codes you are prepared to sacrifice.

Doesn’t the government want universities to be doing more than manipulating research classifications? Hopefully, yes!

A flawed procedure

For some strange reason, as with electrons orbiting atomic nuclei, the ARC wants to force collections of researchers into “quantum states”. So although the raw scores might have left your ERA grouping at 4.49, you’ll probably get truncated back to a 4.

One more publication might have made you a 4.51 and delivered the magical 5 rating!

Ideally the ARC could publish a histogram of each individual’s own rating within a discipline, between 1.0 and 5.0, and then averages, standard deviations, medians, maybe even skewness. This would avoid quantisation errors, and allow a truer representation of research excellence from each discipline.

Leo Reynolds

Extending this further, one could imagine a dot on a scatter diagram that showed the impact of each publication within a code on the y-axis and the individual researchers on the x-axis.

Then we could get a feeling of whether a group’s outputs were dominated by one individual, dragged down by a few part-timers, or of high quality but limited in number.

But the deeper you start to look at these measures, the more you realise the flaws.

In some areas impact is relatively easy to measure from citations on short timescales but, alas, not in all.

Within a discipline, some areas and activities cite extremely well, and in others, such as instrumentation, not so well. In some disciplines, such as mathematics, citations are almost meaningless.

And once we start talking about “esteem” factors – such as editorial boards, members of the academy, the relative worth of a Nobel Prize to a Fellow of the Royal Society – it all starts getting a bit arbitrary.

How many Nature papers are the equivalent of a Nobel Prize or membership of an editorial board? Do two ten-citation papers become equivalent to one 20-citation paper? Is a 5 researcher plus a 1 researcher equal to two 3 researchers?

Tough measures

Ultimately, the government might have to accept that, as with the momentum and position of a subatomic particle, research excellence is impossible to quantify.

And yet in 2012 a portion of the university’s income to help with the indirect costs of research, the so-called “Sustainable Research Excellence Threshold 2 funding”, used the ERA results to allocate funding. ERA 5s being seven-times the value of a 3 and 1s and 2s being worthless.

Leo Reynolds

The problem for the younger institutions was that an ERA 5 at a poorly performing “average research income per EFT” university got much less funding than an ERA 5 at a uniformly excellent one.

Why? Well the total dollar amount for each ERA grouping was multiplied by the average category 1 research income per academic, so poor areas contributed nothing, and diluted the income rewards of the stronger ones.

So much for the incentive for the fledgling research universities to be rewarded for concentrating their research efforts!

This meant the Go8 universities cleaned up with this funding change, as the majority of their codes were ERA 4 and above, and their average research income much higher. If one differences the SRE 2012 and 2011 numbers, we see that the University of Queensland pocketed an extra A$5.8 million this year – the Queensland University of Technology just A$15,000.

Teaching kills

The other, major problem with the whole ERA concept is that most university academics teach. And this, ladies and gentlemen, is the killer. Why? Because, unlike research quality, the government doesn’t seem to mind what your teaching quality is like.

Leo Reynolds

University funding is independent of teaching quality.

And quality teaching takes time, and that time makes it harder to do research, especially quality research.

So the incentive is clear: if you are going to do research, heavily concentrate in one area and minimise any time those people spend on teaching.

Don’t set teaching assignments such as essays because they take time to mark. Go for multiple choice answers instead and perhaps lump most of the assessment into the end-of-term exam?

That will get your Q-index (a daily measure of an individual’s research worth used at the University of Queensland) firing on all cylinders!

It’s also worth using sessional staff or teaching-only academics wherever possible to not dilute your research effort.

Just do what you can to propel your university up the research rankings and gain as many ERA 5s as possible!

That will make your VC truly happy, and might even allow them to keep your music schools and arts faculties open.

Professor Matthew Bailes is a member of the ERA-5 rated Centre for Astrophysics and Supercomputing at the Swinburne University of Technology.

Join the conversation

56 Comments sorted by

  1. Alex Holcombe

    logged in via Twitter

    Thanks for this. Frightening to see our universities increasingly caught up in these perverse games.

    report
  2. rory robertson

    logged in via email @gmail.com

    Good morning all. Speaking of excellence in research, is anyone else concerned about the lack of genuine quality control in science research at the University of Sydney. The Vice-Chancellor seemed unconcerned about various serious errors - including a spectacularly false conclusion - featured in a recent obesity study: http://www.australianparadox.com/pdf/SydneyUniVC%20LETTER070612.pdf

    Regards,
    Rory

    report
  3. Michael J. Lew

    Senior Lecturer, Pharmacology and Therapeutics at University of Melbourne

    Am I alone in thinking that the managements of universities in Australia are letting us down by playing along with these ridiculous bureaucratic games?

    It seems fairly clear that the measurements of research quality are nearly useless the funding outcomes are ill-suited to rational purposes and the system invites gaming.

    Why did the whole university sector not simply say "No"? Have we been corporatised top the extent that the various Vice Chancellors view each other as competitors or enemies?

    report
    1. Dennis Alexander

      logged in via LinkedIn

      In reply to Michael J. Lew

      I am sorry to say Michael that Matthew is really only dealing with the prospective tip of an iceberg. If one looks at the news of the World Rankings in the USA, "gaming" the ranking/system is de rigeur - not to do so would be to disadvantage one's institution.

      The core problem is that we, as a society (as opposed to individuals), have bought or been saddled with the myth of performance measurement and reward as an economic necessity. The myth works fine for dogs, laboratory rats and sports people…

      Read more
    2. Michael Ashley

      Professor of Astrophysics at UNSW Australia

      In reply to Dennis Alexander

      Excellent points Dennis. Our universities were doing just fine before this obsession with measurement/reward came in vogue. I'm not aware of any of my colleagues at UNSW who are motivated by KPIs.

      And thanks Matthew for an interesting article.

      report
    1. Diane Lester

      logged in via Facebook

      In reply to Matthew Bailes

      While I agree it’s the research bureaucrats who are ultimately the problem, researchers themselves could bring about change. If they simply refused to review papers for subscription-based journals the whole outdated (and unethical, I might add) publishing system would collapse overnight.

      report
    2. Diane Lester

      logged in via Facebook

      In reply to Matthew Bailes

      Are you seriously saying that busy publicly funded academics should continue to give away and review content for free for the powerful wealthy private publishers so that the publishers don’t fold?
      If researchers stopped supporting subscription-based journals the publishers would ‘innovate’ and migrate to the Open Access business model. Even if the publishers did fold the databases could not be closed off because many library licences include perpetual access. Besides, there is the CLOCKSS initiative…

      Read more
    3. Daryl Adair

      Associate Professor of Sport Management at University of Technology, Sydney

      In reply to Diane Lester

      I must have reviewed 20 papers this year for journals, most of which are for commercial publishers. This is something we academics do as a service to the "profession". Perhaps we should be charging them a fee-for-service, which could then be put into university research accounts and used to sustain our own scholarly endeavours? At present I receive kudos from colleagues around the world for reviewing papers, but this is not understood - under ERA - as a demonstration of my contribution to research…

      Read more
  4. Gavin Moodie
    Gavin Moodie is a Friend of The Conversation.

    Adjunct professor at RMIT University

    As with previous posters, I agree that this is a useful piece. However, its usefulness is reduced by raising too many big issues in so short a compass, resulting in the conflation of several issues.

    Yes, a big problem in Australia and many other countries is the lack of a rigorous measure of the quality of teaching and/or student attainment. The Office of Learning and Teaching and its predecessor the Australian Learning and Teaching Council supported some useful work on this, but a much bigger…

    Read more
  5. Jack Bowers

    Learning Adviser

    Good article. As someone who focuses primarily on teaching, I feel like I'm wasting my time and doing nothing for my career. But I really enjoy teaching, and I love teaching when I'm given the space and time to do a good job.

    And our students deserve it!

    report
  6. Dr. Chandana Hewege

    logged in via LinkedIn

    What a wonderful piece of critique Mathew! Performance measures are to be taken seriously as they modify, direct, pivot, and distort behaviours of actors involved in the process. It is said that ‘what gets measured is rewarded. The rewarded behaviour is repeated. When behaviours are repeated, they become routines and habits’. Therefore, it is of paramount importance that we set our measures/rankings in a manner that they modify our behaviour to reach the ‘desired future state’ of our universities…

    Read more
  7. Igor Bray

    Head of Physics, Curtin University

    Well done Matthew. This reminds of the wonderful quote attributed to Einstein: "Not everything that can be counted counts, and not everything that counts can be counted". Another quote that applies is: "You do not fatten a pig by weighing it". The ERA process is rather clumsy, with costs being a considerable driver. Greater accuracy could have been achieved if the FOR codes assessed were at the six digit level rather than four. However, the cost of this activity would be much greater. Would it be…

    Read more
    1. Michael Ashley

      Professor of Astrophysics at UNSW Australia

      In reply to Gavin Moodie

      Gavin, do you know how self-citations are determined? E.g., if you have two papers, each with several hundred authors, but only one author in common, and one cites the other, is this self-citation? One would hope not.

      report
    2. Igor Bray

      Head of Physics, Curtin University

      In reply to Gavin Moodie

      Gavin, can you point to a document where this is stated? I recall Margaret Sheil explicitly saying that self-citations will not be excluded. The reason given is that this would make a complex situation even more complex. I asked a current ERA panel member, and they also were unaware of your claim.

      report
    3. Gavin Moodie
      Gavin Moodie is a Friend of The Conversation.

      Adjunct professor at RMIT University

      In reply to Igor Bray

      Sorry, I can't. I was relying on my recollection of Scopus' method for analysing citations for a different purpose, or perhaps on Thomson Reuters' description of its method for calculating Eigenfactor scores. I subsequently looked thru the ARC's materials and could see no thorough explanation of the method for collecting and analysing citations and couldn't find anything relevant on Scopus' web site. So I'm probably wrong, for which I apologise.

      ‘In earlier studies by the authors, basic regularities…

      Read more
  8. Don Aitkin

    writer, speaker and teacher

    In 2014 it will be fifty years since the ARGC, the predecessors of the ARC, was established, and in that time three basic facts have been true:

    (1) University research is based on public funding that could go to any number of other worthwhile ends.

    (2) The spending of that money has to be governed by rules.

    (3) Researchers study the rules intently, and construct what they propose in terms of the current rules to get the money they need to do the research they want to do.

    I can remember…

    Read more
  9. Frank Marino

    logged in via Facebook

    Finally, someone prepared to be critical of this messy and time consuming ERA. The question I want answered is why one of the most powerful groups in Australia (The Vice Chancellors) acquiesced to this? What would happen in a similar corporation if the minor stakeholder called the shots? They would be told to go away and let the major shareholders make the big decisions.

    report
    1. Gavin Moodie
      Gavin Moodie is a Friend of The Conversation.

      Adjunct professor at RMIT University

      In reply to Frank Marino

      I suspect every vice chancellor would be surprised that they form 'one of the most powerful groups in Australia'! However, most not only acquiesced but supported the introduction of ERA, for 2 reasons.

      Coalition and Labor ministers responsible for research insisted that there was no prospect of getting more funding for research unless there were a rigorous assessment of its quality. So the very substantial additional funds that former minister Kim Carr won for university research in the sustainable…

      Read more
  10. Kerry Carrington

    Professor, Head of School of Justice at Queensland University of Technology

    Matthew,

    Thank you for a refreshing deconstruction of the ERA exercise and for pointing out how it favours the big 8 well established universities over the new comers.

    I still wonder why they scrapped the impact measures as trialed under the RQF. While I accept that governments need to have measures to assure themselves that they research is worthwhile funding, suring impact can be measured by much more than citations.

    What about policy impact? Impact on public debate (i.e. the measures for the conversation just might count towards something then)?
    Impact on the real lives and livihoods of real people, real communities, real law, really anything more than just who else has cited your article. The narrowness of this impact measure is truly astounding and would not pass the taxi driver test, that's for sure.

    Kerry

    report
    1. Gavin Moodie
      Gavin Moodie is a Friend of The Conversation.

      Adjunct professor at RMIT University

      In reply to Kerry Carrington

      The Australian Government did not proceed with assessing research impact because the impact measures currently available require much more work than measures of research quality and their method isn't well established.

      The 5 Australian Technology Network universities, 4 of the Group of 8 universities and 3 other universities are currently trialling the research impact measures that will be used in the UK's research excellence framework in 2014. See

      https://theconversation.edu.au/universities-to-explain-benefit-of-research-to-end-users-7478

      report
  11. Alex Smuts

    Academic

    There are many other problems with this system. Humanities and social science disciplines in particular are often internally divided into schools each with their own set of methods, values and purposes, and judgment of the quality of work may differ quite markedly depending on which school the assessor and assessed comes from. (E.g. an analytic philosopher judging Derrida-style work). In fact there is often no agreement possible over "quality" at least in the social sciences and humanities. If there…

    Read more
    1. Don Aitkin

      writer, speaker and teacher

      In reply to Alex Smuts

      These are good points, Alex. But it's still public money that could go anywhere. So your task is to design a system that gives the good outcome that you want, is acceptable to other disciplines, and satisfies accountability.

      It's not easy!

      report
    2. Gavin Moodie
      Gavin Moodie is a Friend of The Conversation.

      Adjunct professor at RMIT University

      In reply to Alex Smuts

      To argue that 'there is often no agreement possible over "quality" at least in the social sciences and humanities' leads ultimately to solipsism. How would a university appointments committee, promotions committee or internal grants committee know how to appoint, promote or fund anyone in the social sciences and humanities?

      Universities can't be self referential refuges of people engaged in an arcane activity like the players of Hesse's Glass Bead Game.

      report
    3. Alex Smuts

      Academic

      In reply to Gavin Moodie

      Strong departments who can pick and choose and who are not intimidated by audit bureaucracy don't hire according to some objective notion of quality, which, as I say, doesn't exist. They hire in relation to how people will extend a particular intellectual culture: particular departments have particular cultures, e.g. Chicago economics; in the old days Sydney philosophy. That's how it still works in the best UK and US universities.

      report
    4. Alex Smuts

      Academic

      In reply to Don Aitkin

      There need to be limits on what satisfying accountability means: at current levels and with current audit mechanisms, public accountability shades into something like state control, with ultimately totalitarian implications. That sounds extreme but actually I don't think it is an exaggeration. As I say, the academy needs a fair degree of autonomy to prosper. The public is a chimera here: it doesn't really give a damn about university management, does it? The current policy settings are determined by various other factors, amongst which something like fashion plays a big part.

      report
    5. Gavin Moodie
      Gavin Moodie is a Friend of The Conversation.

      Adjunct professor at RMIT University

      In reply to Alex Smuts

      But how would one know whether a department with one intellectual culture should be allocated an additional member of staff or more research funding than a department with another intellectual culture?

      It is possible to claim that there are criteria for comparing performance in one discipline with performance in another without insisting that the criteria are objective.

      report
    6. Alex Smuts

      Academic

      In reply to Gavin Moodie

      I think the answer to this differs across continents. In the US where departments still exist (they don't in Australia) deans add or take away lines to particular departments for a number of reasons, none of which relate to performance criteria as measured in the English or Australian systems. Among these reasons would be an informal sense of the department's reputation among peer institutions; success in placing grad students; success in attracting and keeping strong hires and high quality grad…

      Read more
    7. Gavin Moodie
      Gavin Moodie is a Friend of The Conversation.

      Adjunct professor at RMIT University

      In reply to Alex Smuts

      Departments are retained in some Australian and British universities.

      You are conceding or claiming that there is little rational basis for allocating resources to research in many disciplines, which is a very weak ground for justifying research time and funding for those disciplines to the people providing the funding for it, taxpayers.

      It would be far better to discard epistemological relativism, which is in any case barren on other grounds.

      report
    8. Alex Smuts

      Academic

      In reply to Gavin Moodie

      I don't concede, nor does anything I say imply, that there is little rational basis for allocating resources to research, though we may disagree exactly on what "rational' (and what 'research') means. You are resorting to cheap and dismissive rhetoric! I think the continuing presence of academic disciplines in our society is extraordinarily important to those societies for reasons that are various and complex, and not all of which depend on those disciplines being public utilities or open to the…

      Read more
    9. Gavin Moodie
      Gavin Moodie is a Friend of The Conversation.

      Adjunct professor at RMIT University

      In reply to Alex Smuts

      'And of course in actual fact politics etc play an important role in making these decisions. . . . I suspect that in Australian institutions, academic politics plays a still bigger role in these kinds of decisions than it does in the States . . .'.

      The procedure for justifying expenditure to taxpayers is so obvious that it hardly bears spelling out. Each Australian Government minister responsible for research since Barry Jones has stated explicitly their need to justify research funding to their…

      Read more
    10. Alex Smuts

      Academic

      In reply to Gavin Moodie

      "The procedure for justifying expenditure to taxpayers is so obvious that it hardly bears spelling out." That is not true and the fact that Australian government ministers might say and think it doesn't make it so. Words like 'taxpayer', 'quality' etc are abstractions or fictions used to simplify so as to "manage" complex realities. I am quite aware of how the system works and have a sense of why we have the system we now have. I have lived through important changes in Australia, as I suppose have…

      Read more
    11. Gavin Moodie
      Gavin Moodie is a Friend of The Conversation.

      Adjunct professor at RMIT University

      In reply to Alex Smuts

      Almost all of the US colleges and universities which have more autonomy than Australian public universities are private. Many US public colleges and universities have considerably less autonomy than Australian public universities. I don't think it is fair or even appropriate to expect the Australian Government to grant Australian public universities the level of autonomy enjoyed by private US colleges and universities.

      report
    12. Alex Smuts

      Academic

      In reply to Gavin Moodie

      Well that is where we disagree. And why not grant more autonomy to universities, if that is what is required for them to prosper? That's how it was in the past in the UK and Australia after all. It seems to me that you are committed to a particular form of statism, whose final implications are, in fact, totalitarian. Totalitarianism after all names that form of governmentality that denies institutional autonomies and doesn't value independent and critical thought etc. As you may be aware many…

      Read more
    13. Don Aitkin

      writer, speaker and teacher

      In reply to Alex Smuts

      I think you are missing the point — or perhaps I didn't express myself well. On the whole I agree with Gavin: public funds are contestable every day inside the government system. Gavin is right about a Minister's need to argue his case, and I would put the starting point much earlier than Barry Jones.

      Once no one knew much about universities, but it was agreed that they were important, because they produced doctors and teachers and engineers, and we had to have them; and their teachers argued…

      Read more
    14. Gavin Moodie
      Gavin Moodie is a Friend of The Conversation.

      Adjunct professor at RMIT University

      In reply to Alex Smuts

      I think that it is at least arguable that Australia grants its private universities as much autonomy as the US grants its private universities, and that it grants its public universities more autonomy than most US States grant their public universities.

      I suggest that currently in Australia there is a trade off between autonomy and funding. Australian universities had more autonomy from 1974 to 1988, but during that period Australian governments spent much less money on universities. Before 1974 and particularly before WWII Australian universities were supervised very closely by State governments which determined the fields in which chairs were established and sometimes their occupants, but that is probably too early to be a useful comparison to current times.

      report
    15. Michael Ashley

      Professor of Astrophysics at UNSW Australia

      In reply to Gavin Moodie

      For what it is worth, I very much agree with what Alex Smuts has been saying here. Trying to quantify academic output, and having accountability to taxpayers as a primary goal, will not lead to excellent universities. The world's top 10 universities did not gain their reputations through such a method.

      report
    16. Alex Smuts

      Academic

      In reply to Gavin Moodie

      Yes. I think it is right that the government has traded more funding (and at the same time democratization of the university system) for less autonomy since the seventies. The funding per student didn't always appear though, did it? That, of course, doesn't mean that that policy has been successful either. But I agree too with the implication of your remarks that the extension of a university education across, let's say, half the population (which is itself desirable in my view) makes guarantees of autonomy more administratively and politically difficult. The answer might be to hierarchize the system in more open ways, and, as it were, distribute autonomy unevenly, and that too is politically and administrively difficult but not I think impossible.

      I don't know how much influence State governments had chair appointments in the past: by the time I arrived in the system just before the Dawkins reforms they were not playing that role as I recall.

      report
    17. Alex Smuts

      Academic

      In reply to Don Aitkin

      Yes. We are talking at cross purposes here. I am idealistically trying to think about how the system might be radically changed so as to enable better universities, or really better disciplines ("disciplines that can prosper") given the real and greivious failings of the forms of governmentality that we have. You seem to be thinking about how the current system works.
      As you and Gavin know, universities today are complex beasts with various purposes and constituencies. In particular different disciplines…

      Read more
    18. Don Aitkin

      writer, speaker and teacher

      In reply to Alex Smuts

      It depends on what you mean by 'autonomy'. When CTEC was in charge universities and CAEs expanded and diversified only as CTEC agreed. CTEC came to an end in 1988, and that is no longer the case. Universities can do pretty much what they like. Now universities are more able to set their own directions, but they are also held accountable through a much more complex set of rules — they are, and have to be, 'compliant'. This is one reason why there are more administrative staff than there used to be. And universities are expected to note government 'priorities' — there were once eight separate priorities with respect to students (Indigenous, women, rural and regional, handicapped, and so on).

      In my experience, both real (my own) and secondhand, I have never heard of a State Government's interesting itself in a chair appointment, though I understand that the Queensland Government once took a keen interest in the appointment of vice-chancellors.

      report
    19. Don Aitkin

      writer, speaker and teacher

      In reply to Alex Smuts

      Thanks, and I agree we are rather at cross purposes. Indeed, I agree with much of what you have written and can only say that I have been wrestling with these questions at practical level since the early 1980s. I am always trying to get the system to work better, and while I think it is often intellectually useful to think in terms of a clean slate, the change we can make today, or tomorrow, is to the existing system. As an incrementalist, I look for the least expensive, most effective small change…

      Read more
    20. Alex Smuts

      Academic

      In reply to Don Aitkin

      What I mean by autonomy is precisely not being accountable in the ways you describe. And not being driven into the cycle of emulation and fashion of managerial modes which universities pretty much have to join in Australia. Automony is also not being forced into the competitive relation with other institutions. Universities produce disciplines which are in the keeping of teachers, researchers and scholars around the world for whom institutional affiliations are, and should be, secondary. That is…

      Read more
    21. Don Aitkin

      writer, speaker and teacher

      In reply to Alex Smuts

      After reading your thoughtful and well-written contributions here it is still not clear to me what you want, though it sounds like what a former head of Telecom labs once said, that universities simply wanted the cheque pushed under the door and the giver's then creeping silently away.

      You need to come up with a design of what you want, I think, a design that you can explain both to those working in the system and those who will be providing the money. I wish you well.

      report
    22. Michael Ashley

      Professor of Astrophysics at UNSW Australia

      In reply to Don Aitkin

      Here is an analogy:

      Suppose that the Government is interesting in funding a ballet company.

      Model 1: the Government gives the ballet company a $1M cheque each x-months. The ballet company employs 10 dancers, 5 production crew, and 1 manager. It is very successful and widely regarded as the best in the country.

      Model 2: the Government gives the ballet company the same $1M cheque each x-months. However, to ensure that the taxpayer is getting value for money, and to help the Minister argue…

      Read more
    23. Alex Smuts

      Academic

      In reply to Gavin Moodie

      I very much like Michael's analogy. If Gavin or anyone else has difficulties figuring out what the better universities are (most of us who are inside them, I think, don't) it shows that the way to go about clearing those difficulties up is not to carry out audits whose procedures and demands alter and slow down what universities properly do.
      And am I wrong in detecting a certain moralism or resentment in Gavin's language? Prima donnas?? It as if audits were a kind of punishment ...Many of my colleagues do think the current governmental/managerial culture does in part express resentment against the academy....I myself am not so sure, but comments like this make one wonder....

      report
    24. Gavin Moodie
      Gavin Moodie is a Friend of The Conversation.

      Adjunct professor at RMIT University

      In reply to Alex Smuts

      Sigh. My point is not that I have difficulty identifying 'the better universities', but that many more are identified as 'better' than the country can possibly fund to the level they aspire. The country has to have a way of choosing between them, or more correctly, of choosing between the research centres and departments promoted as being better.

      Most of the champions of the 'better' universities, departments and centres are their own researchers. I used the term 'prima donna' as a play on leading performing artists who have no doubt about their worth - referring to Ashley's ballet company. But I promise not to attempt banter in future posts.

      report
    25. Alex Smuts

      Academic

      In reply to Gavin Moodie

      Point taken. I think I didn't get your prima donna joke. Sorry. (I am actually in Germany right now, and am only waking up.)

      The problems with your analysis is that it relates not so much to the real world as to an idealized state-administered system.

      In the real world universities exist with a certain number of faculties, centres and schools (in the old days departments) etc, many with quite long histories and with particular cultures. Universities and departments are approximately hierarchized…

      Read more
    26. Don Aitkin

      writer, speaker and teacher

      In reply to Gavin Moodie

      I must have missed some of these exchanges. Ashley only has two models. I could give him half a dozen more, from experience. In model 3 the artists fight amongst themselves, and embarrass those otherwise sympathetic; in model 4 despite all this money they are not regarded as the best in the country, and attention shifts to a struggling and underfunded group that does really interesting things; in model 5 the director spends too much money on bringing in a prime donna assoluta from overseas, others…

      Read more
    27. Gavin Moodie
      Gavin Moodie is a Friend of The Conversation.

      Adjunct professor at RMIT University

      In reply to Alex Smuts

      I didn't think we were discussing project grants awarded by the ARC, NHMRC and others, but institutions' block research grants. There has to be a way of allocating those in different amounts to different institutions.

      I think institutionalisation of research is important, but I don't think it should be substantially self referential and self perpetuating. And regardless of what I prefer, I don't think there is a realistic prospect of the Australian Government continuing to provide substantial block research grants to universities without some external demonstration of research quality in a way acceptable to the minister and explainable by them to their fellow ministers and to their constituents.

      report
    28. Alex Smuts

      Academic

      In reply to Gavin Moodie

      What Australian government ministers do and don't do is open to change. (Once they discussed reserve bank interest rate settings!) So although I do agree there is no likely prospect of changes in this regard in the near future, I'd be surprised if one day more autonomy weren't granted to the universities.

      Ministers and civil servants can't do much about what you call "research quality", that is really up to the universities themselves. And once it is understood that universities better serve…

      Read more
  12. rory robertson

    logged in via email @gmail.com

    Good afternoon Associate Professor Adair,

    Thanks, Daryl, for sharing your recent experience in the mysterious "peer review" process that is supposed to be protecting our "scientific record". Yes, I agree that it would be good for academics to be paid for the time they devote to "quality control" for "peer reviewed" journals. Say maybe $200 per hour to encourage involvement, and reviewers names acknowledged at the end of the article to encourage accountability. For example:

    Reviewers…

    Read more