Twelve academics today received awards as the most prolific and most-cited researchers in fields deemed to be strong areas for Australian research, at a ceremony in Canberra.
However, the prestigious Thomson Reuters Australia Citation and Innovation Awards has raised the eyebrows of education and humanities academics who argue that the system used to calculate those fields and their best and brightest scholars is - like the Federal Government’s National Research Priorities - skewed to the sciences.
Thomson Reuter’s Senior Director for IP and Science in Australia and New Zealand, Jeroen Prinsen, said that while the awards were intended to be be cross-disciplinary, the claims of a methodological bias against the humanities were correct.
“It is a challenge. The humanities does have a different citation patterns, so using one methodology to apply equally across the board wouldn’t give you appropriate results. That is a correct ascertation of it,” Mr Prinsen said.
The citation section of the Thomson Reuters Australia Citation and Innovation Awards, which had no good news for fields including the arts and education, went to the following scholars:
• Professor Karl Glazebrook (Astronomy & Astrophysics) - Swinburne University of Technology
• Professor Terry Speed (Biochemistry & Molecular Biology) - Walter and Eliza Hall Institute of Medical Research (WEHI)
• Dr Jane Elith (Biodiversity Conservation) - University of Melbourne
• Professor Ove Hoegh-Guildberg (Ecology) - University of Queensland
• Dr Thomas Wiedmann (Economics) – CSIRO
• Dr Shaobin Wang (Environmental Studies) - Curtin University of Technology
• Professor Kurt Lambeck (Geosciences) - Australian National University
• Professor Fabienne Mackay (Immunology) - Monash University
• Professor Greg Stuart (Neurosciences) - Australian National University
• Professor Rana Munns (Plant Sciences) – CSIRO
• Professor Colin MacLeod (Psychology) - University of Western Australia
• Dr Evie Leslie (Public, Environmental & Occupational Health) - Flinders University
Thomson Reuters' online account of the methodology it uses states:
This exploration of Australian research began with an identification of fields which were strengths for the country, by a combination of three criteria: first, how many papers Australian researchers published; second, how great a proportion of the field globally Australian research represented; and third, the level of impact, measured in citations, relative to global performance in the field.
After the fields of apparent strength were identified, Thomson Reuters then identified scholars who shone in those disciplines, again via output and citation metrics:
All Australian-affiliated authors from papers published between 2002 and October 2011, in journals indexed in Thomson Reuters’ Web of Science, were reviewed. A shortlist of researchers in each field was established by finding those authors who had published a given number of papers, each of which had been cited that given number of times – a metric called the h-index. This meant that only researchers who had made a strong contribution in terms of both quality and quantity were considered.
Such methods are blind to the way humanities works and will necessarily push its disciplines down the list of fields of strength, said the executive director of the Australian Academy of the Humanities, Christina Parolin.
“What’s reflected most in these awards is that they’re based on a particular form of publishing practice that just doesn’t really exist in the humanities,” Dr Parolin said.
“Journal articles aren’t seen as the main form of publishing output for the humanities; it’s still the book, the monograph, that is the principal output. No citation indices can capture that form of publishing. Yes, humanities academics publish in journals, but the principal output remains the scholarly monograph,” she said.
Furthermore, the teamwork common to the sciences puts researchers names into play much more often than does the more isolate nature of humanities research and writing, Dr Parolin said.
“In the sciences, you’ll get a research team or a lab and everybody’s name will get on the article, from the PhD students through to the head of the research team. So in the sciences you get people who can claim - not illegitimately - 15 journal articles a year. That practice just doesn’t exist in the humanities, where even though there are collaborations you very rarely see a team of 10 authors for an article. There are very marked differences in publishing practices,” she said.
This was why the Australian Research Council does not rely on methods such as Thomson Reuters uses when assessing the quality of research, Dr Parolin said. “Citations doesn’t come into play in the humanities,” she said.
“Most of our work is done in the public sphere: the best work is done through The Conversation, where we have enormous reach to the general public. In my field, if I have five or 10 citations over five years I’m doing really, really well. But I might have 15,000 readers at The Conversation and I know I’m having huge impact,” Dr Zyngier said.
Dr Zyngier cited his most recent article at The Conversation, saying that in less than 24 hours since publishing it had drawn several hundred readers and had prompted emails from schools around Australia giving feedback.
“That kind of impact is never in any of the metrics. Citation indices are all skewed towards the sciences,” Dr Zyngier said.
Both Dr Parolin and Dr Zyngier questioned the
makeup of the Commonwealth’s National Research Priorities, from which the humanities and education are largely excluded.
Dr Zyngier said that this marginalisation had unfortunate consequences. “It comes out in the Australian Research Council grants - education gets very few. As an early career researcher, which I am, only three Discovery early career research grants were awarded in education in the first round.”
“Without education what we’d have is a dumb society. You don’t get scientific research, you don’t get breakthroughs in medicine, you don’t get new architecture and new engineering, if children aren’t able to understand what they need to know early on in public schools,” Dr Zyngier said. “Education is spoken about as a priority by our politians, but that’s not reflected in grants for research.”
Thomson Reuters would be happy to work with the humanities sector to try and develop a better method of identifying highly esteemed research in their disciplines, Mr Prinsen said.
Meanwhile, the Commonwealth has started a process to “refresh” the National Research Priorities, and Dr Parolin said she hoped the refreshment would deliver a solid place in them to the humanities.
Comments welcome below.