Academics from a dozen universities will be required to explain to industry experts the economic and social value of hundreds of research projects from the past 20 years, under guidelines for a trial designed to measure the wider benefits of taxpayer-funded academic work.
Twelve universities – four from the Group of Eight, five of the Australian Technology Network of Universities, and the University of Newcastle, Charles Darwin University and the University of Tasmania – will take part in the Excellence in Innovation for Australia (EIA) trial, which is expected to influence funding decisions for future research projects.
Each university has until the end of August to prepare submissions for up to 20 case studies that show how their research provided “end-user benefits” in one of four “Socio-Economic Objective” (SEO) areas defined by the Australian Bureau of Statistics: defence (or national security), economic development, society and culture, and the environment.
The impact of the research is to be gauged over the past five years.
Crucially, the guidelines specify that the case studies “should be in layperson’s language and free from jargon” so they can be understood by people who are not experts in the research field.
The trial is part of a push to break from narrow measurements of research impact that count peer-reviewed publications in academic journals and citations in articles by other academics.
“In this exercise researchers are being asked to focus on a clearly identified impact or public good and then explain how their research contributed to this outcome – telling the story via a succinct case study,” said Vicki Thomson, National Director of the Australian Technology Network of Universities.
Within each SEO area there are up to 12 categories. Case studies must demonstrate the impact of the research across three.
For each SEO area, two assessment panels made up largely of industry experts will judge the case studies on the “reach and significance” of the research before assigning an overall mark, from A – signifying that the “adoption of the research has produced an outstanding social, economic, environmental and/or cultural benefit for the wider community, regionally within Australia, nationally or internationally” – down to E, for scholarship of limited or no impact.
The results will be announced in November.
“Linking research with research outcomes is imperative for industry, governments and the community to understand and see value in university research,” Ms Thomson said.
The guidelines add that “the transfer of knowledge between universities, industry and the community, and the impact of that knowledge on the development of new technology, new policy and economic, cultural, environmental or societal outcomes is an important focus for many Australian universities”.
The trial will draw heavily on the experience of the UK’s new Research Excellence Framework, which will include a 20% “impact component” in funding decisions from 2014. The British scheme was modelled in turn on the Research Quality Framework impact trial conducted by the ATN and Murdoch University in Australia in 2005 and 2006 – but later abandoned by the Rudd Government.
For the latest trial, impact is defined as “an effect on, change, benefit to the economy, society, culture, public policy or services, health, the environment or quality of life beyond academia. It includes, but is not limited to, an effect on, change or benefit to:
The activity, attitude, awareness, behaviour, capacity, opportunity, performance, policy, practice, process or understanding
Of an audience, beneficiary, community, constituency, organisation or individuals
In any geographic location whether locally, regionally, nationally or internationally."
The EIA will complement the Federal Government’s Excellence in Research for Australia program, conducted in 2010 and this year, which measures the standing of research within academe but not in the wider community.
Gavin Moodie, Principal Policy Advisor at RMIT University, welcomed the decision to assess research from as far back as 1992. “Some of the best research takes a long time to have a big impact. It is good that assessment panels will be chaired by and have a majority of ‘end users’ since this is likely to concentrate attention of the significance of the research’s impact on users rather than other researchers.
“It is true that impact is an even more retrospective measure of research importance than the standard measures of funding – publications and citations. However, this seems inescapable if impact is to be taken into account since some of the best research takes along time to have a big impact.”
In the 15-page submission for each case study, universities must list the authors of the work, explain how much the research project cost, identify the problem the project aimed to solve, specify the findings, and demonstrate exactly how those findings led to the impact claimed.
“The Assessment Panels realise that providing data to conclusively verify each claim of high value research impact would impose a large administrative overhead for institutions and could result in submissions which are excessively lengthy,” the guidelines state.
The submissions, therefore, should include a statement “to the effect that institutions have the data, testimonies, material or other information as relevant, to verify claims made in submissions and would be able to produce such information should they be so required to do.”