I love my job. I’m trying to understand how plants build themselves out of thin air. It’s exciting, it’s creative, it’s beautiful and on top of all that it’s important and useful. I like working with other people with different perspectives and I like the sharing of ideas and the piece-by-piece building of understanding from careful observation, experiment and analysis. Then there are those rare eureka moments when suddenly something that was obscure makes sense and unconnected ideas fit together to make a satisfying whole.
All these motivations for life as a researcher are evident in the results of a survey conducted as part of a project led by the Nuffield Council on Bioethics to examine the culture of scientific research in the UK. The 970 survey respondents, most of whom work in universities, picked improving their knowledge, making discoveries for the benefit of society and satisfying their curiosity to describe what motivates them in their work; and they identified collaboration, creativity, openness and multidisciplinary working as hallmarks of a high quality research culture.
But the project has also uncovered threats to the vibrancy of this intellectual melting pot. While participants in the project acknowledged the positive influence of competition in driving up the quality of research, they also expressed concerns about the current criteria used to find the competition winners.
Competition in science
Science has always been competitive. There are more ideas for new research projects than there is money to fund them – and there are more people wanting to pursue careers in research than there are jobs for them to fill.
With the expansion of the scientific enterprise, the current squeeze on resources and the drive toward more assessment at all levels, researchers are spending increasing amounts of time competing for funding and for jobs, and assessing the applications of their peers for funding and for jobs. Some aspects of research assessment are reasonably objective: have these experiments been designed rigorously? Does this researcher have an established track record in using these techniques?
However, many aspects are fundamentally subjective and inexact. Is this project exciting? Will this person revolutionise the field? All these judgements take time and careful perusal, and all of them require the judges to accept the subjectivity of the exercise. But time is in short supply and scientists don’t like basing their decisions on subjective criteria.
As a result a range of easy-to-collect metrics have been widely adopted as proxies for scientific excellence.
Publish or perish
Researchers are now assessed almost entirely on the research papers that they have published in peer-reviewed journals. These are easier to assess than important but less-tangible qualities such as public engagement and training and support provided to colleagues. But even assessing papers is time-consuming and subjective. So instead, it is now common to use proxies such as the number of times other people have cited the papers.
However, garnering significant numbers of these is only possible for papers published some time ago, so often quality is assessed by the perceived prestige of the journal in which the paper is published. Although it is widely agreed that using journal prestige in research assessments has serious limitations, the researchers who took part in our study were clear that publishing in prestigious journals is still thought to be the most important element in determining whether researchers gain funding, jobs and promotions.
The wrong incentives
A relentless focus on publishing papers in prestigious journals can lead to a wide range of non-ideal practices, such as over-claiming the significance of research findings, sticking to trendy areas of science and leaving important but confirmatory results unpublished due to lack of incentives to spend the time writing them up.
This can erode the quality of science in the short term, but the long-term effects are even more worrying. If research stops being about finding out how the world works for the benefit of society, and becomes being about competing to get your work published in a particular journal, then the most creative and brilliant people will go and do something else. The people who stay in research will be those mostly motivated by wanting to look good according to some semi-arbitrary yardstick. This is causing widespread unease in the research community.
So what should be done about it? The good news is that since assessment processes are implemented almost entirely by the very researchers who are worried about them, it should be possible to change them. However, another clear result from the Nuffield project is that everyone in the system – funders, universities, publishers and editors, professional bodies, and researchers – claim to be powerless to change things, believing that it is someone else’s responsibility.
We hope the findings of the project will stimulate discussion and debate about how to shift the culture back to its roots in creativity and innovation, coupled with rigour and openness. If left unchallenged, the current trends will inevitably influence what science gets done and therefore what we learn about the world, what problems we’re able to solve and whether public funding is well spent. This is not just some arcane academic debate, it matters to everyone.