If researchers go to the effort of getting funds, recruiting patients, and following them up, you would think that they would be keen to publish the results. So it’s surprising that our best estimates show around half of all completed clinical trials have never been published in academic journals. And the half we have is biased towards trials with positive results.
Non-publication is both a waste of research resources, and a problem for decision makers attempting to use the biased set of studies they can access.
But this is not only problematic for decision makers, it also poses a dilemma for people who volunteer for medical research. Most do so because they believe they will be contributing to medical knowledge, and might be horrified to learn that the research they participated in has gone unpublished.
The latest revision of the World Medical Association’s Declaration of Helsinki – Ethical Principles for Medical Research Involving Human Subjects now rightly suggests public reporting of results is an ethical obligation.
Sometimes non-publication and selective publication occurs for commercial reasons. This is illustrated by the incomplete information in the Cochrane review of drugs for treating influenza, which has been hampered by the lack of access to the results of all studies. Roche, the manufacturer of the drug, oseltamivir or Tamiflu, has repeatedly refused to give researchers full access to the data.
One of the authors of the review, Tom Jefferson, recently said, “We identified that a large number of studies, including data from 60% of the people who have been involved in randomized, placebo-controlled phase III treatment trials of oseltamivir, have never been published. This includes the biggest treatment trial ever undertaken on oseltamivir that on its own included just over 1,400 people of all ages.”
Governments around the world have spent billions of dollars on stockpiling oseltamivir (Tamiflu) but the Cochrane review authors conclude that there’s no clear evidence of the effect of this anti-flu drug. “The evidence supports a direct oseltamivir mechanism of action on symptoms but we are unable to draw conclusions about its effect on complications or transmission,” the authors said.
But non-publication is almost as frequent among non-commercial trials. Sometimes lead authors can move jobs, become ill, retire, or (as with a 68,000 person colorectal cancer screening trial) die as the research is finishing.
Occasionally authors give up after several attempts to publish, but more often the results are never submitted for publication. A randomised trial of deworming treatment of almost a million children in India, for instance, has not been published even though it was completed in 2005 (rumour suggests in will be published early this year).
Given the extent of deworming programs in developing countries, this non-publication of crucial evidence is hampering rational decisions about such program investment – should they stop or be extended?
The problems of non-publication have been known for at least three decades. Indeed, John Simes, the director of the Clinical Trials Centre at The University of Sydney, demonstrated the problem and proposed universal registration of trials in the 1980s. Since then trial registration has improved but unfortunately, most national and international clinical trials registries are currently voluntary.
While awareness has spread and some action has been taken, a number of recent events – and the publication of Bad Pharma by Ben Goldacre – have resulted in greater awareness and willingness to act at a regulatory and political level.
It seems that we’re at a critical juncture, and pushing for more action now is vital. Hence a coalition of several organisations – Bad Science, Sense About Science, BMJ, James Lind Initiative and the Centre for Evidence-based Medicine at Oxford University – have put together the AllTrials campaign.
The alltrials website sets out the details of why action is important now, and includes a number of proposals, including a petition that everyone is urged to sign. So if you want the waste of non-published research to stop, and if you want doctors and health policymakers to base their decisions on all the research rather than a selected sample, I suggest you to sign up.
Tim Scanlon
Debunker
I agree whole heartedly with this.
But to play devils advocate, it has to be remembered that trials and funding are on a tight rotation, and publishing failed trials to the high standard of peer review is a lengthy and frustrating endeavour. So often the reason for not publishing is a lack of time, resources and incentive to publish. It is also worth remembering that some trials aren't published for years after the work is done, usually in the free time during the next project.
A good halfway would be to have methodology and results data available openly or have publishing as a requirement that has its time factored into the funding cycle (and subsequent cycles to allow the last round to be published).
Sean Lamb
Science Denier
"But to play devils advocate"
Oh dear, Dr Scanlon, I think the Devil will be saying "I demand the right to defend myself, your honor."
A data repository just requires you upload your data set with a description of the protocol - which you presumably wrote a long time ago in the pre-planning and ethics approval phases.
The use of data repositories in life sciences is well established and there is no reason why it can't be applied to clinical trials
Tim Scanlon
Debunker
Bit more complicated than that Sean.
Which is why publishing takes so long.
Sean Lamb
Science Denier
Golly, so how on earth to Genbank manage to function?
The Devil is slumped in dock and muttering "this is what happens when you are forced to rely on legal aid."
Sebastian Poeckes
Retired
Having been caught between two stools some years ago when two research programs I was leading had not been published when I achieved a significant promotion, I can understand how difficult it can sometimes be to meet expectations. In my case I finally delivered a summary of outcomes to a professional conference which were incorporated in the proceedings. I had no time to do more, but recognise that this was quite unsatisfactory. Sometimes circumstances militate against optimal distribution of results.
If I data repositories had been available then I would have felt more comfortable about the way things panned out.
Sue Ieraci
Public hospital clinician
I agree with Tim and Sebastian - having also been extremely slow to write up research reports and seek publication while also doing a full-time clinical job. Many studies remain unreported just due to lack of time or motivation - not any sort of conspiracy.
There is, however, a responsibility for those trials that use human subjects to justify their use through accessible results which can contribute to further knowledge. Data repositories are a good solution for those who never get around to writing for publication.
It's important to realise how many studies are done by students or as part of professional post-graduate training - research is not all sponsored by multinationals - far from it.
Sean Lamb
Science Denier
"I agree with Tim and Sebastian - having also been extremely slow to write up research reports and seek publication while also doing a full-time clinical job."
Read moreHaving checked out pubmed I can see that must indeed be the case - too busy even to read that Tim and Sebastian were arguing substantially different to almost opposing positions. One said data repositories would be very useful, the other arguing for unspecified reasons that this was too simplistic.
In general Professor Glaziou wasn't raising…
Tim Scanlon
Debunker
Stop with the strawman Sean.
You stated that it would be dead easy to have a data repository because it is all done before the trial. I was pointing out that it isn't that easy at all and would take effort to process the information into a usable form, with analysis, so that it isn't just a bunch of numbers.
Sue, Sebastian and I are pretty much on the same page and just adding to Paul's proposal.
Sean Lamb
Science Denier
Well wrong on both counts.
Read more1. The analysis phase is generally the shortest phase and least work of any clinical trial. Usually it is when statisticians try to get too clever that they start spinning spurious findings out of over-analyzed data. Statistics are powerful tools to reveal hidden trends, but also powerful tools for self-deception. Where it may take time is when the data set is proving refractory to producing strong positive publishable results - and the researcher is unwilling to accept…
Tim Scanlon
Debunker
Sean, you have quite clearly never been involved in any actual science of any substance.
You also seem to want to twist what I say and create strawman arguments.
You also don't seem to understand how hard it is to find time to publish results, let alone how much science is sitting in filing cabinets and on desks around the world unpublished.
Sue Ieraci
Public hospital clinician
Sean Lamb - you must be very busy yourself. Unable to look beyond PubMed, you are limited in the number of throw-away lines you can deliver me.
My point of agreement with Tim and Sebastian is that research is often completed but not published due to lack of time or effort. Where did the "conspiracy card" come from? Not this article, but the common allegation that unpublished research is hiding something.
I am merely describing reality: a lot of mundane research is done in the course of study or work - often by people whose primary job is not research. Writing up the results and seeking publication takes time and effort.
If people wanted data on why people walk out of ED, they could get it from the published paper. SImilarly, the impact of a complexity-based fast-track system on ED waiting times. IF they wanted data on how triage nurses allocate an urgency category according to historical and clinical characteristics, however, they would need to ask for the report.
Monika Merkes
Honorary Associate, Australian Institute for Primary Care & Ageing at La Trobe University
"So if you want the waste of non-published research to stop, and if you want doctors and health policymakers to base their decisions on all the research rather than a selected sample, I suggest you to sign up."
I've signed up. For the above reasons, and because publication of all clinical trial data might also stop unnecessary duplication of research involving animals.
Sonia Hines
Internerd at Queensland University of Technology
How much part do the journals themselves play in this? It's been my experience that many journals are reluctant to publish research with findings of no effect. If you can't find an outlet for your work in a reasonable time period, at some point you just have to move on with new work.
Tim Scanlon
Debunker
Journals are also reluctant to publish all sorts of things that really should be available. It would be good if a database and open journal were part of the solution. Then data could be used for meta-analysis and an open journal would fill in those studies.