Why data from published trials should be made public

TRANSPARENCY AND MEDICINE – A series examining issues from ethics to the evidence in evidence-based medicine, the influence of medical journals to the role of Big Pharma in our present and future health.

Here Ian Haines makes an argument for research data from published papers to be made public.

It’s time for medical journals to mandate data release and independent audits for all clinical trials submitted for publication. And here’s why.

Over the past few years, editorials and commentaries in some high-profile medical journals and magazines have heralded the exciting revolution in personalised cancer therapy. They talk of treatments involving combinations of antibodies and enzyme inhibitors specifically designed and tailored for each individual’s cancer.

But most of these many new treatments cost between $40,000 and $120,000 per patient per year. And results of clinical trials so far show miniscule benefits that require extraordinary statistical gymnastics to achieve the magical “significance” level.

The problem is that these usually commercially-sponsored studies are designed by the sponsor and the statistical endpoints are no longer one or more of the traditional “gold standards”.

Those gold standards include:

extending overall average survival;
improving “quality of life” measures;
reducing toxicity of treatment and;
reducing the cost of treatment.

So many contemporary clinical trials have stopped measuring things that matter, such as quality of life and overall survival. Instead, they measure things that are easier to achieve positive results for, such as how long a patient’s X-ray stays stable in the very subjective opinion of an investigator.

These measures are clinically inconsequential, often unverifiable because of unavailability of the “raw data” and are far too subjective.

In addition, the raw data from these studies are protected – seen and analysed only by the sponsor’s data team. The results are then usually written by “medical writers” employed by the sponsor together with clinicians who often have significant financial conflicts of interest with the sponsor.

The final analyses may have little relation to the originally stated primary and secondary endpoints of the study and may emphasise other, often inconsequential, positives. And the abstract, which is a summary of the results, often emphasises positives that are not even supported by the data.

This is akin to a large public company producing its end-of-year financial results but keeping the primary data hidden, and then doing an audit on itself.

So despite hyperbole of medical “revolutions”, the US Food and Drug Administration (FDA) recently withdrew its previous approval of the world’s biggest selling multi-billion dollar breast cancer drug, bevacizumab. This step is almost unprecedented.

And the National Institute for Health and Clinical Excellence (NICE), the equivalent body in the United Kingdom, issued the shock announcement that it didn’t consider the use of any of the three “blockbuster” new targeted therapies for bowel cancer – cetuximab, bevacizumab and panitumumab – to be a cost-effective use of National Health Service (NHS) funds.

In fact, when the FDA examined the raw data of the key trial that had produced the initial approval for bevacizumab, its independent reviewers had to reverse the initial assessment in 51% of the cases.

When I completed my medical training in 1978, the first cures of solid tumour cancers by chemotherapy (as distinct from acute blood cancers or acute leukaemias) had just been reported in the medical literature. Hodgkin’s disease, aggressive non-Hodgkins lymphomas and advanced testicular cancer, which had hitherto usually been fatal, had disappeared in many patients with combinations of new and older drugs.

This was a watershed moment for cancer treatment in the twentieth century and generated great optimism for the future.

Clinical trials needed to prove the benefits of new treatments could easily be done by well-organised research groups. We didn’t need statisticians to dissect the p-values and hazard ratios of multiple opaque statistical analyses to tell us that these were significant advances in treatment.

And the drugs were affordable.

The breakthroughs of the time spawned the new speciality of medical oncology even though we still didn’t have effective treatments for the common and usually fatal advanced solid cancers such as bowel, lung, breast, melanoma or prostate.

And as we did our daily ward rounds, seeing the many referrals with these conditions, our most common note in the medical record was that “chemotherapy had nothing meaningful to offer”.

To prevent us returning to those days, it’s time to mandate data release and independent audits for all clinical trials. This will increase the chance of trials that are designed to measure relevant endpoints, of producing results that are verifiable, and of generating analyses and conclusions that match the original design endpoints.

To avoid mirages in our quest for cancer treatment oases, our ongoing need for innovation, hard work and entrepreneurship will need to be carefully balanced with more transparency and accountability to independent reviewers and assessors.

This is the tenth part of Transparency and Medicine. You can read previous instalments by clicking the links below:

Part One: Power and duty: is the social contract in medicine still relevant?

Part Two: Big debts in small packages – the dangers of pens and post-it notes

Part Three: Show and tell: conflicts of interest undeclared for clinical guidelines

Part Four: Eminence or evidence? The ethics of using untested treatments

Part Five: Don’t show me the money: the dangers of non-financial conflicts

Part Six: Ghosts in the machine: better definition of author may stem bias

Part Seven: Clearing the air: why more retractions are good for science

Part Eight: Pharma’s influence over published clinical evidence

Part Nine: Insight into how pharma manipulates research evidence: a case study

Part Eleven: Open disclosure: why doctors should be honest about errors

Part Twelve: Reaching full and open disclosure for universities, medical schools and doctors