Over the past 25 years we have seen an explosion in top-level clinical trials in China. But only a tiny proportion of that research is available on the main databases used by doctors and researchers in the UK, Australia and the US. It means we could be missing out on evidence for potential medical breakthroughs.
There have been some high-profile cases in China of dodgy research, such as one poorly conducted trial (that didn’t even have ethical approval), but the Chinese have taken steps to improve research integrity. And we have to recognise research misconduct as a global issue.
I conducted a survey of low to middle-income countries randomised control trials in mental health. These studies are modelled very carefully and considered the gold standard of evidence for medical treatments. The survey involved laboriously hand searching databases in each region and in each language. Pretty dry stuff but as it turns out, very important. In China they increased by more than seven times between 1991 and 2000.
Prolific rise in Chinese trials
The prolific rise of Chinese trialling is unlikely to be unique to mental health and follows a trend in scientific research in general. In 2011, China was already taking over from the UK as the second most productive country in terms of academic citations in research publications and is predicted to take over from the US as the world’s leader by 2020.
Despite this, I was very surprised to find the number of Chinese trials on PubMed (the main database used by doctors and researchers in the UK, US and Australia) had actually decreased from six out of seven trials in 1991 (about 86%) to only two out of 53 trials in 2000 (less than 4%). But whether or not a trial was included in PubMed was unrelated to its quality – so just because it was not there doesn’t mean it was of poorer quality than trials that were.
The proportion of non-English language trials that could be found on the Cochrane Central Library, “the largest collection of records of randomised controlled trials in the world” also decreased.
We need to actively seek information
In mental health, the Cochrane schizophrenia group is unusual (even within the organisation) for actively searching out Chinese literature. When I conducted a systematic review on a potential treatment for schizophrenia, 14 of the 15 randomised control trails on this subject were from China. Sadly the trials were not of good enough quality to allow us to make recommendations regarding treatment.
But evidence suggests that the quality of trials of traditional Chinese medicine is improving.
China not the only one
Three academics were sacked from Zheijang University, in China, for “plagiarism and fake research” after a major investigation. It’s the kind of story that fuels the scepticism of Chinese research and questions of integrity.
But it’s not like we have a perfect record of integrity either. One infamous case was the widely published Richard Borison and Bruce Diamond who lied to embezzle millions in the name of trialling.
And let’s not forget about the the enormous impact of under-reporting or biased reporting that happens in trials outside China all the time. Up to a third of researchers from high income countries have admitted to “questionable research practices” according to one survey.
Let’s hope that the pharmaceutical industry, which is expanding massively into China, operates ethically, and that the Chinese are less accepting than we have been in allowing it to conceal unfavourable results.
Ignore China at our peril
There is still plenty of room for improvement, but the quantity – and quality – of trials from China is increasing while our databases have a diminishing number of them.
Systematic reviews are about evaluating all of the evidence available. The beauty of randomisation is that bias is minimised, especially when trials are done well with large numbers of people. Putting all of the trials together into a systematic review lets us compare the trials and cut down on any bias introduced by individual trials even more.
By ignoring Chinese trials we risk missing interventions that might work that we haven’t even looked at or, by not taking into account all of the evidence, coming out with biased results leading to wrong conclusions. We also risk implementing clinical guidelines and policies based on these incorrect conclusions.
It’s no time for high mindedness and dismissal of relevant research. What we should be doing is finding all these trials and then assessing their quality rather than ignoring a whole body of evidence without properly evaluating it. It’s the prerogative of scientific reviewers to evaluate the quality of trials and decide whether to include them, not simply exclude a mass of research because of where it’s from or the language it’s in.
Tightened research budgets also mean that we can ill afford ignore extra work. As time goes on it’s going to become more important to find and evaluate trials reported in languages other than English.