Paracelsus' poison

Missing the Point About Evidence Based Medicine (teaching and doing)

Memo to CAM enthusiasts, try not to shoot yourself in the foot publicly next time.

I am, of course, referring to the recent article in the Medical Journal of Australia “The Legitimacy of Academic Complementary Medicine” (conversation article here). This is a supposed reply to the article “Tertiary education institutions should not offer pseudoscientific medical courses” (see the Conversation piece here).

The “Legitimacy” article pretty comprehensively misses the point of MacLennan and Morrison’s original. However, my article “What CAM courses at universities should look like” covers most of those misconceptions, so I won’t revisit them here. Instead I want to focus on the following statement, because it effectively destroys the credibility of the entire article.

“Complementary medicine, like conventional medicine, is riddled with poor scientific methodology and lacks a comprehensive evidence base. The BMJ compiles and updates the evidence for medical interventions (3000 to date) and currently considers that 51% of medical treatment is of unknown effectiveness and that only 11% is definitively beneficial, with another 23% probably beneficial.7”[1]
Wow, that looks pretty bad doesn’t it, 51% of all treatments are of unknown effectiveness. Well, it does until you actually go to the referenced site (indicated by the 7). Why don’t you go there now? Scroll down to to the diagram the “Legitimacy” article authors got their figures from.

Now read the caption.

The “Legitimacy” article strongly implies that 51% of all current therapies have no evidence of effectiveness. But the diagram doesn’t mean what the “Legitimacy” authors think it means. As the caption makes clear, the scary figures do not relate to all, or even most, current therapies.

The site with the scary diagram is part of the journal Clinical Evidence, which publishes reviews of clinical trials that are performed to determine if therapies work (or are better than existing therapies, or don’t interact with existing therapies and so on). Not surprisingly, a lot of new therapies have not yet accumulated enough evidence to show they work (or fail to work) and more research is needed.

There is also the situation in which a given therapy works perfectly well for disease A, but the patient subsequently develops disease B. It is currently unclear if the drug for disease A will interfere with the therapy for disease B. However disease A is sufficiently serious you don’t want to stop the drug.

Not surprisingly, these sorts of situations are also under research. Importantly, most of these therapies (not all are drugs) have not progressed to general clinical use.

As a reality check, I looked at the Clinical Evidence summaries for heart failure (you’ll have to trust me on this unless you have a subscription to Clinical Evidence). Anti-platelet agents are listed as “unknown effectiveness”, the 51% category the “Legitimacy” authors imply are being used without good evidence.

Now, let’s go to the Australian Heart Foundation’s guidelines for treating heart failure. Are anti- platelet drugs being used as treatment for heart failure?


This is just one example, but I checked quite a few to be sure. The “unknown effectiveness” treatments (not just drugs [2]) are mostly treatments that are not used in therapy, unlike what the “Legitimacy” authors try to imply.

This is the whole point of doing clinical trials, to find out if new treatments work, or are better than existing treatments, or can be combined with other treatments. Most complimentary and alternative medicine has nowhere near this level of evidence. In the cases where CAM has been extensively tested, most of it doesn’t work (see here, here, here and here).

To present a site which reports results of treatments being tested to ensure they work before they are used as a sign of the failure of modern medicine is to so profoundly misunderstand how medicine is conducted as to invalidate the rest of the argument in the “Legitimacy” article. At least next time read the caption to the diagram before you use the figures.

[1] Leaving aside, of course, the question of how the argument “some medicine is rubbish, so lets teach more rubbish” even makes sense.

[2] Exercise is only “likely to be beneficial” for heart failure. But you would want to include exercise for a range of other reasons (depending on how severe the heart failure as). This again illustrates how the “Legitimacy” authors fail to understand what the Clinical Evidence figures represent.

Here’s another interesting link of some relevance.