UK United Kingdom

Evidence-based medicine is broken. Why we need data and technology to fix it

Guideline for preventing cardiovascular disease

A recent paper in the British Medical Journal suggests that evidence based medicine is in crisis. Evidence based medicine is based on the practice of employing treatments that have scientific research that backs up their effectiveness. It is usually set against medical practice that is based on anecdotal experience or simply doing things because that is the way they always have been done.

The ways evidence based medicine is broken

The authors of the paper point out however that there are a number of problems with evidence based medicine that together, significantly compromise its effectiveness. The first problem is that evidence based medicine plays right into the hands of companies and organisations with a vested interest in seeing a treatment recommended. Of particular concern are drugs that are recommended for treating pre-diseases. The classic example of this is the use of statins to lower cholesterol in the attempt of reducing cardiovascular disease.

The National Institute for Health and Care Excellence (NICE) has offered evidence based guidance on prescribing statins that has reduced the required level of risk before treatment is offered, effectively increasing the number of people taking statins significantly. The bar for being put on statins is very low and cholesterol levels can be considered normal and the person could still have a risk profile that puts them on the pathway for treatment.

Drug companies have a vested interest in funding trials and journals are more likely to publish positive results than negative ones. The authors of the BMJ paper cite “One review of industry sponsored trials of antidepressants showed that 37 of 38 with positive findings, but only 14 of 36 with negative findings, were published.”

Another major problem with evidence based medicine is the fact that statistical significance in a clinical trial does not necessarily mean very much at an individual level. This works to overemphasise benefits and to underestimate risks.

People writing the guidelines to implement evidence based medicine face a growing challenge of dealing with ever increasing amounts of evidence. At the end of this are the clinicians that have to deal with an explosion in the guidelines themselves which can be complicated and even more so when they have to be applied to a patient with all of the real-life complexity and idiosyncrasies that they bring.

The authors are not suggesting abandoning evidence as the basis for clinical treatment. What they end up recommending are modifications of how the evidence is gathered, reported and implemented.

Technology and data as a solution

What the authors don’t mention though is the role that technology has to play in providing a solution for the problems that they have outlined. The first element of a technology based solution actually runs counter to the author’s point of crisis. What is needed is more evidence and data, not less. Large data sets providing detailed monitoring of patients combined with genetic information will eventually be able to reveal specific information about what makes a treatment work in one patient and not another. This type of information goes much further than the limited range of measures normally collected as part of current clinical trials. To enable this collection of data, we will need to be monitoring people pervasively and collecting that information on a continuous basis.

Clinical analytics of this so-called “clinical big data” can potentially reveal much more information about the effectiveness of particular drugs and treatments given a specific individual with a set of symptoms and illnesses. What we will then need is sophisticated artificial intelligence as portrayed in the fictional film “Her” that guides clinicians through a range of true evidence based treatment options that have been personalised for the specific patient at that moment in time.

It is possible that as a society we are not ready to trust the likes of Google CEO Larry Page with the task of collecting and analysing patient data on this massive a scale. He declared our ability to save thousands of lives through this approach. Despite our reticence, it is clear that he is right.

Companies like PatientsLikeMe have also faced skepticism when arguing their ability to contribute to clinical research through the massive, but self-reported data they collect from patients.

Despite the resistance, sites like PatientsLikeMe, CureTogether and the personalised genetic profiling company 23andMe are part of the future of medicine.

These companies have the ability of providing extensive information about the effectiveness of treatments, symptoms and side effects of people outside of the restrictions, and some would say artificial settings, of a clinical trial.

All of this does not deny the needs for the skills of a doctor, at least until the technology has advanced to the point where it makes those skills redundant.

Microsoft culls a massive 18,000 staff. Does this spell the end of its devices?

Satya Nadella

The pain, it seems, is not over for former Nokia workers as their new employer, Microsoft, prepares to cut its workforce by a massive 18,000. Although Microsoft has not announced where all of these cuts will come from, but 12,500 are expected to be from the newly acquired Nokia mobile business which added an extra 25,000 staff this year to swell Microsoft’s staff numbers to 127,000.

It is not surprising that Microsoft would want to reduce its staff numbers. In a simple comparison of productivity of net revenue per staff between Microsoft and other tech leaders like Google and Apple, Microsoft was lagging even before it took on the extra Nokia staff (see table below). At 127,000 staff, profitability would have taken a big hit.

Net Revenue per Staff (2013) Data from Google Finance and Wolfram Alpha

The layoffs are expected to cost Microsoft between $1.1 and $1.6 billion over the next year in pre-tax charges which makes the Nokia purchase even more expensive.

The New Direction

Recently, Microsoft CEO Satya Nadella sent an email to Microsoft staff announcing the new direction for the company. No longer would it be a simple “devices and services” company but it would become a “productivity and platform company for the mobile-first and cloud-first world”. It was no secret that former CEO Steve Balmer completely misread the iPhone’s potential and consequently consigned Microsoft’s role in the mobile market to that of an extra. Microsoft however have always believed in the mythology that even though they come to a market late, they could always catch up through sheer will and the declaration that they are suddenly all about devices and services. This was certainly true when they held a monopoly in the PC market, but it is unfortunately not the case in markets for which they no longer have any influence.

Satya Nadella is now doing what most new CEOs do and declaring a new vision for Microsoft which is all about productivity. This may guide Microsoft into deciding what they don’t want to be doing but unfortunately says very little about what they can do in a market in which they have no presence.

The massive cuts perhaps signal that Nadella has no confidence that Microsoft will be able to make something of the mobile phone business it bought for $7.2b. This wouldn’t be the first time that Microsoft has written off a massive purchase. In 2012, it wrote off most of the $6.2 billion it had spent on online advertising company aQuantive.

Nokia’s mobile phone business was already struggling before Microsoft bought it and with the layoffs and change of direction, it seems clear that this is unlikely to change.

Xbox not core

As part of the re-definition of Microsoft, it seems that some of the staff cuts will come from the no-longer-core Xbox division. At one point, Microsoft used the Xbox as the spearhead in the fight for the living room. Again, they viewed the battle to be about PCs. Prior to the Xbox, Microsoft had developer Windows Media Centre with the view that people would have a PC in their living room and this would replace all other types of media devices.

What happened however was that people didn’t want PCs in their living rooms and opted instead for smart TVs and a plethora of small inexpensive media players that include the Apple TV and Google’s Chromecast. If they wanted to surf the web at the same time, they used their tablet and/or their mobile phone.

As far as the Xbox went, this was never going to make it out of the gamer of the family’s bedroom. It was certainly never going to make it to be the centrepiece of the living room.

The Microsoft legacy

However Microsoft decides to “tweak” or restate its business, its fundamental challenge will be that it has no presence in the mobile market. As Apple and Google continue to eat away at the dominance of Windows on the PC, even Microsoft’s two principal cash cows, Windows and Office, are at risk. If mobile defines our future productivity, Nadella’s vision for his company seems unrealistic and it is hard to see a future where Microsoft continues to play anything other than a legacy role.

Universities are still standing. The MOOC revolution that never happened.

MOOCs replace lectures

It is two years since Coursera began offering massive open online courses (MOOCs) that threatened the very existence of Universities and the increasingly expensive education they offered. It was really only a matter of time before bricks-and-mortar universities faced the same digital battle-to-the-death that has transformed other industries like newspapers and music. Universities faced a MOOC “tsunami” and “avalanche”.

Two years on however, the avalanche/tsunami/revolution never came and universities are not only still standing, they have, by-and-large, been remarkably unaffected by the free courses now offered by a couple of hundred universities around the world. Arguably MOOCs have spurred a renewed interest in using what is called a “blended learning” approach to university courses, offering students a mix of online and face-to-face tuition. But beyond this, one can argue that MOOCs have had limited impact on the day-to-day business of universities.

The logic of why MOOCs presented such a threat to the existing higher educational model was reasonably sound. Faced with an option of quality courses that were free, why would people continue to saddle themselves with, what in some cases is a lifetime of debt, to take courses from second or third-tier universities and colleges?

The problem with free

Well it turned out that free has consequences. The first is that the drop-out rate of MOOCs was amazingly high. In fact, when retention in MOOCs is measured by how many students watch videos each week or take any of the set quizzes, the drop-out rate is remarkably constant and results in something like 2 - 14% of students who initially enrolled finishing the course. In fact, on average, only 55% of people who enrolled actually watch the first video.

This attrition is remarkably constant across all MOOCs and has little to do with quality or any obvious forms of engagement of the students. It is simply a fact that frictionless entry into a course makes frictionless exit just as easy. There are no consequences for people not completing a course and so simply put, they don’t.

The problem with change

But actually, the major issue with why universities haven’t been affected by MOOCs is to do with a range of factors that are integral to the way universities work. Universities are not free enterprises in the same way as music companies and news media companies are privately owned companies. Universities are governed by legislation and quality requirements that restrict who can award a degree and how they have to go about ensuring that students achieve a particular standard in order to earn that degree. This has made any change to this model a matter for governments as well as universities and their customers. So far, although there has been enthusiasm shown by governments for the general principles of MOOCs, none, nor their proxy institutions have shown any interest in allowing anyone other than universities to grant degrees based on MOOCs. Even allowing credits for individual MOOCs to count towards a degree has had limited acceptance.

The problem with the business model

The original model of academics simply putting up recordings of powerpoints on their computer screens as a MOOC have rapidly disappeared. Harvard spends between $75,000 and $150,000 on each MOOC it produces. Even those universities that don’t directly spend this sort of money on production of MOOCs end up relying on academics and other staff to create the MOOCs in their spare time which represents a hidden cost. To date, Coursera’s attempts to charge for verified certificates of completion of MOOCs have not generated enough revenue to make a profit, let alone cover these costs.

Georgia Tech has teamed up with Udacity to create an inexpensive Masters degree programme in computer science but even here, they have had to limit the number of enrolled students to a few hundred and provide enough staff support to ensure that the students have some chance of completing. It is consequently hard to see how this will scale and make money.

The future of universities

Universities are not homogenous organisations that take a consistent approach to how they produce students with degrees. Each academic teaches in a largely unique way and students all approach their learning in an equally non-uniform manner. At the end of the day, we haven’t found any consistent way of getting around the fact that in order to learn, students are required to put in a great deal of effort and they can only be guided and supported in this endeavour. The motivations for students, especially young ones, to do this on their own without the incentive of obtaining a degree that means something substantial in economic and social terms, are simply not there.

If universities do eventually experience a revolution, it will not be because of MOOCs.