If big data is to solve Britain’s productivity crisis, then we need to get cracking now

Big data, what can it do for us - and when? Novelo/shutterstock.com

Britain has a serious productivity problem – levels have not risen since 2008, and are now around 20% lower even than in strike-prone France. The gap with our competitors in Europe and further afield is estimated to cost the UK almost £100 billion a year, and addressing this problem has been a source of angst for politicians for decades.

The problem has been that the proposed solutions are often rooted in a 20th century view of the economy, such as adopting Japanese approaches to reducing defects (“total quality management”), replacing outdated machinery, and improving transport infrastructure. But a more 21st century solution is already at hand: using big data could be a game-changing approach that could tilt the scales back in Britain’s favour. At least, that’s what a recent parliamentary science and technology select committee report has suggested, by extracting more value from the vast amounts of data generated across all areas of industry and society.

Computing has always been focused on processing data, so why is this now talked of as being “revolutionary”? The change comes from how increasingly dependent we are on software for all aspects of our lives. Most businesses depend on computers, public services are increasingly administered electronically using online platforms, our health is monitored by risk-detection algorithms, and even our social lives are now filled with the social media we use on smartphones – the computers in our pockets. All this software generates vast amounts of data – and this is where the new opportunities arise.

Consider an e-commerce website. Every time a customer accesses the site, information on what they viewed and bought is captured. Over time, this builds a pool of data with huge potential value. The company can discover whether an advertising campaign is effective by analysing changes in customer purchasing. It can build a picture of each customer’s interests so that adverts can be personalised, and help predict in advance what stock levels are needed for different products at different times of the year. The data need not just be used to optimise the sale of existing products: analysing what people search can reveal opportunities for new services and products to fill gaps in what’s available.

The potential to exploit data isn’t restricted to retail alone, but to any industry or field that has been computerised, either internally or in the way it interacts with its customers. Some of this data has already been made open for use by others. For example, vast quantities of data on chemical properties are now freely available to anyone on the web, waiting for an innovative company to use it to design an effective new drug, or for a clever researcher to make a Nobel-prize winning discovery.

Big data: everyone’s talking about it. What are we doing with it? vodafone-institute, CC BY-ND

So why aren’t all organisations taking advantage of this data to transform their fortunes? As the select committee report suggests, the problem is a lack of expertise. Management consultants PwC’s 2015 Annual CEO survey found that almost 90% of CEOs in Britain see digital technologies as vital to their success and a source of competitive advantage, but almost as many – 84% – are worried about the lack of skills to make it happen. The report also highlights that Europe has fallen behind the US and Asia, and that the UK is increasingly importing R&D from different regions of the world. It concludes that “if the UK is to address these CEO concerns and thrive as a growing knowledge economy, the long-term decline in UK R&D investment identified in this survey must be reversed”.

One of the real problems in tackling this problem and reversing the skills shortage is that very few existing university courses produce graduates with the new expertise that is needed. Our own experience is that the ideal combination is Mathematics plus Computer Science plus background knowledge of the subject area that the skills will be applied to, whether that’s banking or science or retail or any other. This approach goes against the traditional single subject-based approach to research and teaching common at universities.

We’re trying this approach at Newcastle University, training researchers in these skills and putting them together with industry experts to tackle real problems. The Cloud Innovation Centre, set up with Newcastle City Council, is one way we then help to get these skills into other organisations in order to help transform their products and services.

If the UK is to succeed then we need many more centres that can train students in the right combination of skills, and set them to work addressing organisations’ real problems. If we do this then we positively can transform the vast majority of UK organisations, from manufacturing through to healthcare. If we don’t, then we will have missed this opportunity to raise productivity and so benefit the UK economy and society.