The recent publication of the government’s strategic case for HS2 has added to mounting concerns about the strength and validity of evidence put forward to support the project.
Previously, the business case report written by KPMG stated that the high-speed rail link could benefit the UK by £15 billion a year – a claim made just days before a critical vote in parliament that secured further funding.
The KPMG report’s authors have been accused of “cherry-picking” the evidence to exclude any that did not support the project. It listed, for example, the areas that would benefit – such as Greater London by £2.8 billion and the West Midlands by £1.5 billion – but omitted details of those that would end up worse off. This was discovered in response to a Freedom of Information request.
A panel of academic experts subsequently told the Treasury select committee that the KMPG report overstated the HS2’s regional economic benefits by six to eight times. Dan Graham, professor of statistical modelling at Imperial College London, told the committee of MPs: “I don’t think the statistical work is reliable.” The findings were widely cited by the government, including in its new strategic case document, yet used a procedure that was “essentially made-up”, according Henry Overman, professor of economic geography at LSE. It seems critics have every reason to be sceptical about the evidence being presented.
The wider picture
There are few relevant studies that can provide statistically significant evidence of cost performance in transport infrastructure projects. But one that does is by Flyvberg et al. (2003). The sample used is the largest of its kind, covering 258 projects in 20 nations worth approximately US$90 billion (at 1995 prices). The findings from the study were as follows:
It shows rail projects incur the highest difference between estimated and actual costs, no less than 44.7% on average. Based on the available evidence it seems rail projects are particularly prone to cost escalation, followed by fixed links such as bridges and tunnels. Road projects seem to suffer this relatively less, although actual costs are still higher than forecast costs much more often than not.
Flyvbjerg and his colleagues then subdivided rail projects into high-speed rail (the study included HS1, the Channel Tunnel Rail Link), urban rail and conventional rail, finding that high-speed rail tops the list of cost escalation at on average 52%, followed by urban rail at 45% and conventional rail at 30%. According to Ricard Anguera’s 2006 study, “the British economy would have been better off had the tunnel never been constructed”. For all three project types, the evidence shows that it is sound advice for policy and decision-makers as well as investors, stakeholders, media and the public to take any estimate of construction costs with a pinch of salt, and especially for rail projects and fixed links.
Full HS2 job projections:
Cost escalation and benefit shortfalls
Cost performance has not improved over time. The tendency for costs to escalate today is the same as it was ten, 30 or 70 years ago. If our ability to estimate and forecast the costs of infrastructure projects has improved over time, this does not show in the data. In other words, no one seems to have learnt from past experience.
Is this due to the influence of pressure groups, which seem to have some influence over the decision-making process? For example, environmentalists’ desire to safeguard the natural environment demands that roads or railways are buried in expensive tunnels, rather than above ground. The historical period that Flyvbjerg draws his data from goes back long enough to include projects from before pressure groups could wield influence over decision-making and costs. Then, as now, cost estimates were as inaccurate and cost escalation as large.
Costs shrunk or inflated to order
Another explanation is that cost underestimations and escalations are intentional. They are part of power games played by project promoters and consultants aimed at getting projects started. Cost underestimation is used strategically to make projects appear less expensive than they really are in order to gain approval from decision-makers. Such behaviour best explains why cost escalations are so consistent over time, space and project type, as Flyvbjerg and Martin Wachs have described extensively.
One of the reasons that the evidence is so easily contested is due to its nature. The data presented in the Strategic Case for HS2 is predominantly predictive. How are we to know in 20 years’ time what the expected rail demand will be? Or how commuters will behave? The evidence is a forecast which leaves it open to much criticism and debate.
Those campaigning for the project need to accept that their data will be questioned and look at other ways to justify HS2. It is unlikely that someone whose home must be destroyed to make way for the new high-speed network is going to be sympathetic to evidence that suggests large cities will benefit by billions of pounds. Perhaps a new approach to justifying these large transport infrastructure projects is needed.
Hard Evidence is a series of articles in which academics use research evidence to tackle the trickiest public policy questions.