There have been a lot of questions following Australia’s below-expectation performance at the London Olympics. But the question I keep asking myself is, how did so many “experts” get their medal tally predictions so wrong?
Prior to the Olympics, the Australian Olympic Committee (AOC) boldly predicted Australia would win 15 gold medals and finish 5th on the medal table. They were right about one thing – 15 gold medals would have placed Australia 5th on the gold-medal table.
But history will show that in contrast to these lofty predictions, Australia scraped into 10th place with less than half that number: seven gold medals (and 35 medals in total).
The AOC were not alone in getting it wrong. Economists at Goldman Sachs used a raft of economic, political, productivity and growth information to also predict Australia would finish with 15 gold medals.
Similarly, economics researchers at the Ruhr University Bochum, Germany used economic, demographic, and cultural factors to predict Australia would win 43 medals (they did not predict the number of gold medals). Australia finished the Games with 35 medals.
So, how did so many experts get it so wrong? I can think of at least three reasons.
First, most of these predictions take current trends (such as past Olympic results) and project them into the future. In fact, despite what seems like the use of sophisticated modelling, for most countries the difference between the predictions of Goldman Sachs and the actual results from the 2008 Olympics was less than one gold medal.
The only exceptions were China and Great Britain where they factored in a home-games effect.
Relying on current trends to continue is generally a pretty safe bet, but not always. Trends end, surprises happen, and predicting the future by projecting the past is like driving with your eyes shut – a good chance of success on the Gunbarrel Highway, but try it on the London M5 motorway and you’re headed for trouble.
The second reason is that none of the predictions took into account the effect a resurgent Great Britain would have on Australia’s medal chances.
Research by Goldman Sachs has shown that hosting the Olympic Games can increase the host nation’s medal tally by more than 50% with respect to what they would have otherwise attained. Given the shared sporting traditions of Great Britain and Australia, it seems logical that more medals to Great Britain is precisely the subtle curve in the road that can disrupt past trends and reduce Australia’s medal haul.
In fact, on five occasions British athletes relegated Australians to the silver medal (women’s pair, women’s double sculls, and men’s four in rowing; men’s team cycling pursuit and the men’s long jump).
A third, and much-discussed possibility, is that Australia underperformed. I won’t enter into this debate here but, of all the countries at the 2012 Olympics, Australia had the largest difference between their actual and predicted number of gold medals (except for the USA who won nine more than predicted). Underperformance is obviously something that is very difficult to include into a model.
A final question worth asking is what purpose do these (inaccurate) predictions serve? A famous rebuttal to Nobel laureate Kenneth Arrow, when he was a young statistician during the Second World War, was:
“The Commanding General is well aware the forecasts are no good. However, he needs them for planning purposes.”
So, apart from a little harmless fun, can we actually use these predictions to plan for future Olympics?
Refreshingly, sport remains unpredictable, and it is this not-knowing who will perform at their best in four years’ time that makes the Olympics so compelling. Undoubtedly there are some trends – the host nation normally improves their medal haul, and the previous host nation typically has a slight decrease in their number of medals.
Apart from that, in the words of retired English soccer star Paul Gascoigne: “I never make predictions and I never will”.