Unpredictability is omnipresent in life, from dealing playing cards, through contracting an illness or getting promoted, to the rise or fall of equity markets. For those attempting to make the calls on how economies will fare it is important to distinguish between three different forms of unpredictability. Rolling dice conveniently illustrates all three.
With a well-balanced six-sided die, the probabilities of all possible outcomes can be calculated in advance. For example, getting three sixes in a row should happen on average once every 216 times. We call this intrinsic unpredictability, as it depends only on the random properties that are intrinsic to rolling the dice.
Getting ten sixes in a row is very unlikely, once in 60 million tries, but it can occur – and would (probably) if a billion people played dice games every day. When such a rare event happens, it is sometimes called a “black swan”: we call it “instance unpredictability” because no one could predict in advance when such an instance might happen or how large it might be.
Suppose you were unaware that a cheat had fixed a magnet to increase the chances that a six turned up when they rolled, but switched it off at your turn. The probabilities of outcomes are then not what you calculated, and have shifted from the first case. We call this extrinsic unpredictability, as it depends on events outside the usual calculations. In a sense, flocks of black swans suddenly arrive; outcomes occur that would be very unlikely under random sampling in the previous situation. Unless the cheat was careless, you could take a long time to learn that the odds had altered.
Model behaviour
Innumerable, often unexpected, changes have occurred historically in legislation, science, medicine, climate, politics and economic regimes among others. Despite the many pioneering developments in both economic theory and econometric methods over the last century, macroeconomic models often fail to deliver reliable forecasts and policy analyses. To quote Alan Greenspan: “The Fed’s models fell apart” during the financial crisis – and the Fed was not alone. The kind of forecasting debacles we have seen in the financial crisis and Great Recession are almost all due to these unanticipated shifts.
The implications of extrinsic unpredictability for economic analyses, econometric modelling and forecasting are surprisingly different: major theoretical problems arise when unanticipated shifts occur and forecasts can be systematically wrong. When an unanticipated location shift occurs (as in the figure below where the mean shifts from 0 to -5), today’s expectation can be a poor estimate of tomorrow’s outcome.
Extrinsic unpredictability wrecks people’s plans from one period of time to the next as their expectations for the next period can be far from what transpires. That then makes it irrational to form future expectations based on previous evidence, and invalidates most inter-temporal analyses.
The Bank of England recently abandoned its policy analysis system called BEQM (the bend it like BEQM system that bent too much?), albeit replacing it with another of the same type. We see a major danger with that replacement. The mathematical basis of such dynamic stochastic general equilibrium models (known by their acronym DSGEs and designed to describe aggregate economic trends) assumes expectations formation that ignores unanticipated shifts, so fails precisely when an economic model is most needed. During a crisis, DSGEs crash.
Assuming the present distribution remains constant when it does not is about as useful as changing the measurements of ingredients in a recipe from metric units to imperial when half way through. In a world plagued by intermittent unanticipated shifts, economists cannot rely on theory-based models alone, unless that theory uses a new mathematics which can adapt and evolve.
Changeability
However, once shifts have occurred, they can be modelled empirically, and robust forecasting devices can correct rapidly to a new equilibrium, so “traditional” economic analysis is the main victim from sudden shifts. When one of the authors of this piece, David Hendry, was delivering a course on economic forecasting in the face of shifts at the Bank of Japan in March 2009, news arrived that Japanese exports had fallen 40% year-on-year: that dramatic drop was completely unanticipated and the Bank of Japan’s forecasts were badly wrong.
Asked what our methods would forecast for the next month, Hendry proposed another 40% fall – to the horror of those attending, who felt a “reversion to the mean” would occur. It did not: exports eventually fell 70% year-on-year before recovering as the graph above shows (the solid red line is actual and the dashed lines are the alternative forecasts). The “conventional forecasts” of Japanese exports are systematically wrong (above during the fall, below during the recovery), as happens typically with equilibrium correction formulations. Our robust method is a bit more erratic, but avoids systematic failure. Put simply, a conventional forecast extrapolates past behaviour, whereas a robust forecast adapts quickly to incorporate the effect of a change once it has happened.
There is some poetic justice that those DSGE models for making economic predictions crash and burn when the black swans arrive. The most strident advocates of these models were the same people who hailed the death of Keynesian models when they failed after the first Oil Crisis. It turns out both classes of model were the victims of the unanticipated shifts that play havoc when economists start rolling the dice.