Monday, January 14, 2013

Steve Keen on "bad weathermen"

I've made quite a lot of the analogy between the dynamics of an economy or financial market and the weather. It's one of the basic themes of this blog, and the focus of my forthcoming book FORECAST. I don't pretend to be the first one to think of this at all. I know that the head of the Bank of England Mervyn King has talked about this analogy in the past, as have many others.

But the idea now seems to be gathering more popularity. Steve Keen even writes here specifically about the task of economic forecasting, and the entirely different approaches used on weather science, where forecasting is now quite successful, and in economics, where it is not:
Conventional economic modelling tools can extrapolate forward existing trends fairly well – if those trends continue. But they are as hopeless at forecasting a changing economic world as weather forecasts would be, if weather forecasters assumed that, because yesterday’s temperature was 29 degrees Celsius and today’s was 30, tomorrow’s will be 31 – and in a year it will be 395 degrees.

Of course, weather forecasters don’t do that. When the Bureau of Meteorology forecasts that the maximum temperature in Sydney on January 16 to January 19 will be respectively 29, 30, 35 and 25 degrees, it is reporting the results of a family of computer models that generate a forecast of future weather patterns that is, by and large, accurate over the time horizon the models attempt to predict – which is about a week.
Weather forecasts have also improved dramatically over the last 40 years – so much so that even an enormous event like Hurricane Sandy was predicted accurately almost a week in advance, which gave people plenty of time to prepare for the devastation when it arrived:

Almost five days prior to landfall, the National Hurricane Center pegged the prediction for Hurricane Sandy, correctly placing southern New Jersey near the centre of its track forecast. This long lead time was critical for preparation efforts from the Mid-Atlantic to the Northeast and no doubt saved lives.

Hurricane forecasting has come a long way in the last few decades. In 1970, the average error in track forecasts three days into the future was 518 miles. That error shrunk to 345 miles in 1990. From 2007-2011, it dropped to 138 miles. Yet for Sandy, it was a remarkably low 71 miles, according to preliminary numbers from the National Hurricane Center.

Within 48 hours, the forecast came into even sharper focus, with a forecast error of just 48 miles, compared to an average error of 96 miles over the last five years.

Meteorological model predictions are regularly attenuated by experienced meteorologists, who nudge numbers that experience tells them are probably wrong. But they start with a model of the weather than is fundamentally accurate, because it is founded on the proposition that the weather is unstable.

Conventional economic models, on the other hand, assume that the economy is stable, and will return to an 'equilibrium growth path' after it has been dislodged from it by some 'exogenous shock'. So most so-called predictions are instead just assumptions that the economy will converge back to its long-term growth average very rapidly (if your economist is a Freshwater type) or somewhat slowly (if he’s a Saltwater croc).

Weather forecasters used to be as bad as this, because they too used statistical models that assumed the weather was in or near equilibrium, and their forecasts were basically linearly extrapolations of current trends.
How did weather forecasters get better? By recognizing, of course, the inherent role of positive feed backs and instabilities in the atmosphere, and by developing methods to explore and follow the growth of such instabilities mathematically. That meant modelling in detail the actual  fine scale workings of the atmosphere and using computers to follow the interactions of those details. The same will almost certainly be true in economics. Forecasting will require both lots of data and also much more detailed models of the interactions among people, firms and financial institutions of all kinds, taking the real structure of networks into account, using real data to build models of behaviour and so on. All this means giving up tidy analytical solutions, of course, and even computer models that insist the economy must exist in a nice tidy equilibrium. Science begins by taking reality seriously.

5 comments:

  1. Have you read the book from Nate Silver, "The signal and the noise" already? It has many analogies between these two, too.

    ReplyDelete
  2. Mark, I just found your blog. Quite interesting!

    I have little experience with these modern weather models (Other than seeing their output on TV screens). Are they actually geospatial system dynamic models with stocks and flows? Is there a Monte Carlo component -- a random number generator and assumed distribution from which it's drawn?

    Also, it seems as if you assume that stock price time series will be as forecastable as the track of an already-formed hurricane. What if it's more like an earthquake: a critical-state system with an upcoming phase change at an unforecastable time?

    More importantly, how do we know when we have one type of forecasting problem vs. the other?

    ReplyDelete
  3. Hi Ken,

    Good questions! Weather-inspired economic/financial models of the kind I envision don't really exist yet, although we have quite a few simplified models that show the kind of things that might be done. A good example is the model I wrote about here (http://physicsoffinance.blogspot.co.uk/2011/07/leverage-control-for-market-stability.html) which looks at how leverage can grow in a market through competition as investment funds try to attract investors. The ultimate outcome is closely akin to a storm - the market becomes highly unstable and you have an eventual crash. The high likelihood of such a crash is predictable, although the precise timing is not (this does depend on a random triggering fluctuation, creating a kind of nucleation event).

    Now, this is just a conceptual model -- akin to something a computational fluid dynamics person might run to explore fundamental aspects of some complex flow pattern. Using this kind of model to actually do something more like weather forecasting would require the availability of much better real time data on the behaviours and strategies in use by firms in the market. This would require new regulation to make this kind of data available.

    On the matter of predicting actual stock movements and rice trajectories, I'm not sure that will be possible. The kinds of prediction I think will be possible is just what you mention - prediction of more global properties and collective modes of behaviour. For example, we might use models to probe and predict the existence of boundaries of stability for markets or economies that would be dangerous to cross (too much leverage, too much clustering of different people or funds in the space of investing strategies, etc). And, with real data, it might be possible to track an economy or market and get early warnings of an approach toward boundaries.

    ReplyDelete
  4. Actually, economists are better at forecasting the weather than meteorologisits: Economic Logic post.

    ReplyDelete