Tuesday, December 6, 2011

Power laws in finance

My latest column in Bloomberg looks very briefly at some of the basic mathematical patterns we know about in finance. Science has a long tradition of putting data and observation first. Look very carefully at what needs to be explained -- mathematical patterns that show up consistently in the data -- and then try to build simple models able to reproduce those patterns in a natural way.

This path has great promise in economic finance, although it hasn't been pursued very far until recently. My Bloomberg column gives a sketch of what is going on, but I'd like to give a few more details here and some links.

The patterns we find in finance are statistical regularities -- broad statistical patterns which show up in all markets studied, with an impressive similarity across markets in different countries and for markets in different instruments. The first regularity is the distribution of returns over various time intervals, which has been found generically to have broad power law tails -- "fat tails" -- implying that large fluctuations up or down are much more likely than they would be if markets fluctuated in keeping with normal Gaussian statistics. Anyone who read The Black Swan knows this.

This pattern has been established in a number of studies over the past 15 years or so, mostly by physicist Eugene Stanley of Boston University and colleagues. This paper from 1999 is perhaps the most notable, as it used enormous volumes of historical data to establish the fat tailed pattern for returns over times ranging from one minute up to about 4 days. One of the most powerful things about this approach is that it doesn't begin with any far reaching assumptions about human behaviour, the structure of financial markets or anything else, but only asks -- are there patterns in the data? As the authors note:
The most challenging difficulty in the study of a financial market is that the nature of the interactions between the different elements comprising the system is unknown, as is the way in which external factors affect it. Therefore, as a starting point, one may resort to empirical studies to help uncover the regularities or “empirical laws” that may govern financial markets.    
This strategy seems promising to physicists because it has worked in building theories of complex physical systems -- liquids, gases, magnets, superconductors -- for which it is also often impossible to know anything in great detail about the interactions between the molecules and atoms within. This hasn't prevented the development of powerful theories because, as it turns out, many of the precise details at the microscopic level DO NOT influence the large scale collective properties of the system. This has inspired physicists to think that the same may be true in financial markets -- at least some of the collective behaviour we see in markets, their macroscopic behaviour, may be quite insensitive to details about human decision making, market structure and so on.

The authors of this 1999 study summarized their findings as follows:


Several points of clarification. First, the result for the power law with exponent close to 3 is a result for the cumulative distribution. That is, the probability that a return will be greater than a certain value (not just equal to that value). Second, the fact that this value lies outside of the range [0,2] means that the process generating these fluctuations isn't a simple stationary random process with an identical and independent distribution for each time period. This was the idea initially proposed by Benoit Mandelbrot on the basis of the so-called Levy Stable distributions. This study and others have established that this idea can't work -- something more complicated is going on.

That complication is also referred to in the second paragraph above. If you take the data on returns at the one minute level, and randomize the order in which it appears, then you still get the same power law tails in the distribution of returns over one minute. That's the same data. But this new time series has different returns over longer times, generated by combining sequences of the one minute returns. The distribution over longer and longer times turns out to converge slowly to a Gaussian for the randomized data, meaning that the true fat tailed distribution over longer times has its origin in some rich and complex correlations in market movements at different times (which gets wiped out by the randomization). Again, we're not just dealing with a fixed probability distribution and independent changes over different intervals.

To read more about this, see this nice review by Xavier Gabaix of MIT. It covers this and many other power laws in finance and economics.

Now, the story gets even more interesting if you look past the mere distribution of returns and study the correlations between market movements at different times. Market movements are, of course, extremely hard to predict. But it is very interesting where the unpredictability comes in.

The so-called autocorrelation of the time series of market returns decays to zero after a few minutes. This is essentially a measure of how much the return now can be used to predict a return in the future. After a few minutes, there's nothing. This is the sense in which the markets are unpredictable. However, there are levels of predictability. It was discovered in the early 1990s, and has been confirmed many times since in different markets, that the time series of volatility -- the absolute value of the market return -- has long-term correlations, a kind of long-term memory. Technically, the autocorrelation of this time series only decays to zero very slowly.

This is shown below in the following figure (from a representative paper, again from the Boston University group) which shows the autocorrelation of the return time series g(t) and also of the volatility, which is the absolute value of g(t):



Clearly, whereas the first signal shows no correlations after about 10 minutes, the second shows correlations and predictability persisting out to times as long as 10,000 minutes, which is on the order of 10 days or so.

So, its the directionality of price movements which has very little predictability, whereas the magnitude of changes follows a process with much more interesting structure. It is in the record of this volatility that one sees potentially deep links to other physical processes, including earthquakes. A particularly interesting paper is this one, again by the Boston group, quantifying several ways in which market volatility obeys several quantitative laws known from earthquake science, especially the Omori Law describing how the probability of aftershocks decays following a main earthquake. This probability decays quite simply in proportion to 1/time since the main quake, meaning that aftershocks are most likely immediately afterward, and become progressively less likely with time. Episodes of high volatility appear to follow similar behaviour quite closely.

Perhaps even better is another study, which looks at the link to earthquakes with a somewhat tighter focus. The abstract captures the content quite well:
We analyze the memory in volatility by studying volatility return intervals, defined as the time between two consecutive fluctuations larger than a given threshold, in time periods following stock market crashes. Such an aftercrash period is characterized by the Omori law, which describes the decay in the rate of aftershocks of a given size with time t by a power law with exponent close to 1. A shock followed by such a power law decay in the rate is here called Omori process. We find self-similar features in the volatility. Specifically, within the aftercrash period there are smaller shocks that themselves constitute Omori processes on smaller scales, similar to the Omori process after the large crash. We call these smaller shocks subcrashes, which are followed by their own aftershocks. We also show that the Omori law holds not only after significant market crashes as shown by Lillo and Mantegna [Phys. Rev. E 68, 016119 2003], but also after “intermediate shocks.” ...
These are only a few of the power law type regularities now known to hold for most markets, with only very minor differences between markets. An important effort is to find ways to explain these regularities in simple and plausible market models. None of these patterns can be explained by anything in the standard economic theories of markets (the EMH etc). They can of course be reproduced by suitably generating time series using various methods, but that hardly counts as explanation -- that's just using time series generators to reproduce certain kinds of data.

The promise of finding these kinds of patterns is that they may strongly constrain the types of theories to be considered for markets, by ruling out all those which do not naturally give rise to this kind of statistical behaviour. This is where data matters most in science -- by proving that certain ideas, no matter how plausible they seem, don't work. This data has already stimulated the development of a number of different avenues for building market theories which can explain the basic statistics of markets, and in so doing go well beyond the achievements of traditional economics.

I'll have more to say on that in the near future.

19 comments:

  1. Why exactly is power-law distribution for asset returns inconsistent with EMH? It is trivial to write "standard" economic model where returns have fat tails, e.g. if we assume that stochastic process for dividends / firm profits has fat tails. That of course may not be very satisfactory explanation, but it still shows that EMH != normal distribution.

    In fact, Fama wrote about non-gaussian returns back in 1960's (and Mandelbrot before him), so the idea is not exactly new. The work you describe here is certainly useful and interesting, but pure patterns in data (or "stylized facts", as economists would call them) by themselves are not enough - we need some theory to make sense of them, and it would be interesting to hear more about contributions from econophysics in that area.

    ReplyDelete
  2. It's also worth pointing out that EMH, as I understand it, doesn't assume or dismiss that returns follow some specific distribution. Rather, EMH simply posits that prices reflect known information. For many years, analysts presumed that EMH implies a random distribution, but the empirical record says otherwise. But the random walk isn't a condition of EMH. Andrew Lo of MIT has discussed this point at length. The market may or may not be efficient, but it's not conditional on random price fluctuations.

    Separately, ivansmi makes a good point about models. You need a model to reject EMH. But that only brings you so far. Let's say we have a model of asset pricing that rejects EMH. Then the question is whether EMH or the model is wrong? That requires another model. In short, it's ultimately impossible to reject or accept EMH, unless of course you completely trust a given model. But that brings us back to square one. Welcome to economics.

    ReplyDelete
  3. Considering the autocorrelation, from 10 minutes to 10 days, this can still be reflect efficient market pricing assuming prices reflect the probabalistic uncertainty as to the return persistence, even if it is known that there will be some form of persistence?

    In one of Taleeb's books, didn't he make mention that something cannot be proven true, only disproven? I think it was the whole swan thing - if you have an appropriate sample and count 100% white swans does not prove there are ONLY white swans, while a sample that has a black one proves that there are not ONLY white swans.

    ReplyDelete
  4. You make a very good point, epistemologically, as regards the purpose of hypothesis testing. One can only seek to dismiss the null hypothesis, which of itself causally does not prove that alternative hypothesis is therefore true. And hence data does not prove that certain ideas must be true. It confirms, alternatively, that certain ideas are statistically unlikely to be the case. Perhaps that has been part of the problem - the statistically rejection of a null hypothesis in financial models has led to a belief that the alternative hypothesis must hold good?

    ReplyDelete
  5. By using the title "The physics of Finance", the author intends nothing but fraud. There is nothing close to physics that authors or anybody else have derived from finance. Large number of data and modern computers make it very easy to observe data. But to conclude anything in terms of physical law or Boltzman style observation has never been achieved.

    When the author using fat tail without even bother the mathematical equation (it has to be universally accepted), it deserves a big laugh.

    ReplyDelete
    Replies
    1. I think you are misreading the idea of using Physics. It is not we drive Physical laws from finance but rather apply known physical laws in finance, which is shown by distinguished mathematician like Mandelbrot. The physics of finance is one of the developing fields in research what is called 'econophysics'. There is now large literature on statistical mechanics of complex systems. Financial systems are not totally artificial as it is thought to be but manifest similarities. I understand your critique but calling it total fraud is little unfair and you need back up your argument with literature and evidence. I would first approach with a good faith to other people's work before accusing them with fraud.

      Delete
  6. Dear Anon...
    This sort of analysis is long overdue for the financial markets, for the markets in stocks and bonds, and for the instruments derived from those basic elements.
    The last twenty years or so have seen endless amounts of networked computing power applied to these markets. The tools of statistical physics, of dynamical systems theory, and of network theory, as well as those of mathematical ecology - all of which have emerged to deal with complex systems - are exactly the right ones to be used in these analyses.
    It's high time that the untested notions of economic theory, such as the efficient market hypothesis, be subjected to real scientific scrutiny. Those notions emerged in a data-poor past. To carry them forward blindly turns them into religious dogma.
    I'm frankly appalled at what passes, in much of economics and finance, for knowledge. It is nothing of the sort. Those who called themselves quants blindly implemented ideas, such as the Gaussian copula, that proved disastrous in the current computing environment. In particular, the assumption of uncorrelated assets was completely destructive of the marketplace. It shows an appalling lack of understanding about the reality of this environment. Time to get down to facts.

    ReplyDelete
  7. Medical accountants:Health professionals can take the best tax solutions from professional medical accountants. It is not possible to manage big accounts without professional accountant.

    ReplyDelete
  8. Thanks for this post. It provides some great insight. Lawyers In Kent

    ReplyDelete
  9. A particularly interesting paper is this one, again by the Boston group, quantifying several ways in which market volatility obeys several quantitative laws known from earthquake science, especially the Omori Law describing how the probability of aftershocks decays following a main earthquake.
    gold ira accounts

    ReplyDelete
  10. Well, truly said, finance sector can be seen by many aspects. Even it can be analyzed not only with physics but it can be analyzed through various other sectors.
    --Moises Maionica

    ReplyDelete
  11. There are many probs beginning in this law.. So I think we should study first the previous law records.

    ReplyDelete
  12. It has come to my attention that a solution to the puzzle of why large fluctuations in prices have an inverse power law distribution goes unanswered. This is an easy corollary to the principal result of my 1999 book, Axiomatic Theory of Economics, Theorem 12, the Law of Price Adjustment.

    My theory describes a single instant in time. Of course, instants have the habit of following one another, eventually forming something called history. So it is natural to inquire, not just what the price is at a given instant, but what the distribution of price changes is over history. This I did not do in 1999.

    Addressing this issue requires an additional axiom: the parameter called importance, μ, must have an exponential distribution, exp(-μ/θ)/θ.

    In point of fact, the only restriction that I placed on μ is that it be a positive real number. But negative monotonic distributions with this support and moments of all orders are not that numerous. Why not the exponential? The fact that an exponential distribution of μ implies an inverse power law distribution of price fluctuations is motivation enough for most economists to accept this new axiom, and my other results require only that the distribution of μ be negative monotonic on [0,∞).

    ReplyDelete
  13. These people want to end up their suffering of low living standard by working of Finance business on small-scale but usually because of poverty they are not able to put capital to start their prize bond business,
    so they invest their small savings into prize bonds and then by winning high prize bond money,

    ReplyDelete
  14. Cool blog site friend I'm about to suggest this to all my listing contacts.faxless cash loans

    ReplyDelete
  15. This is very important part to be served in the market to gain the law of finance in the market for the good one trend.
    Free Stock Tips on Mobile

    ReplyDelete
  16. The NSE BSE Tips have been proved to be very much helpful in gathering the personal finance.

    ReplyDelete
  17. Here your post about Power laws in finance is very helpful for me. I have never read this type of informative article before.Hope you will update us form all relevant information in future.

    ReplyDelete