The infallible portfolio?

In this short essay, Ricardo Fenholz of Columbia University makes what seems (to me at least) to be a rather incredible claim: that it's relatively easy to construct a portfolio that is guaranteed to outperform the S&P 500 over one year (or any other interval you like) and also has a limited downside during that year. The idea is the take the S&P 500 Index and tweak it a little, creating a portfolio with less weight in stocks with higher capitalization, and more weight in those with less capitalization, and presto -- you have something guaranteed to outperform the S&P 500, he claims. Is that possible? That easy? Here's a little more detail:

To understand how this works, let’s consider the S&P 500 U.S. stock index. Suppose that we wish to invest some money in S&P 500 stocks for one year. Currently, Apple has a total market capitalization of roughly $500 billion, making it the largest stock in the S&P 500 and equal to approximately 4% of the total capitalization of the entire index. Suppose that we believe it is very unlikely or impossible that either Apple or any other corporation’s capitalization will be equal to more than 99% of the total S&P 500 capitalization for this entire year during which we plan to invest. As long as this turns out to be true, then it is actually pretty simple to construct a portfolio containing S&P 500 stocks that is guaranteed to outperform the S&P 500 index over the course of the year and that has a limited downside relative to this index. In essence, we can construct a portfolio that will never fall below the value of the S&P 500 index by more than, say, 5% and that is guaranteed to achieve a higher value than the S&P 500 index by the end of the year.[1]

This is not a trivial proposition. If we combine a long position in this outperforming portfolio together with a short position in the S&P 500 index, then we have a trading strategy that requires no initial investment, has a limited downside, and is guaranteed to produce positive wealth by the end of the year. According to standard financial theory, this should not be possible.[2] Furthermore, the assumptions that guarantee that our portfolio will outperform the S&P 500 index appear entirely reasonable. After all, not for one day in the more than 50-year history of the S&P 500 has one corporation’s market capitalization come anywhere close to equaling even 50% of the total capitalization of the market. A 99% share of total market capitalization would essentially amount to there being only one corporation in the entire U. S. for an entire year. This seems like neither a likely outcome nor one that investors should take seriously when constructing their portfolios.

What does a portfolio made up of S&P 500 stocks that is guaranteed to outperform the S&P 500 index look like? There are many different ways in which such a portfolio can be constructed, but one feature common to all such portfolios is that relative to the S&P 500 index itself, they place more weight on those stocks with small total market capitalizations and less weight on those stocks with large total market capitalizations. The weight that an index such as the S&P 500 places on each individual stock is equal to the ratio of that stock’s total market capitalization relative to all stocks’ total market capitalizations taken together. In the case of Apple, then, the S&P 500 index would place a weight of roughly 4% in this individual stock while those portfolios that use HFT to outperform this index would instead place a weight of less than 4% in Apple stock.

The only condition for this to work, he suggests, is that the assumption that no stock in the market comes to dominate the market in the sense that its market capitalization comes to be a high fraction of that of the entire market. This is, as he notes, a fairly weak assumption, although the weaker you make the assumption, the longer the time interval over which this idea apparently works.

Now, I'm not doubting the veracity of this claim. I'm just stunned that such a simple recipe could work, and can't see the intuition behind it. What if the high cap stocks happen to perform brilliantly next year, relative to the lower cap stocks? Wouldn't this portfolio with underweighted high cap stocks then underperform the S&P Index? I've had a quick look at the paper Fernholz references as a detailed support of his claim, and there he explains the conditions for the theorem to hold in slightly different terms:

The conditions mandate, roughly, that the largest stock have "strongly negative" rate of growth, resulting in a sufficiently strong repelling drift away from an appropriate boundary; and that all other stocks have "sufficiently high" rates of growth.

That sounds very different from the quite plausible assumption about no market dominance of a single stock. Indeed, this seems like saying that, if one assumes that large cap stocks will perform poorly, and small caps stock better, then we can build a portfolio guaranteed to outperform the S&P 500 Index by weighting small cap stocks more heavily. Isn't that like assuming we know the future?

But maybe I'm wrong. I'd be interested in the thoughts of others. The paper is quite dense and light on intuitive discussion of the logic. Fernholz suggests that perhaps the existence of these superior portfolios -- which require continuous rebalancing by high-frequency buying and selling of many stocks -- explains some of the very high profits consistently earned by quantitative high-frequency hedge funds such as Renaissance Technology's Medallion Fund. I think I find more convincing the analysis of Lo and Khandani which seemed to suggest that much of the performance of quant hedge funds over the past decade or so can be accounted for by fairly vanilla long short equity strategies, with increasing use of leverage in the mid 2000s (used to maintain high reported earnings even as raw earnings fell off due to competition).

Why time matters

I'm still thinking about the ideas of Ole Peters and the importance difference between time and ensemble averages. A few comments suggest that some people I have "lost the plot," but I'm convinced this issue is indeed extremely important and generally underappreciated. A few things to add for now:

1. This post by economist Lars Syll from earlier this year does an excellent job of laying out the main issues and linking them to the Kelly criterion: a practical criterion for playing risky gambles that is based explicitly on time averages. Lars couldn't have explained the basic ideas more clearly.

2. From some comments on other blogs, similar to some I've seen here, many people familiar with probability theory find it hard to accept that a time average expected return of a random multiplicative process is just not equal to the (usual) expected return of a single round. It isn't. Start with any number you like, multiply it by a long sequence of numbers, each either 0.9 or 1.1 drawn with equal probability, and you will find that the number tends to get smaller. In the limit of an infinite sequence, the result heads to 0. And the result quoted in the Towers and Watson paper, a 1% decline on average per period, is correct.

 3. I came across an interesting comment from Tim Johnson, writing on Rick Bookstaber's blog:

The model Peters develops appears to be remarkably similar to the one Durand proposed in 1957 (The Journal of Finance, 12, 348–363) and is discussed by Szezkely and Richards (The American Statistician, 2004, Vol. 58, No. 3).

I do not disagree with your assessment that there has been an error in economics for the past 77 years (just one?) but mathematicians working in finance have generally ignored Samuelson's attacks on logarithmic utility. Poundstone's book on the Kelly Criterion is a good description of the battle in the 1960s and I there is a rich contemporary literature that develops Kelly's ideas...

The paper by Szezkely and Richards is indeed worth a read, although I'm convinced that Peters has gone considerably further than Durand.

Ultimate Limits to Growth

My latest essay in Bloomberg touched on ultimate limits to energy growth (and quite possibly economic growth) due to the accumulation of waste energy in the environment. It's really just an exercise in taking basic physics into account while extrapolating trends in energy use into the future. The conclusion is that continued exponential growth in energy use -- which we've experienced over the past few centuries (and possibly much longer) -- cannot last for much more than a century or so. What about economic growth? We don't know. Economists theorize about a great "decoupling" of energy from economic productivity, but that hasn't happened so far in any country. My conclusion is that economic growth must also end, fairly soon (by which I mean, say, 100 years) -- unless we transform our economic activity to involve far less energy and in a way we have never done before.

I noticed that Noah Smith has a post criticizing physicist Tom Murphy, who I cited in my article. I love Noah's blog and read everything he writes, but I don't think this is his fairest criticism, although parts are fair. Certainly, I can see that economists might feel that their best arguments weren't put forward in the dialogue Murphy recalled between himself and an economist, the subject of Murphy's most widely read post. (Note: I didn't cite that particular post Noah refers to, but this one of Murphy's which looks at trends in energy growth alone.) But reading Noah, I'm led to believe that my understanding of the prevailing view among modern economists on economic growth is hugely mistaken. Indeed, he makes it sound as if economists generally accept that growth must end, and fairly soon (if true, I' very happy about that).

Murphy made the point that, if we extrapolate our current and past energy growth into the future, then we will actually boil the oceans in 400 years (with 2.3% energy growth; sooner with fast growth). To this Noah responds,

This is correct. And in fact, Murphy didn't even need to mention waste heat or anything like that to make his argument; he could have just said "Hey, eventually the Sun will explode, and then the whole Universe will degrade into heat, and where will your economy be then?" So what if that happens 500 million years in the future, or 10^100 years? What's the difference? One way or another, the human race is kaput!

Yes, of course. But the point is that 400 years is not very long. And we don't need the oceans boiling before we would see important temperature change (and other associated environmental changes) that would make life rather uncomfortable. I think Murphy is right that most people do not appreciate how soon in the future (soon on a timescale of human history) continued growth of energy use brings problems. This isn't a problem set some 5 billion years in the future. This is one reason I think so many people have found Murphy's posts worth reading: this seems really surprising to them.

Noah goes on:

Are economists ignoring this basic fact? Do economists' models crucially hinge on the idea that economic growth will continue forever and ever and ever? No. The "long term trend growth" that is used in growth and business cycle models is only meant to represent a trend that lasts longer than the business cycle - so, longer than a decade or two. No economist - I hope - thinks that currently living humans are making economic decisions based on what they think is going to happen in 400 years, or 2500 years, or 500 million years.

Again, I think this is just the point Murphy is trying to make -- that if these effects are looming only 400 years (or significantly less) in the future, then perhaps contemporary humans ought to be taking them into account now in making their economic decisions. Certainly it would be appropriate for our leaders to be casting an eye on this long term, and to seek advice from our best economists who might help them think clearly about how our society could manage to change in response. Does economics end with the business cycle? Nothing longer term than that? 400 years is perhaps only 20-30 business cycles away.

These are the main points I think Murphy was trying to make. I do agree with Noah that other parts of Murphy's original post are much less convincing. When he moves into proper economic territory, discussing prices, scarcity of future energy, etc., my own feeling was to take that all with a grain of salt, as quite a lot of speculation.

In any event, I'm glad Noah has brought the attention of more economists to the quite short timescale on which continued energy growth leads to problems. If this is already well understood in economics, and built into theories of growth, then great, I've learned something. In my experience a lot of people think we'll be fine and continue growth if we can only find some cheap, infinite and non-polluting energy source to power our future. That's not the case.  

Why expected value is a mistake

I want to take a closer look at the very interesting work of Ole Peters I mentioned in my last post. He argues that the ensemble averages typically used in economics and finance to compute "expected" returns are, in many cases, inappropriate to making decisions in the real world; in particular, they severely underestimate risks. Peters begins with a simple gamble:

Let’s say I offer you the following gamble: You roll a dice, and if you throw a six, I will give you one hundred times your total wealth. Anything else, and you have to give me all that you own, including your retirement savings and your favorite pair of socks. I should point out that I am fantastically rich, and you needn’t worry about my ability to pay up, even in these challenging times. Should you do it? ... The rational answer seems to be “yes”—the expected return on your investment is 1,583 1/3% in the time it takes to throw a dice. But what’s your gut feeling?

As he notes, almost no real person would take this bet. You have 5 chances out of 6 of being left destitute, one of being made very much wealthier. Somehow, most of us weight outcomes differently than the simple and supposedly "rational" perspective of maximizing expected return. Why is this? Are we making an error? Or is there some wisdom in this?

Peters' gamble is a variation on the famous St Petersburg "paradox" proposed originally by Nicolas Bernoulli, and later discussed by his brother Daniel. There the question is to determine how much a rational individual should be willing to pay to play a lottery based on a coin flip. In the lottery, if the first flip is heads, you win $1. If the first is tails, you flip again. If the coin now comes up heads, you win $2, otherwise you flip again, and so on. The lottery pays out 2^n (^ meaning exponent) dollars if the head comes up on the nth roll. An easy calculation shows that the expected payout of the lottery is infinite -- given by a sum that does not converge (1*1/2 + 2*(1/2)^2 + 4*(1/2)^3 + ...) = (1/2 + 1/2 + 1/2 + ...). The "paradox" is again why real people do not find this lottery infinitely appealing and generally offer less than $10 or so to play.

This is a paradox, of course, only if you have some reason to think that people should act according to the precepts of maximizing expected return. Are there any such reasons? I don't know enough history of economics and decision theory to say if there are -- perhaps it can be shown that such behavior is rational in some specific sense, i.e. in accordance to some set of axioms? But if so, what the paradox really seems to establish, then, is the limited relevance of such rules to living in a real world (that such rules capture an ineffective version of rationality). Peters' resolution of the paradox shows why (at least for my money!).

His basic idea is that we live in time, and act in time, and have absolutely no choice in the matter. Hence, the most natural way to consider the likely payoff coming from any gamble is to imagine playing the gamble many times in a row (rather than many times simultaneously, as in the ensemble average). Do this indefinitely and you should encounter all the possible outcomes, both good and bad. Mathematically, this way of thinking leads Peters to consider the time average of the growth rate (log return) of the wealth of a player who begins with wealth W and plays the gamble over N periods, in the limit as N goes to infinity. In his paper he goes through a simple calculation and finds the formula for this growth rate:

               
The third line here is explicitly for the St Petersburg lottery, while the second line holds more generally for any gamble with probability p_i of giving a return r_i (with the sum extending over all possible outcomes).

This immediately gives more sensible guidance on the St Petersburg paradox, as this expected growth rate is positive for cost c sufficiently low, and negative when c becomes too high. Most importantly, how much you ought to be willing to pay depends on your initial wealth w, as this determines how much you can afford to lose before going broke. Notice that this aspect doesn't figure in the ensemble average in any way. It's an initial condition that actually makes the gamble different for players of different wealth. Coincidentally, this result is identical to a solution to the paradox originally proposed by Daniel Bernoulli, who simply postulated a logarithmic utility and supposed that people try to maximize utility, not raw wealth. This idea reflects the fact that further riches tend to matter relatively less to people with more money. In contrast, Peters result emerges without any such arbitrary utility assumptions (plausible though they may be). It is simply the realistic expected growth rate for a person playing this game many times, starting with wealth W. Putting numbers in shows that the payoff becomes positive for a millionaire for a cost c less than around $10. Someone with only $1000 shouldn't be willing to pay more than about $6.

It's also useful to go back and work things out for the simpler dice game. One thing to note about the formula is that the average growth rate is NEGATIVE INFINITE for any gamble in which a person stands to lose their entire wealth in one go, no matter how unlikely the outcome. This is true of the dice gamble as laid out before. I was wondering whether this really made any sense, but after some further exploration I now think it does. The secret is to again consider that the person playing has wealth W and that the cost of "losing" isn't the entire wealth, but some cost c. A simple calculation then shows that the time average growth rate for the dice game takes the form shown in the figure below, showing the growth rate versus the c/w, the cost as a fraction of the players' wealth.

Here you see that the payoff is positive, and the gamble worth taking, if the cost is less than about 60% of the player's wealth. If more than that, the time average growth rate is negative. And, if becomes strongly more negative as c/w approaches 1, with the original game recovered for c/w=1. Again, everything makes more sense when a person's initial wealth is taken into account. This initial condition really matters and the question of the likely payoff of a gamble depends strongly on it, as lower wealth means higher chance of going bankrupt quicker and then being out of the game entirely. The possibility of losing all your wealth on one turn, no matter how unlikely, becomes decisive because this becomes certain in the long run.

Again, this way of thinking likely has significance far beyond this paradox. It's really pointing out that ensemble averages are very misleading as guides to decision making, especially when the quantities in question, potential gains and losses, become larger. If they remain small compared to the overall wealth of a person (or a portfolio), then the ensemble and time averages turn out to be the same, giving a formula in which initial wealth doesn't matter. But when potential gains/losses become large, then the initial condition really does matter and the ensemble average is dangerous. These points are made very well in this Towers Watson article I mentioned in an earlier post.

Which brings me to one final point. Ivan in comments suggested that perhaps Peters has changed the initial problem by looking at the time average rather than the ensemble average, and so has not actually resolved the St Petersburg paradox. I'm not yet entirely sure what I think about this. The paradox, if I'm right, is why people don't act in accordance with the precepts of expected return calculated using the ensemble average. To my mind, Peters' perspective resolves this entirely as it shows that this ensemble average simply gives very poor advice on many occasions. In particular, it makes it seem that a person's initial wealth should have no bearing on the question. If you face gambles, and face them repeatedly as we all do throughout life in one form or another, then thinking of facing them sequentially, as we do, makes sense. But that's not, as I say, my final view..... this is one of those things that gets deeper and deeper the more you mull it over....

Ergodicity -- the Biggest Mistake in Economics?

I'm increasingly convinced that Ole Peters has identified the nub of an utterly essential problem in the framework of contemporary (i.e. last 50 years) economics. In a series of recent papers (here, here, here), he has argued with impressive clarity that the usual ensemble averages used to compute "expected" returns in finance are, in many cases, simply inappropriate to making decisions in the real world. Take a risky gamble, and the usual average over different outcomes mixes potential worlds in which we go broke with others in we get rich, and, importantly, takes the often irreversible consequences of these outcomes (bankruptcy, for example) out of the picture. If you make hugely risky investments, this average gives you full credit for all the wonderful possible outcomes, weighted appropriately for their likelihood, which of course seems sensible. What it doesn't do is account for the very real fact that the bad outcomes may effectively wipe you out entirely and take you out of the game, making it impossible to play again -- in which case you will never get to experience those eventual big payoffs.

Maybe the best thing to read about this is this wonderful paper by people from the financial firm Towers Watson (credit: I learned of this from Rick Bookstaber's blog). The potential implications of this are really huge, as Peters' perspective suggests that the standard way of assessing risk versus reward in financial economics is wrong and systematically underestimates risks (and not merely because it ignores fat tails). The paper above, the first paper of Peters I mentioned above, and this interview with Peters are among the most interesting things I've read this year.

I'm going to do an in depth post on this stuff soon, but I must admit that I need to study it in detail a little more. I'm convinced that Peters insight -- which brilliantly resolves the centuries old "St Petersburg paradox" of probability theory proposed originally by Bernoulli -- also has a lot to do with the work of Doyne Farmer and John Geanakoplos on economic discounting, which I've written about before. Both suggest that our basic thinking about probability in time series suffers from some terrible misconceptions, and generally makes us underestimate risks. More coming on this soon.