Friday, September 16, 2011

Milton Friedman's grand illusion

Three years ago I wrote an Op-Ed for the New York Times on the need for radical change in the way economists model whole economies. Today's General Equilibrium models -- and their slightly more sophisticated cousins, Dynamic Stochastic General Equilibrium models -- make assumptions with no basis in reality. For example, there is no financial sector in these model economies. They generally assume that the diversity of behaviour of all an economy's many firms and consumers can be ignored and simply included as the average behaviour of a few "representative" agents.

I argued then that it was about time economists started using far more sophisticated modeling tools, including agent based models, in which the diversity of interactions among economic agents can be included along with a financial sector. The idea is to model the simpler behaviours of agents as well as you can and let the macro-scale complex behaviour of the economy emerge naturally out of them, without making any restrictive assumptions about what kinds of things can or cannot happen in the larger economy. This kind of work is going forward rapidly. For some detail, I recommend this talk earlier this month by Doyne Farmer.

After that Op-Ed I received quite a number of emails from economists defending the General Equilibrium approach. Several of them mentioned Milton Friedman in their defense, saying that he had shown long ago that one shouldn't worry about the realism of the assumptions in a theory, but only about the accuracy of its predictions. I eventually found the paper to which they were referring, a classic in economic history which has exerted a huge influence over economists over the past half century. I recently re-read the paper and wanted to make a few comments on Friedman's main argument. It rests entirely, I think, on a devious or slippery use of words which makes it possible to give a sensible sounding argument for what is actually a ridiculous proposition. 

The paper is entitled The Methodology of Positive Economics and was first published in 1953. It's an interesting paper and enjoyable to read. Essentially, it seems, Friedman's aim is to argue for scientific standards for economics akin to those used in physics. He begins by making a clear definition of what he means by "positive economics," which aims to be free from any particular ethical position or normative judgments. As he wrote, positive economics deals with...
"what is," not with "what ought to be." Its task is to provide a system of generalizations that can be used to make correct predictions about the consequences of any change in circumstances. Its performance is to be judged by the precision, scope, and conformity with experience of the predictions it yields.
Friedman then asks how one should judge the validity of a hypothesis, and asserts that...
...the only relevant test of the validity of a hypothesis is comparison of its predictions with experience. The hypothesis is rejected if its predictions are contradicted ("frequently" or more often than predictions from an alternative hypothesis); it is accepted if its predictions are not contradicted; great confidence is attached to it if it has survived many opportunities for contradiction. Factual evidence can never "prove" a hypothesis; it can only fail to disprove it, which is what we generally mean when we say, somewhat inexactly, that the hypothesis has been "confirmed" by experience."

So far so good. I think most scientists would see the above as conforming fairly closely to their own conception of how science should work (and of course this view is closely linked to views made famous by Karl Popper).

Next step: Friedman goes on to ask how one chooses between several hypotheses if they are all equally consistent with the available evidence. Here too his initial observations seem quite sensible:
...there is general agreement that relevant considerations are suggested by the criteria "simplicity" and "fruitfulness," themselves notions that defy completely objective specification. A theory is "simpler" the less the initial knowledge needed to make a prediction within a given field of phenomena; it is more "fruitful" the more precise the resulting prediction, the wider the area within which the theory yields predictions, and the more additional lines for further research it suggests.
Again, right in tune I think with the practice and views of most scientists. I especially like the final point that part of the value of a hypothesis also comes from how well it stimulates creative thinking about further hypotheses and theories. This point is often overlooked.

Friedman's essay then shifts direction. He argues that the processes and practices involved in the initial formation of a hypothesis, and in the testing of that hypothesis, are not as distinct as people often think, Indeed, this is obviously so. Many scientists form a hypothesis and try to test it, then adjust the hypothesis slightly in view of the data. There's an ongoing evolution of the hypothesis in correspondence with the data and the kinds of experiments of observations which seem interesting.

To this point, Friedman's essay says nothing that wouldn't fit into any standard discussion of the generally accepted philosophy of science from the 1950s. But this is where it suddenly veers off wildly and attempts to support a view that is indeed quite radical. Friedman mentions the difficulty in the social sciences of getting
new evidence with which to test an hypothesis by looking at its implications. This difficulty, he suggests,
... makes it tempting to suppose that other, more readily available, evidence is equally relevant to the validity of the hypothesis-to suppose that hypotheses have not only "implications" but also "assumptions" and that the conformity of these "assumptions" to "reality" is a test of the validity of the hypothesis different from or additional to the test by implications. This widely held view is fundamentally wrong and productive of much mischief.
Having raised this idea that assumptions are not part of what should be tested, Friedman then goes on to attack very strongly the idea that a theory should strive at all to have realistic assumptions. Indeed, he suggests, a theory is actually superior insofar as its assumptions are unrealistic:
In so far as a theory can be said to have "assumptions" at all, and in so far as their "realism" can be judged independently of the validity of predictions, the relation between the significance of a theory and the "realism" of its "assumptions" is almost the opposite of that suggested by the view under criticism. Truly important and significant hypotheses will be found to have "assumptions" that are wildly inaccurate descriptive representations of reality, and, in general, the more significant the theory, the more unrealistic the assumptions... The reason is simple. A hypothesis is important if it "explains" much by little,...   To be important, therefore, a hypothesis must be descriptively false in its assumptions...
This is the statement that the economists who wrote to me used to defend unrealistic assumptions in General Equilibrium theories. Their point was that having unrealistic assumptions isn't just not a problem, but is a positive strength for a theory. The more unrealistic the better, as Friedman argued (and apparently proved, in the eyes of some economists).

Now, what is wrong with Friedman's argument, if anything?  I think the key issue is his use of the provocative terms such as "unrealistic" and "false" and "inaccurate" in places where he actually means "simplified," "approximate" or "incomplete."  He switches without warning between these two different meanings in order to make the conclusion seem unavoidable, and profound, when in fact it is simply not true, or something we already believe and hardly profound at all.

To see the problem, take a simple example in physics. Newtonian dynamics describes the motions of the planets quite accurately (in many cases) even if the planets are treated as point masses having no extension, no rotation, no oceans and tides, mountains, trees and so on. The great triumph of Newtonian dynamics (including his law of gravitational attraction) is it's simplicity -- it asserts that out of all the many details that could conceivably influence planetary motion, two (mass and distance) matter most by far. The atmosphere of the planet doesn't matter much, nor does the amount of sunlight it reflects. The theory of course goes further to describe how other details do matter if one considers planetary motion in more detail -- rotation does matter, for example, because it generates tides which dissipate energy, taking energy slowly away from orbital motion. 

But I don't think anyone would be tempted to say that Newtonian dynamics is a powerful theory because it is descriptively false in its assumptions. It's assumptions are actually descriptively simple -- that planets and The Sun have mass, and that a force acts between any two masses in proportion to the product of their masses and in inverse proportional to the distance between them. From these assumptions one can work out predictions for details of planetary motion, and those details turn out to be close to what we see. The assumptions are simple and plausible, and this is what makes the theory so powerful when it turns out to make powerful and accurate predictions.

Indeed, if those same predictions came out of a theory with obviously false assumptions -- all planets are perfect cubes, etc. -- it would be less powerful by far because it would be less believable. It's ability to make predictions would be as big a mystery as the original phenomenon of planetary motion itself -- how can a theory that is so obviously not in tune with reality still make such accurate predictions?

So whenever Friedman says "descriptively false" I think you can instead write "descriptively simple", and clarify the meaning by adding a phrase of the sort "which identify the key factors which matter most." Do that replacement in Friedman's most provocative phrase from above and you have something far more sensible:
A hypothesis is important if it "explains" much by little,...   To be important, therefore, a hypothesis must be descriptively simple in its assumptions. It must identify the key factors which matter most...

That's not quite so bold, however, and it doesn't create a license for theorists to make any assumptions they want without being criticized if those assumptions stray very far from reality.

Of course, there is a place in science for unrealistic assumptions. A theory capturing core realities of a problem may simply be too difficult to work with, making it impossible to draw out testable predictions. Scientists often simplify the assumptions well past the point of plausibility just to be able to calculate something (quantum gravity in one dimension, for example), hoping that insight gained in the process may make it possible to step back toward a more realistic theory. But false assumptions in this case are merely tools for getting to a theory that doesn't have to make false assumptions.

Of course, there is another matter which Friedman skipped over entirely in his essay. He suggested that economic theories should be judged solely on the precision of their predictions, not the plausibility of their assumptions. But he never once in the essay gave a single example of an economic theory with unrealistic or descriptively false assumptions which makes impressively accurate predictions. A curious omission.

8 comments:

  1. Herbert Simon (1963) countered Friedman by stating the purpose of scientific theories is not to make predictions, but to explain things - predictions are then tests of whether the explanations are correct.

    Both Friedman and Simon's views are better directed to a field other than economics. The data at some point will always expose the frailest of assumptions; while the lack of repeatable results supports futility in the explanation of heterogeneous agents.

    ReplyDelete
  2. Comparing economics and physics doesn't really make sense. Physical involves observations of the real world where agents aren't blessed (or cursed by) the existence of a thought process. Economics doesn't rely on "observation" so much as it relies on its agents "actions". Since the basis of economics depends on agents acting in a manner which reinforces the system, assumptions are always both nonsense and conceivable reality all at once. Reflexivity is a real phenomenon in the world of economics - one that doesn't exist in physics.
    We assume zero cost of transportation as a component of classical trade theory. Alas, it isn't so and the "system" of trade produces results that could be contested with alternate assumptions on the cost of transportation. As it is, transport costs have gone down over time and trade has led to greater prosperity. Noble lies are our foundation.
    Every economic theory carries assumptions which may or may not represent reality, but if the system works, they start to. Self fulfilling prophecies are the only alternative to a natural equilibrium in a field that carries agents whose actions are determined by a thought process and not a natural system.

    ReplyDelete
  3. There are certainly financial theories with patently false assumptions. For example, the Capital Asset Pricing Model:

    > all investors are rational
    > all investors have perfect information
    > all investors can borrow and lend at the risk-free rate
    > all investors can buy and short the market in unlimited quantities

    We know none of these assumptions are true. How many of us can borrow at the risk-free rate? Yet they are some of the assumptions that underlie Nobel Prize winning theories.

    As suggested in the blog, these false assumptions are made because they are ancillary to the main point of the theory, which speaks to asset pricing being a function of risk vs. return, and how these assets together comprise portfolios.

    For these items the above does not matter.

    However, if we were to go about modeling the stock or bond market for a month to assess our own portfolio, the false assumptions would matter greatly.

    I wrote a brief blog post along similar track a couple months back if you are interested - http://treasurycafe.blogspot.com/2011/07/capm-interlude-theory-of-theory.html

    ReplyDelete
  4. fascinating stuff sir.

    ReplyDelete
  5. Hi Mark.

    What I think Friedman is doing is what you'll see when you enter the "nature" vs. "space" debate in the philosophy of physics. For this type of discussion, I always turn to Yves Simon's "The Great Dialogue of Nature And Space."

    Friedman's approach is like a Cartesian defending critics of Newton's inverse square law. Let me present the inverse square law of Newton, a la Friedman - Newton does an excellent job of describing a universe of two stationary point masses as being attracted to each other via an inverse square law that closely agrees in its predictions with the visual evidence. Now, the critic - But the universe isn't just two stationary point masses! Newton failed to describe the actual planets and solar system! Watch his system fall apart as velocities increase! What you are describing can't be found anywhere! Now back to Friedman - Aha! You must idealize the system you study in order to make progress via successful predictions. In fact, idealization - stripping away the inessentials in order to be able to describe the whole (now dessicated of most of its actual, realistic qualities) mathematically - is the way to go for a scientist.

    This is the Cartesian approach of "space." The critic of this tends to take the Aristotelian approach of "nature," which is to preserve the whole of the qualities (a planet's "nature") and provide a description of the whole thing, at the expense of making progress via a mathematical description that provides predictions. To a "naturist" a "descriptively simple" approach is always an unrealistic approach. To a Cartesian, the naturist can never get to the good stuff, which is an (idealized) description capable of prediction based on the math of points, sets, and lines.

    It is hard to see both approaches at once for what they are, because students are often trained in one or another, creating two schools where never the twain shall meet. In my description above I used idealizations, which are correct within their limitations, but are open to criticism that "no one is a real Cartesian through and through and so you've described nothing and can predict nothing." Oh well, the trick is to be able to see both approaches for their values and shortcomings. Cartesians tend to forget to describe the shortcomings of their predictions because of the assumptions inherent in their theory, while naturists tend to ignore the utility of predictive science. The book I mentioned above attempts to see both at the same time, and so does Martin Heidigger in "What is A Thing," though he spends a lot of time on Kant (yuck!).

    ReplyDelete
  6. "it doesn't create a license for theorists to make any assumptions they want without being criticized if those assumptions stray very far from reality."

    And how does one decide what "very far" means? Why describing a planet as infinitesimally small point mass is plausible, but describing it as a cube is not, when both assumptions are clearly wrong? Why is DSGE model with agents maximizing utility subject to market-clearing prices unrealistic, yet ABM model with agents randomly exchanging money like gas molecules is fine, when reality is much more complicated than both of them?

    I think that the main point of Friedman's article was precisely that you cannot say which assumptions are "too unrealistic" just by looking at them. Instead, you should analyze predictions of the theory and reject an assumption if it turns out to be too restricting (i.e. its implications don't agree with reality). Thus, criticizing an assumption for its lack of realism without considering the model in which it's used, the model's purpose and it's empirical tests is meaningless.

    We don't know what did those economists write to you exactly, but my guess is that they meant something similar as above. In which case I don't really see what the problem is...

    ReplyDelete
  7. So according to Friedman, if you assume there are economic gods that rule over the economy, we have the perfect theory. It explains everything perfectly, by little, its assumption is descriptively false and it would predict things just as accurately as the current theory that completely missed the 2008 crash, maybe even better.

    We have angered the gods of economy!! Priest say we must give more money to the banks!!

    ReplyDelete
  8. David K. Waltz

    I read your link and have to disagree. As a psychology graduate student, using Freud as an example of good science and theory is ludicrous. Freudianism is not scientific, in fact he makes the exact same mistakes as Friedman, which is to make unrealistic assumptions. Modern psychology is leagues beyond Freud and recognizes that most his assumptions were wrong. When did you go to school? The early 20th century?

    Lost in this discussion is the fact that Friedman's predictions are the exact opposite of what has occurred.
    Therefore, its not a stretch to look at the assumptions of his theories to see if they have any relationship with reality. Low and behold, they don't. Then to find out that he thinks the more unrealistic an assumption is, the better? WTF? To think that other economists use that passage as a justification to assume whatever the hell they want about humans and how we behave is unbelievable.

    Neoliberal economics is actually a lot like Freudianism in that it is not predictive of reality and therefore not a good scientific theory. One only has to look at neoliberal economic policies across the world to see this obvious truth.

    ReplyDelete