Wednesday, November 23, 2011

Economic models in theory and practice

Michael Woodford has a nice essay at INET where he responds to John Kay’s plea for a changed economics. In it, Woodford presents a number of arguments in favor of economic models that I think are valid and useful, but I don’t think he successfully defends the way economists use models in practice. Instead, he defends a different way of using models that would be more defensible.
Let’s consider his arguments in favor of mathematical models.
Argument 1: Precision
Models allow the internal consistency of a proposed argument to be checked with greater precision;
True, if the argument allows for translation into mathematical form. There’s an old Keynes quote on this:
Much economic theorizing to-day suffers, I think, because it attempts to apply highly precise and mathematical methods to material which is itself much too vague to support such treatment.
Sometimes, surely, the “vagueness of the material” is a shortcoming that makes an argument sound more sensible than it is. In that case, forcing it into mathematical form forces us to clarify what we actually mean and makes it harder to “weasel.” In other cases, however, formal methods force us to “sharpen” assumptions in a way that changes the argument itself. There’s a difference between saying that people have some thoughts on how gasoline prices will be in the future, and saying that people have subjective beliefs about future gas price trajectories that can be defined as a probability distribution over all possible price paths.
Argument 2: Differentiation/clarification
Back to Woodford:
they allow more finely-grained differentiation among alternative hypotheses
This is true – in so far as all the alternative hypotheses can be translated into this common language. For instance, the philosopher Jon Elster has written on Gary Becker’s rational addiction work that
Although I disagree sharply with much of it, it has raised the level of discussion enormously. Before Becker, most explanations of addiction did not involve choice at all, much less rational choice. By arguing that addiction is a form of rational behavior, Becker offers other scholars the choice between agreeing with him or trying to identify exactly where he goes wrong. Whatever option we take (I'm going to take the second), our understanding of addiction will be sharpened and focused.
This sounds fine, until you try to read the literature and discussions and realize that economists rarely find it interesting (or possible?) to discuss the specification of a model taken as a serious hypothesis about a causal mechanism in the world. As Woodford says elsewhere in his essay, when you want to conduct economic analysis with a mathematical model,
An assessment of the realism of the assumptions made in the  model is essential --- not, of course, an assessment of whether the model literally describes all aspects of the world, which is never the case, but an assessment of the realism of what the model assumes about those aspects of the world that the model pretends to represent. It is also important to assess the robustness of the model’s conclusions to variations in the precise assumptions that are made, at least over some range of possible assumptions that can all be regarded as potentially of empirical relevance. These kinds of critical scrutiny are crucial to the sensible use of models for practical purposes.
However, in many parts of economics, this kind of discussion is seen as irrelevant or silly. If you persist in trying to discuss “the realism of what the model assumes about those aspects of the world that the model pretends to represent” you’ll just be a person who doesn’t “get” economics.
Consider the rational addiction model of Becker and Murphy and the others that followed in its wake. As I’ve written with a colleague here,
The core of the causal insight claims from rational addiction research is that people behave in a certain way (i.e. exhibit addictive behavior) because they face and solve a specific type of choice problem. Yet rational addiction researchers show no interest in empirically examining the actual choice problem – the preferences, beliefs, and choice processes – of the people whose behavior they claim to be explaining. Becker has even suggested that the rational choice process occurs at some subconscious level that the acting subject is unaware of, making human introspection irrelevant and leaving us no known way to gather relevant data
In addition: Trying to examine whether the causal mechanisms described are at all plausible or consistent with evidence, is seen as irrelevant or weird. It’s an exercise only philosophers like the above quoted Jon Elster and oddballs like myself seem to be interested in (I first wrote on this in an article called “Taking absurd theories seriously”). “Real” economists seem content to wave their hands and say “as-if” or “these are just standard assumptions.”
Argument 3: Enables complexity
[Models] allow longer and more subtle chains of reasoning to be deployed without both author and reader becoming hopelessly tangled in them.
This claim by Woodford is similar to Krugman’s claim in “Two cheers for formalism” (a piece originally published in the Economic Journal):
Most of the topics on which economists hold views that are both different from "common sense" and unambiguously closer to the truth than popular beliefs involve some form of adding-up constraint, indirect chain of causation, feedback effect, etc.. Why can economists keep such things straight when even highly intelligent non-economists cannot? Because they have used mathematical models to help focus and form their intuition.
This sounds sensible: One could argue that individuals face relatively simple problems (“how much milk do I feel like drinking now?”), but that we need formal tools to understand what happens when they interact in markets or firms or whatever. However, this wouldn’t really be true. The argument is particularly off if you’re into rational expectations – in which case you want to impose the requirement that the agents in the model understand the model they are in and optimize in light of the real constraints they face. In that case, they need the same tools you do.
The argument is also off in other contexts. As I’ve argued elsewhere, it is actually an argument against all sophisticated, mathematical theories of individual choice. If mathematical modeling is a tool necessary for economists to reason their way through, say, rational addiction theory, how on earth do they expect “even highly intelligent non-economists” to discover that becoming a junky is their best shot at happiness? You might want to avoid the question by saying that they are able to do this “subconsciously”, but even that is a testable claim (hint: it’s empirically false). Also – if we really did solve such problems easily in our subconscious – wouldn’t these models seem intuitive and in line with our gut feelings? Put differently, it seems odd to develop a tool to overcome human cognitive frailty and then claim that this tool used at “full power”
Argument 4: Enables critical evaluation
Woodford’s essay again:
Often, reasoning from formal models makes it easier to see how strong are the assumptions required for an argument to be valid, and how different one’s conclusions may be depending on modest changes in specific assumptions. And whether or not any given practitioner of economic modeling is inclined to honestly assess the fragility of his conclusions, the use of a model to justify those conclusions makes it easy for others to see what assumptions have been relied upon, and hence to challenge them.
Here I’ll just refer back to the discussion on argument 2. Once again, I agree with Woodford in principle – but would argue that this is descriptively inaccurate in terms of how academic discussions in economics are actually conducted. When I examined the justification of specific assumptions in rational addiction theories (in “Taking absurd theories seriously”), I found that this was a lackadaisical affair: The most weird and unbelievable stuff was left uninterpreted in the model, other weird assumptions were justified by telling whatever anecdote would support it, or by giving loose evidence that would support a different but related assumption. The models, to sum up, were
poorly interpreted, empirically unfalsifiable, and based on wildly inaccurate assumptions selectively justified by ad-hoc stories.
Time to wrap up.
I realize that I’ve sounded critical of Woodford, but hope the “in principle”/”in practice” distinction is clear. My problem with his essay is that it’s framed as a defense of current economic practice. Evaluated in that regard, the arguments fails: He actually defends a form of “best practice” in modeling that is neither widespread nor widely recognized as such in economics today (as far as I can tell).

No comments:

Post a Comment