Fighting publication bias #1

Short version: By having a hierarchy of journals that accept work partly based on a prediction of how important/novel the work seems to be to a few referees and an editor, researchers will

• try too hard to find results that will seem to be novel/important
• try too hard to reproduce new results and show that they too have found this new novel/important thing
• shelve their work (because it seems flawed or because will at best be publishable only in less interesting lower-tier journals) if they fail to reproduce the new novel/important things

The current academic publishing system with peer-reviewed journals is an attempt to achieve a lot of different goals at the same time:

• Facilitate scientific progress, by
• ensuring quality of published research by weeding out work that is riddled with errors, poor methodology etc. through anonymous peer-review by relevant experts
• assessing/predicting importance of research and thus how “high up” in the journal hierarchy it should be published,
• making research results broadly accessible so that disciplines can build their way brick-by-brick to greater truths
• promoting a convergence towards consensus by ensuring reproducibility of research and promoting academic dialogue and debate
• Simplify the evaluation of individual researchers (given the above, the number of articles weighted by journal type is a proxy for the importance and quality of your research)
• Generate huge profits for publishing houses (To quote an article from Journal of Economic Perspectives, “

The six most-cited economics journals listed in the Social Science Citation Index are all nonprofit journals, and their library subscription prices average about $180 per year. Only five of the 20 most-cited journals are owned by commercial publishers, and the average price of these five
journals is about $1660 per year.

Now, clearly, not all of these goals are compatible – most obviously, it is hard to square rocketing subscription costs with the goal of making research results more accessible. However, the ranking of academics based on where in a hierarchy of journals they have published seems likely to lead to issues as well.

If you want to get ahead as a researcher, you need to be published, preferably in good journals. If you want to be published in a good journal you need to do something surprising and interesting. You need to either show that something  people think is smart is stupid, or that something people think is stupid is smart. As a result, you get a kind of publication bias that can be illustrated by a simple thought experiment:

Imagine that the world is exactly as we think it is. If you drew a number of random samples, the estimates for various parameters of interest would tend to be distributed rather nicely around the true values. Only the researchers “lucky” enough to draw the outlier samples whose estimated parameters were surprising would be able to write rigorously done research that supported new (and false) models of the world that were in line with these (non-representative) results. This is actually not a very subtle point: One out of twenty samples will by definition have results that reject a true null hypothesis at 5% significance level.

OK, so let us say ideological bias, fashions and trends in modeling approaches etc. are irrelevant, so the result is published. Right away, this becomes a hot new topic, and anyone else able to reproduce it (read: anyone else drawing random but non-representative samples) get published. And then, gradually, the pendulum shifts – and the interesting and novel thing is to disprove the new result.

Now, clearly the above thought model is too simple. For one thing, we don’t know the truth. But the recent New Yorker essay on “The decline effect” sounds like this might be part of what’s going on:

all sorts of well-established, multiply confirmed findings have started to look increasingly uncertain. It’s as if our facts were losing their truth: claims that have been enshrined in textbooks are suddenly unprovable. This phenomenon doesn’t yet have an official name, but it’s occurring across a wide range of fields, from psychology to ecology. In the field of medicine, the phenomenon seems extremely widespread, affecting not only antipsychotics but also therapies ranging from cardiac stents to Vitamin E and antidepressants

The essay discusses a number of explanations (some of them sort of mystical and new-agish), but also notes the explanation above. When biologist Leigh Simmons failed to replicate a new interesting result, he failed to replicate it:

“But the worst part was that when I submitted these null results I had difficulty getting them published. The journals only wanted confirming data. It was too exciting an idea to disprove, at least back then.” For Simmons, the steep rise and slow fall of fluctuating asymmetry is a clear example of a scientific paradigm, one of those intellectual fads that both guide and constrain research: after a new paradigm is proposed, the peer-review process is tilted toward positive results. But then, after a few years, the academic incentives shift—the paradigm has become entrenched—so that the most notable results are now those that disprove the theory.

It seems to me that this is an almost unavoidable result of the current journal system, but not an unavoidable result of peer-reviewed journals as such. The problem seems to me to stem from the hierarchy of journals, and from the two tasks we give to referees (assess quality and assess importance/interest). The new open-access mega-journals (PLOS One, Sage Open, etc) that aim to publish all competently done research independently of how “important” it seems should at least mitigate the problem. Not necessarily by making it less important to have a “breakthrough” paper with a seemingly important result, but by making it easier to publish null-results.

Rewards and incentives can be OK

There’s a result from behavioral economics that increasing rewards and incentives for a behavior (e.g. to get kids to read more books) “crowds out” intrinsic motivation and leaves them less interested in books than before once the rewards dry out. Barking up the wrong tree notes a study that fails to find this when it comes to getting kids to eat vegetables. Good to know for those of us with kids.

Liking and intake of the vegetable were assessed in a free-choice consumption task at preintervention, postintervention, 1 month after intervention, and 3 months after intervention. Liking increased more in the three intervention conditions than in the control condition, and there were no significant differences between the intervention conditions. These effects were maintained at follow-up. Children in both reward conditions increased consumption, and these effects were maintained for 3 months; however, the effects of exposure with no reward became nonsignificant by 3 months. These results indicate that external rewards do not necessarily produce negative effects and may be useful in promoting healthful eating.

Why support free trade?

Economists are usually in favor of free trade. I myself am both an economist and usually in favor of free trade. But I thought this post on “Kids prefer Cheese” which Mark Thoma recently re-blogged had a good and valid point that economists do well to remember: Even if trade benefits the trading partners, that does not mean that a large number of people in a country may not be hurt by allowing free trade. And even if the monetary gains of the winners are bigger in sum than the monetary losses of the losers, that isn’t always a big help for the losers since there is no redistribution automatically triggered making everyone at least as well off as before.

Economists usually defend their stance on such issues by talking about Pareto efficiency, saying that making someone better off is always good provided someone else isn’t made worse off by it. Then they switch from talking about Pareto improvements (probably rare in actual policy) to talking about potential Pareto improvements, where the winners could compensate the losers and achieve a true Pareto improvement. Of course, they won’t do so in actuality, which makes the policy also have a redistributive element. A common reply is that redistribution should not be solved through trade measures, but through redistributive policies. But, guess what, most of those aren’t that popular amongst economists either: They distort incentives and reduce efficiency and involve moral hazard problems and, besides, inequality isn’t that horrible anyway. I may be completely wrong, but my guess is many of the economists most adamant about the glories of completely free trade are also amongst those staunchest in opposition to redistributive taxation and public welfare schemes. Though, being a guess, that may be just based on stereotypes and shouldn’t be given too much weight.

Anyway, here’s an excerpt:

People, the United States is not a person! Only in DSGE models do we assume that all individuals are identical! There is no "our" to which general statements can be attached.
Yes, going from autarky to free trade will raise the GDPs of both nations, but that is a very far cry from saying that a large number of individuals will not be made worse off in the process. I figure that NGM is familiar with the Stolper-Samuelson theorem, so I guess he is assuming the political process always provides adequate compensation for the losers??

ROFLMAO, anyone?
Here's a case for free trade:

Individuals should be allowed to contract with whoever they wish, without government interference based solely on geography.

Now, that is not much of an economic argument, but, to tell the ugly truth, THERE ISN'T MUCH OF AN ECONOMIC ARGUMENT.

Once you factor in agent heterogeneity, imperfect competition, increasing returns, and an arbitrarily large number of traded goods, the welfare economics of free trade is murky at best.

More good stuff making the same point here

Manipulating maths for whose amusement?

Amplify’d from www.technologyreview.com

Q&A: The Experimenter

Gary Loveman, the CEO of Caesars Entertainment, says there are three ways to get fired from the hotel and casino company: theft, sexual harassment, and running an experiment without a control group.

Loveman, who has a PhD in economics from MIT and was a professor at Harvard Business School, has impressed the importance of data analysis on his employees, who are expected to quickly scale small tests into company-wide initiatives. For example, they might test which is likelier to get customers to spend more: a free meal or a free night in a hotel.

When you got your economics PhD from MIT in 1989, subdisciplines like behavioral economics and experimental economics had a mixed reputation. Now—a couple of Nobel Prizes in the field later—they seem to be cornerstones of how many businesses and industries try to innovate.

My impression is that when I got my PhD, we were really manipulating mathematics for our own amusement, and we weren't producing all that much to help real people make real decisions. That was dissatisfying to me and, frankly, frustrating. The notion that we could do experiments based on the central tenets of economics and have that make a real-world difference was exciting. Of course, with Freakonomics and Predictably Irrational these themes have become more popularized and accessible. It's a very heartening development, and it's increased my enthusiasm for my own discipline enormously.

What do you like to tell your academic colleagues about the challenges of real-world experimentation and innovation?

Honestly, my only surprise is that it is easier than I would have thought. I remember back in school how difficult it was to find rich data sets to work on. In our world, where we measure virtually everything we do, what has struck me is how easy it is to do this. I'm a little surprised more people don't do this. Read more at www.technologyreview.com

 

See this Amp at http://amplify.com/u/bqqn5

Experimental evidence on infinitely repeated games??? Infinity is a loooong time!

Surely an ongoing study by definition, reporting on some results from a work in progress. And from the most prestigious economics journal - the American Economic Review - no less.

My own criticism of the predictions from the theory of infinitely repeated games would be more directed towards their lack of applicability in my (AFAIK) finite life.

However, if I could gain immortality only by agreeing to spend it sitting in a laboratory playing prisoner's dilemma for ever - then I think I would pass. My guess is they have a sample selection problem.

And yes, I know I'm being dumb.

And no, I'm not being serious.

Amplify’d from www.ingentaconnect.com

The Evolution of Cooperation in Infinitely Repeated Games: Experimental Evidence

Authors: Bó, Pedro Dal; Fréchette, Guillaume R.

Source: The American Economic Review, Volume 101, Number 1, February 2011, pp. 411-429(19)

Abstract: A usual criticism of the theory of infinitely repeated games is that it does not provide sharp predictions since there may be a multiplicity of equilibria. To address this issue, we present experimental evidence on the evolution of cooperation in infinitely repeated prisoner's dilemma games as subjects gain experience. We show that cooperation may prevail in infinitely repeated games, but the conditions under which this occurs are more stringent than the subgame perfect conditions usually considered or even a condition based on risk dominance. Read more at www.ingentaconnect.com

See this Amp at http://amplify.com/u/bqofr

Economists should not be unduly concerned with reality?

It’s “Quotes out of context” day today. Here’s a couple of interesting quotes by prominent economists that I came across in a blog-post I stumbled onto. None of them really say anything factually wrong, but they seem (out of context, at least) indicative of an attitude valuing logically correct, sophisticated and elegant mathematical systems over pragmatically useful and informative, well-supported theories about the world. One danger of this is that if we use the word “economic theories” about both logical systems and theories-of-the-world, and if we also say that logical systems are correct or true when they are logically consistent and valued by economists, then it is only a small slip of the mind before we allow our views of the world to be colored and influenced by the logical systems that have yet to be related to reality.

There’s one by Samuelson:

Nobel Prizewinner Paul Samuelson's conclusion in his famous 1939 article on "The Gains from International Trade":

"In pointing out the consequences of a set of abstract assumptions, one need not be committed unduly as to the relation between reality and these assumptions."[3]

This attitude did not deter him from drawing policy conclusions affecting the material world in which real people live.

And one from

the textbook Microeconomics by William Vickery, winner of the 1997 Nobel Economics Prize:

"Economic theory proper, indeed, is nothing more than a system of logical relations between certain sets of assumptions and the conclusions derived from them... The validity of a theory proper does not depend on the correspondence or lack of it between the assumptions of the theory or its conclusions and observations in the real world. A theory as an internally consistent system is valid if the conclusions follow logically from its premises, and the fact that neither the premises nor the conclusions correspond to reality may show that the theory is not very useful, but does not invalidate it. In any pure theory, all propositions are essentially tautological, in the sense that the results are implicit in the assumptions made."[4]

Should we see it coming?

Michael Lewis has a wonderfully engaging, well-written (long) article about the Irish economic catastrophe in Vanity Fair. Worth reading for all sorts of reasons.

Here, I just want to point out the simple arguments and observations used by an economics professor during the boom to argue that there was a housing bubble. It’s puzzling how something that seems obvious in retrospect, based on simple, big-picture statistics that were easily googled at the time, could be so ignored or downplayed or rejected by economists and others alike at the time. The sense that “this time is different,” “past cases don’t apply,” and that all sorts of more or less good “small” arguments are enough to (psychologically?) weaken the impact of the big-picture items. Sometimes, the difficult thing is to just keep pounding on the big, strong, clear argument instead of allowing yourself to get derailed into lots of smaller-scale discussions of all sorts of details that don’t really count for much in the big picture. (It seems to me, for instance on the basis of this graph, that Norwegian house prices are grossly inflated today (the red curve is Norway, the blue US, both in real terms and normalized to 1890 levels)– but when I present this graph to others I constantly get derailed into side-tracks like “building standards are more stringent now than in the past, which might have increased costs”)

Morgan Kelly is a professor of economics at University College Dublin, […] Kelly saw house prices rising madly and heard young men in Irish finance to whom he had recently taught economics try to explain why the boom didn’t trouble them. And they troubled him. “Around the middle of 2006 all these former students of ours working for the banks started to appear on TV!” he says. “They were now all bank economists, and they were nice guys and all that. And they were all saying the same thing: ‘We’re going to have a soft landing.’ ”

The statement struck him as absurd: real-estate bubbles never end with soft landings. A bubble is inflated by nothing firmer than expectations. The moment people cease to believe that house prices will rise forever, they will notice what a terrible long-term investment real estate has become and flee the market, and the market will crash. It was in the nature of real-estate booms to end with crashes—just as it was perhaps in Morgan Kelly’s nature to assume that, if his former students were cast on Irish TV as financial experts, something was amiss. “I just started Googling things,” he says.

Googling things, Kelly learned that more than a fifth of the Irish workforce was employed building houses. The Irish construction industry had swollen to become nearly a quarter of the country’s G.D.P.—compared with less than 10 percent in a normal economy—and Ireland was building half as many new houses a year as the United Kingdom, which had almost 15 times as many people to house. He learned that since 1994 the average price for a Dublin home had risen more than 500 percent. In parts of the city, rents had fallen to less than 1 percent of the purchase price—that is, you could rent a million-dollar home for less than $833 a month. The investment returns on Irish land were ridiculously low: it made no sense for capital to flow into Ireland to develop more of it. Irish home prices implied an economic growth rate that would leave Ireland, in 25 years, three times as rich as the United States. (“A price/earning ratio above Google’s,” as Kelly put it.) Where would this growth come from? Since 2000, Irish exports had stalled, and the economy had been consumed with building houses and offices and hotels. “Competitiveness didn’t matter,” says Kelly. “From now on we were going to get rich building houses for each other.”

The endless flow of cheap foreign money had teased a new trait out of a nation. “We are sort of a hard, pessimistic people,” says Kelly. “We don’t look on the bright side.” Yet, since the year 2000, a lot of people had behaved as if each day would be sunnier than the last. The Irish had discovered optimism.

Their real-estate boom had the flavor of a family lie: it was sustainable so long as it went unquestioned, and it went unquestioned so long as it appeared sustainable. After all, once the value of Irish real estate came untethered from rents there was no value for it that couldn’t be justified. The 35 million euros Irish entrepreneur Denis O’Brien paid for an impressive manor house on Dublin’s Shrewsbury Road sounded like a lot until a trust controlled by the real-estate developer Sean Dunne’s wife reportedly paid 58 million euros for a 4,000-square-foot fixer-upper just down the street. But the minute you compared the rise in prices to real-estate booms elsewhere and at other times, you re-anchored the conversation; you biffed the narrative. The comparisons that sprung to Morgan Kelly’s mind were with the housing bubbles in the Netherlands in the 1970s and Finland in the 1980s, but it almost didn’t matter which examples he picked: the mere idea that Ireland was not sui generis was the panic-making thought. “There is an iron law of house prices,” he wrote. “The more house prices rise relative to income and rents, the more they subsequently fall.”

Why scientists are liberals – some speculative comments

The Freakonomics blog discusses political bias in sciences, and quotes a social psychologist who estimated that 80% of attendees at a conference were liberals (he asked for a show of hands). Dubner seems worried that political views will shape research conclusions, and writes:

How can it be that an academic field is so politically homogeneous? What kind of biases does such homogeneity produce? What sort of ideas get crowded out? And how homogeneous are other disciplines?

I have to say that I was surprised at the overt political (leftward) bias exhibited by several prominent economists at the recent American Economics Association meetings, although my sample set was quite small.

It is interesting — and sobering — that two fields, psychology and economics, that we rely upon to describe and amend bias in the world are themselves so susceptible to bias within the ranks of their practitioners.

Krugman disagrees, implying that research conclusions probably push attitudes towards the liberal side, saying

Biologists, physicists, and chemists are all predominantly liberal; does this reflect discrimination, or the tendency of people who actually know science to reject a political tendency that denies climate change and is broadly hostile to the theory of evolution?

Now, I don’t mean to say that political bias in the academy is absent, although it’s not consistent: I can well imagine that it’s hard to be a conservative in some social sciences, but in economics, the obvious bias in things like acceptance of papers at major journals is towards, not against, a doctrinaire free-market view. But the point is that doing head counts is a terrible way to assess that bias.

It might be that the most recent amusing statistical post on the blog for dating service OK Cupid has the answer. The post analyzes its database to identify the most unthreatening, innocent questions that best predict characteristics that you may not want to ask about directly (whether they’re religious, would have sex on a first date, their political ideology etc.). Based on their national US data they write that the question identifying politics is

• Do you prefer the people in your life to be simple or complex?

Because...

We were very surprised to find that this one question very strongly predicts a person's ideas on these divisive issues:

Should burning your country's flag be illegal?

Should the death penalty be abolished?

Should gay marriage be legal?

Should Evolution and Creationism be taught side-by-side in schools?

In each case, complexity-preferrers are 65-70% likely to give the Liberal answer. And those who prefer simplicity in others are 65-70% likely to give the Conservative one.

Seems to me that this is pretty consistent with the “bias” in academia. Academia is often very much concerned with complexity – finding nuances in interpretations and methods, considering alternative explanations for patterns in data, etc. If you prefer simplicity as a general trait in people and thoughts you would probably be pretty frustrated as an academic. And a 2:1 ratio is roughly 66%, which isn’t that far away from the estimate of 80% that we started with.

Sidenote: Interesting that Dubner is so worried by the left-wing attitudes of economists he encountered, given that his freakonomics podcast on how the world would look if it was driven by the gloriously rational economists basically said they would implement Milton Friedman’s pretty libertarian proposals. His (presumably unbiased and representative?) economist picked to answer on the behalf of the profession was Russ Roberts at George Mason University, who answered that his policy program would

start with some obvious things. I would get rid of the Department of Commerce. The Department of Commerce doesn’t do anything except subsidize exports, which is just a way of saying it makes certain companies rich at the expense of the rest of us. So I don’t think the Department of Commerce does anything particularly useful, I would get rid of that. I’d get rid of the Department of Education. I don’t think that the Federal Government has any productive role to play in the school system. I’d get rid of all tariffs. I’d let people be free to buy whatever they wanted from all around the world. What else? I would get rid of the minimum wage law, which I think makes it hard for low-skilled people to find work; it makes them artificially expensive. I’d change the Federal Reserve. We spend a lot of time trying to find the right interest rate. That’s a fool’s game that has contributed to the current crisis. So I would change the Federal Reserve. I would certainly at a minimum require it to only care about price stability. Right now it cares about price stability, unemployment, the health of the stock market, Wall Street salaries, evidently. So I would get all of those things out. It’s going to be hard to do legislatively, so I would probably replace the the Fed with a Friedmanite fixed growth and money supply or just abolish it entirely and let private money emerge. I’m getting out of control here.

You don’t say…

Update: More links and discussion here from McArdle in the Atlantic. Her take seems to be that there is a bias, that it is amusing to see conservatives (usually dismissive of bias accusations) believe it and liberals (usually sympathetic to bias accusations) dismissive, that it is unsolvable, and that we should all just try harder to get along and see each other's point of view.

Why intuitive stories are important and dangerous

Below are some excerpts from a blogpost on the importance of "simple" stories/models that Paul Krugman praised on his NYT blog. I fail to find a simple moral to the story as it seems (to me) to involve a lot of different views on this issue, such as (in my formulations):



"Simple case-stories/thought-experiments are a necessary adjunct to sophisticated models/theories - because we cannot reason using models but need simplified versions that our brains can grasp"

"Simple case-stories/thought-experiments are rhetorically convincing in discussions/debates"

"Simple case-stories/thought-experiments trigger the psychological feeling of understanding/insight which is a better signal of truth than other kinds of evidence"

"Formal/standard models in economic theory are accepted because other economists accept them (emperor's new clothes) but people who accept them don't really understand the mechanisms they involve"

"Economists are confused concerning what it takes to evaluate claims about the real world"



Here's part of Krugman's comment on the same post (http://krugman.blogs.nytimes.com/2011/02/02/models-plain-and-fancy/ ):



"I have nothing against mathematical models and econometrics. But my experience is that many misunderstandings in economics come about because people don’t have in their minds any intuitive notion of what it is they’re supposed to be modeling. The whole notion of an economy-wide shortfall in demand is just hard to grasp — by famous economists as well as the lay public; quite a lot of our hopeless public debate reflects the fact that many people, some of them imagining themselves to be sophisticated about the issue, just can’t visualize what Keynesian ideas are about. But the baby-sitting coop offers a human-scale example, and makes the whole thing clear."

Amplify’d from modeledbehavior.com

more than any other analysis the baby-sitting coop story made me a confident Keynesian. Before then I could parrot the New Keynesian models and understood that this was more or less what a smart economist was supposed to say.

However, I didn’t know how to counter the logic of Laizze Faire except to say, “well there are sticky prices and an Euler equation and so the household will adjust consumption . . . “  This is compelling to virtually no one – not even, on a deep level, to myself.

When it really came down to it, I would have been left with “Great Depression! Want it to happen again? No? Then we need to spend more money or cut taxes! Why? Because I am very smart and I have a whiteboard. Do you have a whiteboard?”

However, a simple story about baby-sitting and it all fell into place Read more at modeledbehavior.com

 

See this Amp at http://amplify.com/u/aoawp

Ethical economists again...

Alex Tabarrok at marginal revolution applauds Glaeser's take on the ethical basis of economics and quotes a text-book he has co-authored with Tyler Cowen which he claims makes a similar point (see below).



In this case, I'd make the point that their "take" is only superficially similar. It presupposes more. Glaeser's point was that you make an assumption when you jump from "the person chose A over B" to "A is better for the person than B," and that "preferences" before you make this jump refer to nothing more than what you would observe the person choosing. Cowen and Tabarrok, on the other hand, write as though they've already made this jump.More specifically, they seem to beg the question when they state that economists don't second-guess people's "preferences" and do "not regard some preferences as better than others" and don't mind it if people "like" wrestling better than opera. Choice, here, is already taken as (always??) an expression of what serves the choosing person's actual tastes and judgments best.

Amplify’d from www.marginalrevolution.com via Google Reader

Even though the predictions of economics are independent of any ethical theory, there are ethical ideas behind normative economic reasoning. An economist who rejects the idea of exploitation in kidney purchases, for example, is treating the seller of kidneys with respect—as a person who is capable of choosing for himself or herself even in difficult circumstances.

Similarly, economists don’t second-guess people’s preferences very much. If people like wrestling more than opera, then so be it; the economist, acting as economist, does not regard some preferences as better than others. In normative terms, economists once again tend to respect people’s choices.

None of this it to say that economists are always right in their ethical assumptions. As we warned you in the beginning, this chapter has more questions than answers. But the ethical views of economists—respect for individual choice and preference, support for voluntary trade, and equality of treatment—are all ethical views with considerable grounding and support in a wide variety of ethical and religious traditions. Read more at www.marginalrevolution.com

 

See this Amp at http://amplify.com/u/antcb

Why economics should not have a "moral core"

Here's a nice take on the Glaeser piece I noted below (http://freakynomics.blogspot.com/2011/01/fundamental-leap-of-welfare-economics.html )

Amplify’d from noahpinionblog.blogspot.com

going beyond the facts is precisely what Glaeser's "moral compass" would have us do. Suppose psychologists find that people are less happy when they have the choice to become addicted to heroin. Should economists refuse to accept this fact? If I were an economist studying heroin addiction, I'd say: "Hey, policymakers and voters, here's the deal. If you let people do heroin, their happiness will go down, but they'll have more freedom of choice. I'll let you guys decide what to do."

Glaeser disagrees. He seems to think the economist's duty is either to A) recommend the alternative that entails more freedom of choice, or B) disbelieve the finding that allowing heroin use reduces happiness. (A) is saying that economists, as a class, have a fixed and definite role in deciding society's morality, kind of like a priesthood. (B) is saying that intellectual honesty and scientific integrity must take a back seat to a faith-based belief system.

Whichever he is saying, I highly disapprove.

If we limit our set of economic theories to those that seem to recommend individual freedom of choice - if we give economics a "moral compass" - we are refusing to take an honest look at the way the world really operates. That, in my moral opinion, is bad science.

And don't think that this doesn't happen in practice. Many of the same economists who espouse a belief in individual freedom of choice are biased against theories that imply a need for collective decision-making. They routinely refuse to believe in the existence of public goods, demand fluctuations, and other phenomena that imply a role for government. Behavioral economic theories, which assert that people are sometimes irrational, are routinely pooh-poohed by "conservative" economists, regardless of the mountain of laboratory evidence in those models' favor. In short, the widespread belief that economists should act both as scientists and as priests has made them less effective as scientists. Read more at noahpinionblog.blogspot.com

 

See this Amp at http://amplify.com/u/anr42

Tim Harford pokes fun at rational addiction

He twittered my rational addiction video late last year and includes it in a side post (http://timharford.com/2011/01/are-the-economists-on-drugs-too/ ), so maybe I can take a small crumb of credit for the idea? (His is much more accessible to non-economists, though...)

Amplify’d from timharford.com

I wasn’t always an alcoholic tramp. I am a man of letters. I studied Philosophy, Politics and Economics at Oxford, like that David Cameron fellow. But when I looked at the options open to me – over-worked banker, castrated civil-servant or, worst of all, parliamentarian – I decided that the optimal course of affairs would be to begin building up my stock of addictive capital.

I don’t want to romanticise life as a rough-sleeping bum. It gets cold and lonely. I’m not sure what is keeping my underpants together, though I’m sure they wouldn’t survive contact with suds and warm water. But my boozy existence has a cool, calculating logic. I know that seems odd, but thankfully Gary Becker and Kevin Murphy, two of the University of Chicago’s most celebrated economists, have worked out the details in their theory of rational addiction.

For sure, not everything is perfect. But I’m a rational addict; a utility-maximising old soak. I drink because it makes sense to do so – by following an ex-ante optimal inter-temporal consumption plan, as they say. Speaking of which, let me crack open a bottle of strong cider … that’s better. Read more at timharford.com

 

See this Amp at http://amplify.com/u/anr31

The fundamental leap of welfare economics

I like this. This is honest. It's stupid, but it's honest. And clear: "Improved welfare" means nothing more than "new, previously not available choice option was picked". That's all. All of standard welfare economics presupposes that the best (only?) way to identify the welfare-maximizing choice available to an individual is to see what he chooses when left alone.

Maybe someone who didn't believe me will believe it when it comes from a Harvard economist praised by both Akerlof and Gary Becker... Here's Ed Glaeser

Amplify’d from economix.blogs.nytimes.com

Teachers of first-year graduate courses in economic theory, like me, often begin by discussing the assumption that individuals can rank their preferred outcomes. We then propose a measure — a ranking mechanism called a utility function — that follows people’s preferences.

If there were 1,000 outcomes, an equivalent utility function could be defined by giving the most favored outcome a value of 1,000, the second best outcome a value of 999 and so forth. This “utility function” has nothing to do with happiness or self-satisfaction; it’s just a mathematical convenience for ranking people’s choices.

But then we turn to welfare, and that’s where we make our great leap.

Improvements in welfare occur when there are improvements in utility, and those occur only when an individual gets an option that wasn’t previously available. We typically prove that someone’s welfare has increased when the person has an increased set of choices.

When we make that assumption (which is hotly contested by some people, especially psychologists), we essentially assume that the fundamental objective of public policy is to increase freedom of choice. Read more at economix.blogs.nytimes.com

See this Amp at http://amplify.com/u/anr2u

The consensus is...

The blog below notes some instances of stated "consensus" in science in the last 25 years that are no longer the consensus. This kind of thing is tricky, though. It has to do with the optimal level of trust towards people in our (and other) disciplines. If we always take other people's results and claims on good faith, scientific progress would slow and maybe halt as false results were accepted on authority. On the other hand, if we never accept other people's results and claims, we would open ourselves to lots of beliefs that are wrong or even ridiculous with a high probability - especially outside our own domain of specialization. We want a diversity of views challenging each other in order for the scientific process to work, but we also want the scientific process to lead to "consensus" views that we can feel reasonably confident in.

Amplify’d from falkenblog.blogspot.com

Robin Hanson reminds us that the scientific consensus is often wrong. Ron Bailey did a Nexis search of the phrase 'scientific consensus over the past 25 years, and found the following: saccharin causes cancer in humans dietary fiber appeared to reduced the incidence of colon cancer. agents found to cause cancer in animals should be considered suspect human carcinogens fusion energy reactors would produce more energy than it consumed within five years acid rain is destroying lakes and forests These are no longer consensus findings. He did find the phrase 'scientific consensus' in regards to uncertainty about when life starts, which probably still stands. Yet in all, that's a pretty weak record for the consensus. Read more at falkenblog.blogspot.com

 

See this Amp at http://amplify.com/u/angyg

Would people accept a market in organs?

Here's a guy riled up about how the liver-transplant Steve Apple-and-Pixar Jobs got a couple of years ago after having had pancreatic cancer may have been "wasted" if his recent health leave is due to his cancer returning. The reason he got the liver, according to this guy and his sources, was that the had the financial resources to "shop around" in the different state health systems.



Makes me curious how a purely market driven system would work. On the one hand, the fact that such "gaming" is EXTREMELY expensive now may make each individual case stand out more. In a market system, seeing a rich guy get his organ first would be an everyday occurence. On the other hand, if rich people could always buy their way to the front of the line, there would likely be many more cases of organs going to people who for other medical reasons may be poor recipients. If you have the cash and you're willing to gamble - here's your organ, even if the chances of success are low.



Maybe there would be supply-side reactions - people signing up for donation-under-specified-criteria? Or maybe the market revenue at stake would become a force for changing the default donation rules (which has enormous impact on donation rates - with roughly 20% changing from the default we can choose having 80% donation rates or 20% if I remember "Nudge" correctly)? Such factors would also make "gaming" and "medically suboptimal transplants" less salient. And I guess we should never underestimate the ability market players have for making things less transparent - whether it involves hiding how blood-stenched Coltan from Congo is in your cell-phone, how some financial sector legislation gets designed to benefit the financial sector, or how factory farming of hogs, turkeys and chickens is... well, evil (read Safran Foers new book on eating animals)

Amplify’d from www.slate.com

Two years ago, Jobs gamed the transplant allocation system to get a liver that could have saved somebody else. At the time, skeptics doubted that he should have received the organ, since he'd been treated for pancreatic cancer—in fact, he may have sought the liver because of the cancer—and the likelihood of the cancer's recurrence made him a bad bet for putting the liver to best use. If his health is now failing because of the cancer, that suspicion may be vindicated.

Jobs lives in Northern California, but he got his liver in Tennessee. Why? Different parts of the country have different waiting lists, and the wait in Northern California was three times longer than the wait in Tennessee. In fact, the median wait in the Tennessee area where Jobs snagged his liver was around 15 percent of the national average. Jobs confirmed last year that this is why he went to Tennessee: "My doctors here advised me to enroll in a transplant program in Memphis, Tennessee, where the supply/demand ratio of livers is more favorable than it is in California here."* Legally, you're allowed to get on multiple waiting lists around the country. That's how you game the system.

So why doesn't everybody do this? Because they can't. First you have to show up for an extensive in-person evaluation. Then you have to be available for a transplant in the area within hours of an organ becoming available. And while one jurisdiction might accept you as a charity case, if you want to play the field you'll have to prove you can pay for the transplant yourself. You also get priority points for being able to guarantee follow-up medical care, since this assures transplant allocators that the organ will be well cared for. Ordinary people can't compete with billionaires at meeting these tests. They can't go to multiple states for evaluations. They don't have private jets. Their insurance doesn't cover multiple evaluations and may not cover much of the half-million dollar transplant, much less the follow-up care.

there were roughly 16,000 people on the national liver waiting list when Jobs got a liver. He was one of 1,581 people who got livers in the United States in the first quarter of [2009]. Almost none of those people had any form of cancer. In fact, if Jobs' tumor has spread from his pancreas into his liver as is likely, some transplant surgeons say that they would not recommend a liver transplant because there is no data that shows a transplant will stop or even slow the spread of the cancer. This raises the question: Is this the best use of a liver? Read more at www.slate.com

 

See this Amp at http://amplify.com/u/bmxkx

Predictions - don't trust the odd one out.... but don't trust the rest either...

Yesterday I noted that the guys who are good at predicting extreme outcomes suck at predictions anyway because they’re crying “Wolf! Wolf!” all the time. Reminded me of something I read in a book which argues that “the rest” just say the same thing as each other without being correct. From Mark Buchanan’s “The Social Atom”:

A few years ago, for example, the economics consultancy London Economics assessed the recent predictions of more than thirty of the top British economic forecasting groups, including the Treasury, the National Institute, and the London Business School. They concluded:

It is a conventional joke that there are as many different opinions about the future of the economy as there are economists. The truth is quite the opposite. Economic forecasters . . . all say more or less the same thing at the same time; the degree of agreement is astounding. The differences between forecasts is are trivial relative to the differences between the forecass and what happens . . . what they say is almost always wrong . . . the consensus forecast failed to predict any of the most important developments in the economy over the past seven years […]

Today's best prediction is that things are gonna stay mostly the same...

...and don't you let anyone tell you otherwise...

Amplify’d from www.boston.com

We reserve a special place in society for those who promise genuine insights into the future — who can predict what will happen in business, in sports, in politics, technology, and so on. The media landscape is rich with these experts; Wall Street pays millions of dollars every year to analysts to put a precise dollar figure on next year’s company earnings. Those who manage to get a few big calls right are rewarded handsomely, either in terms of lucrative gigs or the adoration of a species that so needs to believe that the future is in fact predictable.

But are such people really better at predicting the future than anyone else?

To find the answer, Denrell and Fang took predictions from July 2002 to July 2005, and calculated which economists had the best record of correctly predicting “extreme” outcomes, defined for the study as either 20 percent higher or 20 percent lower than the average prediction. They compared those to figures on the economists’ overall accuracy. What they found was striking. Economists who had a better record at calling extreme events had a worse record in general. “The analyst with the largest number as well as the highest proportion of accurate and extreme forecasts,” they wrote, “had, by far, the worst forecasting record.”

Their work is the latest in a long line of research dismantling the notion that predictions are really worth anything. The most notable work in the field is “Expert Political Judgment” by Philip Tetlock of the University of Pennsylvania. Tetlock analyzed more than 80,000 political predictions ventured by supposed experts over two decades to see how well they fared as a group. The answer: badly. The experts did about as well as chance. And the more in-demand the expert, the bolder, and thus the less accurate, the predictions. Research by a handful of others, Denrell included, suggests the same goes for economic forecasters. An accurate prediction — of an extreme event or even a series of nonextreme ones — can beget overconfidence, which can lead to making bolder and bolder bets, and thus, more and more errors.

There’s no great, complex explanation for why people who get one big thing right get most everything else wrong, argues Denrell. It’s simple: Those who correctly predict extreme events tend to have a greater tendency to make extreme predictions; and those who make extreme predictions tend to spend most of the time being wrong — on account of most of their predictions being, well, pretty extreme. There are few occurrences so out of the ordinary that someone, somewhere won’t have seen them coming, even if that person has seldom been right about anything else. Read more at www.boston.com

 

See this Amp at http://amplify.com/u/f6t7

Macroeconomics yet again: The disinterest in reality

DeLong from last year with two questions: Why do good macroeconomists seemingly find "patently unrealistic" theories acceptable? And why don't they feel they need to make their theories consistent with the evidence described and collected by economic historians?



Again, I feel the answer has to involve the strategies and attitudes towards empirical facts, knowledge and data that economists too frequently allow. The types of arguments and challenges economists face in seminars and from referees and editors make it necessary to be consistent with current theoretical fads and remove the need to take into account certain types of evidence and arguments. Provided you know the right incantations and spells ("this is just an as if theory," "these are standard assumptions," etc.), then I'm confident you can ward off even the economic history bootcamp that DeLong proposes.

Amplify’d from delong.typepad.com

two questions:

First, it does not seem to me that it is the case that nobody really believes these just-so stories. Ed Prescott of Arizona State University really does believe that large-scale recessions are caused by economy-wide episodes of the forgetting of the technological and organizational knowledge that underpins total factor productivity—with the exception of episodes like the Great Depression, which Prescott says was caused by the extraordinary pro-labor pro-union policies of Herbert Hoover that pushed real wages far above equilibrium values. Casey Mulligan of the University of Chicago really does appear to believe that large falls in the employment-to- population ratio are best seen as “great vacations”—and as the side-effects of destructive government policies like those in place today, which are leading workers to quit their jobs so they can get higher government subsidies to refinance their mortgages. (I know; I find it incredible too.) Things that strike Kocherlakota as “patently unrealistic” are not viewed as such by many of his modern macroeconomic peers and colleagues. Why not? Why do they find these just-so stories satisfactory?

Second, whether modern macroeconomics attributes our current difficulties either to causes that I agree with Kocherlakota are “patently unrealistic” or simply confesses ignorance, why do they have such a different view than we economic historians do? Whether they have rejected our interpretations and understandings or simply have built up or failed to build up their own in ignorance of what we have done, why have they not taken and used our work?

The second question is particularly disturbing to me. There is, after all, no place for economic theory of any flavor to come from than from economic history. Someone observes some instructive case or some anecdotal or empirical regularity, says “this is interesting; let's build a model of this,” and economic theory is off and running. Theory is crystalized history—it can be nothing more. After the initial crystalization it does develop on its own according to its own intellectual imperatives and processes, true, but the seed is still there. What happened to the seed?

This situation is personally and professionally dismaying. I do not say that the macroeconomic model-building of the past generation has been pointless. I don’t think that it has been pointless. But I do think that the assembled modern macroeconomists need to be rounded up, on pain of loss of tenure, and sent to a year-long boot camp with the assembled monetary historians of the world as their drill sergeants. They need to listen to and learn from Dick Sylla about Cornelius Buller’s bank rescue of 1825 and Charlie Calomiris about the Overend, Gurney crisis and Michael Bordo about the first bankruptcy of Baring brothers and Barry Eichengreen and Christy Romer and Ben Bernanke about the Great Depression.

If modern macreconomics does not reconnect—if they do not realize just what their theories are crystallized out of, and what the point of the enterprise is—then they will indeed wither and die. Read more at delong.typepad.com

 

See this Amp at http://amplify.com/u/bm974

"The profession danced around the wrong models..."

More macro-criticism from last year - this time quotes from Joseph Stiglitz. And again, it's a case of "those guys used these models, which I think are stupid. Luckily, other people used these models which I think are smart - and these are the ones we should start using."



Again - I miss a focus on evidence and methodology: If these critics are right that our profession allowed madness to reign - how can we avoid this in the future? What should we demand from researchers who claim that they can guide policy, explain society, etc? Surely we need to do better than "they should employ the assumptions and modelling approaches that I find reasonable and that lead to the conclusions I am comfortable with"?

Amplify’d from economistsview.typepad.com

It is hard for non-economists to understand how peculiar the predominant macroeconomic models were. Many assumed demand had to equal supply – and that meant there could be no unemployment. (Right now a lot of people are just enjoying an extra dose of leisure; why they are unhappy is a matter for psychiatry, not economics.) Many used “representative agent models” – all individuals were assumed to be identical, and this meant there could be no meaningful financial markets (who would be lending money to whom?). Information asymmetries, the cornerstone of modern economics, also had no place: they could arise only if individuals suffered from acute schizophrenia, an assumption incompatible with another of the favored assumptions, full rationality.

Bad models lead to bad policy: central banks, for instance, focused on the small economic inefficiencies arising from inflation, to the exclusion of the far, far greater inefficiencies arising from dysfunctional financial markets and asset price bubbles. After all, their models said that financial markets were always efficient. Remarkably, standard macroeconomic models did not even incorporate adequate analyses of banks...: even a cursory look at the perverse incentives confronting banks and their managers would have predicted short-sighted behavior with excessive risk-taking. ...

Fortunately, while much of the mainstream focused on these flawed models, numerous researchers were engaged in developing alternative approaches. ... With a few exceptions, most central banks paid little attention to systemic risk and the risks posed by credit interlinkages. Years before the crisis, a few researchers focused on these issues, including the possibility of the bankruptcy cascades that were to play out in such an important way in the crisis. This is an example of the importance of modeling carefully complex interactions among economic agents (households, companies, banks) – interactions that cannot be studied in models in which everyone is assumed to be the same. Even the sacrosanct assumption of rationality has been attacked: there are systemic deviations from rationality and consequences for macroeconomic behavior that need to be explored.

Changing paradigms is not easy. Too many have invested too much in the wrong models. Like the Ptolemaic attempts to preserve earth-centric views of the universe, there will be heroic efforts to add complexities and refinements to the standard paradigm. The resulting models will be an improvement and policies based on them may do better, but they too are likely to fail. Nothing less than a paradigm shift will do. Read more at economistsview.typepad.com

 

See this Amp at http://amplify.com/u/bltx1

Ronald Coase on good and bad economics

This post contains no argument or data or big insight. File it under "Hey! Somebody famous said something I like the sound of!"

Amplify’d from economistsview.typepad.com

RC: The bad or wrong economics is what I called the "blackboard economics". It does not study the real world economy. Instead, its efforts are on an imaginary world that exists only in the mind of economists, for example, the zero-transaction cost world.

Ideas and imaginations are terribly important in economic research or any pursuit of science. But the subject of study has to be real. Read more at economistsview.typepad.com

 

See this Amp at http://amplify.com/u/blrt4