Dangerous Interpretation of Data

It is generally considered that a theory must be supported by facts. The problem with the use of data as a support of a theory is the interpretation of such data.

An example helps to understand this. If a historian discovered that due to an increase in the supply of a product X, the price of the product increases instead of decreasing, is that enough to discredit the law of supply and demand ? Of course, no. Confounding factors may have come into work. For example, a dramatic drop in the supply of a product Y, which is a substitute to the product X. The very idea of accepting a theory because it receives some ’empirical support’ is absolutely ridiculous. In the given example, this means that the law of supply and demand is erroneous. It would be difficult to believe that when the supply of a good increases, it will not exert any downward pressure on prices. No one would believe that ​​the infinite use of printing money wil not diminish the value of the money. And no one would believe that one seller facing a countless number of buyers will not exert any bargaining power.

Likewise, if one makes the assumption that incarceration rate decreases criminality, and that statistics have once shown that there was an increase in criminality following an increase in the prison population, does that mean that the incarceration rate tends to increase criminality ? This assumption makes absolutely no sense – there are confounding factors that came into play, which are not captured by data. In other words, if crime has increased following an increase in the number of prisoners, we should say that crime would have been higher if incarceration rate has not increased. The facts and historical evidence are meaningless if we fail to interpret it correctly.

Accepting an illogical theory just because it is supported by data is a mere fallacy.

This entry was posted in Divers.

6 comments on “Dangerous Interpretation of Data

  1. Are you familiar with Oskar Morgenstern’s “On the Accuracy of Economic Observations”? Personally I wouldn’t trust any data without error bars.

  2. 猛虎 says:

    Hmm, not really. Is it so interesting ?

    My argument here, anyway, is that data does not necessarily capture all the confounding variables. Suppose that theory A sounds logical, and B sounds illogical. Can I consider theory B as being more relevant than A juste because data “fits” theory B ?

  3. Yes, it is very interesting. The problem is that in the human sciences, so-called, there are no variables. I.e, no measurable relationships. The laws of supply and demand, for example, express qualitative, not quantitative, relationships. Even if we could get accurate price information (we can’t) the best we can make is a qualitative judgment: all else equal, a lower price will mean a higher demand. It is possible to have judgment, based on historical experience and personal experience and a knowledge of these putative qualitative relationships, on the basis of which you can make qualitative judgments about the future. Just don’t expect to much agreement with other “experts,” of which there are none.

    This is not a counsel of despair however. Adam Smith was still correct (in my judgment, based on historical experience). Economics is a logic and an art, not a science. It’s goal is to discover policy that seems likely to increase the welfare of this and the succeeding generation. It in inherently political, and hence rhetorical. Which explains the success of people like Smith, Keynes, and Friedman. But good economist are rare. Sophists abound. A good example of a good economic policy is the case of natural gas price controls here in the US. Back in the 1970’s it was predicted (by some economists, but by no means all) that if you removed price controls the supply would increase. That judgment proved correct.

    That said, the answer is yes: you can eliminate illogical and inconsistent “theories.” But that doesn’t mean there are logical and consistent “theories.” You should never write equations with equal signs in them, or with parameters in the form of ax + by = c. Stick to tildens. Use ordinary language. A simple geometrical diagram is fine as long as you use it for heuristic purposes only and don’t take it too seriously.

    It’s probably not a bad idea to know calculus however. Why? Because then you can understand why it does not apply.

    But back to Morgenstern: read the book and you will understand why not even statistics applies to most economic data. There are no Gaussians. And certainly no way to show there are Gaussians unless you assume them in the first place. The only thing that works is inside knowledge.

    That said I think there is a good case to be made for a graduated expenditure tax in place of a graduated income tax or, indeed, any other kind of tax. But you will have to be a genius to make it so persuasive it actually gets implemented. What kind of genius? Psychological, political, rhetorical, historical. Smith and Keynes both had that ability.

    All this is just my personal opinion of course.

  4. 猛虎 says:

    This is astonishing. Morgenstern seems to be highly influenced by austrian economics. This is exactly what Mises said in his Human Action. See for example :
    http://mises.org/humanaction/chap2sec1.asp

    The experience with which the sciences of human action have to deal is always an experience of complex phenomena. No laboratory experiments can be performed with regard to human action. We are never in a position to observe the change in one element only, all other conditions of the event remaining unchanged. Historical experience as an experience of complex phenomena does not provide us with facts in the sense in which the natural sciences employ this term to signify isolated events tested in experiments. The information conveyed by historical experience cannot be used as building material for the construction of theories and the prediction of future events. Every historical experience is open to various interpretations, and is in fact interpreted in different ways.

    The postulates of positivism and kindred schools of metaphysics are therefore illusory. It is impossible to reform the sciences of human action according to the pattern of physics and the other natural sciences. There is no means to establish an a posteriori theory of human conduct and social events. History can neither prove nor disprove any general statement in the manner in which the natural sciences accept or reject a hypothesis on the ground of laboratory experiments. Neither experimental verification nor experimental falsification of a general proposition is possible in its field.

    Complex phenomena in the production of which various causal chains are interlaced cannot test any theory. Such phenomena, on the contrary, become intelligible only through an interpretation in terms of theories previously developed from other sources. In the case of natural phenomena the interpretation of an event must not be at variance with the theories satisfactorily verified by experiments. In the case of historical events there is no such restriction. Commentators would be free to resort to quite arbitrary explanations. Where there is something to explain, the human mind has never been at a loss to [p. 32] invent ad hoc some imaginary theories, lacking any logical justification.

  5. 猛虎 says:

    Oh, I have just come across this review of Morgenstern :
    http://mises.org/daily/2280/The-Problem-of-Accuracy-of-Economic-Data

  6. Yeah, a nice review/summary.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s