Patrick O'Donnell uses the occasion of a review by Thomas Frank of Thomas Picketty's new book Capital in the Twenty-first Century. He takes the opportunity to bash the pretensions of economists. It reminds me of the problems of statistical measures in epidemiology and debates about how to make reasonable inferences of causation of disease. This was a central concern of mine in the pre-Daubert and the post-Dabubert era. There defense attorneys for drug companies took a kind of Popperian (nothing can be proven) stance. Probabilities are predictions and cannot be used to prove causation was the argument. I took that on in a 1995 essay that I still like: Against the Odds - Proving Causation of Disease with Epdemiological Evidence. - gwc
Ratio Juris: The Imperialist and Scientistic Pretensions…or The Secret and Not-So-Secret Sins, of Economics:
by Patrick O'Donnell
Quoting Deirdre MCloskey:
“It is not difficult to explain to outsiders what is so dramatically, insanely, sinfully wrong with the two leading methods in high-level economics, qualitative theorems and statistical significance. It is very difficult to explain it to insiders, because the insiders cannot believe that methods in which they have been elaborately trained and which are used by people they admire most are simply unscientific nonsense, having literally nothing to do with whatever actual scientific contribution (and I repeat, it is considerable) that economics makes to the understanding of society. So they simply can’t grasp arguments that are plain to people not socialized in economics.”
'via Blog this'
George,
ReplyDeleteTo be clear, McCloskey’s treatment of the misuse of statistics by her colleagues (and my endorsement of her views in this regard), has no (direct or obvious) bearing whatsoever for questions or problems (or lack thereof) that arise or might arise in epidemiology or evidence-based medicine generally. Any skepticism or critique articulated by her is specific to her profession and is intrinsically related to the fact that explanations and questions about causation in social sciences like economics are far more difficult and intractable than they are in the natural sciences (e.g., reductive exercises are easier in the latter than the former, even though sometimes they’re perilous in the natural sciences as well). In other words, the problems that arise in the social sciences with regard to the use of statistics are unique if only because more varied and recalcitrant than any similar such issues that might arise in epidemiology. Thus one should not infer from her critique of the manner in which statistical significance is used in economics has any relevance whatsoever for epidemiology or any of the natural sciences.
The kind of deductive modelling used in the social sciences for statistics is rather crude, most notoriously exemplified in the case of rational-choice models. (The endeavor to make these models more sophisticated is, in itself telling, which is not to say they’re completely bereft of value; I’ll refrain here from citing the relevant literature.) Jon Elster provides us with a brief inventory of some of the (often insuperable) difficulties that arise with statistical analysis in the social sciences, these being especially acute and fairly commonplace: data mining, curve fitting, arbitrariness in the measure of dependent and independent variables, distinguishing correlation from causation, and the difficulty of identifying the direction of causation. The use of “lagged” values of variables presents further opportunities for mischief, as might the heterogeneity of units of analysis. Prediction, determinacy and precision are far more elusive and frequently illusory in the social sciences than is true of the natural sciences. Elster goes into some detail as to why this is so, including the fact that identification of beliefs and preferences still leaves us with a significant degree of indeterminacy with regard to accounting for action or behavior in the real world. And to further complicate matters, Elster reminds us of our meager understanding of the mechanisms of preference formation. Abstraction from context in the social sciences more often than not leaves us with results (and multiple ceteris paribus clauses) that end up far removed from real-life situations and scenarios (there may be a corresponding problems here in identifying the possible causes of diseases, which means we’re forced to treat symptoms and not causes, in particular—but not only—for many forms of mental illness).
In addition to Elster’s several books on “nuts and bolts” for the social sciences, Richard W. Miller’s Fact and Method: Explanation, Confirmation and Reality in the Natural and Social Sciences (Princeton University Press, 1987) might also be read for its insights into the aforementioned issues (and the faddish fondness for Bayesian model of statistical inference or probability does not cure what ails us). See too Steven Horst’s Beyond Reduction: Philosophy of Mind and Post-Reductionist Philosophy of Science (Oxford University Press, 2007).