ASA Calls Time on ‘Statistically Significant’ in Science Research

It's a rare pleasure to be able to say "I told you so".  In this case I did. Almost 30 years ago I was arguing against over-reliance on measures of statistical significance and Relative Risk in making inferences of causal relationship in asbestos related disease.  After a few years I discussed my views in, among other things, a 1995 article titled "Against the Odds - Proving Causation of Disease with Epidemiological Evidence."  I relied on two leading thinkers - Sir Austin Bradford Hill, an architect of the studies linking smoking and disease; and South Africa born Columbia University epidemiologist and theorist Mervyn Susser.  Now the mountain is moving in my direction.   - gwc

American Statistical Association Calls Time on ‘Statistically Significant’ in Science Research

Scientists should stop using the term “statistically significant” in their research, urges the authors of an editorial in a newly published special issue of The American Statistician.

The issue, Statistical Inference in the 21st Century: A World Beyond p<0.05, calls for an end to the practice of using a p-value of less than 0.05 as strong evidence against a null hypothesis or a value greater than 0.05 as strong evidence favoring a null hypothesis. Instead, p-values should be reported as continuous quantities and described in language stating what the value means in the scientific context.

Containing 43 papers by statisticians from around the world, the special issue is expected to lead to a major rethinking of statistical inference by initiating a process that ultimately moves statistical science—and science itself—into a new age.

In the issue’s editorial, Ronald Wasserstein, executive director of the ASA; Allen Schirm, retired from Mathematica Policy Research; and Nicole Lazar of the University of Georgia said:

Based on our review of the articles in this special issue and the broader literature, we conclude that it is time to stop using the term ‘statistically significant’ entirely.

No p-value can reveal the plausibility, presence, truth, or importance of an association or effect. Therefore, a label of statistical significance does not mean or imply that an association or effect is highly probable, real, true, or important. Nor does a label of statistical nonsignificance lead to the association or effect being improbable, absent, false, or unimportant.

For the integrity of scientific publishing and research dissemination, therefore, whether a p-value passes any arbitrary threshold should not be considered at all when deciding which results to present or highlight.