[Update 5/8/2016]
Back in March, the American Statistical Association (ASA) released statements on statistical significance and p-values, "....The ASA statement is intended to steer research into a ‘post p<0.05 era.’"
This statement contains 6 principles that address the misuse of the p-value:
- P-values can indicate how incompatible the data are with a specified statistical model.
- P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone.
- Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.
- Proper inference requires full reporting and transparency.
- A p-value, or statistical significance, does not measure the size of an effect or the importance of a result.
- By itself, a p-value does not provide a good measure of evidence regarding a model or hypothesis.
Details: "The ASA's statement on p-values: context, process, and purpose".
[End Update 5/8/2016]
First are some background information.
This article "The Odds, Continually Updated" from NY Times happened to catch my attention. To be short, it states that
Besides above, perhaps the most famous paper criticizing p-value is this one - "Scientific method: Statistical errors" by Regina Nuzzo from Nature, in which a lot of scientific issues raised by p-value approach has been discussed, like reproducibility concerns, p-value hacking, etc.
[Update 6/92015]
Just noticed the news, and thought it would be good to put it here for discussion.
Psychology journal bans P values
A controversial statistical test has finally met its end, at least in one journal. Earlier this month, the editors of Basic and Applied Social Psychology (BASP) announced that the journal would no longer publish papers containing P values because the statistics were too often used to support lower-quality research.
Along with a recent paper, "The fickle P value generates irreproducible results" from Nature, about P value.
[End Update 6/9/2015]
[EDIT 2]Update:
Check Andrew Gelman's blog about this: No, I didn't say that! (Credits to @Scortchi, @whuber).
[END EDIT 2]
[EDIT]Update:
I would be particularly interested in whether there are cases that Bayesian approach is more reliable than frequentist's p-value approach. By "reliable", I mean the Bayesian approach is less likely to manipulate data for desired results. Any suggestions?
###Update 6/9/2015
Just noticed the news, and thought it would be good to put it here for discussion.
Psychology journal bans P values
A controversial statistical test has finally met its end, at least in one journal. Earlier this month, the editors of Basic and Applied Social Psychology (BASP) announced that the journal would no longer publish papers containing P values because the statistics were too often used to support lower-quality research.
Along with a recent paper, "The fickle P value generates irreproducible results" from Nature, about P value.
###Update 5/8/2016
Back in March, the American Statistical Association (ASA) released statements on statistical significance and p-values, [END EDIT]"....The ASA statement is intended to steer research into a ‘post p<0.05 era.’"
This statement contains 6 principles that address the misuse of the p-value:
- P-values can indicate how incompatible the data are with a specified statistical model.
- P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone.
- Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.
- Proper inference requires full reporting and transparency.
- A p-value, or statistical significance, does not measure the size of an effect or the importance of a result.
- By itself, a p-value does not provide a good measure of evidence regarding a model or hypothesis.
Details: "The ASA's statement on p-values: context, process, and purpose".
