Image modified from “Death of a Salesman”. Image from Wikipedia
In February, a social psychology journal, Basic and Applied Social Psychology , made the bold (and extreme) move to ban the use of p-values, F-statistics, T-values, and any other form of Null Hypothesis Testing (NHT) method. This major move generated a lot of buzz in the press and on social media: with some praising the move and others urging that we proceed with caution.
In a recent comment in Nature, Jeffrey Leek and Roger Peng point out this ban is only treating one symptom, while we should really be trying to treat the root cause: problems and errors in judgement during any other stage of experimental design and analysis.
“Arguing about the P value is like focusing on a single misspelling, rather than on the faulty logic of a sentence.”
If you’re a longtime reader of my contributions (which is probably a long list of one – me) then you’ll know that I agree with Leek and Peng’s sentiments: all researchers need a solid statistical background in order to design, report, interpret, and review research. There are plenty of statistical courses at most universities and, importantly, online, that can guide researchers in determining how and when to appropriately engage certain statistical and computational tools. This point should ring true with many readers of TME who are interested in genomics (where p-values are still alive, I might add, but come with their important companion the q-value, a false discovery rate measure).
What is the alternative?
While they banned the use of NHT statistics, the editors of Basic and Applied Social Psychology provide no alternatives. They require that papers be published with “strong
descriptive statistics, including effect sizes” and “larger sample sizes than is typical in much psychology research”. Were authors not publishing their descriptive statistics, effect sizes, and sample sizes before?! It doesn’t necessarily need to be an “either-or” scenario – present it all!
We shouldn’t throw the baby out with the bathwater with respect to p-values. Value is in their name, and they have just that. In one number they capture the combination of both the effect and sample sizes. Without those other two numbers, they aren’t as meaningful, but with them, they have true value.
What do you think about the p-value ban? Would you vote “yay” or “nay” on “Prop p”?
REFERENCES
David Trafimow & Michael Marks (2015) Editorial, Basic and Applied Social Psychology, 37:1, 1-2, DOI:10.1080/01973533.2015.1012991
Leek JT & Peng RD (2015) Statistics: P values are just the tip of the iceberg. Nature 520, 612–612.