by Jennie Dilworth, Ph.D
Keywords: statistics, research, integrity
Gotcha! I knew that title would get you to read further! But, seriously, statistics is an important topic in psychology. The quote, which is commonly attributed to Mark Twain, points to the truth that statistics can be used to prove almost anything (because numbers are impressive).
The complete quote goes something like this: “there are three kinds of lies: lies, damned lies, and statistics.” This quote is frequently attributed to Mark Twain who claimed to be quoting someone else.
According to Joel Best, the author of “Damned Lies and Statistics,” statistics have received a bad rap over the years because they can be misquoted, erroneously analyzed, and used to “prove” (quotes are used intentionally) things that may not actually be true.[1]
It is also well known in social science research (and probably in research done outside of social science) that when publishing your research, you can simply report the results that confirm your hypotheses or research questions while leaving out data that does not. This is not the same thing as intentionally altering data, which constitutes fraud. It’s simply a matter of highlighting results that make you (and your research) look good.
People, believing it is easy to manipulate statistical data to fit one’s pre-conceived hypothesis, may propose that all statistics should be ignored. Best explains that this is the wrong approach: “the solution to the problem of bad statistics is not to ignore all statistics, or to assume that every number is false. The solution is . . . to become better judges of the statistics we encounter” (p. 53).
One of the important goals of psychology is to make psychology students better consumers of psychology and of psychological data they will encounter. Even if you aren’t looking up data on psychological phenomenon, you will encounter this data in your social media feed, while watching the news, or even when listening to the radio. Having a basic foundation in statistics and statistical tests (for example, why can’t a correlation prove a cause-effect relationship between the variables?) will help you navigate the numbers being thrown at you every day. And, hopefully, when you encounter data on topics that interest you, you will be motivated to look up more information about how that data was obtained.
As an example, I want to share with you an interesting study result that I heard reported on the radio a couple of decades ago. At the time, I was interested in fitness and weight loss, so I was particularly primed for this “new finding” about weight loss. In the sound bite, it was reported that people who drink cold water throughout the day will burn more calories than those who drink the same amount of water at room temperature. Thus, people could lose more weight by simply drinking colder water. Of course, no details were shared in the sound bite, so I was curious to find out how MANY more calories cold water would burn—10, 100, 500—each day. To me, it was worth the time to follow this up by reading the study.
Because I had access to online databases of scholarly publications, I was able to look up the study [2] referenced on the radio. I read the article carefully and was quite flabbergasted that a study with such an insignificant outcome would ever be reported on the radio. I also wondered how many others who heard the same sound bite might think that the best weight-loss strategy should involve nothing more than drinking colder water. The study found that drinking 16 ounces of cold water burns about 17 calories more than drinking the same amount of room-temperature water. Therefore, a person drinking the recommended number of 64 ounces of water per day would burn only 68 calories more per day by drinking cold water.
Self magazine did a good critique of this study and its flawed design, here is a link to the Self magazine article.[3]
If you would enjoy a laugh or two about the odd titles sometimes given to scientific publications, check out this list on the Slate magazine website.[4]
[1] Best, J. (2011). Telling the truth about damned lies and statistics. In G. Massey (ed.)., Readings for sociology (7th ed.)., (pp. 51-56). New York, NY: WW Norton.
[2] Boschmann, M., Steiniger, J., Hille, U., Tank, J., Adams, F., Sharma, A. M., Klaus, S., Luft, F. C., & Jordan, J. (2003). Water-induced thermogenesis. Journal of Clinical Endocrinology and Metabolism, 88(12), 6015-9. https://doi.org/10.1210/jc.2003-030780.
[3] Barnes, Z. (2016, Sep 4). Yes, technically, drinking cold water burns more calories—but there’s a catch. Self. https://www.self.com/story/drinking-cold-water-calorie-burn
[4] Carpenter, M., & Fritz-Laylin, L. (2015, Dec 15). Scientists’ silly, dark, and sometimes inappropriate humor. What’s in a (scientific publication) name? Slate. https://slate.com/technology/2015/12/the-best-funny-clever-or-offensive-science-paper-titles.html