On October 12, Jezebel ran a story with the headline: “Study Says People Who Take Coffee Black Have Psychopathic Tendencies.” Reporting on the same story, Popular Science titled its piece: “Love Beer and Coffee? You Might Be a Psychopath,” and social media’s preferred source of science news, I fucking love science, reported: “Study Claims People Who Like Their Coffee Black Are More Likely To Have Psychopathic Tendencies.”
The study, having gone viral after the initial headline, seems to have evolved into a completely different study by headline alone—“If You Drink Black Coffee, Experts Say You Might Be a Psychopath” says Delish; “Like Your Coffee Black? Congratulations, You Could Be a Psychopath,” says Eater; “Love dark chocolate and black coffee? You could be a psychopath,” says Telegraph UK; “Like coffee or a gin and tonic? You could be a psychopath,” says The Daily Mail.
Of course that’s not really what the study (led by Professor Christina Sagioglou from Innsbruck University in Austria and published in the journal Appetite) concluded, but I’m hoping you’ve gathered that by now. (Twin Peaks fans can rejoice that quintessential gentleman Dale Cooper most likely does not have psychopathic tendencies, at least not due to his coffee-drinking habits, but that’s neither here nor there.) Slate writer L.V. Anderson explains the study’s actual methodology here, which asked for people’s flavor preferences and found a slight association between people who said they liked bitter foods—without being asked about coffee, beer, dark chocolate, gin and tonic, or any other specific bitter food—and “traits associated with psychopathy, narcissism, sadism, and Machiavellianism.”
Anderson also argues that it’s possible participants didn’t take the study very seriously, given that survey takers were recruited through Amazon’s Mechanical Turk, a platform that recruits and pays individuals minimal amounts to perform “human intelligence tasks.” Participants in this survey were paid $0.60-$1.00 for their participation and, according to Anderson, were asked less-than-precise questions:
Self-reporting is notoriously unreliable, and the study included a bunch of personality-test questions that I, personally, would have no idea how to answer. For instance, participants were asked to rate this statement on a scale from 1 (extremely uncharacteristic of me) to 5 (extremely characteristic of me): “I often find myself disagreeing with people.” What constitutes disagreement? What constitutes “often?” Does “people” include Donald Trump or just people I actually know and hang out with? If I were being paid $0.60 to answer fifty-two maddeningly vague questions like this, I’d probably just mark a number arbitrarily and move on as quickly as possible.
The study certainly seems flawed, but don’t take my word for it or Anderson’s word for it. Read the study yourself. I don’t have to tell you that I fucking love science and other online outlets aren’t telling the whole truth (or skew dangerously close to inaccurate) when they report stories like “Ebola Confirmed To Be Sexually Transmitted Disease,” “Cat Parasite Linked To Development of Mental Illness In Owners,” or “More Than 200 Life-Extending Genes Identified.” I don’t need to caution you to remember that whether coffee causes or protects you from cognitive impairments depends on whether you read about it on Forbes.com (“Here’s More Evidence That Coffee Is Good For Your Brain”), CBS Atlanta (“Study: Increasing Coffee Intake Harmful to Brain”), or in the Washington Post (Yesterday’s coffee science: It’s good for the brain. Today: Not so fast…*”)—simply because how a headline is written creates the story’s angle and frames your perception of the issue.
Unfortunately, most stories aren’t clicked on for the value of their information but because the headline appeals, and journalism isn’t evaluated by the accuracy of its reporting as much as it is by the number of advertisers reaching a satisfactory number of people.
This is just a friendly reminder that an article with Twin Peaks in the headline may have little at best to do with the TV show in the same way that an article about coffee and psychopaths may have little to do with either. Stay skeptical. Stay (or get!) statistically literate.