How to Read the Science News #5: Who Published the Study?

When you're reading the popular science news – that gee-whiz stream of “Can you believe this?” “Miracle cure!” and “Everything you know is wrong” – it helps to ask a few questions. Here's #5 on the list: Where was the original research published?

Who do you trust?

Who do you trust?

This might sound dry and boring, but it's important. And pretty easy to answer. Real-world science doesn't matter, at least from a professional standpoint, until it's been published in a scientific journal. The standard process goes like this: A researcher does some science – conducts a series of experiments, crunches some numbers, makes a series of observations, whatever – and finds something that looks like a valuable new addition to their field. They don't blog it or go to their local newspaper. No, the first thing they do is write it up in the form of a formal research article and submit it to a scientific journal. The journal editors then review it and send it out to a few other researchers in the field for some critical feedback. It's called “peer review,” and it's a very important step. Letting other scientists look over the unpublished work in private allows them to comment on experimental shortcomings, question methods, analyze data, put it into context. It's an important quality control step. Only after the paper passes this peer review (often after several rounds of back-and-forth editing) is it published.

Most real scientists (as opposed to company PR people, marketers, or makers of diet supplements, skin rejuvenation creams, and other sciencey-sounding products) will not talk to the media until their work has been published in a scientific journal. If the news story you're reading doesn't mention an article published in a journal, be suspicious. Either the “scientist” whose work is being touted is not a scientist at all, or the work is not worthy of publication.

And even if a scientific journal is cited, it doesn't mean that the work is legit. Today, a number of publishers have figured out that you can start a publication that looks and sounds like a scientific journal, but is really a place where just about anybody can get just about anything published if they pay a fee. There is little quality control. 

How can you tell a legit journal from one that's not? Scientists have a pretty strong sense of what the top journals are. There is a sort of pecking order, with the most respected ones at the top, for every field of science. If you can get your research published in a top journal, it will have greater impact on the field. Their quality control is good, and the competition to get into them is fierce. In theory, top journals publish only top papers. 

There are different ways of measuring the best journals, but almost always a few major players are at the top of the general science list: Science (and its various sub-journals), Nature (and its sub-journals), The Proceedings of the National Academy of Sciences (PNAS), the publications of the Royal Society in the UK (the oldest of the group), and PLOS ONE (the youngest). Anything published in these journals can be trusted as a legit advance in science. You can find out more about top science journals here and here.

So it pays to know (a) that the research was published in any sort of scientific journal and (b) what that journal was. If the news was not based on a journal article – say it was based on interviews instead, or some “doctor” making claims – don't buy it at face value. You have to ask a few more questions: What are their credentials? What is their agenda? Will they profit from their statements?