Is the peer review system broken?

The New York Times ran an article yesterday with the following opening:

Recent disclosures of fraudulent or flawed studies in medical and scientific journals have called into question as never before the merits of their peer-review system.

The system is based on journals inviting independent experts to critique submitted manuscripts. The stated aim is to weed out sloppy and bad research, ensuring the integrity of what it has published.

Now, I can’t figure out how peer review can ever achieve the lofty goal of “ensuring the integrity of what [is] published.” I mean, as a reviewer, one can raise questions about research methodology, connections between the research results and conclusions, lack of attention paid to alternate hypotheses, etc. How can one catch someone who willfully fabricates research (a la Hwang Woo Suk)? The entire article seems to be barking up the wrong tree.

The article does contain some interesting stuff, particularly about some frauds that I didn’t know about. Here’s an example:

None of the recent flawed studies have been as humiliating as an article in 1972 in the journal Pediatrics that labeled sudden infant death syndrome a hereditary disorder, when, in the case examined, the real cause was murder.

Twenty-three years later, the mother was convicted of smothering her five children. Scientific naïveté surely contributed to the false conclusion, but a forensic pathologist was not one of the reviewers. The faulty research in part prompted the National Institutes of Health to spend millions of dollars on a wrong line of research.

There are interesting discussion threads over at Adventures in Ethics and Science and Uncertain Principles.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s