Is the peer review system broken?

The New York Times ran an article yesterday with the following opening:

Recent disclosures of fraudulent or flawed studies in medical and scientific journals have called into question as never before the merits of their peer-review system.

The system is based on journals inviting independent experts to critique submitted manuscripts. The stated aim is to weed out sloppy and bad research, ensuring the integrity of what it has published.

Now, I can’t figure out how peer review can ever achieve the lofty goal of “ensuring the integrity of what [is] published.” I mean, as a reviewer, one can raise questions about research methodology, connections between the research results and conclusions, lack of attention paid to alternate hypotheses, etc. How can one catch someone who willfully fabricates research (a la Hwang Woo Suk)? The entire article seems to be barking up the wrong tree.

The article does contain some interesting stuff, particularly about some frauds that I didn’t know about. Here’s an example:

None of the recent flawed studies have been as humiliating as an article in 1972 in the journal Pediatrics that labeled sudden infant death syndrome a hereditary disorder, when, in the case examined, the real cause was murder.

Twenty-three years later, the mother was convicted of smothering her five children. Scientific naïveté surely contributed to the false conclusion, but a forensic pathologist was not one of the reviewers. The faulty research in part prompted the National Institutes of Health to spend millions of dollars on a wrong line of research.

There are interesting discussion threads over at Adventures in Ethics and Science and Uncertain Principles.

Advertisements

One thought on “Is the peer review system broken?

  1. Ah yes, I have just found this rather interesting blog and am prodded to make my second comment regarding peer review. Allow me to fall back once again on personal but factual anecdote. A very close friend with a long and distinguished track record in the field of reproductive biology (awarded a lifetime achievement award from the Society) has been struggling to publish a scientific article in a leading publication, perhaps the 60th or 70th such article he has published over a distinguished career. In this case, however, publication has been refused for over two years despite half a dozen re-submissions.

    Why, you might be tempted to ask? An acknowledged leader in the field denied publication. Really?

    Yes.

    It turns out the editorial team of this journal has been taken over by a highly partisan group advocating the primacy of one side of a long-standing argument (between advocates of oxytocin vs. advocates of interferon as the key driver of important processes) and for all intents and purposes suppressing dissenting views.

    Shame shame shame.

    Peer-review, it should be understood, does NOT mean fair review.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s