Bad papers

Dear Mole,

I just finished reading your recent piece "When Papers Go Bad – Part I" (J. Cell Sci. 117, 5953-5954) and as usual found it trenchant and entertaining. This column reminded me of an incident I was recently told of, and I thought it might be worth mentioning. The concept of people reading abstracts from high impact journals and accepting them at face value is probably more widespread than can be dismissed, and this practice may influence reviews of new manuscripts. An investigator working in an area in which reagents are freely distributed, often before publication, provided a second investigator with reagents that allowed the second investigator to pursue a similar line of experimentation. The second group then quickly published a paper in a high-impact journal, reporting conclusions that were new, unexpected and wrong. Having taken the time to get things right, the first investigator subsequently faced the problem that reviews of their manuscript stated "this work all looks fine, but don't we already know from the paper in the high-impact journal that it works a different way?" and then parroted back the abstract of the competing group's paper.

I realize that this is a single example of something only related to the main point of the column. My point is that, yes, all papers will ultimately stand or fall on their merits, and while bad papers can sometimes get published in top journals and readers should decide for themselves what is true, I believe this tale suggests that it may be increasingly difficult for bad papers to fall in the court of public opinion when even reviewers may accept abstracts from high-impact journals as received wisdom.Go



View larger version (41K):
[in this window]
[in a new window]
 
 

Robert S. Krauss

Dear Robert,

Your story is, sad to say, not unique in the annals of what many of us in the profession refer to as `really horrible publication stories', and it is depressing to think that otherwise normal humans (`normal' except for being scientists) behave this way. But the subtext, that once something is published in a high-impact journal, it must be right (or at least must take precedence) is deeply disturbing. Not only because of our horrendous frustration at being blocked from publishing contradictory work (or, maybe worse for our egos, publishing it where it won't be seen) but for a more insidious reason. As you mention, papers ultimately stand or fall on their merits, and the `field' can judge these unkindly. But if you aren't in the core group of that field, the result is utter chaos as you try to get your head around what may or may not be true, and then, when you try to publish based on a wellknown (to other people not you) `bad' paper, you are lost. We should, all of us, push for a change in the way in which papers that contradict other papers are treated – we should encourage their publication and revel in any resultant controversy that arises.

But there is something else of interest hidden in your letter that I might have missed, had not I been using a printed version to write notes on (we must all of us do what we might to save the forests – I do this by carrying scraps of paper in my pockets until they disintegrate). You mentioned that the authors of the quickly scribbled, high-impact paper had gotten it wrong. But here is the very heart of the problem. Wrong? Right? How do we know? Eventually we will learn from experience and experiment, but that will take ages. And even then, how will we know who is right. So maybe it isn't about wrong and right, and the antiheroes have got the thing sussed: "It's only about winning and losing". They got the prize, they got the big one. Promotions, kudos, photos in the paper. It was never about being right. Okay, I'm being sarcastic. No-one should feel this way. It's a terrible way to think about science.

So what to do? The key, I suggest, is to remember this. There were many big prizes and awards and much kudos given for the lobotomy. Nobody remembers who invented it (or at least, not with any fondness). And we'll forget the authors of that high-impact paper, too. Because they were wrong. Right?

Mole

Dear Mole,

I was in Capri, toward the end of my holiday, when I made a fundamental error: I checked my email. Besides spam, confirmations of confirmation, random greetings, etc., there was a referee request. It was from a respected journal, and the title was hot and exciting. Relaxed and in a good mood, I jumped on the abstract and, indeed, the paper looked potentially interesting. So far so good. Then I made the mistake of looking at the actual data. Disappointment, disillusion, desolation. I leafed quickly through the Results, and the once-promising manuscript popped like a bubble. Interest zero. I went to the piazzetta for a granita. Here's the problem: I am in pain. How could I spend an hour of my vacation writing a negative report on a manuscript that I would have thrown away. All that excitement transformed into pure disappointment. I felt betrayed. How could someone start with good faith, show the ability to formulate a nice title, an exciting abstract, identify a hot topic, and then... no controls, erratic experiments, an illogical procedure, and a wholly unsupported conclusion. Are we on the same planet? Okay, maybe they didn't do this on purpose, but, after reading such a disappointing and erratic manuscript, why should I write a politically correct and logical report? Why should I referee at all?

Despondently,

Gerry Melino

Dear Gerry,

As a great US president once said, "I feel your pain!" I don't think he was talking about reviewer angst, but he might have been. How hard for you to be sipping granitas at the piazetta, watching the parade of all the lovely people who are not reviewing papers. Yes, why do we referee, and usually for free, with all the suffering that goes with it? Editors should give us a better deal: when we review ten papers we get one of our own published free. We'd fight for the chance to review! But, of course, we do it because we aspire to a higher ideal, that by reviewing we can maintain standards in science. There are, however, some things you can do to make this easier to handle. First, write an honest letter to the editor, saying how you really feel (after you are polite and respectful to the authors). Editors ultimately make the decision (or should) and they need to know your feelings about the paper. And if you're really honest, you'll make them laugh (and thus feel good that you brought some sunshine into someone's life, even if it isn't the authors). Be strong, be commanding. "Dear Editor, I banish this paper!"

Mole





This Article
Figures Only
Full Text (PDF)
Alert me when this article is cited
Alert me if a correction is posted
Services
Email this article to a friend
Similar articles in this journal
Similar articles in PubMed
Alert me to new issues of the journal
Download to citation manager
Articles citing this Article
PubMed
PubMed Citation