Skip to content
August 15, 2016 / jpschimel

Single vs. Double-Blind Review: Is it really bad to let reviewers know who you are?

 

A concern with peer review has always been prejudice; prejudice born of reviewers knowing who the authors are but not vice versa. This raises a clear potential for abuse. Shit happens, and I think all experienced researchers have had some experience with inappropriate or personally charged reviews. More recently, the concern has shifted to covert prejudice—quite possibly unconsciously—against women, minorities, other nationalities, or even junior colleagues. A paper authored by John Smith or even J. Smith might review more favorably than one by Jill Smith, Juan Herrera, or Shujin Zhu. Prejudice, whether overt or covert, degrades peer review and scientific publication.

To avoid this, some disciplines and journals are moving to double-blind review in which the names and affiliations of the authors are removed from the paper. In some areas, double-blind is considered a necessary and fundamental requirement of a fair peer review system.

However, in other areas, the counter-argument has been that double-blind is pointless, because reviewers can figure out who the authors are. For example, in environmental field sciences, the combination of topic, approach, and research site can limit the possible research group to such a degree that the reviewer is able to “peek past the blindfold.” If someone is doing work on summertime soil biogeochemistry of California grasslands, working at the Sedgwick Reserve, it wouldn’t be much of a stretch to guess that the work came from my lab. If the paper noted that isotope samples were analyzed in the University of California Santa Barbara Marine Science Institute analytical lab, you’d have it nailed.

Thus, even with double-blind systems, reviewers are often sure they know who the authors are. But research suggests they are regularly wrong; I can vouch for that—I once reviewed a paper and noted to the editor: “This is a really nice paper out of so-and-so’s group.” The paper was covered with “fingerprints” such as personal communication and unpublished data references, but the editor wrote back to tell me I was wrong. My response was that it was clear that there was some relationship between the actual authors and the group I tagged, but that more importantly, it didn’t matter that I was wrong! If I had prejudices, they would still have tainted my review. The counter is that if reviewers are at all uncertain about the authors, it could at least diminish the effects of any prejudice they hold; but in my case, I wasn’t uncertain—I was just wrong. Oops.

In any event, all the discussions I have ever seen have always focused exclusively on eliminating potential bias in the assessment of the manuscript itself, trying to ensure that decisions on the fate of a paper are not a function of who wrote it, but solely on what they wrote.

But submitting a paper is also a form of professional networking. As I mentioned in a previous blog post, “the Editors and reviewers who run the journals are your professors and your colleagues—people you want to be your friends (and maybe your postdoc advisor).” Early career scientists have an interest in becoming known to their senior colleagues. Yet, the papers I read most carefully and pay closest attention to are those I review; I’m likely to register who wrote a paper when I review it. When I get a double-blind manuscript, I may be able to guess where the paper came from, but I can’t know which student or postdoc actually wrote it. Having me know that Dr. Loreau’s group just produced a nice piece of new work may benefit Dr. Loreau, but it does nothing for Ms. Sylvain who actually wrote the paper.

Sometimes, useful relationships even develop from the review process—I started working with Stefano Manzoni, now one of my most valued collaborators, as a result of a review I wrote (and signed) of one of his first papers. He took some ideas I’d included and developed a new model that elaborated on them; he then invited me to be a co-author and we’ve worked together since. Such Cinderella stories may be rare, but they do occur.

If that had been a double-blind review, I couldn’t have told that it was from a group that was newly moving into soil biology and might well not have invested so much in the review. Would I have signed it? I suspect not—anonymity breeds anonymity. And I know I said things that I wouldn’t have said in a completely open review system. Signing that review has benefited both of our careers. Letting the reviewers know who the authors are can help find the glass slipper.

The networking and advertising benefits in classical single-blind review may be modest and occasional; but they are real, and eliminated by double-blind. The debates over single vs. double-blind I’ve seen consider only the balance of risks from prejudice in single-blind vs. the hassles or inefficiencies of double-blind. They don’t consider any overt potential benefit to the authors in classical single-blind. They should.

In some fields the cost-benefit balance of review systems will clearly come down on the side of double-blind. In others (particularly I suspect field-based sciences such as ecology) the balance might well shift to single-blind.

Importantly though, the discussions should consider that the review process is more than a simple evaluation of a manuscript. It also builds relationships among people.

Advertisements

6 Comments

Leave a Comment
  1. Meredith Warshaw / Aug 15 2016 1:10 pm

    That’s a very interesting perspective. I hadn’t thought of the positives of unblinded reviews before.

    As both a reviewer and reviewee, I can see one other advantage to double-blind reviews. Even though one can often tell which group wrote a paper, having it double-blind means the reviewer can’t tell if it was written by a grad student or non-Ph.D. researcher at the lab, or junior versus senior faculty. Many reviewers have conscious or unconscious biases based on status – both against the junior researchers and too slanted in favor of senior ones. The level of bias may vary by field; I know it’s strong in my area of research.

    • jpschimel / Aug 15 2016 1:50 pm

      In my area, the vast majority of papers have, as first author, a junior scientist–student or postdoc. I am almost never first author on an experimental paper, as I’m never the one in the lab doing the work, and the person who did the work writes the paper and is first author. The things that I am sometimes first author on are syntheses. Occasionally I will pick up and finish off the submission process for a student’s paper, in which case I will be the corresponding author.

      But yeah, I want to know who the up-and-coming talent is, if only to start asking them to review papers I handle as editor!

      But cultures differ among fields. I would never assume because a junior researcher is lead author that that means that the paper is in any way lesser than if it were lead authored by a senior person. I assume that the senior will ensure that the quality is up to snuff. But that goes along with the philosophy that as a scientist it is my job to produce both science and scientists. Certainly papers that come from my lab might be written a little better (or at least differently) if I wrote them myself, but I don’t have time to do that, it would be a violation of the mentoring responsibility to do so, and they don’t go out at all until the science and the story are strong.

  2. mngwa / Aug 15 2016 2:47 pm

    Very different in medicine, where authorship lists are long, people have a gazillion publications, and the research world is very hierarchical. When I explained to my mathematician son that in medicine one wants to be first author or last (senior) author, he looked at me like I had two heads. Of course, in math there are seldom more than two authors and they’re always listed alphabetically.

  3. Nadine Forget-Dubois / Sep 18 2016 4:50 pm

    You can always avoid looking at the authors’ names. I’m occasionally asked to review a paper but, as a lab coordinator pretty low in the academic hierarchy, I fear I could be impressed by the name of a bigshot among the authors and question my ability to read the paper critically. I check the authors’ names only after a first reading. There’s an inconvenient, though. I once had to admit to an editor that I knew the bigshot author of the paper I was reviewing.
    Glad I found your blog, Dr. Schimel. I cite your book abundantly in my own (French language) blog about scientific writing.

    • jpschimel / Sep 18 2016 5:29 pm

      Looks like I can use working through your blog as a tool for improving my French. When I was in Montpellier for the summer, I worked through Harry Potter.

      • Nadine Forget-Dubois / Sep 18 2016 6:13 pm

        That’s funny because a lot of French-speakers improve their English by working through Harry Potter.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: