Why not double-blind grant reviews?

In some academic fields, double-blind reviews of manuscripts for peer-reviewed publication is the norm. It’s no surprise that people who study human behavior use double-blind review. They must be on to something that most of us in the “hard” sciences haven’t picked up yet.

Some journals in my field have double blind review. Behavioral Ecology has been doing it for a good long while now. My latest paper in Animal Behaviour was a double-blind review, but I don’t know when they started. A couple years ago, American Naturalist went double-blind, too.

Researchers studied the consequence of Behavioral Ecology going double-blind, and found that it increases representation, resulting with more women as first authors, with no detectable negative effects.

A lot of journals have yet to go double-blind, even though the positive effects are there. That’s a long game to have in mind if you’re in a position to have an influence on editorial policy.

Two years ago, NIH said they were “testing — over the next year — the utility of anonymizing grant applications prior to review.” So, what happened with that? I don’t have any idea. Does anybody know?

It’s been five years since there was a paper in Science Magazine showing that there is a funding bias against African-American scientists who seek NIH support. (Here’s a writeup about the results from Science meant for a broader audience.) I can’t really imagine that the situation has magically improved between then and now. Are they still piloting out that double-blind review process?

One common quibble that people raise is that you can often guess who the person is. But there’s a big difference between guessing and knowing, and sometimes people guess wrong. Also, don’t forget that we know going double blind actually makes a difference. So saying that it doesn’t matter is just not true.

Are grants harder to double-blind than manuscripts? I suspect it’s easier. While reviewers are supposed to evaluate whether the person submitting the proposal is qualified to do the work, and whether the facilities at a particular institution are adequate for the work, that’s something that an appropriately trained program director can do before that information is excluded from the review. The probability that a reviewer can guess the author of a grant is probably not that different than the probability of guessing the author of a paper, if anything it’s lower because grants are about new stuff that hasn’t been done yet. (Using my own anecdote, I don’t think very close colleagues of mine would ever attribute my most recent grant to me.)

As far as I can tell from a few minutes on the google machine, NSF has experimented with having people competing for the same pool of money review one another, but not with double blind. Is that the case? This does seem to me to be one thing that can have a quantifiable effect on improving equity, and I’d be interested in hearing any compelling reasons, if they exist, why we haven’t made the move to double-blind review throughout our academic community.

[Update: check out the the follow-up post that discusses the track record issue, and the whole issue in more depth.]

_{(As a postscript, I should add that in the absence of double-blind review, in my opinion, no-blind review is worse than single-blind.)}