I'm hearing how people are now getting the reviews back from their GRFP applications.

Getting these reviews back is really hard, and can be a rough introduction to the funding process. I'd like to contextualize these reviews with some insights from the process.

First of all, it's okay to be upset and annoyed. Even if other people think the reviews are even-handed and fair, it's okay to be mad. Usually when I get negative remarks in reviews, I take a quick glance and then let them sit for a while, just to collect myself.

What makes the NSF-GRFP reviews feel even more personal is that these actually are reviews of actual people. Other grant proposals are reviewing the project as a whole package. But this is a fellowship that funds the person, and not the project per se. So, yeah, this is rough.

Because of what I've written about this program on here and on my blog, I've been made privy to a lot of reviews from students who perceive some outrageous digs and biases in their reviews. And in many cases, I agree with that. Some reviews have inappropriate and biased remarks.

I think NSF generally does an outstanding job in their review process, and I have more confidence in the fairness and deliberation they use to distribute research funds than any other agency. But, of course, some reviewers are going to have biases. That's just a fact.

NSF works strenuously to remove biases. They work to train reviewers to avoid biases, program officers are highly professional and equity-minded about their work, and there is a great amount of attention to the composition of review panels to represent all kinds of scientists.

Panels can have only so many people, but every panel I've been on has had robust representation in so many axes of diversity that matter: ethnicity, gender, disability, institution type, career seniority, disciplinary expertise. I'm always amazed at how great the panels are.

Given the many thousands of people who apply for the GRFP, I can imagine how hard it is to assemble these panels. It really relies on the community of scientists to volunteer their time. And these panels are done remotely, not in person, so that's another factor.

So I just want to let you know that when you see a bigoted remark in a review - there are people on the panels, and also the program officers, are probably just as mad about it as you are. If you see it, there are others seeing it too.

The way the review process works, the individual reviews get written before the panel meets. And once the panel starts discussing the proposals, things can evolve very quickly. I wouldn't ignore what is said in a review, but many remarks in reviews will have little/no weight.

What really matters is how the panel uses the reviews in the context of the panel discussion. If there's evidence of bias in the summary statement -- now that's a real cause for concern. But usually the horrible crap is just in reviews.

The bummer is that on the submission side of the proposal, you won't be explicitly told that the biased or bigoted or ignorant remark that you get was dismissed out of hand.

All I can say is that when I've been on panels, I've seen reviews with inappropriate remarks. And either the panel ignores them, or the panel talks about the substantive issues in a proper manner.

I don't fully understand why NSF doesn't redact partial or complete reviews that reveal inappropriate biases. But they don't. They let the panel do its job and the summary statement reflects what the panel does.

I don't mean to minimize the crappiness of bias that can pops up in reviews. But I'd like to reassure you that, there are people there who see what is going on and are fighting for equity. And the program officer is specifically trained to deal with these situations.

In other words, evidence of clear bias in a review doesn't necessarily taint the entire review process. The review process is designed knowing that bias exists and attempts to deal with it. Often it works, though obviously, it's not a perfect process.

So, in summary: If you had something horrible in your review, I'm so, so sorry. It's not right and it's not fair. I hope I've given you some information that can help put this in context, to see how it may or may not have affected the review process.

For more information about how the review process works and what this means for writing better proposals (not specifically for the GRFP), here: https://smallpondscience.com/2017/01/02/lessons-from-serving-on-nsf-panels/