[1/n] Dear PL Community,
Would you please indulge me, a tenure-track prof., as I share some thoughts about rejections, quality standards, reviewing processes, and our ultimate scientific goals?
I really want to hear your thoughts!
But please don't begin with "In my experience".

[2/n] We often talk about how to deal with rejections, as we should. It's important to learn to absorb constructive criticism and also to learn when to trust your own instinct about your work. It's important to remind oneself that research is far more than the pursuit of papers.

[3/n] Btw, I don't think that learning to toughen up and march on disregarding the nature of the rejection, the backdrop, and one's current circumstances is the right or sustainable way.

To deal with rejections the right way, one needs some time, space, freedom, power. Agreed?

[4/n] So do we as a community and as part of the larger Academy provide researchers all this? Well, that's exactly what PhDs, tenure, full-professorships are for.
Great! But how does one get through these stages?
We do some high-impact work in a certain amount of time.

[5/n] How does one measure high impact in a meaningful, consistent way? This is obviously a hard problem and we typically use some "top" conferences/grants/awards as measures of quality and impact. We avoid bean counting and work hard to regularize reviewing procedures.

[6/n] Ok this clearly makes it hard to focus on research without thinking about publications. But let's set that aside for now.

What sets apart top conferences? Besides historic reasons, low acceptance rates and high visibility (high number of submissions, attendees, etc.)

[7/n] And here's where I begin to have more questions I can't answer myself.
In our gatekeeping, why do we tend to look past smaller conferences with a narrower focus and size, which are likely to have higher acceptance rates because of their self-selection?

[8/n] Within top conferences, why exactly do we want to want to keep the acceptance rates low? I don't buy the "not enough space" constraint---we can come up with creative solutions and it's clear that many conferences that are going to be virtual are still targeting low rates.

[9/n] Why do we, as a community, not worry that the low acceptance rate targets (along with broader scopes and PCs) are making decisions increasingly stochastic and inconsistent? Do we not see that the push for lower acceptances makes it easier for reviewers to "kill" papers?

[10/10] Ultimately, how does all this gatekeeping based on subjective and inconsistent notions of quality/impact benefit science in the long term? Hasn't science always benefited from an expanse and heterogeneity of ideas? Hasn't science always been hurt by narrow-mindedness?

@michael_w_hicks @ksmckinley @RanjitJhala @rupakmajumdar @suresh12345 @rg9119 @alexandra8silva @JAldrichCMU @djg98115 @emeryberger

I hope some of you senior PL folks will weigh in when you have time. I really think we need to introspect some more as a community.

Thanks all for an awesome thread! Here are some highlights.
1. Most shared my concerns about low acceptance rate targets. It would have been good to hear some counter-arguments though. We need to talk about how exactly to withstand external pressures to keep rates low.

2. There was a lot of support for @rupakmajumdar's policy at POPL'16: if a paper had a champion after discussion it would go in even if another reviewer was negative. This can be refined to account for negative reviews with technical concerns.

3. There wasn't a clear convergence about how to define "quality" and ensure consistency. Some counterpoints were raised for my proposal of requiring reviews to be technical "reports" as opposed to opinions....

....We first need more evaluations such as the NIPS consistency expt. or https://peerj.com/articles/cs-299/. There is also the question of how to enforce any reviewing criteria guidelines.

4. There was not much discussion on the value of smaller conferences and the impact on science.

I hope this thread (and many others before it) will help @sigplan and SCs focus a bit more on what voices and ideas are being left behind. I know the incentives for survivors to do this are low, which is why I thought a tenure-track prof. should raise these questions.