I understand you're annoyed about that rejection from NeurIPS. As an AC, I can confirm that I recommended rejection for a bunch of perfectly adequate papers. They were just not very exciting.

Of course, all of the rejected papers has some kind of issue. Like, the wrong benchmark, insufficient ablations, forgot to cite some important work, or something. But all papers have faults. Think of the ML papers that influenced you the most. Are they perfect? Didn't think so.

If you think a paper is really exciting, you're willing to overlook its weaknesses (within reason). But most people don't think most people are exciting. In fact, when bidding for NeurIPS papers, even using all the fancy sorting functions, I struggle to find anything I care about

One big problem is the low acceptance rates. They turn reviewing into a fault-finding exercise. And the demands of tenure and promotion committees keep acceptance rates low. So many authors write papers to be bulletproof, in the process making them when more boring.

A potentially even bigger problem is that conferences are so big. They bring together people with very different interests, and all but ensure that reviewers are matched largely with papers there are not interested in. So these reviewers will look for reasons to reject the papers

In my experience, small conferences and workshops with high acceptance rates are where the action is at. That's where I find papers that interest me. Part of the reason is that if you write without far of rejection you write more interesting papers.

But a probably bigger reason is simply that I'm more interested in the fields of the small conferences and workshops I attend. I mean, why waste your time on a conference where you're not interested in 95% of the sessions?

I so wish that there was no institutional pressure/incentives to submit to the big, selective conferences. If only people who actually wanted to submit to, say, NeurIPS, submitted there, it would be a much better conference. Also smaller.

(For full disclosure: I did not submit anything to NeurIPS this year. My lab had already submitted all our mature papers to venues we care about. I do reviewing and ACing for these kind of conferences out of some kind of academic duty.)

Continuing the discussion here, as people are reading this thread who are not familiar with the process for this kind of conferences. https://twitter.com/togelius/status/1309995515271098368