I've been hearing a lot about pollsters' interest in new modes, in particular, mail-based surveys (face-to-face even!). I'm naturally curious about these experiments but I'm not as confident there's a magic bullet there. Why? 3 words: the 2016 @DispatchAlerts Poll, & here's why:

As you might guess, the Dispatch poll (which wasn't conducted post-2016, curious why @darreldrowland @dispatcheditor?), was conducted entirely by mail & had been praised in the past by academics for its data quality, high response rates, and accuracy

https://pprg.stanford.edu/wp-content/uploads/1996-Dispatch-POQ-article.pdf

This was also the case in 2016 when the poll was last conducted (I think you all know where this is going). The poll had a _great_ response rate (~10%) & was sampled off the OH voter file database which has great address coverage.

https://www.dispatch.com/article/20161104/NEWS/311049729

The Dispatch provided paid postage to voters & had humans transcribe the survey results for all 1,151 survey respondents (quite the feat!). Alas, here were the results. Nailed the Senate race, but missed (badly) in the Presidential race (by nearly 10 points).

Now, there are places to quibble w/ the survey weighting, etc (50% of weighted respondents have a college degree), but kind of hard to look at this data & think this is the polling industry's savior.

https://www.dispatch.com/content/stories/local/2016/11/06/dispatch-poll-finds-presidential-race-too-close-to-call.html

I'm open-minded that mail is something researchers should examine (& some already are!), & looking at underused modes (even face to face) seems like a useful exercise. But if there's anything to learn from (recent) history, it's not to place too much hope on it

I'm worried that, especially among public pollsters, many are "stumped" on new weighting schemes & variables to account for the continued non-response bias we saw in 2020. Better data would be nice, but the pragmatic case is doubling down on improved data & adjustment techniques