Read on Twitter
Post on polling in 2020. TLDR: projections based only on past voting returns better predicted 2020 results than poll averages in battleground states. I review how we used this to decide to spend in GA, then ask what might went wrong with polling in 2020? https://solomonmg.medium.com/past-vote-data-outperformed-the-polls-how-did-it-go-so-wrong-sol-messing-a76b7f9702aa
Back in April I had the notion to make some projections to see if it might make sense to expand where we at @anotheracronym might spend on mobilization in 2020.
I put together some projections inspired by findings from @yghitza, which if I understood them correctly, suggested that the best election forecasts look a lot like regularized swing projections from past electoral contests.
We used these projections, other extant data (including that there are two Senate races in play), to make what turned out to be a very lucky decision to start spending money in Georgia. We were AFAIK the only IE group in GA, & targeted low-info, low propensity voters.
The 2020 polls underestimated Trump’s support by anywhere from a 4-7 point margin–a significantly worse miss than in 2016, when state polls were off but the national polls did relatively well.
The polling from 2018 looked encouraging, convincing many pollsters that the post-2016 reconing had fixed many issues called out in the 2016 AAPOR report on election polling. After 2018, FiveThirtyEight wrote that the “Polls are Alright”. https://fivethirtyeight.com/features/the-polls-are-all-right/
But the second Miami-Daude reported results from the 2020 election, we knew something was probably wrong with the 2020 polls. Unlike 2016, both state and national polls appeared to underestimate Trump’s support, as this early (Nov 7) analysis from @thomasjwood shows:
Evaluating some projections from April that we used internally at Acronym to help evaluate where we might want to spend, I pulled in the NYTimes polling averages and compared them with the latest state-level presidential results from the AP. https://www.nytimes.com/live/2020/presidential-polls-trump-biden
Turns out our projections were significantly more accurate than the polling averages:
What are these projections? I simply took the last two state-level Presidential and U.S. House election totals, estimated each state’s “trajectory,” and added that to each state’s Democratic margin from the previous cycle. More details here https://solomonmg.medium.com/past-vote-data-outperformed-the-polls-how-did-it-go-so-wrong-sol-messing-a76b7f9702aa
Humble-brag aside, it’s worth asking what might have gone wrong with polling in 2020?
The 2016 AAPOR report on election polling provides some guidance for how we might start to examine issues with the 2020 polls.
https://www.aapor.org/Education-Resources/Reports/An-Evaluation-of-2016-Election-Polls-in-the-U-S.aspx
https://www.aapor.org/Education-Resources/Reports/An-Evaluation-of-2016-Election-Polls-in-the-U-S.aspx
1. Undecided voters: Undecideds broke toward Trump late in the election in 2016–as many as 13 percent of voters were undecided or 3rd party on election day. There were half as many of these voters in 2020, so it's unlikely to be as big a factor as in 2016. https://www.poynter.org/fact-checking/2020/2020-is-not-like-2016-heres-whats-different/
2. Low education non-response & adjustment: In 2016, individuals lower levels of education were much less likely to answer polls but still voted, and broke for Trump. National polls adjusted for this but state level polls did not.
Pew Research Center still found problems with state level polling in 2020, failing to adjust for race and education simultaneously–non-college whites are far more likely to support Trump than non-college non-whites. https://www.pewresearch.org/methods/2020/08/18/a-resource-for-state-preelection-polling/
What’s more, pollsters focused on college/non-college in 2020, which may not have been enough. They might need to account for whether respondents have a high school degree and a college degree, + error/missing data.
Volunteerism & civic engagement: Even after adjusting for education, pollsters still may have problems reaching low civic engagement voters, which persists after modeling/weighting adjustments. https://www.pewresearch.org/fact-tank/2015/07/21/the-challenges-of-polling-when-fewer-people-are-available-to-be-polled/
Likely voter models: Some pollsters ask a battery of vote questions, some use models, some recruit off the voter file. But there’s only a weak relationship between who votes and who scores high on the likely voter battery.
To make matters worse, 2020 was a very high-turnout election, which could have introduced even more instability into likely voter models.
COVID-19: I wrote about this back in June. It’s possible that COVID-19 made lines long and kept people home in urban areas and non-white communities. Yes we had record turnout but... https://solomonmg.github.io/post/what-the-polls-got-wrong-in-2020/
all it takes is a few percent of people who encounter a bit of voting friction, who fail to register in person, don’t get in person canvassing/gotv contact, don’t vote by mail early, and/or don’t vote in vote in person.
Forecasts: even when forecasters get it right, forecasting can create firm expectations that one candidate will win, which in 2016 was complicated by destiny-narrative driven by media coverage of election forecasting. We didn't have the same kind of coverage in 2020, but
Forecasts still create the strong impression that you should be very certain about the race. @seanjwestwood, @ylelkes & I recently published work in JOP showing how much confidence forecasts give us https://solomonmg.github.io/pdf/aggregator.pdf
And wrote about some of the implications for the 2020 election in a recent USA Today op ed. https://www.usatoday.com/story/opinion/2020/10/01/election-forecasts-can-wrong-you-still-need-vote-column/5857993002/
I believe it was the sharp violation of expectations that was so disappointing to Clinton supporters and so invigorating for the MAGA crowd—the Washington elite had underestimated “real Americans” yet again.
AGGH, sorry @thomasjwood the preview image was supposed to be a different vis
Definitely want to make sure we acknowledge other’s efforts in GA + amazing on the ground organizing there. I regret the error https://twitter.com/danancona/status/1325483942561017856
Correct link to my post discussing how COVID might make Trumps chance of re-election significantly better than it looked in June https://solomonmg.github.io/post/trumps-chances-are-better-than-they-look/
Additional folks in GA https://twitter.com/jenancona/status/1325507591930216448
A similar observation from @gelliottmorris https://twitter.com/gelliottmorris/status/1325522151005450247
And should add the @TatendaCheryl did insanely amazing work this cycle, mobilizing voters in GA & doing digital outreach in the right ways https://twitter.com/tatendacheryl/status/1323405254038327298
Should also give credit where credit is due—conversations I had w @gelliottmorris & his forecast helped convince me that going into GA was important
