Read on Twitter
1/x The polls weren't wrong, you just read them wrong.
The spread in a poll matters some. But a 50-47 lead (+3) is FAR MORE ROBUST than a 46-40 lead (+6)
People see a poll avg at 46-40, and when the result is 47-49, they say "pOlL wAS WrONg"
From +6 to -2?!? Off by 8!!!
Nope.
The spread in a poll matters some. But a 50-47 lead (+3) is FAR MORE ROBUST than a 46-40 lead (+6)
People see a poll avg at 46-40, and when the result is 47-49, they say "pOlL wAS WrONg"
From +6 to -2?!? Off by 8!!!
Nope.
2/x
If the polls were "off" by 3 or 4 but the result of the election didn't swing because of it, no one would care, except pollsters. We wouldn't be talking about it.
Here's the most important question: what do polls measure?
If the polls were "off" by 3 or 4 but the result of the election didn't swing because of it, no one would care, except pollsters. We wouldn't be talking about it.
Here's the most important question: what do polls measure?
3/x
This seems like a silly question, but think about it. What do polls measure?
Polls measure preference and plans.
What do polls not measure?
How/if undecided voters will vote.
It seems that people are criticizing polling for not measuring things it doesn't attempt to.
This seems like a silly question, but think about it. What do polls measure?
Polls measure preference and plans.
What do polls not measure?
How/if undecided voters will vote.
It seems that people are criticizing polling for not measuring things it doesn't attempt to.
4/x
This goes back to high school. Remember studying validity & reliability?
When we talk about validity, we ask: "does this study actually measure what we're using it to measure?"
In the case of polls: they don't measure preferences or tendencies of undecided voters.
This goes back to high school. Remember studying validity & reliability?
When we talk about validity, we ask: "does this study actually measure what we're using it to measure?"
In the case of polls: they don't measure preferences or tendencies of undecided voters.
5/x
Polls are intended to measure HOW MANY voters plan to vote for each candidate, and how many are undecided
That's it
For aspiring political and data scientists:
If you figure out a way to calculate or measure how/if undecided voters will vote, I have a blank check for you.
Polls are intended to measure HOW MANY voters plan to vote for each candidate, and how many are undecided
That's it
For aspiring political and data scientists:
If you figure out a way to calculate or measure how/if undecided voters will vote, I have a blank check for you.
6/x
The problem with the polling wasn't a problem with the polling at all. Simply how people read them.
A 45-40 lead (+5) isn't simply a +5.
More undecideds = more uncertainty = more variability.
2016 had a historic number of undecided voters. https://www.cnbc.com/2016/06/20/25-of-voters-in-2016-race-remain-undecided-survey.html
The problem with the polling wasn't a problem with the polling at all. Simply how people read them.
A 45-40 lead (+5) isn't simply a +5.
More undecideds = more uncertainty = more variability.
2016 had a historic number of undecided voters. https://www.cnbc.com/2016/06/20/25-of-voters-in-2016-race-remain-undecided-survey.html
7/x
Clinton's leads in these swing states (+1, +3, even +6) have her polling no higher than 46.8%.
And while I'm not a fan of lumping declared "third party" voters with "undecided" voters, it doesn't take a mathematician to see the gap could feasibly be made up.
Clinton's leads in these swing states (+1, +3, even +6) have her polling no higher than 46.8%.
And while I'm not a fan of lumping declared "third party" voters with "undecided" voters, it doesn't take a mathematician to see the gap could feasibly be made up.
8/x
The assumption seemed, that the undecideds would break somewhat evenly, not impact Clinton's lead. Other than the openly and augury @FiveThirtyEight "Trump still has a chance, undecided voters could swing the election"
The assumption seemed, that the undecideds would break somewhat evenly, not impact Clinton's lead. Other than the openly and augury @FiveThirtyEight "Trump still has a chance, undecided voters could swing the election"
9/x
2016 introduced me to a term I hadn't heard before: double haters.
People who didn't like EITHER candidate.
These folks were mostly moderate or conservative, older, and low-middle income (and cared about jobs!)
These double haters broke for Trump by a whopping 20 points.
2016 introduced me to a term I hadn't heard before: double haters.
People who didn't like EITHER candidate.
These folks were mostly moderate or conservative, older, and low-middle income (and cared about jobs!)
These double haters broke for Trump by a whopping 20 points.
10/x
Now, hindsight bias and all, people say "duh, of course conservative undecided voters would favor Trump"
But that's not the point!
Those undecided voters ended up VOTING for Trump.
Many undecided voters who favored Hillary didn't vote. https://www.washingtonpost.com/news/politics/wp/2018/03/12/4-4-million-2012-obama-voters-stayed-home-in-2016-more-than-a-third-of-them-black/
Now, hindsight bias and all, people say "duh, of course conservative undecided voters would favor Trump"
But that's not the point!
Those undecided voters ended up VOTING for Trump.
Many undecided voters who favored Hillary didn't vote. https://www.washingtonpost.com/news/politics/wp/2018/03/12/4-4-million-2012-obama-voters-stayed-home-in-2016-more-than-a-third-of-them-black/
11/x
Major differences, looking at 2020.
Double haters tend to be younger and liberal (disclosure: it me) with a notable uptick in black double haters (disclosure: it not me).
2020 has *far fewer* undecided voters and there's a case that those undecideds ACTUALLY FAVOR BIDEN
Major differences, looking at 2020.
Double haters tend to be younger and liberal (disclosure: it me) with a notable uptick in black double haters (disclosure: it not me).
2020 has *far fewer* undecided voters and there's a case that those undecideds ACTUALLY FAVOR BIDEN
12/x
Trump won double haters by a 20 point margin in 2016.
As of April, Biden was winning these "double haters" BY 50 POINTS.
The "I like neither, but I prefer the jobs jobs jobs businessmam outsider to the dynastic (lady) neoliberal" edge seems gone. https://www.bloomberg.com/news/articles/2020-04-24/new-poll-shows-a-hidden-danger-for-trump-double-haters
Trump won double haters by a 20 point margin in 2016.
As of April, Biden was winning these "double haters" BY 50 POINTS.
The "I like neither, but I prefer the jobs jobs jobs businessmam outsider to the dynastic (lady) neoliberal" edge seems gone. https://www.bloomberg.com/news/articles/2020-04-24/new-poll-shows-a-hidden-danger-for-trump-double-haters
13/x
So back to the polls. Stop looking at the MARGIN and actually look at the NUMBERS.
Not all margins are equal. A 7 point margin is great but is 44/37 the same as 50/43? Of course not!
Think of it this way: the undecideds gotta go somewhere. AND POLLS ONLY CAPTURE DECIDEDS.
So back to the polls. Stop looking at the MARGIN and actually look at the NUMBERS.
Not all margins are equal. A 7 point margin is great but is 44/37 the same as 50/43? Of course not!
Think of it this way: the undecideds gotta go somewhere. AND POLLS ONLY CAPTURE DECIDEDS.
14/x
When people - analysts included - talk about "polling error" they refer to margin. This is not accurate. A candidate going from +4 in polls to -2 in the election (or vice versa) DOES NOT INSTANTLY INDICATE polling error, and it's wildly erroneous to call it a 6 point one.
When people - analysts included - talk about "polling error" they refer to margin. This is not accurate. A candidate going from +4 in polls to -2 in the election (or vice versa) DOES NOT INSTANTLY INDICATE polling error, and it's wildly erroneous to call it a 6 point one.
15/x
Back to before - what do polls measure?
Preferences of decided voters, and HOW MANY undecided voters.
What do polls NOT measure?
Preferences of undecided voters.
Using polls (with margins that include undecided voters) versus results (which do not) is INTERNALLY INVALID
Back to before - what do polls measure?
Preferences of decided voters, and HOW MANY undecided voters.
What do polls NOT measure?
Preferences of undecided voters.
Using polls (with margins that include undecided voters) versus results (which do not) is INTERNALLY INVALID
16/x
What you've implied, by comparing margins in polls to margins in results, is that a poll is an attempt to try and measure not only how decided voters apportion themselves, not only how many undecided voters there are but also the whims of those undecided voters.
What you've implied, by comparing margins in polls to margins in results, is that a poll is an attempt to try and measure not only how decided voters apportion themselves, not only how many undecided voters there are but also the whims of those undecided voters.
17/x
No one would contend that a candidate polling with a 38-34 (+4 margin) lead with 28% undecided is less safe than a candidate with a 48-44 (+4 margin) lead and only 8% undecided.
If you agree that all margins are not the same, you can't use margins to measure poll validity.
No one would contend that a candidate polling with a 38-34 (+4 margin) lead with 28% undecided is less safe than a candidate with a 48-44 (+4 margin) lead and only 8% undecided.
If you agree that all margins are not the same, you can't use margins to measure poll validity.
18/x
Now for some data.
Here are the 2016 pre-Election poll data, including third party and undecided
@DrewLinzer noted the historic number of undecided voters.
Think of it this way:
Undecideds have to go somewhere!
Not predicting where they go isn't a weakness of polling.
Now for some data.
Here are the 2016 pre-Election poll data, including third party and undecided
@DrewLinzer noted the historic number of undecided voters.
Think of it this way:
Undecideds have to go somewhere!
Not predicting where they go isn't a weakness of polling.
19/x
Here is the overlay of polling results to actual results, with how undecided voters aligned.
Note that third partys lost a small but not insignificant amount in the polls compared to the election.
Nowhere did polls overstate either of the major party candidates support.
Here is the overlay of polling results to actual results, with how undecided voters aligned.
Note that third partys lost a small but not insignificant amount in the polls compared to the election.
Nowhere did polls overstate either of the major party candidates support.
20/x
I'll preface this by saying that there's 2 months until the election and things change.
HOWEVER
No candidate underperformed their poll number in 2016.
No candidate underperformed their poll average in 2016.
That yellow undecided bar can only make up for so much.
I'll preface this by saying that there's 2 months until the election and things change.
HOWEVER
No candidate underperformed their poll number in 2016.
No candidate underperformed their poll average in 2016.
That yellow undecided bar can only make up for so much.
21/x
No state better illustrates what I mean about margins vs raw numbers like Wisconsin
@RealClearNews had Hillary's margin +6.5 there in 2016, and they have Biden's margin there now
The difference? In 2016 the poll was 46.8-40.3
In 2020 the poll is 49.8-43.3
Not the same!
No state better illustrates what I mean about margins vs raw numbers like Wisconsin
@RealClearNews had Hillary's margin +6.5 there in 2016, and they have Biden's margin there now
The difference? In 2016 the poll was 46.8-40.3
In 2020 the poll is 49.8-43.3
Not the same!
FAR FEWER undecideds this year
Could make a case (as I briefly touched on earlier) that undecideds could actually break IN BIDEN'S FAVOR this year
But even if we assume undecideds again overwhelmingly break R:
OH, FL, NC close
Biden comfortably wins MI, PA, WI, AZ, MN for 270+
Could make a case (as I briefly touched on earlier) that undecideds could actually break IN BIDEN'S FAVOR this year
But even if we assume undecideds again overwhelmingly break R:
OH, FL, NC close
Biden comfortably wins MI, PA, WI, AZ, MN for 270+
Fixed Trump-leaning forecast.
Earlier version had even distribution of undecideds.
This one distributes undecideds proportionally to 2016. In most swing states, that's 60%-90% Trump, ~ best case scenario for him.
In 2016, that was enough to swing election. Not so in 2020.
Earlier version had even distribution of undecideds.
This one distributes undecideds proportionally to 2016. In most swing states, that's 60%-90% Trump, ~ best case scenario for him.
In 2016, that was enough to swing election. Not so in 2020.
