Today I'll be live tweeting (Clinton et al. June 29, 2020)
"Partisan Pandemic: How Partisanship and Public Health Concerns Affect Individuals’ Social Distancing During COVID-19" which has a summary thread here: https://twitter.com/marctrussler/status/1278731748277465089

Unit of observation is the survey respondent. On average 7,422 different persons interviewed each day between April 4 and June 7. 467,025 total. Survey collector Survey Monkey

They post-stratify to try to align their random-ish survey sample to the real national population. This is tricky because we know that surveys don't fully reach the population, non-response is correlated with ideology and partisanship.
http://www.stat.columbia.edu/~gelman/research/published/what_learned_in_2016_5.pdf

The outcome is yes/no to self reported mobility in the last 24 hours:
1) going to a restaurant
2) visiting family or friends
3) taking a walk
4) exercising
5) getting groceries
6) receiving medical care
7) going to work

They ask about the last 24 hours to try to focus the respondent on their own personal behavior an not policy more broadly. This should look like a saw tooth over time for weekends on things like going to work.

Indeed that's what we see here with self reported going to work, little two day dips. But one thing to note, is only 25% report going to work rising to 35%ish by the end of the period? If we took this literally 3/4 of Americans are either unemployed or working remotely right now?

Their correlation between this and the google mobility to workplace measure is a week .41. How google calculates that is opaque and so not necessarily a dis-positive, just another weird sign.

Gallup has the % of workers reporting having worked from home as high as 62% so maybe that's not an unrealistic number
https://news.gallup.com/poll/306695/workers-discovering-affinity-remote-work.aspx

One thing to note is just how little variation there is in the measure though. Even a treatment as large as "the weekend" only drops the % who says they went to work by 4-5% from 25%ish to 20%ish. What does it mean for 20% of the sample to say they work 7 days a week for a month?

Similar questions for the other measures
1/20 rising to 1/10 going to the doctor each day? About half getting groceries each day? So within every two days, the entire population gets groceries? What's the literal interpretation of these measures?

The biggest variation is across these, two seeing a friend and visiting a restaurant. Restaurant especially at near zero between closures and personal choice and then crawling back up to 1 in 5ish.

They get stronger correlations with google mobility measures on the other sectors. Would be good to include Apple and Safegraph's measures later as well.

The main IV is self reported partisanship. They first ask whether you identify as democrat or republican, and if you refuse to specify, they ask if you lean one way or the other, and then group you with whoever you lean. Only if you refuse twice do you get independent.

There's some mechanics behind Survey Monkey and behind their weighting scheme that could be made clearer. For example, was party affiliation asked at the time of the COVID questions, or was that asked at some earlier date? Is party both an IV and a post-strat weight?

This question wording makes me think it was at the same time as the covid questions, but still not totally clear

The summary statistics on that right hand side are totally insufficient. There's no information about breakdown in those party questions of the sample, how similar they are to the national level, whether they varied over time, how strongly correlated with other predictors, etc.

Now's a good time to think through what my priors would be on this. The most direct evidence would be other surveys, where we typically see 20-40 point swings between dem and repub on COVID questions

There's also some important geographic factors, republican areas are less dense, more homogeneous, and were hit much later than democratic areas on average. Their threat perception has lagged, but so did their actual threat. How ahead or behind the curve they are is area specific

It's time to think through identification. This is all self reporting, and attitudes can also change party affiliation. If you're a democrat, there is some (possibly small but at minimum unknown) incentive to over report behaving well, just as rep over report behaving badly.

Similarly there was speculation that Trump supporters were embarrassed so under-reported themselves before 2016. That didn't appear to be the case, they were proud to be Trump supporters they were just extra likely to tell a survey firm to go screw itself.

Would be helpful to check against some other ground truth like party registration records or empirically observed voting behavior of that county/zip code just as an extra check on that source of bias.

The cross-tab finds that all 3 political groups increased their mobility over this period. Note that they all started out at almost the same place, so what this is mostly is a story about how republicans increased their activity FASTER than independents or republicans.

The only other un-modeled crosstab we're given is age which shows a nearly uniform increase across age groups. That's pretty unexpected. Though note the variation is small, 0.5-1.0 extra activity

They check a mediating variable/mechanism, asking how worried you that someone in your family will be exposed to the cornavirus

The variation in concern across party is available for an extra month earlier than self-reported behavior. It shows concern tracking pretty closely, peaking for each about 4-1, and then declining for all and just declining faster for republicans.

Concern tracks almost uniformly across ages which again I don't fully understand/expect. This plot also needs a Y axis.

The final variable before getting into the models is the amount of COVID in a county. They use JHS when they should switch to NYT directly which they just import.

Weirdly they don't try lags of COVID19 counts. Just from my experience with family, there's a lag between when models notice an uptick and news starts reporting. By the time I warn them, they've already engaged in risky behavior, and are now stopping because of deaths, too late.

They find nearly no relationship between concerns and prevalence at all which makes me think something is wrong

They also find almost no relationship between behaviors and prevalence which makes me think something is wrong

If we took this literally, it would mean people were not responding almost at all either in attitudes or behaviors based on what was happening in their county. That doesn't seem reasonable to me, if for no other reason than lockdown orders were forcing behaviors in response.

From here on, results are going to be modeled, with a big vector of personal covariates and state fixed effects. And then slowly layer in party affiliation, or COVID counts, or both.

Before trying to interpret signs, significance, or variance explained of any particular parameters we'd first want to ask whether that model does a good job just in general in explaining variation in personal behavior or attitudes. In other words, should we care about this model?

In this draft, there's almost no argument for the model doing a good job. There's exactly one table at the end of the appendix with an adjusted R^2 of 0.14. If that's high despite 27 covariates, 50 state fixed effects, and 425k observations then someone needs to make the case.

Instead the argument is presented as a horse race between partisanship and number of COVID cases. To their credit, it's shown as percent of variation explained by adding the variable to the model and not P-values. Model everything, hold out residuals, predict residuals with var.

But that result is again kind of weird. They have this outcome that doesn't vary much. This model that doesn't explain much. Their pet IV explains only 7-8% of the remaining variation. And actual COVID counts seem to no have no relation to anything at all which is incredible.

Their conclusion I think is an overreach:
"Our results point to an unequivocal conclusion: partisanship is a far more important determinant
of an individual’s response to the COVID-19 pandemic than the impact of COVID-19 in that
individual’s local community. "

Better to say: "Neither individual level covariates, nor county COVID counts do a very good job explaining behaviors or attitudes. In this underfit model, partisan ID does way more than COVID counts. Future research should explain the other 86% of variation and see if this holds"

Conditional on that gripe, the modeled disaggregation of party effect is interesting. Most of the action is in republicans becoming more likely to visit a friend while dems became less likely, and republicans becoming more likely to go to a restaurant while dems stayed same.

That suggests you can't come at this just from the supply side, e.g. closing down restaurants. The desire to go out means they'll hang out with their friends anyway. For things that aren't optional like seeking medical care the difference is the smallest.

My main takeaway from this at this stage in the project is that we still don't really understand why people behave the way they do around COVID. Even the biggest cudgel that people are swinging about partisan ideology doesn't explain that much of the variation in behavior.

I think the non-result on COVID counts is possibly broken. Possibly some lag structure would reveal it. If it really does end up having no effect, either on people's choices or government's restrictions, that would be shocking and should have been the main result.

Where I wish this would go is framing this as a genuine question of curiosity "People are dying, we need to explain their choices and beliefs, let's find things to measure and models that do a good job in time to save lives." A win is finding them and offering interventions.

The string of papers we've seen recently on "can we show republicans, fox news, protests do/don't make people sick or dumb" is great for headlines but isn't what's going to end COVID. There's an opportunity with this data to do better and I look forward to seeing where it goes.

If you found this useful, this 6th COVID-19 paper I've done public review on and you can find them and future ones at this thread here: https://twitter.com/RexDouglass/status/1278115752747253760