I'm excited to finally share a huge project @mitvis on COVID, misinfo, + data vis!

How do COVID skeptics on FB/Twitter use + interpret data? To find out, we analyzed half a million tweets, 41k data visualizations, + conducted a digital ethnography:

the paper has been accepted for publication at CHI '21, and while it's not yet on the ACM website yet, you can find the full version on arXiv now! https://arxiv.org/abs/2101.07993 

data visualizations have played a huge role in helping us make sense of the pandemic: from Flatten the Curve to @jburnmurdoch's *incredible* graphs, they document not only how the pandemic is unfolding, but also how we should adjust our behavior https://twitter.com/jburnmurdoch/status/1347200811303055364?s=20

and while it's tempting to characterize COVID skeptics as simply being "anti-science" (or otherwise ignorant of the data), our study analyzes how these groups discuss, interpret, and make their own data visualizations to make sense of the virus' epidemiological spread

how does this unfold, and what data do they use? What does this mean for public health policies in the US moving forward?

in our Twitter analysis, we created a network graph of *all* users who share and interact with COVID-related visualizations, and we classified them to see the kinds of visualizations that were most prevalent in each group.

here's a quick snapshot of the user network:

a note on the graph: each node is a user + edges that links the users together are RT, likes, or mentions. You'll see that Tr__p is a central node, bc a huge number of tweets about US COVID response will inevitably mention him, regardless of political affiliation

moving thru the data, we found a couple distinct communities: US American media organizations (red, blue); a network centered around @nytimes (green), one around @jburnmurdoch / @FT (orange), one on @CDCgov (yellow), and another on the Indian PM (pink)

[N.B.: these are somewhat loose affiliations, but I would say that these labels generally hold true and help us make sense of these discussions]

so what do we find in all of this?

importantly, one of the largest networks in this data consists primarily of users promulgating anti-mask rhetoric, and they are one of the *most prolific sharers* of visualizations (line charts, maps, area charts, etc).

in fact, it's not just that they share *more* data, but also that these data are visualized in many diff ways! While other networks might tend to share one kind of viz more than others, anti-mask users tend to share more viz types *across the board* with high levels of engagement

this network also has one of the *highest* number of original tweets (as a % of total tweets) AND one of the highest rates of in-network retweets (i.e., is this a tweet of another user who also likes/RT the same content)?

"but Crystal! couldn't these accounts just be bots?"

as far as we can tell, no. They have roughly the same percentage of verified users (as a % of total users) compared to other communities, and none of the other metrics are out of sync w other groups. In fact, the account with the most followers in this network is El*n M*sk.

practically, what do these metrics mean?

(1) anti-mask Twitter users don't need more data to help them understand how bad the pandemic is in the US. They probably have more data than you, and they're sharing it at far higher rates.

(2) they tend to share these visualizations (of all stripes!) primarily among like-minded users. Concretely, 82.17% of all RT with visualizations among anti-mask users are shared in-network.

the Twitter analysis here gives a high-level view at the kinds of visualizations people are sharing (anti-mask and otherwise), but it doesn't necessarily show us *how* anti-mask users talk about, interpret, or make visualizations.

if getting a good handle on the pandemic is about "following the science," what does that actually mean to COVID skeptics? To understand this, let's dive headfirst into the wild world of COVID skeptic Facebook groups!

to really understand these groups, we did a lot of what I call "deep lurking." For six months, day in day out, we were immersed in anti-mask FB groups that actively discussed + shared data about the pandemic ... FOR SCIENCE

(I should have said this earlier, but we use "anti-mask" as a shorthand for a broad spectrum of beliefs: the pandemic is exaggerated, schools should reopen, etc. It is also a term that many of these groups would use to describe themselves, not one we're imposing on them)

what kinds of concerns do these groups have about the data that is used to formulate public policies? How do they talk about the limitations of data, or create visualizations to prove that the pandemic is not a problem?

we found that many of these Facebook groups prize unmediated access to the data, exchange notes on how best to download data from government portals, and they critically assess both the data sources and their representations.

they are mindful of how data analyses can obscure or highlight information, and they develop communities of practice around *how to make or interpret better visualizations.*

I want to underscore, too, that this isn't relegated to Facebook comments (though that was a lot of what we analyzed!). In the course of this research, I went to screencasts where people worked through downloading data together...

...workshops where FB groups brought in facilitators to give "the mom up the street" the skills to convince their city council to reopen schools, interviews w Congressional candidates who wanted to use data *from these groups* for their COVID policy platforms...

this isn't an isolated phenomenon; it's an entire information ecosystem that leverages data-driven rhetoric to challenge the expertise of the scientific establishment.

in other words, diverging from the scientific orthodoxy isn't a matter of *different datasets*, but about a broader epistemological rift about what science is.

fundamentally, the groups we studied believe that *science is a process,* not an institution. They mistrust scientists for many (good) reasons: Big Tobacco has paid off legions of scientists to line their pockets, the CDC backtracked on whether or not to wear masks...

critical thinking is about *not* taking these ideas as given, but about subjecting them to increased doubt. Question the message, the messenger, and (of course!) the data.

(before continuing, I want to be abundantly clear that we are not promoting these views, and that we believe that they run counter to scientific consensus. We also believe that everyone should wear masks and that the pandemic is horrifying.)

my gut instinct, though, is to think about these groups' analyses as fundamentally unscientific, and this is how many public health officials think about it! In an interview with @HHSGov, for example, Dr. Fauci talks about how a major problem in the US is an anti-science bias

in framing this issue as a dichotomy (you either understand the science or not), however, we miss a broader conversation about the contested state of expertise in American democracy

understanding what anti-maskers mean when they say "science" helps us unravel a larger story about what kind of knowledge matters + is valued!

US anti-maskers draw on ideals of intellectual self-reliance to emphasize how scientific knowledge has been usurped by a condescending elite which expects intellectual subservience (science is real!) instead of critical thinking from the lay public (question everything)

so what do we do with this? I sidestepped the question here a bit, by talking about what we *shouldn't* do: https://twitter.com/crystaljjlee/status/1347257503659118594?s=20

more broadly, though, we have a couple prescriptions in the paper: grappling with the social/political dimensions of visualization at the beginning (rather than the end) of projects, and by thinking about uncertainty + objectivity more carefully

I think this thread has gone on *way* longer than I had intended, so I'll wrap there but I wanted to make sure to shout-out to my amazing research assistant co-authors @itstabya + Gaby, and the *best* advisors a grad student could ask for: Graham + @arvindsatya1

if you stuck it out to the end here, you have my eternal gratitude! It's been A Time™ , and I'm grateful to have the opportunity to think for a living, esp supported by @NSF, @ssrc_org, and @DigitalHumMIT. In the meantime, please be safe, wear a mask, and hug your pets :)