NEW: our analysis, showing why Bernie fans are seen as more toxic on Twitter. We show there are *many* more active Bernie fans on Twitter, they reply to other candidates more frequently than other supporters do, and their replies are (slightly) more likely to be toxic. Thread. 1/

Are Bernie supporters reply on Twitter in a more toxic fashion to rivals than those of other candidates? With 2 new papers on how political candidates are attacked online, we realized we have data/methods to shed some light on this question. 2/

Why are Bernie fans (BFs) perceived as the most toxic? Variables that can explain it:

(a) there are more of them online
(b) they engage with other candidates more
(c) they use toxic language more

All these factors would contribute to making BFs' toxicity more visible. 3/

Result: for explaining toxicity in replies to other candidates, it’s mostly (a,b) and a little bit of (c). Outcome: the number of toxic replies from Bernie fans to other candidates seems to greatly overwhelm the num of toxic replies by any other candidate’s supporters. E.g: 4/

Let’s break down @ewarren's case. Of users we could identify as supporting candidates except Warren, 58% support Bernie. But his fans are responsible for 75% of replies from these supporters to Warren, and for 79% of supporters’ toxic responses to her (i.e. more than expected) 5/

These three factors: the number of supporters, the amount of attention they gave @ewarren, and the (marginally higher, within error mostly) toxicity *together* make toxic replies from Bernie fans far outpace the toxic replies Warren’s received from other candidates' supporters 6/

There’s more below! But first, something about the data, and disclaimers for full context.

The data: replies to tweets from candidates between 1/27 to 2/28, pre-Super Tuesday. There are periods of dropped/missing data, but no reason to think these introduce significant bias 7/

We tracked @ewarren @SenSanders @TulsiGabbard @steyer
@PeteButtigieg and @amyklobuchar, but sadly no @joebiden due to a data collection error. More (recent) data with Biden is coming soon -- it’s showing similar trends re: Bernie supporters. 8/

Candidate supporters were determined by retweet patterns: user X supports candidate C if X retweeted candidates at least 5 times, retweeted candidate C strictly more than any other candidates, and in at least 30% of their retweets. We used a similar heuristic in our papers 9/

Toxicity of tweets is determined using @jigsaw's
Perspective API, w/ threshold of toxicity score>0.7 to determine a toxic tweet. To be counted, a tweet has to be tweeted by a user we identified as supporting another candidate. 15% of the tweets were classified. 10/

Both measures (supporter estimate, toxicity) are conservative, reducing the estimates of toxicity but making them less noisy. It's possible that the measures add significant bias, e.g. missing a certain class of toxic tweets, but we do not have a strong reason to believe so 11/

Important detail: we don’t know if these users are bots or trolls, Russian or other. But most people seeing these tweets and replies don't either. In other words: we are studying perceived exposure to toxicity, not intent 12/

Here's some more data, then! Bernie had almost as many identified supporters in our data (20,182) as all the other candidates combined (23,706). This reflects his greater online presence and tracks with the number of Twitter followers the respective candidates have. 13/

Of the candidates, @PeteButtigieg got the most total attention from other candidates' supporters (64K replies), followed by Bloomberg (48K) and Warren (30K). Bloomberg and Pete also saw the highest ratio on toxic replies to their tweets (16% and 14%). 14/

What about @BernieSanders--who attacked him most in Twitter replies? Bloomberg supporters. Warren's fans (40% of all non-Bernie supporters) paid less attention to Bernie's tweets than others (made 25% of supporter replies), and were not more toxic (made 23% of toxic replies) 15/

Guess who paid extra attention to @amyklobuchar? That’s right, Pete’s supporters. They replied to her twice as much as you'd expect based on their numbers, but were also responsible for a similar ratio of the toxic replies (i.e. not more toxic than others). 16/

OK, that’s it for now. Please respond/discuss with respect, and let me know if you have questions about the analysis or questions you want to ask of the data. Next week I will try to post an analysis of the Super Tuesday week, including @JoeBiden’s followers and replies. 17/

Two upcoming peer-reviewed papers describe the methodology and provide some analysis in the context of the 2018 midterms, and you can find them here:

http://yiqing-hua.com/papers/adversarial_user_chi2020.pdf

http://yiqing-hua.com/papers/adversarial_candidates_icwsm2020.pdf

18/

This is work and data collection led by the amazing @yiqqqing. Any errors, misrepresentations, and responsibility for overly-bold claims are with me. 19/

And final note: this analysis is different from @TaylorLorenz's report yesterday, as we directly look at the difference in discourse (replies) instead of comparing general follower behavior. But check that out, too!

https://twitter.com/TaylorLorenz/status/1237373431659802625?s=20 20/20 (hey that was a coincidence)