Read on Twitter
Since we launched Botometer v4 a week ago ( http://botometer.org ), we have seen a few tweets about the changes. Complaints tend to focus on accounts that get a higher score compared to the old model. Here is a thread with a few points to keep in mind. 1/18
First, you might want to read our blog post ( https://cnets.indiana.edu/blog/2020/09/01/botometer-v4/) or updated FAQs ( https://botometer.osome.iu.edu/faq ). 2/18
Botometer v4 uses a new learning model and is trained with new datasets, so changes are expected. Our research to be presented at CIKM 2020 ( https://arxiv.org/abs/2006.06867 ) shows a significant improvement in recall with a very small decrease in precision. 3/18
This means that you may observe some new false positives. There are also fewer false negatives, although they may be less noticeable. Remember, no tool is perfect and you should not let an algorithm replace your judgment. 4/18
There is no universally agreed-upon definition of social bot. Given the huge gray area, we broadly define a bot as an account controlled at least in part through software. A bot need not be completely automated; sophisticated autonomous bots are rare. 5/18
Software may be used to control many accounts even if the actions originate from humans. Most automated bots simply act (reply/post/follow etc.) based on triggers or according to scripted patterns. 6/18
If your behavior follows predictable patterns, such as following many accounts mentioned by your friends, retweeting many tweets posted by your friends, and so on, then your account may get a high score. 7/18
That does not mean you are a bot. It just means that our algorithm has spotted similarities between your behavior and those of accounts labeled as automated in the training data. 8/18
It's tempting to set a threshold and consider accounts with scores above it as bots and everything below as human, but we have always recommended against this approach. We believe it is more informative to look at the distribution of scores over a sample of accounts. 9/18
If binary classification is necessary, we recommend using the CAP. Note that its definition has changed in Botometer v4: it now represents the probability that an account with the given score *or higher* is automated. 10/18
You could set a threshold on the CAP based on a statistical test, for example 95% if you are willing to accept a 5% chance that you will wrongly classify a human as a bot. 11/18
Botometer v4 provides more transparent results by showing scores from specialized classifiers that estimate how similar an account is to different types of bots. Types include fake followers, financial bots, delf-declared bots, spammer bots, & astroturf accounts. 12/18
Several of the accounts that some users are complaining about seem to have high astroturf scores. This means that they are similar to political accounts involved in "follow trains" that systematically delete content. 13/18
We believe these types of astroturf accounts fall under the broad definition of bots as partly automated accounts. We understand some folks may legitimately disagree. You are free to ignore the astroturf score if you so desire. 14/18
We emphasize that Botometer does not have, and has never had, any explicit political bias. It does not consider the content of tweets to determine whether they are of a political nature. It cannot distinguish between "conservative" and "liberal" content. 15/18
All ML algorithms may have implicit bias based on training data. If a majority of bots in the training data ore amplifying some opinion x, the algorithm may learn patterns associated with x. But Botometer is trained on examples across many topics irrespective of politics. 16/18
Finally, we'd like your feedback to improve Botometer in the future (as we have done in the transition from v3 to v4). To let us know about a classification, whether we got it right or wrong, please use the feedback button on the result pane at http://botometer.org 17/18
Sorry we cannot keep track of feedback via Twitter. Also please keep in mind that your feedback will not lead to an immediate correction. It may be incorporated into a future version. So far we have updated Botometer roughly every couple of years. 18/18
