Read on Twitter
Large scale population measurements will often (if not always) have biases. However, dropping such measurements is not the solution: we should acknowledge its limitation, correct for them using statistical methods and find better measurements!
1/
1/
For example, in health, it is known that many of the risk indicators for cardiovascular disease are not good for all populations. BMI + waist circumference does not work as a good predictor for Asian Indians https://www.sciencedirect.com/science/article/pii/S0899900702010833
2/
2/
Does this means we should stop using BMI? No! Does this means we should look for other risk measurements? Yes. But above all, this should aid in the study deswign: if a study tries to assess risk in a population with many Indian Asians with BMI, they have a problem!
3/
3/
I'm saying all this because I believe this is a common mistake I see in research related to hate speech and misinformation. For instance, someone said that using Perspective to measure hate speech is "disqualifying" because it has known biases.
https://twitter.com/lucas_wright_/status/1229900544812056577
4/
https://twitter.com/lucas_wright_/status/1229900544812056577
4/
This is a profound misunderstanding on how setting up a study works. We compared Perspective scores across platforms, populations and in different moments in time (for more than a decade). The criticism gives no reason to believe that the biases are significant for design.
5/
5/
The kind of research that uses large population measurements is often the only way to answer important and difficult answers that can't be addressed by looking at small data. Moreover, this kind of research is reproducible, and very easy to tests for biases
6/
6/
It is important for quantitative researchers to know that their methods are biased. However, the idea of abolishing a measurement because it has known biases is anti-scientific. Critics should be aimed at how these biases invalidate the specific study design!
7/
7/
