Large scale population measurements will often (if not always) have biases. However, dropping such measurements is not the solution: we should acknowledge its limitation, correct for them using statistical methods and find better measurements!
1/

For example, in health, it is known that many of the risk indicators for cardiovascular disease are not good for all populations. BMI + waist circumference does not work as a good predictor for Asian Indians https://www.sciencedirect.com/science/article/pii/S0899900702010833
2/">https://www.sciencedirect.com/science/a...

Does this means we should stop using BMI? No! Does this means we should look for other risk measurements? Yes. But above all, this should aid in the study deswign: if a study tries to assess risk in a population with many Indian Asians with BMI, they have a problem!
3/

I& #39;m saying all this because I believe this is a common mistake I see in research related to hate speech and misinformation. For instance, someone said that using Perspective to measure hate speech is "disqualifying" because it has known biases.
https://twitter.com/lucas_wright_/status/1229900544812056577
4/">https://twitter.com/lucas_wri...

This is a profound misunderstanding on how setting up a study works. We compared Perspective scores across platforms, populations and in different moments in time (for more than a decade). The criticism gives no reason to believe that the biases are significant for design.
5/

The kind of research that uses large population measurements is often the only way to answer important and difficult answers that can& #39;t be addressed by looking at small data. Moreover, this kind of research is reproducible, and very easy to tests for biases
6/

It is important for quantitative researchers to know that their methods are biased. However, the idea of abolishing a measurement because it has known biases is anti-scientific. Critics should be aimed at how these biases invalidate the specific study design!
7/