Thanks to @jovialjoy, I'm reading through this #FairFace paper. There's a lot to unpack but I just want to point out a couple of underlying fundamental problems with respect to the way these authors think about gender. https://openaccess.thecvf.com/content/WACV2021/papers/Karkkainen_FairFace_Face_Attribute_Dataset_for_Balanced_Race_Gender_and_Age_WACV_2021_paper.pdf

The authors developed a new face image dataset and trained a model that they claim performs better than previous models on 'race, gender, and age classification.' However, even putting aside (for now) ethical problems, they made a number of fundamentally incorrect assumptions:

False assumption 1: there are only two genders.

The authors don't even provide a caveat, such as 'for the purposes of this study we reduced gender to a binary.'

I'm not going to waste time here with this one other than to say, look it up. https://en.wikipedia.org/wiki/Gender 

False assumption 2: humans can correctly classify gender from photos (to provide 'ground truth' for model testing). They took a 'best 2 out of 3 Turker guesses' approach.

False assumption 3: humans can correctly classify gender from photos ACROSS ALL RACES, ETHNICITIES, AGES, & GENDERS with equal error rates.

This is especially problematic since it undermines the whole point of the paper.

False assumption 4: it's OK for either humans or machines to classify people's gender without consent.

It's not OK to do this. Don't do it. You need to ask people if you want to know their gender, and get informed consent if you want to use that information to train your model.

False assumption 5: skin = race. It does not. Computer Scientists: Get It Through Your Head. IT DOESN'T. No phenotypical characteristic is a 1-1 match with race because race is not biology. Look it up. https://en.wikipedia.org/wiki/Race_(human_categorization)

False assumption 6: "Latino is often treated as an ethnicity, but we consider Latino a race, which can be judged from the facial appearance."

My friends, Latino is not a 'race,' ESPECIALLY in this context (computer vision).
Here: https://www.vox.com/2016/8/28/12658908/latino-hispanic-race-ethnicity-explained

If you'd like to learn more about the construction of Latino identity & representation in the U.S. I highly recommend the brilliant @arlenedavila1's classic text 'Latinos, Inc.' https://www.ucpress.edu/book/9780520274693/latinos-inc

I have to get back to work now so the thread is done, but there's a lot more to discuss in this paper. Feel free to do so in replies and I'll check back in later to RT.