"Superresolution" AI is weird - it can& #39;t recover info that doesn& #39;t exist in the low-res input so it invents detail.

This is PULSE, whose task is to find a high-res image that matches the low-res input. It& #39;s better at finding some solutions than others.

https://aiweirdness.com/post/622002033086578688/depixellation-or-hallucination">https://aiweirdness.com/post/6220...
The AI tends to find matching faces in certain positions, so it can look weird if you give it something else.

I verified that yes, if you downscale the upscaled image on the right, you’ll get something that looks very much like the picture in the center. It& #39;s technically correct
The AI reconstructs what it’s been rewarded to see, and since it’s been trained to produce human faces, that’s what it will reconstruct. So if I were to feed it an image of a plush giraffe, for example
#PlushGiraffePerturbation
Speaking of what the AI has been rewarded to reconstruct - shortly after the authors released PULSE, people began to notice...THIS https://twitter.com/Chicken3gg/status/1274314622447820801">https://twitter.com/Chicken3g...
Several people, including the authors of the original paper on PULSE, found that the AI had a noticeable tendency to produce white faces, even if the input image hadn’t been of a white person. https://twitter.com/osazuwa/status/1274444300894572546?s=20">https://twitter.com/osazuwa/s...
There could be many reasons that PULSE was biased.

Worth noting: The benchmark for upscaling faces is based on a dataset that is 90% white. An AI that& #39;s great at upscaling white faces but terrible otherwise could still qualify as state-of-the-art. This is definitely a problem.
A related problem: the huge lack of diversity in the field of artificial intelligence. Even an academic project with art as its main application matters & should not have gone all the way to publication before someone noticed that it was hugely biased. https://www.theguardian.com/technology/2019/apr/16/artificial-intelligence-lack-diversity-new-york-university-study">https://www.theguardian.com/technolog...
Several factors are contributing to the lack of diversity in AI, including anti-Black bias.

This report by @databoydg is essential reading. https://twitter.com/JanelleCShane/status/1275289565494775808?s=20">https://twitter.com/JanelleCS...
Ignorance is no excuse. AI& #39;s bias issues are catching up with it, and it& #39;s long past time.

These folks have studied AI bias for years. Follow them.
@timnitGebru @jovialjoy @ruha9 @rajiinio @safiyanoble
@Abebab @mer__edith @YESHICAN @mathbabedotorg
@MiaD @math_rachel @DataSciBae
Also follow @AnimaAnandkumar @minxdragon @databoydg @kdphd @katecrawford @geomblog @artsci00 @BlackAFinSTEM @black_in_ai @AJLUnited

Honestly, follow them ALL.

And if you have other follows to suggest, please let me know. I want to listen to more experts in this area.
You can follow @JanelleCShane.
Tip: mention @twtextapp on a Twitter thread with the keyword “unroll” to get a link to it.

Latest Threads Unrolled: