We tested whether feature visualizations (from @ch402 et al.) really help humans understand CNNs. Our surprising finding: while they do help, they are outperformed by a very simple baseline - natural reference images from the dataset!
Paper @ http://arxiv.org/abs/2010.12606  (1/N)

In our well-controlled human psychophysical experiments, participants are given extremely activating reference images (synthetic or natural) and choose which out of two query images is also a strongly activating image. (2/N)

We find that synthetic images indeed provide helpful information about feature map activations. However, natural images are even better! (3/N)

Participants are not just better, but also faster and more confident for natural images. This holds for both expert and lay participants, and across layers. (4/N)

We argue that future visualization methods should improve over the simple baseline of natural images. (5/N)

Joint work by @judy_borow, @zimmerrol, Judith Schepers, Robert Geirhos, @tsawallis, @MatthiasBethge and @wielandbr. (6/N)