Unbelievable. Absolute disregard of qualitative methods of analysis, historical and art historical expertise. This is peddling bad science.

This is dangerously bad. Others have commented on the art history failings but we also need to flag other errors here. Hop on. 1/ https://twitter.com/baumard_nicolas/status/1308715606196342784

I'm late to this but first I had to read it and look at the data.

First, the basics: No historians of art and no historians co-wrote this paper. Imagine what would happen if historians published a paper on virology and consulted no virologist? That would be fun, wouldn't it? 2/

The assumptions at the beginning of this paper are clearly taken from a bad novel (with no offence to historical novel writers who clearly do more research than has been done here). This is not the fault of the works cited - it's a failure to grasp them and to define periods. 3/

We shall leave the phrenologist assumption of the paper aside as this has been flagged in-depth in other threads. Ok, so: data. The dataset in this paper is the National Gallery collection (n=1962) from 1505 to 2016 expanded to Web Gallery of Art (n=4106) (1360-1918) 4/

Not a WORD has been lost to comment on how the National Gallery collection has been curated and how this collection was assembled. Not even a footnote. We also learn those are "English" portraits. Like, you know this Katherine of Aragon 5/

Now this is a data science failure. As any decent data scientist will tell you a failure to ask how the dataset was assembled and what does it represent deems the analysis essentially worthless from the start. You cannot do this If. You. Don't. Question. Your. Data. 6/

Ok, so the dataset was taken at face value (pun intended) and not checked for biases. Obviously collections are not a result of centuries of curation, they do not reflect the fashions of the time and of now. The issue of digital facsimiles was not even mentioned in passing. 7/

But of course! The way faces are depicted has nothing to do with art styles or personal preferences or methods. Hegel called and he wants his progress of history back. 8/

But now, now we come to the part were the historian (and the data scientist) can get a heart attack. The authors have concluded that... trustworthiness increases with affluence! 9/

This they have done on the basis of datasets spanning almost 700 years, containing portraits mostly depicting the elite, commissioned by the elite and executed in the style reflecting the fashions of the elite. They could have known that if they, you know, asked a historian. 10/

But this is not all! We also learn that "the Middle Ages and the early Modern Period were periods of prolonged economic growth for Europe in general and England in particular". Uh-oh. So, first of all something like GDP doesn't apply to the Middle Ages. 11/

Second, the concept of "economic growth" that we know from the contemporary economics doesn't translate well to the Middle Ages. They would have known that if... you know, they asked a historian. 12/

But the authors claim that their findings support the qualitative findings of historians! How, when? 13/

Oh, we also learn that "direct-gaze Renaissance portraits are more popular than averted-gaze portraits". Because there before the RENAISSANCE nobody looks you in the eye! 14/

(Just as a side-note, the paper that is cited for this sentence claims exactly such evolution, saying "The structure of the “Renaissance gaze shift” bears evidence for the importance of demographic turn-over in cultural change.") 15/

Ok, summing up the methodology part: crucial biases not identified in the dataset, mangled periodisation, absurd economic assumptions (based on a thorough lack of understanding of medieval economy) and absolute shambles made out of correlation. What are the consequences? 16/

This is no longer a case of a bad PR presentation of a study. This paper should never have made it through peer-review. It peddles dangerous ideas that a curated aggregation of digital facsimiles of works of art can reflect a trend like "trustworthiness". 17/

In the age of computer assisted facial recognition this has dangerous consequences. Using historical data to prop claims that allow for segmenting people has dangerous precedents. Linking it to "affluence" equally so. 18/

Failing to understand what a work of art like a portrait is, to take into account its agenda and message and treat is as a "point of data" is always gonna end badly. Such studies have an impact, they will be cited uncritically, they will help to replicate biases. 19/

We need to urge journals and outlets like @NatureComms to *require* relevant humanities scholars to co-peer review any study that uses historical, art or social data. This is a change of policy that should be enacted immediately. 20/

Some post-scriptum with the data. Look at this bar plot of the works of art in the Web Gallery of Art by the country of origin. You see a pattern here? 21/

Sorry for the not-so-pretty graph but it takes literally 2 minutes in R to see that this is essentially a study of European portraiture (with a small admix of USA). Yet another bias of this study. 22/