ANOTHER case of "its worse than another one bites the dust."

Is the Dunning-Kruger Effect* a Statistical Artifact?

*1 of most famous psych 'discoveries' of the last 25 years questioned.

Thread ending in END dripping w/real life ironies. You can find this on my website:

What is the Dunning-Kruger Effect (hence DKE)?
The *claim* that ignorant people are especially overconfident in their beliefs.

Its 1 of the most famous psych "discoveries of the last 25 years.

Its so famous it has a stand-alone page devoted to it at Psych Today.

Like, this is not someone's blog. This is PT's way of saying "this is a really super famous super solid phenomenon that you should know about."
https://www.psychologytoday.com/us/basics/dunning-kruger-effect

You can find the original paper easily enough using Google Scholar. Note that it has garnered>5600 citations in 20ish years. Important stuff!

Why does so much of the world seem to love the DKE?
Is it because it was AN AMAZING HIGH QUALITY SUPER RIGOROUS DEFINITIVE PIECE OF RESEARCH? No.

4 small scale studies, all with undergraduates. BUT the issue here is not "unreplicable." Its more like "premature acceptance before understood." The *finding* is replicable.

Even when lots of findings *are* replicable, they do not mean what their authors claim. This paper is a forced march thru a slew (not DKE), but including:
Erroneously claiming:
gender gaps=discrim
motivation influences perception
IAT scores predict anti-blackl discrim

This problem of replicable but misunderstood findings is everywhere. Race IAT effects are massively replicable but do not reflect what has been called "implicit bias" for the last 20+ years.

So why was the DKE so influential, popular? IDK for sure, but it is a GREAT tool for smugly sophisticated implications of the intellectual superiority of one's self and one's allies. Just look at these Psych Today blogs.

See an agenda here? Naah, nothing to see here.

Goes well beyond Psych Today though.

See an agenda here? Naah, nothing to see here, everyone move along.

This pattern is common. A limited knowledge of the psych of bias often leads people to weaponize that understanding to delegitimize their opponents masquerading as "deep" insights into *other's* biases (which may mainly reveal one's own).

I blogged about this here:
https://www.psychologytoday.com/us/blog/rabble-rouser/201908/confirmation-bias-real-bias-or-delegitimization-rhetoric

Limited knowledge can be a dangerous thing especially when it is weaponized.

Part of what explains this has been well-captured by Gigerenzer's line of research. Like moths attracted to light, people LOVE demonstrations of others' biases. In my experience, not only does this rarely help them see their own it may exacerbate their own biases (see prior tw).

Ok, so what was wrong with the DKE? The key idea is expressed in the conclusion of the original DKE paper:

"...incompetence, like anosognosia, not only
causes poor performance but also the inability to recognize that one's performance is poor."

"Causes." Remember that.

When people are unskilled or unintelligent this *causes* them to be unable to recognize their lack of skill or intelligence.

Know what that means?

It means that smarter people should be *more accurate* in their perceptions of their own intelligence than are less smart people. Or, put differently, the correlation between
actual IQ and self-perceived IQ should be higher for those high in IQ. That did not happen:

SAIQ here refers to "self-assessed IQ." So, as both go up, the pattern should be curvilinear. The line, and the dots, should curve upward. This is from SIMULATED data showing what WOULD happen if the DKE occurred:

Did that happen? No. The pattern was completely linear. Smart people were no better at estimating their IQs than were less smart people. Here are the data from actual (not simulated) people, N>900.

Bottom line: No evidence of DKE in a very large sample test, when tested most rigorously.

This does not mean DKE is "false" -- and the authors are quite clear about that. They only tested it one way, so their work cannot refute other tests.

Their overall conclusions:
"our contention is that the Dunning-Kruger
effects reported in the literature are mostly the result of statistical artefacts, rather than entirely so."

And:
"As studies that test the Dunning-Kruger
hypothesis with the tests recommended in this investigation accumulate in the literature, a more precise partitioning of effects (genuine versus statistical artefact) will be possible."

And:
"When such valid statistical analyses are applied to individual differences data, we believe that evidence ostensibly supportive of the DK hypothesis derived from the mean difference approach employed by Kruger&Dunning (1999) will be found to be substantially overestimated."

It is amusing and delicious that, in response to an early critique of their original paper, Kruger and Dunning make this point:
https://sites.lsa.umich.edu/sasi/wp-content/uploads/sites/275/2015/11/krugerdunning02.pdf

Morals of the story?
1.Definitive studies often aren't.
2. NEVER equate fame with validity
3. It often takes decades to subject even famous studies in psych to the sort of intense skepticism necessary to know whether they should be believed.

4. "Biases" are often weaponized for political purposes, even (especially?) by "scientists." And when they do it? It will masquerade as "I'm just doing science!"

END