Read on Twitter
this is an anecdote I give frequently, but its worth revisiting again.
in August 2017, Robin Wright writes a story for the New Yorker that reports, ominously, that some experts believed America was on the path to civil war https://www.newyorker.com/news/news-desk/is-america-headed-for-a-new-kind-of-civil-war
she claims that " In March, [expert] was one of several national-security experts whom Foreign Policy asked to evaluate the risks of a second civil war—with percentages....The sobering consensus was thirty-five per cent"
![she claims that " In March, [expert] was one of several national-security experts whom Foreign Policy asked to evaluate the risks of a second civil war—with percentages....The sobering consensus was thirty-five per cent"](https://pbs.twimg.com/media/Eyx7SQGW8AMvn98.png "she claims that \" In March, [expert] was one of several national-security experts whom Foreign Policy asked to evaluate the risks of a second civil war—with percentages....The sobering consensus was thirty-five per cent\"")
Sounds scary, right? But what happens if we click through the link?
We get to a blog post by a Foreign Policy writer -- Best Defense is a blog hosted by Foreign Policy, not Foreign Policy *itself* asking a collection of experts to come together https://foreignpolicy.com/2017/03/07/will-we-have-a-2nd-civil-war-you-tell-me/
in the post, the author observes that he got a collection of his friends to talk about the matter over wild boar burgers at an Austin burger joint. No information is given about who attended, how many attended, or how percentages were assigned.
To give a flavor of how informal this convention of experts was, here is how the 35% figure is described: " I was surprised that the range of answers ran from “five percent” to “95 percent.” I would say the consensus was about 35 percent."
Clearly, this was neither an event officially organized by Foreign Policy or a rigorous evaluation of the risks of civil war. It was an informal get-together blogged by an FP author that was totally mischaracterized by the New Yorker reporter.
The New Yorker reporter then built her story around her total mischaracterization of the FP author's blog post, as you can see from the subsequent paragraphs now that you have observed the original source of the 35% statistic.
Why do I give this anecdote? Well, if you had merely trusted the New Yorker's institutional brand and knew little about political science, military history, or social science research methods, you might never have noticed this at all.
Now think about the possibility that things like this are lurking, unnoticed, in stories by similarly prestigious institutional brands in subjects you are either too ignorant of or too distracted to notice
"too distracted" is a key word here. We have neither the time, energy, or resources, to check every source of information we come across. So we default to heuristics. One of these heuristics is institutional integrity.
But institutions in the real world are very different from institutions as we idealize them https://twitter.com/jljcolorado/status/1380641336316067841
This is partly how military organizations, for example, came to rely on a study that boldly claimed that the majority of men did not fire their weapons in combat against the enemy (see SLA Marshall and the "Ratio of Fire" controversy) despite significant uncertainty
over how to verify the original sources of the survey work and allegations that the researcher who had conducted it engaged in fraud. The claim gained credence due to branding and repetition.
It is also how law enforcement's reliance on flawed forensic science put countless innocent people behind bars, or how equally pseudoscientific Recovered Memory treatments in psychology similarly victimized the innocent
Returning back to the anecdote that began this thread, at least the New Yorker story had a reference to an original source that could be checked. A good deal of blockbuster political journalism relies on anonymous sources
You can make a case that the benefits of this outweigh the costs, but even a fuller analysis of Deep Throat's background and motivations may be troublesome for this case https://www.politico.com/magazine/story/2017/09/10/watergate-deep-throat-myth-mark-felt-215591/
For me the point of the original anecdote is a recurring obsession for me about how we talk about disinformation, post-truth, etc. It's something we see as challenging institutions, expertise, what have you. We never see those things as potential *sources* of it
Because at this point we begin to contemplate something truly terrifying. Which is that all of our meaningful knowledge of the world is borrowed from other people, who in turn borrow it from other people, and so forth
We can't really just be skeptical of everything, and self-styled skeptics can sometimes be the most credulous of all https://aeon.co/essays/is-debunking-more-about-the-truth-teller-than-the-truth
But once you consider that knowledge may just be a giant game of telephone -- full of honest errors, bald lies, and a lot of things in between -- it does tend to be scarier than the idea of Macedonian teenagers on Facebook https://twitter.com/Aelkus/status/1381623070729908238
Responses to this often are something along the lines of "yes, but for all of its faults it works well most of the time and its better than the alternatives." And its hard to argue with this, on its face.
But what does "most of the time" here mean? And if failures can sometimes lead to grievous harms, how much failure really should be tolerable?
And, perhaps to put it most bluntly, if you had a close friend that told you bluntly "I will tell you the truth most of the time, but I will lie, fudge, or otherwise misrepresent the truth to you a minority of the time" would that not change how you see everything he says?
