You know that line about judging people by their actions rather than their words? What about people whose only non-trivial output is words? (assume their actions are boringly uncontroversial, like say living basic middle class life funded by job at paper factory)

Here’s a way.

Judge them by topics rather than methods of thought. The thinker’s equivalent of action is choice of topic. Almost always, all the real risk is there (but CRUCIALLY may not be borne by the thinker). Methodology (analytical, empirical, logical, narrative, metaphoric) is secondary

There is a particular kind of motte and bailey maneuver where criticism of the risky part (topic/content) is redirected to an innocent-faced defense of the safe part (method). *Especially when the risk of any erroneous conclusions on the topic is borne by others.*

“I’m just asking questions”, “I’m doing this with normal rigor”, “I’m just following the evidence where it leads” are telltale signs. The part that needs discussion is choice of topic/content. Methods come after. The riskier the topic, the more safety/competence methodology needs

If you study “how stars go supernova” the only justification needed is “I’m curious about it.”

If you, as a male, study “are there gender differences in IQ?” in a world circumstantially loaded with male advantage, the burden of philosophical due diligence is higher. Why and how?

The reason is, *if you screw up and others use your results, others will be hurt far more than you.* Worse, there is prima facie reason to believe sloppiness and errors will likely benefit men. Primum non nocere. First do no harm.

Ask: “Who suffers most if I screw this up?”

You’ve heard “extraordinary claims require extraordinary evidence.” Corollary “extraordinary risk to others from thinking errors requires extraordinary care in how you think.”

Beware the moral hazards of your topic choices.

This isn’t hard. You take more precautions when experimenting with explosives than when experimenting with cookie recipes. You triple check every decision and step if you’re researching “should we bomb this country” but not for “should we hold the event in NY or LA.”

Add another wrinkle. Call it the Incumbency Bias of Social Research Under Sloppiness.

Other things being equal, sloppiness in research about humans is more likely to suffer errors that lead to conclusions that hurt the weak/validate the position of strong. Sewage flows downhill.

Consider a caricature: 2 bad-faith researchers researching “racial bias in police shootings” with intent to find yes and no answers respectively by any means necessary. Are they really equally risky? When should we demand deeper rigor?

There is a theory in complex systems called “normal accidents” about how complex systems like nuclear reactors inevitably (Ie “normally”) suffer failures via multiple failures interacting in unanticipated ways. This means some things are fundamentally more risky.

The author, Charles Perrow, in fact takes the conservative view that sufficiently complex tech like nuclear reactors with sufficiently high negative failure costs should not be used at all. I’m not that radical, but he has a point. And the point applies to “complex thinking” too

If you’re thinking about sufficiently complex topics full of tricky interactions (“Oh Roe vs. Wade led to crime wave ending 20y later...oh wait no, it was taking the lead out of pipes!”) *you WILL make unexpected normal errors*

You may moreover be thinking under deep moral hazard of being nowhere near the reactor meltdown zones. In social research this might be: policing, criminal justice, public schooling, nutrition, education, war-making. Entire communities could be deeply screwed by your errors.

And this shouldn’t need saying but apparently does. The more powerful you are, the more extreme care you need to take because your casual speculative tweeting could cascade into ill-considered action a few degrees away. Think longer per tweet the more powerful you are.

I’m a random D-list blogger. If I tweet speculative dumb shit, very little happens, but there’s more potential for damage than with someone with no following. If you’re a famous academic who has the ear of impulsive CEOs more can happen. If you’re president, wars might start.

If you transpose Perrow’s conclusions about nuclear reactors to social science, you would in fact conclude that some subjects should not be studied at all.

Because the only people with the methodological competence to study it might be under unacceptably high moral hazard.

This is why I’m fundamentally sympathetic to even (say) the most irrational sounding black activists who might want to object to (say) white men studying IQ. The researchers are safe in their nice university jobs. Any errors leading to social policy meltdowns, guess who suffers?

There are two ways to bring these nuclear reactor “normal accident” topics into “safe study” zone.

First: add more methodological rigor burdens in proportion to risk to others. IRB++. But this won’t be enough to bring more responsibility to say casual speculative tweeting.

I like the second approach: increase direct risk exposure (or lower moral hazard). You want a million dollar research grant to study race and IQ? Go for it. We just ask that you live in a black inner city school district while doing so, that will be applying your findings.

You want to speculate about sex markets? Great, do it in a sociology department where you have more female peers than male. Better still: include them in proposal.

Get skin in the game in proportion to the accidental harm you might do to others.

The common response to free-expression absolutism is “freedom of expression is not freedom from consequences”... as in don’t expect people to not yell at you or retaliate. If only it were that simple.

The real messy problem is *others may not be free from consequences*

To bring it back to the opening point, how do you judge a thinker? By topic, not method.

How by topic? If a thinker routinely indulges in morally hazardous thinking where others are more likely to be hurt by erroneous conclusions, I do a double take.

If they aren’t adding extra safety or taking on extra risk to compensate, I flip the bozobit.

I’m doing this more quickly these days. There’s no excuse for putting others at risk with your bad thinking from relative safety.

I’m fine with sloppy speculative spitballing and casual, loud, public thinking. That’s my own modus operandi after all. The trick is to then work on harmless topics and/or ones where you yourself are the one most at risk. If you want to move to meltdown topics, harden you methods