Read on Twitter
I’ve been quietly having conversations w/ tech CEOs this yr re: hate speech.
They all agree it’s a problem - but are unsure of how to manage it & feel uncomfortable deciding what is/isn’t permitted on their platforms.
Enter @Patreon CEO @jackconte: https://nandinijammi.com/manifest-observable-behavior/
They all agree it’s a problem - but are unsure of how to manage it & feel uncomfortable deciding what is/isn’t permitted on their platforms.
Enter @Patreon CEO @jackconte: https://nandinijammi.com/manifest-observable-behavior/
In July 2017, @jackconte banned white nationalist Lauren Southern from @patreon.
She had raised $$ on Patreon for a
to block NGO ships from rescuing refugees off the Italian coast.
Normally, that would be that. But I assume she & her goons began harassing Jack endlessly.
She had raised $$ on Patreon for a
to block NGO ships from rescuing refugees off the Italian coast.Normally, that would be that. But I assume she & her goons began harassing Jack endlessly.
So @jackconte did something unusual: He made a video walkthrough of Patreon’s review process.
He brought 3 things to the mat:
1. Patreon’s Acceptable Use Policy
2. Concrete evidence
3. A review method called “Manifest Observable Behavior”
He brought 3 things to the mat:
1. Patreon’s Acceptable Use Policy
2. Concrete evidence
3. A review method called “Manifest Observable Behavior”
Manifest Observable Behavior = A review method based entirely on observable facts. (E.g. What has video seen/audio recorded/been written online/said to the press?)
In Lauren’s case, there was a mountain of evidence for Patreon to review
In Lauren’s case, there was a mountain of evidence for Patreon to review
Wow this is some pretty heinous shit! But Jack simply points to the facts:
“This violates the section of our content policy prohibits creators from threatening to take or actually taking action that could lead to harm or loss of life.”
Yep, it’s just company policy. Sorry.
“This violates the section of our content policy prohibits creators from threatening to take or actually taking action that could lead to harm or loss of life.”
Yep, it’s just company policy. Sorry.
“We do not allow users who’re trying to get people killed” might seem like a superfluous clause for your AUP.
But neo-Nazis like Lauren actually count on your ambiguous policies, so they can pull out the oldest trick in the book...
Pretend they were just joking! Come on, man.
But neo-Nazis like Lauren actually count on your ambiguous policies, so they can pull out the oldest trick in the book...
Pretend they were just joking! Come on, man.
“We were just kidding!” is the fascist’s equivalent of throwing water on the motherboard. Tech CEO brains melt at the idea of banning a joke. Works like a charm.
But @patreon’s review method accounts for that too: CREDIBLE vs. NON-CREDIBLE statements. https://www.theguardian.com/technology/2017/may/23/alt-right-online-humor-as-a-weapon-facism
But @patreon’s review method accounts for that too: CREDIBLE vs. NON-CREDIBLE statements. https://www.theguardian.com/technology/2017/may/23/alt-right-online-humor-as-a-weapon-facism
In that vein, Lauren claimed she was just there as a “journalist” & Jack was discriminating against her political views.
But footage of her directing the ship + statements of intent to cause loss of life + a crowdfunding campaign to buy a is all that matters to @Patreon.
But footage of her directing the ship + statements of intent to cause loss of life + a crowdfunding campaign to buy a
is all that matters to @Patreon.
Sure, Jack is biased...towards his company values and policies and standards.
Patreon has a right to refuse customers and be associated with assholes like Lauren.
The
here is that Patreon has their values clearly written & standardized.
Patreon has a right to refuse customers and be associated with assholes like Lauren.
The
here is that Patreon has their values clearly written & standardized.
This case study matter bc tech co’s are currently making nonsense “both sides” decisions for fear of appearing biased or liberal.
Last year, @PayPal banned Proud Boys (a violent white supremacist group) AND antifa orgs (anti-racist) in one round.
https://www.theguardian.com/technology/2018/nov/09/paypal-proud-boys-antifa-ban-gavin-mcinnes-criticism
Last year, @PayPal banned Proud Boys (a violent white supremacist group) AND antifa orgs (anti-racist) in one round.
https://www.theguardian.com/technology/2018/nov/09/paypal-proud-boys-antifa-ban-gavin-mcinnes-criticism
I guarantee you the same internal jockeying occurred internally at @PayPal, which yanked the KKK’s account within a week of being notified vs. Stefan Molyneux, which took 8 MONTHS.
Ban a PHILOSOPHER w/ some unpopular views? Mon dieu!
Ban a PHILOSOPHER w/ some unpopular views? Mon dieu!
I have worse examples. @stripe doesn’t have anything in their AUP that prevents them from working w/ Nazis & White supremacists.
So they do. They work with all of ‘em! https://www.rightwingwatch.org/post/notified-of-hate-groups-using-service-stripe-chooses-inaction/
So they do. They work with all of ‘em! https://www.rightwingwatch.org/post/notified-of-hate-groups-using-service-stripe-chooses-inaction/
Basically, tech co’s have no idea how to handle dangerous people & groups internally - let alone explain decisions to the public.
They are flying by the seat of their pants every time. It’s not working for anybody but the bad guys.
They are flying by the seat of their pants every time. It’s not working for anybody but the bad guys.
IMHO, @jackconte was ahead of his time.
An Acceptable Use Policy rooted in company values + a review method that accounts for bad faith arguments is a model that more tech co’s should consider.
Why make Trust & Safety/social media teams suffer thru each case?
An Acceptable Use Policy rooted in company values + a review method that accounts for bad faith arguments is a model that more tech co’s should consider.
Why make Trust & Safety/social media teams suffer thru each case?
I’m not saying @Patreon is perfect, but they are definitely ahead of the pack.
Tech co’s that are looking for binary/global/neutral standards are not going to find them in the chaotic & messy human world.
The best you can hope for is to apply your rules neutrally.
/end
Tech co’s that are looking for binary/global/neutral standards are not going to find them in the chaotic & messy human world.
The best you can hope for is to apply your rules neutrally.
/end
