Read on Twitter
Crowdsourcing can indeed be powerful. 75 pages translated last weekend, lightning speed for a complex technical document. I believe that v0.9 is highly accurate (more on why later), and this will improve further with v1.0. 1/ https://twitter.com/mpbruchez/status/1244773885763104768">https://twitter.com/mpbruchez...
Accuracy was no surprise, as I’m an old hand at crowdsourcing. The shocker was the superficial: the document turned out polished. Tables, diagrams, fonts—they all look amazingly nice for such a rush job. 2/
“What’s the secret?” I’m sometimes asked. My lab has been crowdsourcing for eight years through the citizen science project @eye_wire. https://www.nytimes.com/2015/01/11/magazine/sebastian-seungs-quest-to-map-the-human-brain.html">https://www.nytimes.com/2015/01/1... 3/
I’ve actually been too busy to ever write much about how it’s done. So here’s my attempt to teach Crowdsourcing 101 in a Tweet thread. 4/
#1 Explain significance. Easy here because excellent reporters like @max_fisher @choesanghun @chadterhune @soheefication @heesu_lee @DaslYoon had already explained why other countries should be interested in S Korea’s COVID response. 5/ https://twitter.com/SebastianSeung/status/1243753458903257089?s=20">https://twitter.com/Sebastian...
In the call to action tweet thread, all I had to do was link to their writings and summarize the purpose at the top as “SAVE LIVES by translating Korean→English!” 6/ https://twitter.com/SebastianSeung/status/1243753450074181637?s=20">https://twitter.com/Sebastian...
#2 Set a deadline. I challenged the crowd to finish “by Monday morning.” Deadlines are important for building a sense of urgency. Speed is a universal thrill, and you want people to get that feeling. Video game designers know this. 7/
#3 Quantify progress. “75 pages” made the goal numerical. At the beginning we could count how many pages we had finished, and near the end how many pages were left. 8/
Founding member Alex made a chart of the 75 pages so that we could visualize progress by watching the squares gradually turn orange. 9/ https://twitter.com/SebastianSeung/status/1244311510891089921?s=20">https://twitter.com/Sebastian...
#4 Open call. This was a complex technical document, so we could have asked for people with backgrounds in public health or medicine. We didn’t because projects usually end up requiring a diversity of skills and backgrounds. 10/
#5 Immediate participation. The tweet linked to a Google Doc seeded with an automated English translation of the original Korean document. The Doc was open to editing by anyone. Within seconds, anyone could start contributing. 11/
We had originally considered putting up barriers, like requiring people to email for permission to edit. This would have created delays, and is less motivating. 12/
#6 Trust strangers. Opening the Doc to editing by anyone meant that it could be attacked by vandals. Malicious people are rare, and anyway a Google Doc can be restored from its version history. 13/
More importantly, why would some evil person attack your crummy crowdsourcing project in the beginning, when it looks like it’s going nowhere? Save your paranoia for later on, when the project becomes successful. 14/
#7 Make it social. Online crowdsourcing can be tough, because participants may feel lonely, especially when the project hasn’t taken off yet. You have to provide them with ways to interact, or at a minimum feel each others’ presence. 15/
People come for the cause, but stay for the camaraderie. These are people who want to make a difference. And they like meeting other people who feel the same way. 16/
Here it was easy because Google, to their credit, engineered Docs to be social. Icons for visitors are displayed at the top. Editors’ cursors are visible. People can leave comments and chat. 17/
#8 Attract an audience. You may find that only 1% of visitors end up working a lot on the project. Don’t worry; this is typical. “Many are called but few are chosen.” 18/
But the other 99% are actually part of the project too. They are the audience. Their cheering will keep you motivated. And when you achieve something great, they will tell the world about it. 19/
#9 Quality control. Once the project is on a roll, define the goals of quality control, and project leads will step up to achieve them. Here the goals were technically accurate content, good verbal style, and good visual style. 20/
Quality control requires more organization and communication, facilitated by productivity tools like Slack and Zoom. We tracked progress with a spreadsheet that recorded review of the three QC goals for the 75 pages. 21/
Getting over the finish line is tough. The first draft translation was done by early Sunday afternoon, but careful quality control made us miss our Monday morning deadline. 22/
#10 Release early and often. Technical accuracy of the translation was an important concern, so it was tempting to delay. Instead we decided to release version 0.9 so that people could start benefiting from the information. 23/
#11 Dissemination. The translation was ready by midday on Monday, but we realized it would have no impact unless we could get people to read it. Luckily, people attracted to online crowdsourcing tend to be savvy at social media. 24/
The team sprang into action. In short order, we had a Twitter account. 25/ https://twitter.com/COVID_Translate/status/1244753546769125376?s=20">https://twitter.com/COVID_Tra...
A Facebook page 26/ https://www.facebook.com/covidtranslateofficial">https://www.facebook.com/covidtran...
And a web site. Wow that was fast. What a team! 27/
http://covidtranslate.org"> http://covidtranslate.org
http://covidtranslate.org"> http://covidtranslate.org
#12 Site analytics. Get some idea of your reach by activating Google Analytics for your site. 28/ https://twitter.com/COVID_Translate/status/1244840079626866689?s=20">https://twitter.com/COVID_Tra...
#13 Don’t be discouraged by the naysayers. This person is absolutely correct that accuracy is important. 29/ https://twitter.com/tealeafly/status/1244080692054626306?s=20">https://twitter.com/tealeafly...
But it’s been shown time and time again that crowdsourcing can actually produce superior accuracy, if you do it right. I could cite the studies but the reason is intuitive: recruiting more eyes tends to reduce the error rate. 30/
And anyway the project attracted many volunteers with expertise in public health and medicine, and others with experience in translation. 31/
Another critic tweeted to ridicule us for being too “cheap” to hire professional translators. We finished it over the weekend, before a professional would have even started. 32/
The translation went blindingly fast. By Sunday afternoon, @kj_seung and other public health experts at @PIH were already studying the first draft, and learning from it. 33/
But speed isn’t the only thing. Our volunteers believed in the cause. They weren’t doing it for money. In fact they were paying with their time. Highly motivated people tend to produce the most accurate results. 34/
#14 Have fun. Once you achieve liftoff, crowdsourcing is rewarding in so many ways. On Saturday evening, superstar entertainer @ericnamofficial retweeted. What an honor to be recognized by him and his fans! It was a big boost for the team. 35/ https://twitter.com/ericnamofficial/status/1244055480479014913?s=20">https://twitter.com/ericnamof...
Then a retweet from @siwon_choi of Super Junior fame and regional ambassador for UNICEF. Over the top! We were so grateful for encouragement from him and his 7M followers. 36/ https://twitter.com/siwonchoi/status/1244062322810707968?s=20">https://twitter.com/siwonchoi...
After midnight, so many visitors came to the Doc that we brought Google to its knees :) Editing slowed to a snail’s pace for some, and other editors started getting kicked off the Doc. 37/
It was wondrous to see total strangers coming together to discuss the fine points of translation in the dead of night by leaving comments in the Doc. 38/
As the East Coast participants started flagging, reinforcements arrived from Seoul. The Korean @TEDnews translators to the rescue! 39/ https://twitter.com/SebastianSeung/status/1244137602539806721?s=20">https://twitter.com/Sebastian...
I’ve tried to distill the principles of crowdsourcing for you. Thanks for reading, and I wish you the best of luck with your own crowdsourcing efforts!
