Read on Twitter
1/ Thread: The Wisdom of Crowd
In 2007, @mjmauboussin wrote a paper on the wisdom of crowd, and showed with simple maths and experiments why the crowd can trump individuals.
Some of the takeaways were VERY counterintuitive and it materially influenced my thinking.
In 2007, @mjmauboussin wrote a paper on the wisdom of crowd, and showed with simple maths and experiments why the crowd can trump individuals.
Some of the takeaways were VERY counterintuitive and it materially influenced my thinking.
2/ In the game show “Who wants to be a Millionaire”, contestants can choose to see audience poll or consult with an expert if he/she is unsure on a question.
Audience was right 90% of the time.
The expert?~67%.
Audience was right 90% of the time.
The expert?~67%.
3/ Even if a very small number of individuals know the answer, the crowd will lean to the correct answer.
Why?
Scott Page in his book “The Difference” explains the rationale with a very simple example.
Why?
Scott Page in his book “The Difference” explains the rationale with a very simple example.
4/ Imagine this question was asked to a random crowd:
Which person from the following list was not a member of the Monkees (a 1960s pop band)?
(A) Peter Tork
(B) Davy Jones
(C) Roger Noll
(D) Michael Nesmith
The answer is Roger Noll.
Which person from the following list was not a member of the Monkees (a 1960s pop band)?
(A) Peter Tork
(B) Davy Jones
(C) Roger Noll
(D) Michael Nesmith
The answer is Roger Noll.
5/ Now imagine a crowd of 100 with knowledge distributed as follows:
7 know all 3 of the Monkees
10 know 2 of the Monkees
15 know 1 of the Monkees
68 have no clue
So <10% knows the answer, and over 2/3rd have no idea on the topic. Assume those who have no clue pick randomly.
7 know all 3 of the Monkees
10 know 2 of the Monkees
15 know 1 of the Monkees
68 have no clue
So <10% knows the answer, and over 2/3rd have no idea on the topic. Assume those who have no clue pick randomly.
6/ So what happens in this experiment?
7 who know all the Monkees vote Noll
5/10 who know 2 of the Monkees will vote Noll
5/15 who know 1 of the Monkees will vote Noll
17/68 clueless will vote Noll
The correct answer will get 34 votes whereas the other choices will get 22 each.
7 who know all the Monkees vote Noll
5/10 who know 2 of the Monkees will vote Noll
5/15 who know 1 of the Monkees will vote Noll
17/68 clueless will vote Noll
The correct answer will get 34 votes whereas the other choices will get 22 each.
7/ But not all problems/questions fit this type. There are mostly three kinds of problems:
I. Some people in the crowd know the answer while many, if not most, don’t.
II. The second is a state estimation problem, where one person knows the answer but the group does not...
I. Some people in the crowd know the answer while many, if not most, don’t.
II. The second is a state estimation problem, where one person knows the answer but the group does not...
8/ III. There is a prediction problem, where the answer has yet to be revealed.
The first example addresses the first type of problem just discussed. Let’s discuss the other two now.
The first example addresses the first type of problem just discussed. Let’s discuss the other two now.
9/ Experiment 2: You fill a jar with jellybeans and ask a group of people to guess how many jellybeans there is.
Mauboussin did this experiment a number of times at Columbia. He shared the 2007 results and mentioned the result is remarkably consistent every year.
Mauboussin did this experiment a number of times at Columbia. He shared the 2007 results and mentioned the result is remarkably consistent every year.
10/ The avg. guess of the class was 1,151.
The actual number was 1,116, so avg guess was ~3% shy of the actual.
Here’s what truly shocked me: Only 2 out of 73 people guessed better than average.
The actual number was 1,116, so avg guess was ~3% shy of the actual.
Here’s what truly shocked me: Only 2 out of 73 people guessed better than average.
11/ This can be so counterintuitive that you may want to spend the next minute looking at the table.
Just understand the simple math behind it.
Just understand the simple math behind it.
12/ Three big takeaways from this:
I. Diverse crowd will always predict more accurately than the average individual. So the crowd predicts better than the people in it.
Not sometimes. ALWAYS.
I. Diverse crowd will always predict more accurately than the average individual. So the crowd predicts better than the people in it.
Not sometimes. ALWAYS.
13/ II. Collective predictive ability is equal parts accuracy and diversity. You can reduce collective error by either increasing accuracy by a unit or by increasing diversity by a unit. Both are essential.
14/ III. The collective is often better than even the best of the individuals. So a diverse collective always beats the average individual, and frequently beats everyone.
If it’s still counterintuitive, spend another minute on that table.
If it’s still counterintuitive, spend another minute on that table.
15/ Experiment 3: See the image for the rules of this experiment.
The consensus got 11/12 right.
2 students got 9/12 (best performance).
The avg student got 5/12.
The consensus got 11/12 right.
2 students got 9/12 (best performance).
The avg student got 5/12.
16/ Again, the crowd was much wiser than everyone else!
This was sort of a combination of experiment 1+2.
Some people have better knowledge on pop-culture, and greater diversity of the crowd led to the correct prediction at a higher rate than anyone else.
This was sort of a combination of experiment 1+2.
Some people have better knowledge on pop-culture, and greater diversity of the crowd led to the correct prediction at a higher rate than anyone else.
17/ One caveat about diversity:
While we mostly focus on social diversity (race, gender etc.) and there is indeed high correlation between social identity diversity and cognitive diversity, the actual benefit of diversity stems from cognitive diversity.
While we mostly focus on social diversity (race, gender etc.) and there is indeed high correlation between social identity diversity and cognitive diversity, the actual benefit of diversity stems from cognitive diversity.
End/ Mauboussin also noted that it’s a complicated topic and we certainly don’t understand all aspects of the wisdom of the market.
It is a fascinating read nonetheless. Happy weekend!
Link: https://operators.macro-ops.com/wp-content/uploads/2020/06/ExplainingWisdom-1-1.pdf
It is a fascinating read nonetheless. Happy weekend!
Link: https://operators.macro-ops.com/wp-content/uploads/2020/06/ExplainingWisdom-1-1.pdf
