The Wisdom Of Crowds Turns Into Madness
The "Wisdom Of Crowds" concept is that the average guesses of a crowd will be closer to the truth than a randomly selected individual guess.
The reason this works is that because the crowd has different individuals with different types of systematic error, e.g. prejudices. With more individuals, the prejudices negate each other.
The Swiss study took 144 college students and asked them a series of questions (population of Switzerland, murder rate, etc). It recorded 5 consecutive guesses, as well as the confidence for the first and last guess.
I.
The first interesting finding is that the crowd is sometimes so incredibly wrong that the mean of their responses is just... really wrong. How many assaults were there in Switzerland in 2006? 10? 100? 1000? 10000? 100000? Those are exponentially different guesses, so an arithmetic mean could be way off, factors of ten off.
In such cases, a geometric mean is much closer to the correct answer. So, point number one, when you are crowdsourcing, choose your mean/distribution appropriately.
The diversity of guesses is quite large-- everyone comes to the question with their own prejudices and errors.
But merely by giving the subjects access to the previous round's guesses-- either the mean of the guesses ("aggregated information") or everyone's individual guess, the diversity disappears and everyone's guesses begin to converge.
The first round the guesses were wildly disparate, but as everyone got to see the other guesses, they converge remarkably.
Why did having the full information (all 12 people's individual guesses) seem to cause less convergence than having the mean of their guesses? It didn't, really; but also because the aggregate is only one number that you converge to; having 12 wildly disparate numbers to converge to is harder. But by the third round, it hardly mattered-- a systematic bias had been introduced into the crowd, which is ironic since it is systematic bias that the Wisdom Of Crowds is supposed to negate. Moo.
III.
People following the herd would be boring but not disastrous, except for the other finding.
Since the guesses converge, since other people are converging with you and you can see that, the confidence in these guesses goes up: a false belief of collective accuracy with no increase in actual accuracy. "It's unanimous!" Yikes.
Also remember, these people weren't being given an expert's guess to converge to, just other (regular) people's. As the authors point out, they didn't even attempt to measure group leader effects, persuasion, talking heads on TV, or twitter.
This is not a trivial problem. It isn't just saying that the beliefs converge; it is saying that since the beliefs converge along with greater confidence in their "truthfulness", it becomes more difficult for any individual to not converge as well-- and feel confident about it.
If you do manage to run from the herd you have to climb a high wall. "Can so many people be so wrong, yet so close together in their guesses? So wrong, yet so confident? Is everyone insane?"
You can imagine the social implications of a highly energized crowd, or electorate, or laity, or polity, or tax base, all converging on a "truth" of which they are supremely confident by virtue of the fact that others believe the same (which is the result of similar convergence on their part.) This is probably supercharged when you have a charismatic figurehead leading convergence, and by "charismatic figurehead" I mean media; no one person came up with this, everyone just knows it's true.
IV.
So much for the paper. Now consider the more general implications.
"Well, I'm going to be an independent thinker and not be affected by the herd and make my own educated guess." No, you won't.
The moment you have the other people's guesses, you cannot shake that information. Your "independent" guess necessarily includes that guess in some way, you can't unlearn it. Either your guess converges towards the herd, or your guess is characterized as against the herd. Either way, the herd affected your thinking in ways you don't realize. You're part of the dialectic and you didn't even want to be. That you don't want to be part of it ensures you are part of it.
The existence of the convergence of ideas, knowing that a convergence exists, either attracts further groupthink, or sets up a second groupthink in opposition to the first. Groupthink certainly reinforces one idea; and it can cause the setting up of a second large idea in opposition, but it makes a third independent idea highly unlikely (unless, again, it forms in opposition to ideas 1 or 2.)
In other words, in cases where social influence is impossible to avoid, the wisdom of crowds becomes the madness of crowds even for those who disagree with the crowd. All it takes is one idiot with a megaphone.
---
How to use your own inherent narcissism to guess more accurately
The special circumstance which causes the wisdom of crowds to fail
Digg | 
del.icio.us | 
StumbleUpon | 
Reddit | 
Yahoo! | 
My favourites
Recommend to a Friend
August 31, 2011 12:44 PM | Posted by : | Reply
It's true for just about any piece of extraneous information. If you change the environment and context of just about any question, you get a different answer. E.g. parole board decisions in the morning versus the afternoon. Humans just ain't rational.
Score: 2 (2 votes cast)
August 31, 2011 1:21 PM | Posted by : | Reply
So, when you are in crowds, you don't lose your individuality, but your individuality is changed by the crowd?
Good post, btw.
Score: 2 (2 votes cast)
August 31, 2011 2:07 PM | Posted by : | Reply
TLP: You're part of the dialectic and you didn't even want to be. That you don't want to be part of it ensures you are part of it.
Freud: If you arrive late for the appointment, you're passive aggressive, if you arrive early you're anxious, and if you arrive on time, you're obessive.
There's no escaping it. Sit back, relax, and enjoy the ride.
Score: 11 (11 votes cast)
August 31, 2011 2:11 PM | Posted by : | Reply
So is it *ever* possible to be an "individual"? Or to know that you are not insane?
God, this is depressing ...
Score: 0 (4 votes cast)
August 31, 2011 3:40 PM | Posted by : | Reply
Try playing a game of Paranoid Debating to observe how this works, both within a group and inside your mind.
Score: 1 (1 votes cast)
August 31, 2011 3:54 PM | Posted by : | Reply
Not me man, I think for myself!
Score: 3 (3 votes cast)
August 31, 2011 4:16 PM | Posted by : | Reply
Not sure why I can't just learn from this and try to go beyond it.
"I don't have to go with the group, but I don't have to oppose them either, and I should systematically generate all the possible other answers ...
okay now let me try to figure out which answer makes the most sense ...
and further let me step back and look at my answers in general ... do they tend to line up with one group or another ...
if they do, I can reaxamine more closely to see if I think that's because I'm just following that group, or because those answers actually make the most sense ...
finally I can acknowledge that though I'm still affected by the groupthink, at least I'm not totally dominated by it as I would be if I never considered it ...
Score: 3 (3 votes cast)
August 31, 2011 6:03 PM | Posted, in reply to , by : | Reply
Because no matter how thorough or analytical and self-aware you may be about your opinions and how they were/are affected by others, you will never know what opinion you might have had if you had never known what others thought.
Score: 4 (4 votes cast)
August 31, 2011 6:21 PM | Posted by : | Reply
Also known as an information cascade.
Score: 0 (0 votes cast)
August 31, 2011 7:32 PM | Posted by : | Reply
"it is systematic bias that the Wisdom Of Crowds is supposed to negate" - nope. http://en.wikipedia.org/wiki/The_Wisdom_of_Crowds#Failures_of_crowd_intelligence.The WOC is about averaging out *random* errors.
This paper is like an elixir for demonstrating that people (even quite erudite thinkers like yourself, TLP) don't really grok the idea behind the WOC. E.g., see http://neuroself.com/2011/05/29/jonah-lehrer-is-not-a-neuroscientist/.
Score: 3 (3 votes cast)
September 1, 2011 5:08 AM | Posted by : | Reply
How could one possibly infer anything soundly about the truth of a matter merely on the basis of a collection of opinions or guesses or prejudices? Try doing something sensible, e.g., ask a bunch of people to observe something, think about it, and then come to a conclusion. Now compare that with the truth. This will tell tell us something worth taking seriously about the proper exercise of our faculties and the outcomes yielded by a proper exercise of our faculties.
Score: 2 (2 votes cast)
September 1, 2011 7:28 AM | Posted, in reply to , by : | Reply
It is about bias. If you asked the same person to estimate the value of some similar figure they would likely make an error in the same direction as the first, e.g. if they grossly overestimated the number of rapes they would likely overestimate the number of murders. The idea is that every individual person has a systematic bias within the domain of questions that are being asked, but the biases themselves are treated as a random variable among the population at large; if you randomly select a person, his systematic error becomes a random error in your sample.
That post by Peter Freed is long and rambling, and unfortunately it is terribly unclear in its attempt to reiterate a crucial point stated in the study as follows*: "In the “aggregated information” condition, subjects could reconsider their estimate after having received the average (arithmetic mean) of all 12 estimates of the former round."
But here's what the researchers say about the arithmetic mean, which they report to the subjects after each round of estimates...
"In our case, the arithmetic mean performs poorly, as we have validated by comparing its distance to the truth with the individual distances to the truth. In only 21.3% of the cases is the arithmetic mean closer to the truth than the individual first estimates."
... as opposed to the geometric mean, which they use as a measure the crowd's wisdom after the fact:
"When using logarithms of estimates, the arithmetic mean is closer to the logarithm of the truth than the individuals’ estimates in 77.1% of the cases. This confirms that the geometric mean (i.e., exponential of the mean of the logarithmized data) is an accurate measure of the wisdom of crowds for our data."
Assuming the tendency is to update your estimate in the direction of the aggregated statistic, by reporting the arithmetic mean the researchers ensured that a majority of the subjects would produce a second estimate less accurate than the first, but by reporting the geometric a majority would have given better estimates. It seems that the group was not adjusting its estimates in deference to the wisdom of the crowd, but to the experimenters' authority in choosing a statistic to represent that wisdom. The mistake driving the decrease of their "wisdom-of-crowd indicator" is really appeal to authority as much as it is argumentum ad populum.
From a footnote: "Considering the logarithmic nature of our data, one may argue that the geometric mean would have been a better design choice than the arithmetic mean for the information feedback in the aggregated information condition. However, this measure is hard to understand for most subjects because it necessitates confidence with logarithmic transformations. As the simple average (i.e., arithmetic mean) is known from daily life, this information is more meaningful for subjects. Hence, we decided for the arithmetic mean."
Translation: We assumed these college students were stupid so we gave them data that would prove they were stupid. (The very last footnote also contains more information about why the methods led to the results they got, and in fact implies that they were studying the properties of log-normal distributions more than the properties of human decision-making. It should be the main conclusion of the study.)
Out of everybody involved in this, including the researchers, Freed, and Lehrer, only the lowly undergraduates did an admirable job.
*Also, Freed amazingly fails to notice the main problem with Lehrer's WSJ article: namely that he COMPLETELY fucking lied by saying "each round led to worse guesses." I can't give him the benefit of the doubt here; certainly any Rhodes scholar, or at least the kind of person a Rhodes scholar would hire to read a paper so he doesn't have to, can correctly interpret graph E.
Score: 3 (3 votes cast)
September 1, 2011 7:37 AM | Posted by : | Reply
I'd also crack some jokes about how Freed can't stop referencing Malcolm Gladwell in a post whose thesis is that Lehrer is a journalist and not a neuroscientist, but I really need to go to bed.
Score: 0 (0 votes cast)
September 1, 2011 7:43 AM | Posted by : | Reply
Even if you could learn that this happens and gird yourself to be immune to the effect, it wouldn't necessarily matter because exactly this effect makes much of the world turn, and no individual can swim against that kind of current. How much is a piece of paper, 157mm x 66 mm, 75% cotton/25% linen, worth? Nothing? A pizza? A keg of beer? A Thai teenager to call your own? It can be all of those things, depending on what people collectively agree to be the case.
Sure, the study asked about quasi-natural facts (would be plain 'natural' if Switzerland were a real country), but these mean nothing. They're just trivia. As soon as they start to mean something and start to earn the title 'Truth' (i.e. not 'how many rapes occurred?' but 'how many rapes is so many that we need to start locking women up for their own safety/extending police powers/chopping dangly bits off?"), the answer will be, always has been, and perhaps can only be determined by exactly this mechanism. Yes, it's the crowds doing it. Let's hope that it approximates 'wisdom'.
Score: 0 (2 votes cast)
September 1, 2011 8:24 AM | Posted by : | Reply
"we need to start locking women up for their own safety" People always say it this way... if the stats were "unreasonably high" for the percent of man on woman rape.... wouldn't it make more sense to lock all the men up for the safety of the women? I mean if a bunch of tigers escaped the zoo and were rooming the city and only 6% had ever killed anyone.... would we recapture ALL of the lions for the safety of the people or would we tell people to stay in doors in hiding for the rest of their lives?
I'm just saying it's interesting if one segment of the population was statistically likely to prey on another segment of the population--- you would consider it more ethical to put the potential victims through a life spent in hiding rather than the potential predators. It's unthinkable that innocent men have their free life taken away simply because women keep getting raped, but innocent women's freedom should be expendable "for their own good" if women keep getting raped.
Score: 1 (3 votes cast)
September 1, 2011 8:27 AM | Posted by : | Reply
PS- Guy fox- I'm nitpicking, your comment was otherwise good. ;)
Score: 0 (0 votes cast)
September 1, 2011 8:46 AM | Posted by : | Reply
And remember: psych meds, and not psychotherapy, should always be first-line treatment for any mental disorder. Care should always begin with a psychaitrist's assessment. If that psychiatrist happens to know a local therapist, the psychiatrist might suggest psychotherapy.
If a psychiatrist is not available, any physician will do.
Could 40,000 pharmaceutical reps be wrong?
Score: 2 (4 votes cast)
September 1, 2011 11:18 AM | Posted by : | Reply
While the irony of quoting Monty Python in a comment about individuality is quite obvious, i just can't resist :
Brian: Please, please, please listen! I've got one or two things to say.
The Crowd: Tell us! Tell us both of them!
Brian: Look, you've got it all wrong! You don't NEED to follow ME, You don't NEED to follow ANYBODY! You've got to think for your selves! You're ALL individuals!
The Crowd: Yes! We're all individuals!
Brian: You're all different!
The Crowd: Yes, we ARE all different!
Man in crowd: I'm not...
The Crowd: Sch!
Score: 2 (2 votes cast)
September 1, 2011 11:28 AM | Posted by : | Reply
So your conclusion is what, Hegelian? All important ideas are dialectical responses to older ideas? That in itself isn't a world-shattering claim. Seems pretty reasonable actually.
Score: 0 (2 votes cast)
September 4, 2011 4:32 PM | Posted by : | Reply
You are bearing on (I dare say) a false dichotomy: if by being an individual you differ from the thinking of the crowd, your unique thinking directly depends on the general thinking of the crowd, and therefore, your thoughts no matter how uniquely derived, reflects not you at all but the crowd. This undercuts all the discoveries and inventions made in science. Sorry, individuality remains
Score: 0 (0 votes cast)
September 4, 2011 5:46 PM | Posted by : | Reply
Actually I diverge from the crowd a helluva lot. I'm a high functioning Aspie, and one of the blessings/curses that comes with this is major pattern recognition ability. Lucky me, I recognize behavior patterns.
One of the patterns I recognize very quickly is Bandar-Log... "We all have said it; it must be true."
Soon as I see that little meme eddying about, I can predict the next pattern - "Death to the unbeliever!"
The best response to this is to get the hell away from it. As early as possible in the Bandar-Log stage. Collective insanity indeed...
Wonder if Kipling was Aspie. But more to the point, how the hell has the human race survived this long with the collective mentality of a troop of howler monkeys? I see this pattern on a daily basis, on issues large and small, and the older I get, the more frightening that is.
Score: 0 (4 votes cast)
September 6, 2011 1:23 PM | Posted, in reply to , by : | Reply
I don't think individuality is even possible for us. Humans are first and foremost a social species, perhaps not as lockstep as a colony of ants, but we aren't exactly cats either. And the whole wisdom of crowds thing is more a social reinforcement phenomena than anything. The whole thing seems to revolve around the idea of converging around a mean number. But it doesn't work if you have no idea of the actual facts.
If I asked about how many apples were sold at piggly wiggly today, we'd converge around a mean, BUT it would only be the truth if someone involved knew something about apples and grocery stores. I think the wisdom of crowds is more illusion than anything real. It doesn't give you information about the world out there, just what people think is out there. Wikipedia is great for general interest things that work in a common sense way, but it doesn't work as well when it's something in science or math that doesn't work in a commonsense way.
Score: 1 (1 votes cast)
September 15, 2011 9:58 AM | Posted, in reply to , by : | Reply
--
Good Day, de heer / mevrouw
Heeft u behoefte aan een legitieme en Dringende lening om uw rekeningen te betalen en Clear Uw
Schuld Neem contact met ons lovethloanoffer2002@live.com Vandaag met deze gegevens:
Uw Naam .....................................
Uw land ....................................
Sex ............................
Lening Bedrag dat nodig is ...................................
Duur ................................
Telefoonnummer .....................
Beroep: ...................................
Toegepast voordat ...............................
State ....................................
Maandelijks Inkomen ..................................
heeft u een aanvraag voor een lening voor;
U bent deze te sturen naar ons bedrijf e-mailadres:
lovethloanoffer2002@live.com
dankzij Regard
Mrs liefheeft, samson
Score: 0 (0 votes cast)
November 14, 2011 2:07 AM | Posted by : | Reply
Very happy to see your article, I am very much to like and agree with your point of view. Thank you for sharing. linmei/comment201111
Score: -2 (2 votes cast)
March 28, 2012 5:44 AM | Posted by : | Reply
That you are consequently nice! I just adore realising stunning along with creative giveaways this way. They will really jazz up my own morning, many thanks greatly. liulipingcomment201203blog
Score: -2 (2 votes cast)
October 31, 2012 8:25 AM | Posted by : | Reply
Geachte heer / mevrouw,
Wij bieden particuliere, commerciële en persoonlijke leningen met een zeer minimale jaarlijkse rente zo laag als 3% binnen 1 jaar tot 30 jaar Terugbetaling looptijd een deel van de wereld.
We geven leningen binnen het bereik van $ 10.000 tot $ 200,000,000.00 USD.Our leningen zijn goed verzekerd en maximale veiligheid is onze prioriteit; Geïnteresseerde personen kunnen contact met ons op via e-mail: morganjane04@gmail.com er rekening mee dat als je u jonger dan 18 jaar u niet om al uw gegevens goed gecontroleerd met uw landen hoge commissie toe te passen, dus zorg ervoor dat je echt bent op u van toepassing.
Leners INFORMATIE
{. 1} {Naam. 2} Land [3} Adres: .. [4} Leeftijd: 5.Fax nummer:
{6} Persoonlijk nummer:. {. 7} Beroep: 8.Sex: {. 9} Maandelijks Inkomen:
. Vereiste {10} {11.} Lening duur:
met vriendelijke groet
Met vriendelijke groet,
Mevr. Jane Morgan
Score: 0 (0 votes cast)
Comments