Jul 16 2007
The spending power of crowds
James Surowiecki and Malcolm Gladwell are well known to readers of the New Yorker as the twin purveyors of contrarian gobbledygook. They bring to mind a pair of gymnast contortionists - going through a series of elegant and stylish moves to assume impossible convoluted positions - then adopting serene smiles which might convince a causal observer that they are really comfortable bent double with their feet tucked behind their ears.
Last year Gladwell composed a fanciful but tightly argued piece working to the mischievous conclusion that miniature dachshunds represent a greater threat than pit-bulls. An entertaining read - recommended, provided you do not plan to use it as a blueprint for choosing or training a dog.
The danger is of course that anyone should ever take their arguments seriously.
Unfortunately that is exactly what happened with Sorowiecki’s “The wisdom of crowds”. The folks at Wired set out to prove Sorowiecki’s thesis. Not surprisingly the attempt failed. But it was “a highly satisfying failure”, no-one was killed or seriously injured.
The gullibility, excitability and outright madness of crowds has undoubtedly been responsible for many of the world’s greatest atrocities. But no crowd has ever produced any great work of science, literature. music or art. No crowd has ever performed a difficult surgery or designed a bridge. Why would anyone take the proposition that “wisdom resides in crowds” seriously?
The answer I believe lies in the types of questions we are asking today:
Will Blu-ray beat out HD-DVD?
Will the web obsolete newspapers?
Will the Transformers movie be a hit?
These questions cannot be resolved by informed analysis. Only the crowd can answer. While these types of questions continue to draw so much attention it is perhaps not surprising that even experts defer to the crowd. But it is not their wisdom that commands the deference - it’s their spending power
But Surowiecki never argued that any crowd was wise: the wisdom that came from crowds comes from analysis of aggregated knowledge, not application of collective knowledge.
And equating a work of engineering and design, or surgery — the application of learned skills — with wisdom involves some of that contortionism you write so eloquently about.
Mark I agree. Surowiecki should have titled his book “The knowledge and insights that can be harvested by aggregation and analysis of the behaviors and opinions of certain selected groups containing members from diverse backgrounds”
But he chose instead “The Wisdom of Crowds: Why the Many Are Smarter Than the Few, and How Collective Wisdom Shapes Business, Economies, Societies and Nations. . .”
The promise of that title is never fulfilled, but I suspect it may have helped with sales and speaking fees. His notion of “crowds” stretches the English language beyond its limits. At one point he argues that the NASA engineers who engineered the safe return of Appollo 13 were evidence of the “collective wisdom of the crowd”. NASA mission control a crowd, how so? Evidently they qualified as a crowd because they had worked in many different industries before coming to the space agency.
Real crowds display madness more often than they display wisdom. Surowiecki chose his title in counterpoint to Charles MacKay’s seminal “Extraordinary Popular Delusions & the Madness of Crowds”(1841). To my mind MacKay proves his central thesis far more convincingly than Surowiecki. McKay’s discussions of the South Sea Bubble, the tulip-mania that gripped Holland in the 1600s, alchemy, and witch-hunts have broader application, and yield more profound insight than the cherry-picked examples Surowiecki cites.
Surowiecki is forced to admit evidence that runs counter to his thesis. But he conjures all sorts of conditions to explain these away. So he insists, for instance, that for a collective wisdom to surface a group must be unaware of it: “the irony of group wisdom is that it is only when a group is unaware of its intelligence that it can be effective”.
The Wired experiment seems interesting in this regard - were the participants really unaware of what the project was designed to prove? For an experiment designed to test the hypothesis that the diversity of a group is more valuable than the intelligence/experience of its members, they assembled a remarkably homogeneous crowd.
I would certainly not dissuade anyone from reading “The wisdom of crowds”. But for those interested in a more profound treatise on the contemporary relevance of the issue of the madness/wisdom of crowds I would recommend “When the Press Fails: Political Power and the News Media from Iraq to Katrina (Studies in Communication, Media, and Public Opinion)”
This whole wisdom of crowds/crowdsourcing schtik has run out of control. Most of the arguments advanced in favor of it relate to the value of collaboration, research and statistics, nothing new there. Much of the truth there is in the notion of the Wisdom of Crowds is of little practical value. It’s like the way that certain animals can sense the coming of earthquakes. Lot of scientific evidence that this is the case but no practical way to use that Wisdom. It does have interesting parallels with the sporadic successes of technical analysts in the stock market where the behavior of the market itself is used to predict future share prices. That is probably where Surowiecki came up with the idea. But again that is nothing new. Using past behavior to predict future behavior is something that market researchers have been doing for eons.
An article by Surowiecki in the latest New Yorker suggests that his enthusiasm for extrapolating the “wisdom of crowds” paradigm may be running out of steam. Discussing fuel efficient autos he concludes:
“automakers have listened to car buyers, and put their energy into making vehicles bigger and faster, rather than more efficient. In calling for a law requiring better gas mileage in our cars, then, voters are really saying that they’re unhappy with the collective result of the choices they make as buyers. Sometimes, they know, we need to save ourselves from ourselves”.
an interesting sidenote - what better evidence of the distilled, aggregated wisdom of crowds than the US presidential election?