A TEXT POST

Here comes sloppiness

Clay Shirky is not on the list of my favorite writers. Why? I find many of his arguments to be sloppy, populist and, occasionally, unfair to the people he’s criticizing (see my review of Cognitive Surplus).

I’ve been rereading Cognitive Surplus and found a very good example that encapsulates it all. At one point in the book, Shirky criticizes the elitism of restaurant critics - who, of course, can’t appreciate the wisdom of crowds - by taking on an essay called The Zagat Effect that first appeared in Commentary.

Here is what Shirky says about it: 

One early critical complaint [about the declining power of restaurant critics] was an essay called “The Zagat Effect,” written by Steven Shaw in 2000. Zagat is a restaurant guide that aggregates user-generated reviews and ratings. Shaw complained bitterly about them, focusing in particular on their ranking New York City’s Union Square Café as number one, which he felt was unjust:

[Union Square Café] is number one in the sense that it emerges first in response to this question on the survey: “What are your favorite New York restaurants?” … Union Square Café is, indeed, a very good restaurant, one beloved by many New Yorkers for its compassionate service—it is perhaps the most unintimidating of the city’s better restaurants—and its simple but intensely flavorful food. But with all due respect to that justly popular establishment, it is patently ridiculous to rank it ahead of a dozen other places, and in particular such world-class restaurants as Lespinasse, Jean Georges, and Daniel.

Nowhere does Shaw spell out why preferring Union Square Café to Lespinasse is patently ridiculous—calling Lespinasse world-class simply begs the question. In a world where access to information is open, the critic does a delicate dance. Shaw is unwilling to condemn Union Square as a bad restaurant; it’s just not the kind of restaurant people like him prefer, which is to say people who eat in restaurants professionally and are happy to have a little intimidation with their appetizers. But if he makes that complaint too visibly, he risks undermining his desire to be able to guide his audience. Back when professional reviews were the only publicly available judgment of restaurants, this difference didn’t matter much (and critical contempt for the audience wasn’t so visible), but when we can all now find an aggregate answer to the question “What is your favorite restaurant?” we want that information, and we may even prefer it to judgments produced by professional critics.

No surprises here: all of this fits well with Shirky’s populist message: most institutions are elitist bastards, power to the people! 

Well, compare Shirky’s summary and critique with what the Zagat Effect essay actually said: 

But let us suppose that the democratic procedures were impeccable. Would they then constitute an appropriate means of rating restaurants?

The scores and rankings in the Zagat survey reflect, by definition, an average of the opinions gathered. But if you want to know how good a restaurant is, averages are seriously misleading. Thus, for four years running, according to Zagat, the number-one restaurant in New York has been Union Square Café. It is number one in the sense that it emerges first in response to this question on the survey: “What are your favorite New York restaurants?” It is true that other restaurants score higher than the Union Square Café in the areas of food, décor, and service, and the closest thing to a “best” restaurant in Zagat, based on scores alone, would be Le Bernardin; but this information appears on no chart.

Union Square Café is, indeed, a very good restaurant, one beloved by many New Yorkers for its compassionate service—it is perhaps the most unintimidating of the city’s better restaurants—and its simple but intensely flavorful food. But with all due respect to that justly popular establishment, it is patently ridiculous to rank it ahead of a dozen other places, and in particular such world-class restaurants as Lespinasse, Jean Georges, and Daniel.

The point that the author makes is obvious: according to Zagat, the standard of excellence in the restaurant business is not “food, décor, and service” - what professional critics usually focus on and assess - but, rather, what restaurant comes to mind when a bunch of people are asked to name their favorite restaurant.

The essay then goes on to argue this point in fine detail: 

Another example of what can happen when one averages consumers with different levels of taste and (above all) experience was on display in the 1998-99 edition of the Zagats’ America’s Best Meal Deals. On its nationwide list of “Top Delis,” not a single New York delicatessen was to be found; instead, the guide featured places like d’Bronx Deli in Kansas City (which apparently doubles as a pizzeria) and another establishment in Salt Lake City. Even in New York, where the Zagats draw on a presumably more sophisticated base and a far greater number of ballots, the survey disproportionately rewards what might be called “yokel pleasers” like Café des Artistes (the ninth most popular restaurant in the survey) and River Café.

Worse, it is a simple but distorting truth that people tend to prefer the restaurants they already frequent—witness the similarity between the Zagat “Traffic Report” and its list of “Most Popular Restaurants.” The circular nature of this process has been well pinpointed by the critic Seymour Britchky:

Once you learn to hate a restaurant you never go back, [but] since you do not evaluate a restaurant for Zagat unless you have been there in the past year, those who continue to rate a place are, disproportionately, its admirers—fans—while the opinions of detractors go unrecorded.

The New York Times critic William Grimes has labeled this phenomenon “The Zagat Effect,” adding that once a restaurant gets a good rating, “diners flock to it … and, convinced that they are eating at a topflight establishment, cannot bring themselves to believe otherwise.”

Clay Shirky, somehow, manages to condense all of this to a complaint that the evil Commentary critic didn’t “spell out why preferring Union Square Café to Lespinasse is patently ridiculous.”

But, of course, he did: according to Zagat’s own guide, the odds are that Lespinasse has higher scores for food, service and decor…Cognitive deficit, anyone? 

  1. evgenymorozov posted this