Categories
General

Is Spontaneity Overrated?

It used to be a surprise when people remembered our birthdays, but in the twenty-first century Facebook ensures that we will never forget a birthday. Does that make the happy birthday wishes any less sincere? Or is technology simply providing us with a cognitive assist and helping us express our sincere feelings?

A related question is how we should react to solicited reviews–a topic that was the subject of a recent interview on Mike Blumenthal’s blog. To be clear: I’m not talking about businesses offering incentives to reviewers–most folks seem to agree that incented reviews are a bad idea. And let’s not get started on hiring interns or Turkers to write them! Rather, the question is whether a review is less meaningful because it was solicited by a business  rather than spontaneously volunteered  by the reviewer.

I’m ambivalent. I don’t think the content of a solicited review is inherently insincere–after all, the reviewers have no reason to lie. In fact, soliciting a review from a disgruntled customer may annoy that customer enough to elicit one that is scathingly sincere!

Nonetheless, it’s hard to imagine why a business wouldn’t target a solicitation campaign at a sympathetic set of reviewers, given the opportunity to do so. Given that consumers put a fair amount of trust in aggregated reviews (as documented in the Forrester study about the “groundswell effect“), skewing the population of reviewers can significantly stack the desk.

And even a uniform campaign to solicit reviews raises concerns. Research by Yong Liu supports the adage that all buzz is good buzz–though in fairness I don’t know if he observed causality or just correlation. But I can extrapolate from personal experience that the number of reviews signals the popularity of a product or service. And I doubt I’m alone, given that Yelp, MenuPages, and other review sites let you filter or sort by the number of reviews. A successful campaign to solicit reviews, even if it doesn’t skew the polarity of the reviews, will at least inflate their quantity.

Still, where’s the harm? There’s nothing unethical in a business soliciting private or public feedback. And, back to the birthday example, I haven’t seen anyone upset by Facebook-prodded birthday greetings. Perhaps the online solicitation of reviews serves a similar “reminder” purpose, and we should simply accept its as part of our twenty-first century reality.

But consumers will need to re-calibrate their trust in reviews–or at least in what the numbers signal–if it turns out that a significant fraction of them are solicited. As Yelp CEO Jeremy Stoppelman pointed out in a blog post defending his company against recent legal action:

If a business could garner a top rating on Yelp simply by soliciting 5-star reviews from friends, family, and favored customers, how useful would such a site be?

While I don’t know enough to comment on the legal merits of the lawsuits (or the history of allegations that Yelp extorts advertising from businesses), I can understand how a proprietary review filter is controversial and invites skepticism from businesses whose positive reviews are filtered or demoted–especially given that relevance ranking raises similar concerns. But I can also understand how making such a filter completely transparent could defeat its stated purpose: “to protect consumers and business owners from fake, shill or malicious reviews”. And Yelp does at least disclose that it considers users’ activity level as a signal in its filter.

But let’s face it: it’s hard to draw a clear distinction between a solicited responses and spontaneous ones. Review sites have never claimed to conduct scientific polls, and consumers should be sophisticated enough to expect some degree of sample bias. Moreover, the process does not have to be perfect in order to be useful to consumers–we learn to approach review sites with a calibrated level of cynicism.

Still, my hope is that consumers will start placing less stock in the aggregated opinions of anonymous strangers and shift their trust to people who are more transparent about their identities and motives. The more that reviewers stand behind their opinion and put their own integrity on the line, the less it will matter whether those opinions are solicited or spontaneously expressed. We’ll see how the opinion marketplace sorts this out.

By Daniel Tunkelang

High-Class Consultant.

13 replies on “Is Spontaneity Overrated?”

Mike Blumenthal forwarded a link to this article to me, and for sure, it’s a really interesting one.

Like you, I don’t see anything wrong with a business owner asking customers to consider reviewing them. Our company acquires testimonials to be published on our website in this manner, and when a client is really happy, they are glad to oblige.

I am especially in favor of soliciting reviews because of the fact that many people are unaware that user reviews even exist. The business owner has the chance to educate his customer about this, acting, in a sense, as an entree into this world of user generated content.

But you’ve raised a good point: does lack of spontaneity diminish the value of reviews? Inasmuch as any smart SMB is only going to encourage reviews from his happy customers, then, yes…there might be something slightly artificial about a company’s profile with 100% positive reviews. It also might have the effect of distorting an SMB’s understanding of how people are actually talking about him if the majority of his reviews are ones he’s attained rather than one’s that have occurred without any action on his part. So, there is a bit of shaky ground here.

Overall, though, I think that as time goes by and more people understand that they can review businesses, there will be a healthy mix of owner-sought and voluntary reviews.

A good and thoughtful piece.

Like

A good review should help consumers find a product or service that fits their use cases for the product, and not just rate the product on a linear good/bad scale. For example, a MP3 player may have rave reviews, if one review says that you cannot resume playback in the middle of a track, I can rule it out as a player for listening to audiobooks.
Often, I go deliberately for the few one-star reviews, to see if they point out any flaws that I would consider fatal.
What I really need is a search engine that finds those product reviews that address my particular use case.

Like

The sociolinguistics literature, at least as reported in Nass and Reeves’ The Media Equation, indicates that people provide more positive feedback when asked for the feedback by the entity (not necessarily person) being evaluated. They may not have a reason to lie, but there’s an induced bias.

The “equation” in the title refers to Nass and Reeves’ and others’ findings that people interact with computers socially with many of the same patterns as they react with people.

As a search problem, many companies are interested in finding negative reviews to see if there are issues they can address.

To bring this all back to Endeca-style search, what Gregor’s saying is that we need to be able to auto-facet the reviews (by what is called “aspect” if you’re looking for papers on the topic).

Like

“Still, my hope is that consumers will start placing less stock in the aggregated opinions of anonymous strangers and shift their trust to people who are more transparent about their identities and motives. ”

A big (and growing) field for private investigators is getting the identities of people who post strong, negative opinions of businesses online — hired by the businesses of course.

This lack of “reviewer” transparency clearly has a dollar value to the businesses who are willing to pay to uncover the origin of the review.

Will there ever be a dollar value to uncover positive reviewers? Would consumers be willing to pay to figure it out through some form of an opinion marketplace?

I doubt it…but you pose an interesting question!

Like

Thanks for the comments–especially from folks who are new here!

Miriam, I imagine most businesses would prefer negative feedback in private and positive feedback in public. But I could be wrong. Indeed, responding to negative feedback in public may be a great way to win new customers. I’m curious how many businesses encourage such an open conversation.

Gregor, I agree that there’s a lot more information in a review than a star rating. You might want to take a look at http://www.buzzillions.com/.

Bob, that makes sense intuitively–and only reinforces concerns about bias. As for finding negative reviews on the open web, that seems to be one of the canonical applications of text analytics.

Lecia, you raise an important point about private investigators–and you might appreciate my blog post about public expression, liability, and anonymity. As for the opinion marketplace, I don’t see money changing hands, but rather people using their reputations as social currency. People already do that offline–why should the online world be so different?

Like

By Post Author

This is a great topic. I have a strong opinion about it but hey that’s what makes life interesting. So roll the clock back a few years when people didn’t solicit reviews for every service known to mankind like they do now. What you had was Digg where 50% of the content was submitted by just 100 users who sometimes were caught selling their ability to drive traffic to the highest bidder. if you ask your friends who posts reviews and who reads reviews, you’ll find that nowadays almost everyone read reviews but that only 1 in 15 ever post a review. It takes time and most people are content to consume information and not create it.

So you have a sample bias to some extent because you only get people who are the “talkers” or people who super excited or super upset about products to write reviews. So we encourage our retailers to email their consumers sometime after they have purchased the product. This democratizes the review writing process by soliciting people to write reviews who normally wouldn’t. In addition, it actually generates a critical mass of reviews. Without soliciting reviews, retailers would have one-tenth of the reviews they collect and product reviews would be the Betamax of online retail (only high loyalty retailers would ever get reviews).

In addition, most all of our retailers don’t use coupons or incentives to write reviews. The more the incentive, usually the lower quality the review will be (we do offer a $1,000 prize to one reviewer in our entire network but that doesn’t incent the one-liners that a $20 coupon will). So the survey request is sent to all purchasers and the amount of reviews goes up. In addition we mark these reviewers as Verified Buyers. Though everyone is interested in who the reviewer is, in the product world that information is not as useful. Though users can eat at dozens of restaurants in a year, most people only buy a fridge once every 5-20 years, a digital camera every 2-4 years and so on. So if you see a user who has reviewed tons of cameras then that user is either a super user (whose comments should be read in the light) or someone who’s receiving product from the manufacturer to do a spot review like the way CNet does. These spot reviews are very different from reviews of consumers who actually use the product. Consumers notice that cameras are slippery and experts often focus on specs and features.

In any case, without solicitation, you would not have the volume of Verified Buyer reviews that people find so useful.

As for negative reviews, they are the best reviews of all since they are more predictive than positive reviews (especially negative reviews that are by Verified Buyers). Unfortunately people are too positive in their reviews. I read tons of 4 star reviews that pan a product. In a recent internal survey we found that people thought a product was bad if it had 2 stars or lower. In reality, bad products start around 3.8 stars and lower (though each category has a very distinct average rating).

In the interest of disclosure, I run the Buzzillions.com site and am on the founding team of PowerReviews, a company that collects product reviews on behalf of online retailers.

Like

Jim, thanks for the comment. And I assume you noticed my pub for Buzzillions.com in a previous comment.

I agree that some sort of sample bias is inevitable. But that’s not so bad if, as a user, I can develop a consistent model of that bias and calibrate for it. The problem is when different products reflect different sample biases for their reviews (e.g., because some businesses solicit reviews) and I as a consumer can’t tell which sample biases apply to which products.

Like

By Post Author

[…] should be no surprise to folks who have read my recent posts about distributed trust networks and solicited reviews. Anyway, I decided to go straight to the source and persuaded Unvarnished CEO Peter Kazanjy to […]

Like

The more useful distinction may be objective vs. subjective content. I imagine that, on average, reviews of businesses (and of people) are more subjective than those of things. But I don’t think there’s a clear dividing line.

Like

By Post Author

Wow, unless you log on to facebook all day long, I don’t think facebook reminds me about my friends b’day. It only displays it on the sidebar, but I don’t think I notice that too. So yeah, I don’t really care about my friend’s bday.

Like

Comments are closed.