Comments on: How Recommendation Engines Quash Diversity

By: Beyond Social Currency

Beyond Social Currency — Tue, 06 Jul 2010 20:52:24 +0000

[...] quality, my own conformity of musical taste, or skew on the part of the recommendation system (cf. does everything sounds like Coldplay?). Still, I’m quite sure that I’m not favoring music based on prior knowledge of its [...]

By: Google Follow Finder

Google Follow Finder — Thu, 15 Apr 2010 00:15:46 +0000

[...] a bit of an “everything sounds like Coldplay” effect (e.g., @fredwilson shows up in a lot of the searches I tried), but overall I’m [...]

By: Tuning in to Google Music Search | The Noisy Channel

Tuning in to Google Music Search | The Noisy Channel — Thu, 29 Oct 2009 17:10:25 +0000

[...] my friend (and Princeton sociologist) Matt Salganik and his former advisor Duncan Watts), but even recommendation engines quash diversity. Google really can’t make things that much [...]

By: Bob Carpenter

Bob Carpenter — Fri, 27 Feb 2009 17:16:05 +0000

If you’re playing by relevance, by which I mean give the user the recommendation most likely to be sound, then it in many cases makes sense to give them ColdPlay (or the Jonas Brothers, or Frank Sinatra, depending on their first few ratings).

I like the finding new items problem. It focuses on recall, which we’ve always been arguing is important for many kinds of search. (It’s very hard to balance with knowing when to stop, though, which on paging interfaces is up to the user anyway.) I’ve often argued for diversity in rankings. I think Amazon does much better than Netflix at this, for instance.

By: alltoute

alltoute — Wed, 25 Feb 2009 17:48:14 +0000

When you don’t know what to get you get what everybody get. For the recommendation engine perspective it’s just a question of transparency (as long as the recommendation engine know where are the tradeofs). A best seller list is a kind of recommendation system also. I think when chances are “equal” the recommendation provider should promote best seller stuff way before long tail stuff if he want’s to make money and play safely.

By: jeremy

jeremy — Wed, 25 Feb 2009 15:58:27 +0000

See also Greg Linden’s discussion of avoiding the “Harry Potter” problem:

http://glinden.blogspot.com/2006/03/early-amazon-similarities.html

Btw, Daniel, I switched my blog domain name. Try irgupf.com.

By: Daniel Tunkelang

Daniel Tunkelang — Wed, 25 Feb 2009 14:06:01 +0000

I think that last point is what’s key. I wonder if a big problem with both recommendation engines and search engines is their lack of humility / self-awareness: they don’t know when do say “I don’t know.”

By: MarkH

MarkH — Wed, 25 Feb 2009 14:03:46 +0000

Actually, thinking further. An engine should know when it has insufficient evidence to make a recommendation and not resort to the Coldplay effect.
No one should have to suffer Coldplay unnecessarily

By: Daniel Lemire

Daniel Lemire — Wed, 25 Feb 2009 14:01:00 +0000

Thanks for this great post.

I’m glad I discovered Jeremy’s Algorithmic Mediation for Collaborative Exploratory Search. That is the right general direction, I believe.

By: MarkH

MarkH — Wed, 25 Feb 2009 13:58:27 +0000

So not “*all* roads lead to Coldplay”.

There are a large number of very rarely-travelled roads that may have to resort to Coldplay but there are a big wedge of reasonably well-travelled roads that can suggest useful detours.

Seems a reasonable state of affairs to me. I’m not sure how you realistically expect to build a navigation system that works for the very rarely travelled roads.

By: Vegard Sandvold

Vegard Sandvold — Wed, 25 Feb 2009 13:50:56 +0000

@MarkH
I admit that my explanation of collaborative filtering, which you’re refering to, is overly simplistic. For sure, social recommenders must compensate for popularity bias. But I believe that feedback loops reinforcing already popular items are difficult to avoid.

A particularly interesting nuggets of information from Oscar’s thesis (from my post) – it takes on average 5 links/clicks/jumps to reach from the head to the long tail with a social recommender, while it takes just 2 for expert and content-based recommenders.

By: Daniel Tunkelang

Daniel Tunkelang — Wed, 25 Feb 2009 13:43:10 +0000

Gene, Jeremy, I’ll have to read that paper, since I haven’t looked at your collaborative search work in a while.

By: Daniel Tunkelang

Daniel Tunkelang — Wed, 25 Feb 2009 13:41:27 +0000

I am sure they do, and that’s avoids recommending the #1 song or artist to everyone. But I suspect that, below some threshold, it’s hard to collect enough statistical significance from the data–in which case you have to be in the head to qualify.

To use your search engine example, no one will push you towards stop words, but the words with the most discriminatory value are ones with medium idf scores that are still in the head rather than the long tail of the vocabulary.

By: MarkH

MarkH — Wed, 25 Feb 2009 13:27:58 +0000

“Coldplay becomes very well connected “…

While recommendation engines will always have limitations I think the “posioned by popularity” theory outlined above is an unfair diagnosis.

Any recommendation engine worth its salt would compensate for popularity ( the same way a decent search engine will not necesarily put any statistical significance in the similarly highly-connected word “the”). Identifying significant correlations requires more analysis than simply counting popularity.

By: jeremy

jeremy — Wed, 25 Feb 2009 05:01:57 +0000

This topic really deserves a full blog post. I'll get on that. But until then, Daniel, check out section 5.2.2. We wanted to see how well two explicitly collaborative users could do in terms of finding relevant information, that no one else had found. In other words, non popular, but still relevant, content. And the short of it is that explicit collaboration really got at relevant information that was not found by anyone else. More importantly, this difference was more pronounced for sparse information needs, or queries/topics in which relatively little relevant information was available. It seems like that's a desirable property.. that you have a technique you can use when relevant information isn't as plentiful and easy to come by.

By: Gene Golovchinsky

Gene Golovchinsky — Wed, 25 Feb 2009 02:28:12 +0000

This is the sort of thing we were talking about when contrasting recommender systems with collaborative search in our SIGIR paper. I am sure Jeremy will have more to add! See Algorithmic Mediation for Collaborative Exploratory Search.