Categories
Uncategorized

How Recommendation Engines Quash Diversity

As regular readers here know, I have strong opinions about how recommendation engines should work. So does Daniel Lemire, a regular reader who specifically argues in favor of diversity in recommender systems. Well, this post is for him and all who share his concern.

In “Does Everything Really Sound Like Coldplay?“, Vegard Sandvold explains:

When a lot of people (who may otherwise have very diverse tastes in music) listen to Coldplay, Coldplay becomes very well connected with a lot of other artists, and also becomes a hub in what is known as a small-world network. Such networks are the basis for social recommendations. Oscar shows that these hubs are indeed the most popular artists, who again gets recommended more often than others. That is why all roads lead to Radiohead.

The cited Oscar is Oscar Celma, who recently defended his PhD thesis on “Music Recommendation and Discovery In The Long Tail”. I’ve only had a chance to skim the abstract, but I’m optimistic that people are giving more thought to the limitations of current recommendation systems. Of course, I’d really like it if they embraced transparency and exploration. But emphasizing diversity is certainly a worthy endeavor.

Or maybe everything really does sound like Coldplay…

By Daniel Tunkelang

High-Class Consultant.

16 replies on “How Recommendation Engines Quash Diversity”

This topic really deserves a full blog post. I’ll get on that.

But until then, Daniel, check out section 5.2.2. We wanted to see how well two explicitly collaborative users could do in terms of finding relevant information, that no one else had found. In other words, non popular, but still relevant, content. And the short of it is that explicit collaboration really got at relevant information that was not found by anyone else.

More importantly, this difference was more pronounced for sparse information needs, or queries/topics in which relatively little relevant information was available. It seems like that’s a desirable property.. that you have a technique you can use when relevant information isn’t as plentiful and easy to come by.

Like

“Coldplay becomes very well connected “…

While recommendation engines will always have limitations I think the “posioned by popularity” theory outlined above is an unfair diagnosis.

Any recommendation engine worth its salt would compensate for popularity ( the same way a decent search engine will not necesarily put any statistical significance in the similarly highly-connected word “the”). Identifying significant correlations requires more analysis than simply counting popularity.

Like

I am sure they do, and that’s avoids recommending the #1 song or artist to everyone. But I suspect that, below some threshold, it’s hard to collect enough statistical significance from the data–in which case you have to be in the head to qualify.

To use your search engine example, no one will push you towards stop words, but the words with the most discriminatory value are ones with medium idf scores that are still in the head rather than the long tail of the vocabulary.

Like

By Post Author

@MarkH
I admit that my explanation of collaborative filtering, which you’re refering to, is overly simplistic. For sure, social recommenders must compensate for popularity bias. But I believe that feedback loops reinforcing already popular items are difficult to avoid.

A particularly interesting nuggets of information from Oscar’s thesis (from my post) – it takes on average 5 links/clicks/jumps to reach from the head to the long tail with a social recommender, while it takes just 2 for expert and content-based recommenders.

Like

So not “*all* roads lead to Coldplay”.

There are a large number of very rarely-travelled roads that may have to resort to Coldplay but there are a big wedge of reasonably well-travelled roads that can suggest useful detours.

Seems a reasonable state of affairs to me. I’m not sure how you realistically expect to build a navigation system that works for the very rarely travelled roads.

Like

Actually, thinking further. An engine should know when it has insufficient evidence to make a recommendation and not resort to the Coldplay effect.
No one should have to suffer Coldplay unnecessarily 🙂

Like

I think that last point is what’s key. I wonder if a big problem with both recommendation engines and search engines is their lack of humility / self-awareness: they don’t know when do say “I don’t know.”

Like

By Post Author

When you don’t know what to get you get what everybody get. 🙂 For the recommendation engine perspective it’s just a question of transparency (as long as the recommendation engine know where are the tradeofs). A best seller list is a kind of recommendation system also. I think when chances are “equal” the recommendation provider should promote best seller stuff way before long tail stuff if he want’s to make money and play safely.

Like

If you’re playing by relevance, by which I mean give the user the recommendation most likely to be sound, then it in many cases makes sense to give them ColdPlay (or the Jonas Brothers, or Frank Sinatra, depending on their first few ratings).

I like the finding new items problem. It focuses on recall, which we’ve always been arguing is important for many kinds of search. (It’s very hard to balance with knowing when to stop, though, which on paging interfaces is up to the user anyway.) I’ve often argued for diversity in rankings. I think Amazon does much better than Netflix at this, for instance.

Like

[…] quality, my own conformity of musical taste, or skew on the part of the recommendation system (cf. does everything sounds like Coldplay?). Still, I’m quite sure that I’m not favoring music based on prior knowledge of its […]

Like

Comments are closed.