Comments on: Recommending Diversity

By: jeremy

jeremy — Tue, 03 Mar 2009 07:54:33 +0000

Busy letting the boa constrictors of navigational, known-item search swallow the elephants of relevance?

By: Daniel Tunkelang

Daniel Tunkelang — Tue, 03 Mar 2009 04:52:35 +0000

“Don’t you see–I am very busy with matters of consequence!”

By: jeremy

jeremy — Tue, 03 Mar 2009 03:12:57 +0000

And how does the story continue?

—-
I was very surprised to see a light break over the face of my young relevance assessor:

“That is exactly the way I wanted it! Do you think that this diversity will have to have a great deal of click-throughs?”

“Why?”

“Because where I live everything must be relevant…”

“There will surely be enough click-throughs from it,” I said. “It is a very large diversity that I have given you.”

He bent his head over the SERP.

“Not so large that–Look! It has gone to sleep…”
—-

Non?

By: Daniel Tunkelang

Daniel Tunkelang — Mon, 02 Mar 2009 23:58:40 +0000

"This is the black box. The diversity you asked for is inside." With apologies to Saint Exupéry.

By: jeremy

jeremy — Mon, 02 Mar 2009 23:16:30 +0000

I’ve talked with insiders at Google, and they claim that they strive to automatically generate diversity in their rankings. I believe what I’ve heard, but Google is still such a black box that I have no idea how to evaluate or understand exactly how much diversity is being created.

By: When Recommendations Become a Problem - Things On Top

When Recommendations Become a Problem - Things On Top — Mon, 02 Mar 2009 20:39:44 +0000

[...] so we cover up by design systems that encourage exploration and discovery. We strive for transparency and diversity, designing an experience for the eager and motivated user. But for some of us, the Paradox of [...]

By: Daniel Tunkelang

Daniel Tunkelang — Sun, 07 Dec 2008 04:14:27 +0000

Indeed, it’s hard to do research on a problem that doesn’t have accepted measures for evaluation.

As for putting a weight on each attribute, I suppose you can turn anything into a vector space (e.g., by treating each manufacturer as a binary value). But distance measures can do funny things in high-dimensional spaces.

By: Max L. Wilson

Max L. Wilson — Fri, 05 Dec 2008 13:43:20 +0000

Diversity in results seems surprisingly underrepresented in both academia and industry. Saw a couple of posters at SIGIR07 on it, but they were largely calculating the diff between 2 pages, and not reporting what made each unique! It seems all the more important in faceted search, for example, where every result is often equally related to the selections made by the user. There was a group that presented at SearchSolutions2008 who weight every attribute given to an object in faceted search. Im not sure how they can put a weighting on things like the manufacturer etc.

By: Daniel Tunkelang

Daniel Tunkelang — Tue, 18 Nov 2008 02:52:26 +0000

True. But the nice thing about diversity is that it’s an unsupervised measure. I’m a big fan of such unsupervised set measures, like the query-less version of query clarity that we use at Endeca. There really should be unsupervised analogs of the precision vs. recall trade-off that can be used in the absence of relevance assessments.

By: Daniel Lemire

Daniel Lemire — Mon, 17 Nov 2008 21:13:04 +0000

One nice thing about recall/precision, is that to compute them, you just need to know whether document x is relevant or not to query y. Measuring diversity requires that you also know the “distance” between document x1 and x2. It is not hard, but it makes up an extra layer of math.