Comments on: Catching Up With Hunch

By: Daniel Tunkelang

Daniel Tunkelang — Wed, 13 May 2009 23:15:14 +0000

Looking at a few measures, but the one that I’m most interested in is based on relative entropy between the starting set and expanded set, using distributions on explicit or extracted document annotations. We also can use lexical databases like WordNet, but I (and my fellow Endecans generally) much prefer to work with statistical techniques than linguistic ones. And we don’t expect the measure to be right all the time, so we’re also looking at ways to give users not only fine-grained control of the expansion, but also previews of how expanding the query changes the results. After all, I am an HCIR zealot!

This is work in progress–I’ll say and show more when I can!

By: jeremy

jeremy — Wed, 13 May 2009 05:19:25 +0000

How are you measuring topic drift? Through non-relevance alone, or by some other metric?

By: Daniel Tunkelang

Daniel Tunkelang — Wed, 13 May 2009 01:25:35 +0000

I think most of these sites put more emphasis on precision than recall--it's very hard to make the case for the latter, not that I haven't tried. My colleagues and I are working on some approaches that focus on recall in the face of noisy annotation, exposing a number of records vs. topic drift trade-off (which is an unsupervised analog of recall vs. precision). Demo in progress!

By: Bob Carpenter

Bob Carpenter — Tue, 12 May 2009 17:39:36 +0000

The main problem I have as a consumer with both of these technologies is recall. Even highly faceted sites like newegg.com make decisions about which facets to present to the user in what order, just like a decision tree.

If I go to newegg.com, for instance, and type “2gb memory”, it by default pops me into “Guided Search”. But even if I go to “Advanced Search”, I don’t get any of the subcategories I’d expect, like the type of memory (pin config, DDR2/DDR3), the speed of the memory, etc. I can expand subcategories, useful links (?), price, and manufacturer. I have to make several choices, limiting my choice of products, until I see the “type” and “speed” and “cas latency” facets. Someone (or some algorithm) decided that manufacturer was more important than the type of memory! And I can’t click on the breadcrumb path at the top of my search to just limit myself to DDR2 and forget the other facets I’ve chosen.

The second issue is plain old database errors. With faceting at sites like NewEgg and Amazon, the DB annotation isn’t perfect, so you get false positives and false negatives on faceted search that you tend not to get with plain old text search. If I select DDR2, I have to trust that they’ve entered “DDR2″ for every product that really is DDR2. Amazon also has serious issues faceting by recording artist (granted, it’s a hard problem), but it means that I can’t click on an artist’s name and expect to see all their other albums. The faceting’s actually hurting recall.

Of course, we could hope that they do a better job on the database, selecting facets, etc., but it’s a really difficult problem when there are literally hundreds or thousands of facets in the system and humans doing data entry.

By: Christopher

Christopher — Tue, 12 May 2009 02:36:49 +0000

I was very skeptical (still am) as I don’t think Decision Trees are the best mechanism for this BUT I’m thinking like a researcher/scientist not an average consumer who is their target (I assume) or is their target for this incarnation of Hunch.

Now when I found out who their Chief Scientist is I changed my tune a bit, he has done some great stuff like using common sense to improve aspects of NL parsing while at MIT (I am highly impressed with the concept).

The other reason I am now quite interested is the thing you nailed, the availability of an an API (hello Wolfram!). While as a web site Hunch is not overly exciting to me taking (them or a 3rd party) their data into the deeper NLP realm can allow some very interesting things can be built; I think.