The Noisy Channel


Ellen Voorhees defends Cranfield

April 17th, 2008 · 3 Comments · General

I was extremely flattered to receive an email from Ellen Voorhees responding to my post about Nick Belkin’s keynote. Then I was a little bit scared, since she is a strong advocate of the Cranfield tradition, and I braced myself for her rebuttal.

She pointed me to a talk she gave at the First International Workshop on Adaptive Information Retrieval (AIR) in 2006. I’d paraphrase her argument as follows: Nick and others (including me) are right to push for a paradigm that supports AIR research, but are being naïve regarding what is necessary for such research to deliver effective–and cost-effective–results. It’s a strong case, and I’d be the first to concede that the advocates for AIR research have not (at least to my knowledge) produced a plausible abstract task that is amenable to efficient evaluation.

To quote Nick again, it’s a grand challenge. And Ellen makes it clear that what we’ve learned so far is not encouraging.

3 responses so far ↓

  • 1 Daniel Tunkelang // Apr 17, 2008 at 9:40 am

    More commentary on this debate from my friend Jeff Dalton at Search Engine Caffè.

  • 2 Saracevic on Relevance and Interaction | The Noisy Channel // Oct 29, 2008 at 11:23 am

    […] that they work. Not that there isn’t plenty of room for improvement, but the unmet challenge, as Ellen Voorhees makes clear, is evaluation. We need to address Nick Belkin’s grand challenge and establish a paradigm […]

  • 3 Information Retrievability | The Noisy Channel // Sep 26, 2009 at 2:15 pm

    […] measures still have their place. In fact, I’m looking forward to NIST’s Ellen Voorhees defending Cranfield next month to an HCIR crowd that is, for the most part, deeply suspicious of […]

Clicky Web Analytics