Comments on: Nick Belkin at ECIR ’08

By: Human-Computer Information Retrieval in Layman’s Terms | The Noisy Channel

Mon, 28 Sep 2009 02:12:26 +0000

[...] the occasional ideas I am fortunate enough to conceive–are worthy of broader consideration. I started blogging in order to bring greater visibility to HCIR–to convince people that the choice between human [...]

By: Saracevic on Relevance and Interaction | The Noisy Channel

Saracevic on Relevance and Interaction | The Noisy Channel — Tue, 11 Aug 2009 17:43:12 +0000

[...] but the unmet challenge, as Ellen Voorhees makes clear, is evaluation. We need to address Nick Belkin’s grand challenge and establish a paradigm suitable for evaluation of interactive IR [...]

By: Norbert Fuhr’s Probability Ranking Principle for Interactive Information Retrieval | The Noisy Channel

Mon, 10 Aug 2009 13:15:51 +0000

[...] even heard of this paper, despite Nick Belkin citing it in the ECIR 2008 keynote that inspired my first blog post! In my defense, the citation is a single sentence that offers more of a tease than an explanation [...]

By: SIGIR 2009: Day 2, Interactive Search Session | The Noisy Channel

SIGIR 2009: Day 2, Interactive Search Session | The Noisy Channel — Mon, 27 Jul 2009 01:13:11 +0000

[...] search sessions were the most interesting, and this one was no exception. Ironically, even though many of us (myself included) feel that interaction is marginalized within the SIGIR conference and even the [...]

By: Daniel Tunkelang

Daniel Tunkelang — Fri, 30 May 2008 11:36:00 +0000

Thanks to Jeff for posting a link to Nick's talk, recently published in the SIGIR Forum.

By: fd

fd — Sat, 12 Apr 2008 19:21:00 +0000

Accepting nominations for SIGIR “Unfiltered Crank” award.

By: Daniel Tunkelang

Daniel Tunkelang — Fri, 11 Apr 2008 19:16:00 +0000

Jon, you’re right–I am guilty of oversimplification. I think the crux of the problem, which is hardly unique to IR, is that it’s easier–at least in the academic world–to propose solutions that incremental improve relative to an well-accepted problem statement than it is to propose changing the problem statement. This conservative attitude has some merit: it certainly filters out a lot of cranks. But it also discourages radical innovation.

By: Jon Elsas

Jon Elsas — Thu, 10 Apr 2008 13:37:00 +0000

Daniel — I think the divergence in the two fields is quite a bit more complicated than simply a focus on repeatable experiments vs. user studies. There’s baggage included with IS’s close ties to LS, there’s lots of appeal for CS students to manipulate formulae to eek out incremental improvements, the nuts-and-bolts software engineering expertise doesn’t really exist in many IS departments, and the list goes on. Even though I came through an IS department on my way to CS, I haven’t completely wrapped my head around all the reasons *why* this divide exists.

It is great to see venues like SIGIR and ECIR really elevating the role and visibility of good IS research in the IR community — the most recent best papers at both conferences are perfect examples.

(and, as a side note, IMO if the conclusions of your user study aren’t repeatable, then you’re missing something in your study design or analysis… but that’s probably another discussion)

By: Daniel Tunkelang

Daniel Tunkelang — Thu, 10 Apr 2008 11:35:00 +0000

In fairness to the SIGIR community, the divergence in methodology between the information retrieval and information science communities has made it very hard for the two to collaborate. IR researchers want repeatable experiments, which information and library scientists emphasize user studies that are inherently not repeatable. It is, as Nick said, a grand challenge.

By: FD

FD — Wed, 09 Apr 2008 15:10:00 +0000

From the perspective of a relatively green IR researcher, the IR community started as a combination of computer science and information/library science researchers. Nick’s work—past and present—and the work of information and library scientists is extremely relevant to and, in my opinion, overlooked by the computer science IR community. I’d give my left toe for SIGIR to drop five papers which claim marginal DCG improvement—or better yet every “computational advertising” paper—for an equal number information science papers.

By: Jon Elsas

Jon Elsas — Wed, 09 Apr 2008 14:18:00 +0000

Daniel — thanks for posting this. Sounds like a fascinating talk.