The Noisy Channel


Google Tech Talk: Reconsidering Relevance

January 8th, 2009 · 20 Comments · General

I’m still waiting for Google to post a video of the talk to YouTube (the wait is over!), but in the meantime I’ve posted the slides to Scribd and SlideShare. I’ve included speaker notes designed to make the talk completely self-contained.

I’d like to add that my hosts at Google NYC were very gracious, particularly considering that my material was more than a little critical of their approach to search and information retrieval.

Here is the abstract again as a reminder:

We’ve become complacent about relevance. The overwhelming success of web search engines has lulled even information retrieval (IR) researchers to expect only incremental improvements in relevance in the near future. And beyond web search, there are still broad search problems where relevance still feels hopelessly like the pre-Google web.

But even some of the most basic IR questions about relevance are unresolved. We take for granted the very idea that a computer can determine which documents are relevant to a person’s needs. And we still rely on two-word queries (on average) to communicate a user’s information need. But this approach is a contrivance; in reality, we need to think of information-seeking as a problem of optimizing the communication between people and machines.

We can do better. In fact, there are a variety of ongoing efforts to do so, often under the banners of “interactive information retrieval”, “exploratory search”, and “human computer information retrieval”. In this talk, I’ll discuss these initiatives and how they are helping to move “relevance” beyond today’s outdated assumptions.

20 responses so far ↓

  • 1 Christina Pikas // Jan 8, 2009 at 2:49 pm

    the notes are there under the “notes” tab, if you view on slideshare (aren’t those your notes?)

  • 2 Daniel Tunkelang // Jan 8, 2009 at 3:04 pm

    Those are my notes, and I’m glad those are visible. I thought you’d have to download the presentation to see them. Thanks!

    Unfortunately, there seem to be some quirks in the embedded presentation. I’ve fixed the quirks on SlideShare, but they don’t seem to be working in the embedded presentation. So I’ve removed the embed and urge you all to view the presentation on SlideShare. My apologies for the inconvenience.

  • 3 jeremy // Jan 8, 2009 at 5:39 pm

    What are some of the comments that you got? Yes, they were gracious, but were the Googlers generally interested, dismissive, or uncomprehending of your points?

  • 4 Daniel Tunkelang // Jan 8, 2009 at 5:55 pm

    Questions covered computation (I explained that summary analysis required different data access patterns than filtering and sorting) ; how to recognize the need for disambiguation (I talked about our innovative use of relative entropy in a set retrieval context); and how image search fits in to the HCIR framework (I pointed to Modista as an example of exploratory image search).

    The audience certainly wasn’t dismissive, and I think they were quite interested. They seemed to understand me, but I didn’t quiz them afterward. Not even on the partially-ordered set question. 🙂

    Also, I found out that their VP of Research was in the audience.

  • 5 Google and Search Relevance « NP-Harder // Jan 9, 2009 at 1:46 am

    […] Daniel Tunkelang gave a nice talk at Google yesterday on search relevance.  He posted the slides online.  I wasn’t at the talk, but know Daniel from some previous run-ins. […]

  • 6 Max L. Wilson // Jan 9, 2009 at 6:41 am

    Im looking forward to seeing the video. i imagine its gonna be a lot more enjoyable than skipping through the slides alone, having spoken to you many times in person! Its been interesting to see the little bits they’ve been doing to make it more interactive, like in the image type that you blogged about recently.

    Live Image search is really interesting, aside from having the type of image thing like Google just put in, they have a scratch pad to keep pictures they are interested in. you played with it?

  • 7 Making Whuffie | The Noisy Channel // Jan 9, 2009 at 11:33 am

    […] RSS   ← Google Tech Talk: Reconsidering Relevance […]

  • 8 jeremy // Jan 9, 2009 at 11:45 am

    I agree, Max.. it’s interesting to see the interactivity baby steps that they’ve been taking lately.

    But they’re still only baby steps. And there is so much more that both could and should be done.

    So I know Google famously states how clean and simple they like to keep their interface — almost to the point of obsession. What it appears from the outside is that Google thinks that clean interfaces are more important than interactive interfaces. This is judging purely from their public statements and, well, the interface itself.

    What I was curious about gleaning from Daniel is whether, once he actually got there and talked with people, he also found that to be true. Were there any people in the audience, for example, who raised their hands and said anything to the effect of, “yes, it would be nice to consider other aspects of relevance, to consider faceted retrieval, to consider exploratory retrieval, to consider broader HCI-IR communication paradigms. But in order to implement anything that goes beyond a 2-word text input line, that would involve “cluttering” our interface. So we won’t do it.

    I’m curious about whether this issue was ever raised.

    And even if it wasn’t, I’m curious from Daniel about what his reaction would have been, anyway. What would you have said, Daniel, if someone had said that a clean interface is more important than an interactive interface? And no fair saying that you’d design a “clean, interactive” interface. You’ve watched Google over the past decade+. There really is not much that they will let be added, at all. So how do you overcome that?

  • 9 Daniel Tunkelang // Jan 10, 2009 at 1:07 am

    I’ve played a little with Live Image search, but it didn’t support exploration for many any more than Google. And images are a domain where I often want support for exploration.

    As for Google’s insistence that simplicity trumps power in the interface, no one in the audience brought it up, but I’ve heard it from Googlers before and I alluded to it myself in the talk. And I tried to answer it as follows: make the complexity of the interaction a function of your confidence in system’s understanding of the user’s information need. It’s what we do as human beings.

  • 10 Talking with Daniel Tunkelang about Endeca and Search | Paul Miller // Jan 21, 2009 at 3:25 pm

    […] to give one of Google’s Tech Talks at their New York offices, and with a title of ‘Reconsidering Relevance‘ he (nervously!) set about challenging some of the search behemoth’s approaches. We […]

  • 11 A Warm Reception for “Reconsidering Relevance” | The Noisy Channel // Jan 22, 2009 at 12:12 am

    […] am proud to report that the “Reconsidering Relevance” presentation has been enjoying  a warm […]

  • 12 The Information Triumverate | The Noisy Channel // Jan 24, 2009 at 4:34 pm

    […] that the web has to offer. In fact, that was a major subtext of my recent presentation on reconsidering relevance. I’m not sure that the dominance of the web is itself a problem, but that’s because I […]

  • 13 Obstreperous Minnesota « Network(ed)News // Jan 25, 2009 at 4:06 pm

    […] services to serve us really well when we don’t know we care about something (see especially Dan Tunkelang on HCIR (@dtunkelang)). That something might be fuzzy or specific, obscure or popular, subject to […]

  • 14 Daniel Tunkelang // Feb 1, 2009 at 8:03 am

    Now on YouTube!

  • 15 Remembering What Works « Infovark // Feb 2, 2009 at 9:28 pm

    […] discoverability problem from an information-only perspective i.e. the way Google fixes the problem doesn’t appear to be ideal within the enterprise. People can misinterpret information and corrupt its original intent. […]

  • 16 What the Structure of Content Means for Context « Network(ed)News // Mar 19, 2009 at 9:46 am

    […] then came the interwebs and google and adversarial search, which foisted on journalists the great tyrrany of pageviews. […]

  • 17 Google Offers “More And Better Search Refinements” | The Noisy Channel // Mar 24, 2009 at 12:31 pm

    […] like to get excited about Google embracing HCIR, especially after they were so kind as to let me lecture them about it. And perhaps I’m being too harsh a critic. Their post concludes: Even if you […]

  • 18 WTF! @ k: Measuring Ineffectiveness // Aug 20, 2012 at 6:01 am

    […] between a document and a search query, would  be sufficient to determine relevance was always a crude approximation, but now the diversity of a global user base makes this approximation even […]

  • 19 Relevance of Search Results | Web Search Guide and Internet News // Dec 29, 2012 at 1:29 am

    […] Google Tech Talk: Reconsidering Relevance by Daniel Tunkelang, The Noisy Channel (Jan 8, 2009) […]

  • 20 Tagging | Legal Index // Feb 15, 2013 at 7:11 pm

    […] of Relevance Google Tech Talk: Reconsidering Relevance by Daniel Tunkelang, The Noisy Channel (Jan 8, […]

You must log in to post a comment.

Clicky Web Analytics