Categories
General

Google Tech Talk: Reconsidering Relevance

http://static.slideshare.net/swf/ssplayer2.swf?doc=reconsidering-relevance-1231426605583628-1&stripped_title=google-tech-talk-reconsidering-relevance-presentation

I’m still waiting for Google to post a video of the talk to YouTube (the wait is over!), but in the meantime I’ve posted the slides to Scribd and SlideShare. I’ve included speaker notes designed to make the talk completely self-contained.

I’d like to add that my hosts at Google NYC were very gracious, particularly considering that my material was more than a little critical of their approach to search and information retrieval.

Here is the abstract again as a reminder:

We’ve become complacent about relevance. The overwhelming success of web search engines has lulled even information retrieval (IR) researchers to expect only incremental improvements in relevance in the near future. And beyond web search, there are still broad search problems where relevance still feels hopelessly like the pre-Google web.

But even some of the most basic IR questions about relevance are unresolved. We take for granted the very idea that a computer can determine which documents are relevant to a person’s needs. And we still rely on two-word queries (on average) to communicate a user’s information need. But this approach is a contrivance; in reality, we need to think of information-seeking as a problem of optimizing the communication between people and machines.

We can do better. In fact, there are a variety of ongoing efforts to do so, often under the banners of “interactive information retrieval”, “exploratory search”, and “human computer information retrieval”. In this talk, I’ll discuss these initiatives and how they are helping to move “relevance” beyond today’s outdated assumptions.

By Daniel Tunkelang

High-Class Consultant.

20 replies on “Google Tech Talk: Reconsidering Relevance”

the notes are there under the “notes” tab, if you view on slideshare (aren’t those your notes?)

Like

Those are my notes, and I’m glad those are visible. I thought you’d have to download the presentation to see them. Thanks!

Unfortunately, there seem to be some quirks in the embedded presentation. I’ve fixed the quirks on SlideShare, but they don’t seem to be working in the embedded presentation. So I’ve removed the embed and urge you all to view the presentation on SlideShare. My apologies for the inconvenience.

Like

By Post Author

What are some of the comments that you got? Yes, they were gracious, but were the Googlers generally interested, dismissive, or uncomprehending of your points?

Like

Questions covered computation (I explained that summary analysis required different data access patterns than filtering and sorting) ; how to recognize the need for disambiguation (I talked about our innovative use of relative entropy in a set retrieval context); and how image search fits in to the HCIR framework (I pointed to Modista as an example of exploratory image search).

The audience certainly wasn’t dismissive, and I think they were quite interested. They seemed to understand me, but I didn’t quiz them afterward. Not even on the partially-ordered set question. 🙂

Also, I found out that their VP of Research was in the audience.

Like

By Post Author

Im looking forward to seeing the video. i imagine its gonna be a lot more enjoyable than skipping through the slides alone, having spoken to you many times in person! Its been interesting to see the little bits they’ve been doing to make it more interactive, like in the image type that you blogged about recently.

Live Image search is really interesting, aside from having the type of image thing like Google just put in, they have a scratch pad to keep pictures they are interested in. you played with it?

Like

I agree, Max.. it’s interesting to see the interactivity baby steps that they’ve been taking lately.

But they’re still only baby steps. And there is so much more that both could and should be done.

So I know Google famously states how clean and simple they like to keep their interface — almost to the point of obsession. What it appears from the outside is that Google thinks that clean interfaces are more important than interactive interfaces. This is judging purely from their public statements and, well, the interface itself.

What I was curious about gleaning from Daniel is whether, once he actually got there and talked with people, he also found that to be true. Were there any people in the audience, for example, who raised their hands and said anything to the effect of, “yes, it would be nice to consider other aspects of relevance, to consider faceted retrieval, to consider exploratory retrieval, to consider broader HCI-IR communication paradigms. But in order to implement anything that goes beyond a 2-word text input line, that would involve “cluttering” our interface. So we won’t do it.

I’m curious about whether this issue was ever raised.

And even if it wasn’t, I’m curious from Daniel about what his reaction would have been, anyway. What would you have said, Daniel, if someone had said that a clean interface is more important than an interactive interface? And no fair saying that you’d design a “clean, interactive” interface. You’ve watched Google over the past decade+. There really is not much that they will let be added, at all. So how do you overcome that?

Like

I’ve played a little with Live Image search, but it didn’t support exploration for many any more than Google. And images are a domain where I often want support for exploration.

As for Google’s insistence that simplicity trumps power in the interface, no one in the audience brought it up, but I’ve heard it from Googlers before and I alluded to it myself in the talk. And I tried to answer it as follows: make the complexity of the interaction a function of your confidence in system’s understanding of the user’s information need. It’s what we do as human beings.

Like

By Post Author

Comments are closed.