Beyond that, I haven’t given it tons of thought. But I do like the concept around how we increase the ROI of searchers’ activity.

Indeed, I see informational searches as an area where Google often falls short, and hence one where Microsoft and others would do well to focus efforts. I’m glad to see that you and your colleagues are thinking about this challenge, and looking forward to the solutions you bring to bear on it!

Any thoughts on how to evaluate “value per minute spent searching”?

Second, let me echo Daniel’s first comment. I didn’t mean to imply that search engines should always return a single answer or even that an engine should ‘get it right’ on the first try. As we know, searches are often classed into three major categories of queries: navigational, transactional, and informational. The nav queries should generally be short, to-the-point, and as frictionless as possible as there is generally an ‘answer’ to your question that is more accurate or relevant than others. The informational and transactional (esp the informational) often tend to be more exploratory and today require multiple queries and refinements as people attempt to gather enough info to make a decision. Informational queries tend to be lengthy and complex and we find that people often come back to those queries over the course of time (think days or weeks even). The challenge of making informational and transactional classes of queries return greater “value per minute spent searching” is one we’re really looking at.

So no, I don’t believe that an engine will become prescient enough to return a single, one-query-and-out answer for all queries – primarily because some of the hardest “questions” often can’t be addressed with such an experience. Engines should get out of the way when they can and guide searching to help people make decisions when they need it.

I know that in my case, a better measure of search success would be how many iterations of refining the query string I have to go through, before I find what I am looking for.

To start with the idea was to retrieved the correct document by using a multitude of techniques that were text based, be it NLP, classifiying and so on.

Now we that we have had a lot of experience testing these systems, with billions of queries a day and that as data increases (not only in size but number of different structures…), we see that part of the problem is actually the user.

From tests I have been running on user queries both in natural language systems and on keyword queries, I can say that it appears that people compromise with the results they have been served and the information they were originally after.

Also you can see evidence of them not really knowing what they want but knowing they have found it when they stumble upon it. There are many other examples.

“The job of search engines is to help users figure out what they want and ultimately find it. ”

Absolutely. I think that query intent is a very valuable and interesting research area right now.

The job of search engines is to help users figure out what they want and ultimately find it. Focusing on the one-shot case can only take us so far.