Diego Basch is the CEO and founder of IndexTank, a hosted search service that powers major web sites such as Reddit, Twitvid, blip.tv, as well as providing a WordPress plug-in for blogs (like this one). Diego gained his search experience working with Inktomi, where he wrote some of the world’s first web-scale link analysis algorithms. He is on a mission to make every search box blazing fast and useful.
So much brainpower is spent solving the wrong problems. The world is filled with solutions looking for problems that nobody has — as illustrated by a Google query for [stupidest inventions ever]. More often, people focus narrowly on a particular approach when they should focus on the problem the approach is intended to solve. Or they take a solution for one problem and assume it will apply to another.
Consider the emphasis that search engine developers place on relevance ranking. It is not hard to understand why web-scale search engines emphasize relevance. For example, a search on Google for [emergency locksmith] returns tens of billions of web pages, among which there are only a handful results that you want. Google must filter out the growing number of lead generation companies that spend a ton of money trying to game its results.
Most web and application developers are familiar with the concept of relevance, so they naturally assume that it should be the primary concern when they add search to their own sites or apps. When I talk to people who want full-text search for their 40,000 book titles or 100k classified ads, they ask me about all the ways they can tune relevance. But often they are focusing on a solution, rather than their fundamental problem.
Developers are (or should be!) trying to improve the user experience of their application search. Too often they wrongly assume that relevance is the single most important factor for optimizing this user experience. Let’s surface this confusion in a concrete example.
As a rock climber, once in a while I feel the aches and pains caused by the sport. As the years go by it’s very important to keep your tendons healthy if you do not want to take forced breaks (or type with one hand!). Rockclimbing.com is one of the most popular climbing sites, and I know some medical professionals who occasionally answer health-related questions there. Let’s search there for [tendon injury prevention].
In the above example, part of the problem is that the search results do not have contextual snippets. Maybe there is relevant information hiding behind a click, but the user has no way of knowing. More generally, there’s no hint as to what results could be better. Information such as score of the answer (which is available), the author’s bio (e.g. “climber, physical therapist”) would make the decision easier. If you need to click and scroll, search within the page, go back and try something else, then the search engine is wasting your time.
Which brings us the broader point: when users search, they want to spend the least amount of time possible getting to the information they want. Relevance is a means to this end. In particular, clicks and typing costs users time. That time can come from page load, rendering, repeated use of the back button, and of course typing (and re-typing) search queries.
Some application search engines really nail the user experience. Let’s say we’re looking for the movie Koyaanits-however-you-spell-it. Go to the Internet Movie Database (IMDB) and start typing k-o-y-e — and there it is, as the second result. Notice that there is a ton of irrelevant stuff around it but it doesn’t matter. I see what I want very quickly.
Hopefully these two examples serve to illustrate the broader point: search engines should not focus on relevance as an end in itself, but rather on whatever helps users find the information they want as quickly as possible. That means offering contextual snippets, instant feedback, and of course snappy response times. Give users speed, and you will make them happy.
6 replies on “Guest Post: Diego Basch on The Need for Speed”
when users search, they want to spend the least amount of time possible getting to the information they want…search engines should not focus on relevance as an end in itself, but rather on whatever helps users find the information they want as quickly as possible. That means offering contextual snippets, instant feedback, and of course snappy response times. Give users speed, and you will make them happy.
Mostly yes, plenty of room for no.
If the type of search that one is doing is navigational, fact finding, question-answering, or transactional (where can I buy something), then what you say is true.
However, there is a whole different class of searches, often described by the terms “learning”, “discovery”, “comparison”, “analysis”, “comprehension”, and “investigation”. These are known as “exploratory search” information needs. See this four-page article:
Click to access Marchionini-2006-Exploratory_Search.pdf
And if I am doing investigation, analysis, comparison, etc. then I don’t actually mind if it takes a little bit longer for my search query to execute. Snappy response times and speed are not as important as is gaining insight. And if it takes the algorithm longer to scan and compare lots of different pieces of information, so as to give me as broad of a picture as possible, then I prefer that path over one that gives me less robust, less complete information…more speedily.
Looking up “Koyuki” is not an information need that requires learning, investigation, comparison, and analysis. Therefore, speed is of the essense. I do agree; you want those searches to be snappy. However, looking up information that helps you determine the cause of declining salmon populations in Oregon is a more involved, investigative, exploratory task. And if it takes 3.4 seconds to return a good answer, rather than 0.52 seconds to return a so-so answer, I am more than completely fine with that.
Enjoyed reading your article but like Jeremy did a double take with “search engines should not focus on relevance as an end in itself, but rather on whatever helps users find the information they want as quickly as possible.”
How about relevance first followed by speed?
I can’t speak for Diego, but I think the point is that relevance ranking assumes a user model of scanning through documents in order to find content that satisfies an information need — and thus that the focus on relevance amounts to an attempt to minimize the scanning time. My interpretation of Diego’s argument is that there may be other ways to reduce the time it takes for a user to satisfy his or her information need.
And yes, I agree with Jeremy — this line of reasoning ignores exploratory search.
It never occurred to me to describe interaction design as “speed” (though that’s a good example of taxonomy vocabulary issues). But I’m 100% behind good design for showing results!
The UI changes of the last decade, including the examples in the article, have been all about better understanding of context, which works very well with Information Foraging theory. Even with exploratory search, making the relevance judgements transparent allows the searcher to adjust their strategy to account for search strengths and weaknesses.
I think you’ve hit upon the most important point when it comes to search and that’s time and ease of use. I don’t really care exactly how the search engine works as long as the information I want is in front of me as quickly and easily as possible.
I agree with Tom 100%. What matters is the results bottom line.