Author: Daniel Tunkelang

The 3rd Annual Workshop on Search in Social Media (SSM 2010) will be held on Wednesday, February 3rd at the Polytechnic Institute in Brooklyn, NY. It’s co-located with the WSDM 2010 conference on Web Search and Data Mining. As a co-organizer, I’m proud to announce that the workshop program is now online.

It features a keynote from Jan Pedersen, Chief Scientist for Core Search at Microsoft, as well as an impressive set of posters and panel sessions. Other participants include:

• Sihem Amer-Yahia, Yahoo!
• Jon Elsas, CMU
• Gene Golovchinksky, FXPAL
• David Hendi, MySpace
• LiangJie Hong, Lehigh U.
• Jeremy Hylton, Google
• Matthew Hurst, Microsoft
• Hilary Mason, bit.ly
• Richard McCreadie, U. of Glasgow
• Mor Naaman, Rutgers U.
• Meena Nagarajan, Wright State U.
• Igor Perisic, LinkedIn
• Jeremy Pickens, FXPAL

There’s still time to register if you’re interested!

The other day, I promised in a comment thread that I’d write about what I see as real use cases for real-time search. As it happens, I’m experiencing one right now.

As my wife, daughter, and I were walking home from a playground, we noticed a large number of fire trucks congregating a block away from our house. A quick search on Twitter explained what was going on, particularly by pointing us to this post on Gothamist–which as of this writing seems to be the only reporting about this incident.

I think this example tells us a lot about the utility of real-time search. Most of us don’t need real-time search to tell us about the news in Haiti, since a critical mass of major news providers is covering the story around the clock. Where real-time search matters most is at the personal level–specifically, when our personal urgency to obtain information is higher than that of the general population. In such situations, we’re willing to accept less polished–and even risk less accurate–information, particularly if the alternative is to wait until if and when news providers cover the story. At least to some extent, urgency trumps authority.

Yes, there are other use cases for conversational media like Facebook and Twitter, such as sharing the experience of watching a live event, or simply chatting with friends and strangers about arbitrary topics. But I wouldn’t consider such use of these media to be search. Real-time search, in my view, is about helping users obtain the latest information available–in accordance with their personal needs. Twitter and Google served me well today, and I’m grateful that real-time search gave me real-time peace of mind.

Earlier this week, Peter Morville and Mark Burrell presented a UIE virtual seminar on “Leveraging Search & Discovery Patterns For Great Online Experiences“. It sold out! And I thought Pete Bell and I had done well with our seminar on faceted search!

But I’m hardly surprised. Although I wasn’t able to attend it myself, I gather from Twitter and the blogosphere that it was a great presentation. I enjoyed serving as a reviewer for Peter’s new book on Search Patterns, and I contributed a bit to Endeca’s UI Design Pattern Library while I was there and Mark’s team was developing it.

In reading reactions to the seminar, I was particularly intrigued by a post entitled “Search and Browse” by Livia Labate on her fantastically named blog, “I think, therefore IA“. She raised a question that I think needs to be asked more often: when is (or isn’t) faceted search appropriate?

Her conversation with readers in a comment thread offered some possible answers:

• Faceted search helps users who think in terms of attribute specifications as filtering criteria.
• Faceted search supports search by exclusion, as opposed to by discovery.
• Faceted search requires a set of useful facets that is neither too small nor too large.

I’d like to propose my own answers. Here are the conditions for which I see faceted search being most useful:

• Faceted search supports exploratory use cases, in contrast to known-item search. For known-item search, users are better served by a search box to specify an item by name, or a non-faceted hierarchy to locate it. In contrast, faceted search optimizes for cases where users are either unsure of what they want or of how to specify it.
• Faceted search helps users who need or want to learn about the search space as they execute the search process. Facets educate users about different ways to characterize items in a collection. If users do not need or want this education, they may be frustrated by an interface that makes them do more work.
• The search space is classified using accurate, understandable facets that relate to the users’ information needs. As I’ve discussed before, data quality is often the bottleneck in designing search interfaces. Offering users facets that are either unreliable or unrelated to their needs is worse than providing no facets at all.

Given the above criteria, it’s not surprising that faceted search has been a huge success in online retail: shopping is often an exploratory learning experience, and retailers tend to have good data.

But the success of faceted search in retail overshadows other domains where faceted search may be even more valuable. My favorite example is faceted people search, most recently demonstrated by LinkedIn. I would love to see other entities (locations, businesses, etc.) receive similar treatment, at least in contexts where exploration is a common use case.

I think Livia is right to be skeptical about any interface that introduces complexity–and facets do introduce complexity. I hope that my guidelines help answer her question as to when that complexity is worthwhile and perhaps even necessary to help users satisfy their information needs.

Happy New Year to the Noisy Community and everyone else in virtual earshot! I hope everyone is entering 2010 well-rested and ready for great things. And I don’t just mean shiny new gadgets.

For me, 2009 marked the end of a decade-long run at Endeca, where I focused on bringing HCIR to enterprises. I’m particularly proud of two professional accomplishments: writing a book on faceted search, and organizing the SIGIR 2009 Industry Track.

But past is prologue. I spent the last several weeks of 2009 as a Noogler, and I launch into 2010 living and breathing search on the open web.

What’s on my mind? Here are some top-of-mind questions to which I hope to have better answers by this time next year:

• Exploratory Search: how should we determine that users want a more exploratory search experience, rather than one that minimizes time to a best-effort result? How should we respond to queries that clearly don’t have a single best answers, such as queries of the form [category] or [category location]?

• Mobile Search: should it be just like non-mobile search with a few tweaks to accommodate the device form factor? Or does / should mobile search fundamentally change the way we interact with information?

• Real-Time Search: is it more than real-time indexing plus emphasizing recency as a query-independent relevance factor? What are the use cases, and how should we be addressing them?

• Social / Collaborative Search: should we be looking to microblogging or other social media signals to augment (or even supplant!) link-based citations as authority cues? Should we be supporting mediated search by linking people to people, rather than directly to information?

• Transparency: is it possible to offer more transparency in relevance ranking without losing ground in the battle against spam and black-hat SEO?

To be clear, these are simply the questions that are on my mind–I’m speaking as an individual and not as a Google employee. That said, a great thing about being at Google is that there are people working on all of these areas. So I expect 2010 to be an exciting year!

Curious to hear what problems are on other people’s minds as we enter the new year. Comment away!

Researchers from Microsoft say it’s very challenging. Google is trying, but there’s a long way to go. And Eric Iverson just wrote me to describe his own preliminary efforts to build faceted search on top of Yahoo! BOSS.

I believe there’s a clearly established business case for faceted search inside the enterprise, for site search (especially for retail and media / publishing sites), even for vertical search on the open web. In all of these cases, relevance-ranked results are insufficient to meet a large subset of users’ more exploratory information needs, and HCIR approaches like faceted search are an easy sell.

But it seems much harder to make this case for general web search. The track record of startups in this space isn’t very encouraging. That could be because no one has done it right, but Clayton Christensen’s theory of disruptive innovation would suggest that a successful entrant wouldn’t have to have parity across the board, but would simply need to win on an underserved market segment. Perhaps the increasing use of faceted search for vertical search is how this process is playing out, and faceted search for general web search may end up being a slow agglomeration of verticals.

I’m curious if others have been pursuing efforts like Eric’s. Are the available APIs powerful enough to prototype your own faceted web search engine? If they aren’t, then is this a potential business opportunity for one of the major (or non-major) search engines to promote innovation by offering an open system? Or, if Yahoo! BOSS already offers such an open system, what should we make of the scale of its impact?

Long-time readers may recall my post about visual search startup Modista last November, or this guest post by one of its principals. Unfortunately, the story has a sad ending. I hope that both this technology and its developers find a good home.