Author: Daniel Tunkelang

I’m back from HCIR ’08 and pleased to report that it was a rousing success. We had about 40 attendees, including such HCIR luminaries as Gary Marchionini, Marti Hearst, and mc schraefel. Microsoft Research supplied us not only with space and great food, but also workshop co-chair Ryen White and keynote speaker Sue Dumais, not to mention distinguished attendees Ken Church and Ashok Chandra. With a group like that, it was clear we were in for a great workshop.

And a great workshop we had! Some highlights: 

• Sue Dumais’s keynote on “Thinking Outside the (Search) Box” reviewed a variety of projects she and her colleagues at MSR have pursued in personal and personalized information retrieval.
• Marti Hearst discussed design issues in faceted search interfaces, as well as extensions to the faceted model.
• Steven Voida discussed a novel activitiy-based approach to personal information and task management.

At the beginning of the day, program chair Bill Kules had us write up our top HCIR concerns on post-it notes. We clustered these to form the basis for four breakout groups that discussed interactivity, task / workflow integration, sharing/collaboration, and results presentation. The results of these discussions, as well as the accepted papers, will be published online soon.

The workshop room also served as a space for posters. Posters were displayed throughout the day, and attendees congregated around posters and demos during the various breaks between sessions.

We concluded the workshop by soliciting feedback on how to improve it for next year. A fair number of attendees expressed interest in making the structure less formal, reducing the time spent on presentations and increasing the time available for more informal interaction. A number of folks expressed interest in continuing the discussions online, though there was no consensus on the best forum for doing so.

All in all, I was delighted by the energy of the group, and I believe that these workshops are helping to support HCIR efforts in both academia and industry.

Finally, I was delighted to recognize a number of Noisy Channel readers among the attendees. Conversely, Raman Chandrasekar was nice enough to (inadvertently) advertise this blog by leaving this screen up for a few minutes at the end of his presentation while he took Q&A:

I’ll keep folks here posted as more materials from the workshop become available. And, of course, you’ll be among the first to know where and when HCIR ’09 will take place.

Dave Kellogg has a nice post entitled “Web 2.Over?” in which he eloquently reviews the various reasons that most web 2.0 startups are “in for a reality check”.

But what I liked most about the post was his defense of the spirit of web 2.0:

While a swarm of eyeball-catching, oddly-named, twenty-something-led startups may get obliterated — outside venture circles at least — that wasn’t the point of web 2.0. To me, web 2.0 was, is, and remains an important collection of concepts that will endure:

• A read/write web, where we can participate, update, annotate, comment, etc.
• A social web, where there is awareness of relationships that can be leveraged appropriately
• User-generated content, which is here to stay and always has been (think: radio call-in shows, Kids Say the Darndest Things, or America’s Funniest Home Videos)
• The use of the web for communication and entertainment. People are natural communicators. We will always adapt our tools to that fundamental need.
• A personalized web, that understands what we like and how we like to get it

Amen! The good news is that there is no turning back on this vision of a more interactive online medium. Today it’s blogs and tweets; tomorrow it may be something we haven’t even imagined. But, now that an increasing number of us fancy ourselves as publishers and communicators, I don’t see us giving up that power without a fight.

A while back, I promised to write about tag clouds. I’m a man of my word, and I apologize for the delay in getting to this promise.

First, let’s define tag clouds. A tag cloud is a visual depiction of a set of words or phrases that characterize a set of documents. While “tag” suggests that the words and phrases are user-generated, the contents of tag clouds are often supplied by authors or even automatically extracted. Typically, tag clouds order tags alphabetically and use the size (or some similar typographical aspect) of a tag to indicate its frequency or relevance to the document set.

Tag clouds have been derided as “the mullets of Web 2.0” (I believe the original “mullet” critic was Jeffrey Zeldman). As someone who at least finds himself advising clients about how to improve user experience, I have seen companies clamor for tag clouds without necessarily thinking through how users would benefit from them. Indeed, while a picture may be worth a thousand words, a tag cloud may simply look like a thousand words.

The Good

My favorite example of a tag cloud interface is the ESPN website. Here is a “before” and “after” view of Roger Clemens:

Before:

After:

I know that both Red Sox and Yankees fans read this blog, so I won’t take sides on the accuracy of the Mitchell Report, But the change in the tag cloud clearly and concisely shows how the news about Roger Clemens changed when that report came out.

The Bad

Unfortunately, tag clouds that offer insight are the exception, rather than the rule. Part of the problem is that tag clouds are only as good as the tags they depict:  garbage in, garbage out. Tag clouds can also be so large and heterogeneous. Ryan Turner cites Flickr as such an example in his post, “Tag Clouds Are Bad (Usually)“:

The Ugly

As Greg recently blogged, tag clouds generated by social tagging systems can be worse than unhelpful; they can be actively misleading. Since tag clouds often occupy prime real estate on web site, they are a natural target for what Gartner analyst Whit Andrews calls “denial of insight” attacks.

In summary, tag clouds are a too-often abused but sometimes useful means to communicate information about a set of documents. But sites need to avoid presenting tag clouds simply expose the poverty of their tagging.

Also, while tag clouds may be an appropriate visualization for summarizing of a set of documents, they may not be the best means of presenting users with options for refning it. My colleagues and I discuss this problem in our upcoming HCIR presentation, and I’ll blog about it when I get back from the workshop.

Greg called my attention today to news that Digg is shifting from popularity-based aggregation to personalized news. I can’t say I’m thrilled at the prospect of a system that “would make guesses about what [users] like based on information mined from the giant demographic veins of social networks”. I don’t suppose the results are necessarily worse than showing users stories based solely on their popularity, but at least the latter offers me some transparency.

But it was an older post Greg pointed to that caught my attention: “Combating web spam with personalization“. Here is his argument in a nutshell:

Personalized search shows different search results to different people based on their history and their interests. Not only does this increase the relevance of the search results, but also it makes the search results harder to spam.

In this 2006 post, Greg is specifically referring to the personalized search that Google was beta testing back in 2004. Google has since implemented personalized search, but without sharing much detail about how it works.

Nonetheless, Greg’s argument reminds me of one of the first posts I wrote on this blog. I was criticizing Google’s approach of keeping its relevance approach secret and particularly the argument that Amit Singhal has advanced to justify it–that the subjectivity of relevance makes it harder to develop an open approach to relevance. My response: “the subjectivity of relevance should make the adversarial problem easier rather than harder, as has been observed in the security industry”. 

I suppose personalization can help fight spam even if it is not coupled with transparency to the user. But what a great opportunity to do both by providing more user control over the information seeking process.

On Friday, David Needle of InternetNews published an article with the provocative title, “Google Exec Disses Google’s In-House Search“. The essence of the article: Udi Manber, the Google VP of Engineering who is responsible for core search, evaluated Google’s internal search tools less than enthusiastically, saying “It’s not that good — I’m complaining about it”.

The article is a bit short on details. It quotes Nitin Mangtani describing recent updates to the Google Search Appliance to enable clustering of search results. But the most telling snippet is towards the end of the article, when Manber expresses his views on user interfaces:

While the search giant is constantly tinkering with new user interfaces, Manber said the simplicity of its standard, bare bones design remains tough to beat.

“Google has been very successful by being very minimal,” Manber said. “We’re doing hundreds of experiments with user interfaces; I see two to three new ones everyday.”

He added that Google might offer users the option of different views on its main search page, similar to the way it does so already on its personalized iGoogle page.

“Otherwise, I expect very incremental changes.” He said advanced users appreciate things like 3D and interfaces that offer more detailed views, but for the vast majority, “what happens now works. You type in two words, click and you’re done. You can’t beat that.”

To borrow a popular political slogan, yes we can. In fact, as IDC analyst Sue Feldman (also quoted in the article) said the other day, “One of the problems we have with search is that people ask such lousy questions…anytime tools hand people clues, it helps.”

Google’s success on the consumer web affords them the luxury of hiding their heads in the sand when it comes to enterprise information access. And I understand the appeal that Google’s computer scientists (and others) feel in approaching the information seeking problem as one of optimizing relevance ranking.

It’s not just Manber. Here are some quotes from Google Enterprise Product Manager Cyrus Mistry at a recent presentation:

• “[the ideal search engine] knows exactly what you meant, gives you exactly what you want.”
• “If you think tagging is the way to go, good luck. See me in 10 years.”
• “We’ll decide where to show it” (an explanation of the value proposition of Google’s universal search, which blends results from multiple sources into a single ranking)

It’s easy for Google to be cocky when they’re making $1.35B in quarterly profits. But it doesn’t make them right, especially when it comes to an area that accounts for about 1% of their business. Mind reading may not be impossible; some of my colleagues at CMU are working on it as we speak.

In the mean time, the only practical means our systems have for determining user intent is their input. And, as has been widely reported, the average search query contains 1.7 words. Perhaps the entropy of web search makes it possible to reliably infer intent from such a small signal. But enterprise search–which is to say information seeking in the enterprise–is harder.

At Endeca, we use our own technology in house. Our solution isn’t perfect, and we’re constantly working to improve it. But, most importantly, we’re going after the right problem. To respond to Mistry’s comments:

• The information access tool does not presume to know exactly what you meant or what you want, but instead works with you to establish this understanding through dialogue.
• Tagging can be a very effective way to bring in human expertise, especially when it is distributed across a broad population of users. But the tagging mechanism has to be easy for users, and the system needs to be smart about extrapolating from those tags to fill in the gaps.
• Often the best way to present diverse results is not by blending them into a single ranking but rather exposing that diversity to users in the form of a progressive refinement dialogue.

Google aspires to “organize the world’s information” but admits that its approach falls short when it comes to organizing the information inside their firewall. I commend Manber for his candor. But I hope he and his fellow Googlers take the next step and recognize that they have to think outside the search box.