Author: Daniel Tunkelang

Here are the slides from Marti Hearst’s and Peter Morville‘s keynote presentations at the Enterprise Search Summit:

This morning’s Enterprise Search Summit keynote was by Peter Morville, who has written a number of best-selling books about information architecture. I’ve known Peter for a while and had the pleasure of serving as a reviewer for his latest book, Search Patterns, but had never seen him present this material live. As you can see from his slides, Peter’s presentation style is incredibly visual–almost all of his slides are screenshots or illustrations explaining his concepts. It makes for a great presentation, but a difficult text summary!

The focus of his talk, naturally, was patterns. Specifically, he advocated that we take the behavior patterns of information seekers that library and information scientists have been studying for years, and use them to inform design patterns for search user interfaces.

One point he raised that deserves a deeper dive: number of media (mobile, kiosk, TV) environments push people to browse, partly because of limitations of the medium but also taking advantage of the novelty and relative lack of user habits. Unfortunately, browsing doesn’t always scale in those environments, so search is usually available as a contingency.

Interestingly, while Peter promotes rich interfaces in many of his patterns, he noted that great results ranking plus speedy response (he uses Google “classic” as his example) does allow users to rapidly reformulate their queries while staying in the flow of the information seeking experience. He returned to Google later in his talk, noting that the new interface goes beyond ranking to support a richer user interaction.

And, like me and Marti Hearst (yesterday’s keynote), Peter advocates faceted navigation (I won’t quibble on whether to call it navigation or search) as his favorite search design pattern. He uses the NCSU library as an example not only of a great implementation but also of an organization that continues to experiment with incremental design changes. He also showed faceted search examples from other domains, including Amazon and Buzzilions.

Other patterns he discusses included question answering (his example being Wolfram Alpha) and decision making (his example being Hunch). He didn’t go deep on these, but rather invited the audience to consider a broad palette of strategies for supporting information seeking. Indeed, when I asked him about question answering, he conceded that he was a skeptic and preferred a conversational (i.e., HCIR) approach akin to a librarian’s reference interview.

His closing note was about bridging the gap between physical and digital information, where he offered a potpourri of examples (from Redbox to a tweeting plant). I work in local search, so in my case he’s preaching to the converted. But I think he’s right that everything is only recently coming together–specifically, the ubiquity of digital data on the internet and of mobile devices in the physical world that can both consume and produce that data. Many of us take these developments for granted, but it’s important that we adapt our approach to search to address what is a very recent phenomenon.

Fun stuff! I didn’t get to attend the rest of the summit, but I encourage you to check out the tweet stream at #ESS10.

The Enterprise Search Summit is taking place in New York this week, and I was lucky to be able to attend Marti Hearst’s opening keynote this morning about designing search for humans. If you’ve read her book or heard her present its material, then you’re probably familiar with the pitch she made. Still, it’s great to hear her present it live to a very non-academic audience.

Her major take-aways:

• The user’s emotional response is a key aspect of the information seeking experience.
• There is a double vocabulary problem: different ways to express same concept (cf. Furnas et al.), and users stubbornly anchor on initial query terms (cf. Kahneman, Tversky, et al.)
• Recognition is easier than recall, so interfaces need to support the recognition process.
• Don’t personalize search, socialize it!

She peppered her talk with concrete examples and scholarly references. Given that her book is available online for free, I won’t try to replicate them all here! Still, I’ll single out two Noisy Community members: FXPAL researchers Jeremy Pickens and Gene Golovchinsky (for their SIGIR 2008 work on collaborative exploratory search) and user experience designer Greg Nudelman for his proposal of faceted breadcrumbs as a search user interface.

If you missed her live, you check find a video of a tech talk she gave at Google a few months ago. You can also check out the conference tweet-stream at #ESS10.

Today I celebrate six months of working at Google. I’m having a great time, and I wanted to take a moment to share a bit about my experience thus far.

First, a bit about Google’s New York office. It is a major office–in fact, Google’s second largest office and an R&D powerhouse. New York Googlers played key roles in two of Google’s recent developments: real-time search and the results page re-design. Less visibly but not less importantly, engineers at Google New York contribute to every major aspect of Google’s technology.

My own contributions have been toward improving local search. Local search represents an increasingly large and important fraction of people’s online information seeking. At first glance, it might seem to be an easier problem than general web search, since there are only tens of millions of places, compared to tens of billions of web pages. But local search poses unique challenges–from data quality to ranking to supporting exploratory search.

Another area that I’m especially excited about is the work on structured data. The Magpie team is based in New York. You may be familiar with them as the team that developed Google Squared–which powers the “something different” links for web search.

Of course, there is far more going on at Google New York than I could hope to summarize in a blog post–including lots that I can’t talk about yet. But I hope this at least gives you a taste of what it’s like to work for the world’s best search company in the world’s best city (yes, I know I’ll take some flak for at least one of those superlatives).

If you’re interested in learning more, please don’t hesitate to reach out to me. I may have drunk the kool-aid, but I promise to be candid and as open as I can about what it’s like to go through the hiring process and what awaits you at the end of it.

Sorry for the long delay between posts. Fortunately the blogosphere has been providing ample reading material about the saga of the lost iPhone and the war of words between Apple and Adobe.

I’ve been doing some reading myself. Specifically, I just read F. Randall (“Randy”) Farmer and Bryce Glass‘s recent book on Building Web Reputation Systems. Given that I’ve been thinking a lot about online reviews and reputation systems (e.g., this recent post), I wanted to hear what the experts had to say.

In the book, Farmer and Glass categorize the motivations for user participation as altruistic, commercial, and egocentric. Commercial motives are clearly the most problematic: a review site loses credibility if commercially motivated reviews are disguised to make their commercial motives. Most review site scandals arise from this kind of deception (e.g., this one, this one, and  this one).

Sincerity is a necessary but insufficient condition for a review to be valuable to the person who reads it. There is still the “people like me” problem: sincere reviewers may still be uninformed, unreasonably biased, or may simply not share our tastes. User-generated content is an inherently subjective medium.

Given these challenges, it’s a wonder that online review sites work at all! And yet there are real success stories. My personal favorite is Amazon.com. While it has has its hiccups, Amazon nonetheless serves as a poster child for creating value by aggregating user opinions about products.

Amazon has a well-designed review policy that gets many key elements right:

• Reviewers have identities tied to purchasing history. That encourages disclosure (people use their real names) and discourages abuse.
• The reviews themselves–and even comments in discussion threads about individual reviews–are themselves reviewed as helpful or not. That may seem overly meta, but it does a lot to mitigate information overload.
• Grounding in objective information (product content, sales rank) reduces the ability to manipulate product perception through reviews.

The system isn’t perfect, but it’s good enough to be very useful.

But products aren’t the only reputable entities, to use Farmer and Glass’s term. What about service businesses, such as restaurants, gyms, etc. Or people?

If Amazon exemplifies online product reviews, then Yelp is the canonical example of a review site for service businesses. And, despite its own share of controversy, it is quite successful. But I dare say not quite as successful as Amazon. Part of the problem is that is demographics are less representative of the general online population (here’s what Quantcast says about Yelp and Amazon demographics for their US users). Also, there’s more variance in experiencing a service than in experiencing a product.

But Yelp has also has had  a credibility problem regarding which reviews they allow to be published. Perhaps the root of this problem is that Yelp’s business model depends on paid advertising from the businesses reviewed on the site, while businesses would much rather have unpaid positive reviews. In contrast, Amazon makes its money buy selling products–which at least makes it perceived to be more evenhanded.

But neither Amazon and Yelp have touched the third rail of online reputation: people. LinkedIn dabbles in this space by allowing its members to review one another, but reviewees have veto power over reviews–making the review graph more of a mutual admiration society.

A recent startup, Unvarnished, is trying to create a review site with teeth. Farmer argues on his blog that Unvarnished is breaking some major  rules:

• It displays negative karma–that is, it allows people to write negative reviews of one another and displays those reviews.
• The reviews are not clearly tied to context (e.g., were the reviewer and reviewee co-workers?).
• The anonymity of reviewers does not incent altruistic or even egocentric behavior, and is thus a recipe for abuse.

I’m not as down on Unvarnished as Farmer, but I agree it will have an uphill battle to succeed. Ironically,  for all of the public concern about Unvarnished becoming a trollfest, the reviews skew strongly positive. This is probably an artifact of how Unvarnished is growing its membership: current users ask friends to review them.

I agree most with Farmer that Unvarnished’s incentive structure seems problematic. A person’s friends will probably be inclined to write positive reviews, and may even be annoyed at having to write them anonymously. A person’s enemies may be inclined to write negative reviews as a form of attack or revenge. But it’s less clear what will incent people to write accurate reviews–or what will signal to readers that a review is trustworthy.

All in all, I think that these are early days in the online reputation space, and that there is ample room for innovation. Facebook’s recent release of “like buttons” is an ambitious attempt to boil the ocean of “social objects”. A best poster award at the recent WWW 2010 conference went to Paul Dütting, Monika Henzinger, and Ingmar Weber’s “How much is your Personal Recommendation Worth”.  Hopefully all of these attempts to research and innovate will lead to a world where we can derive real value from others’ opinions and feel incented to contribute our own.