On Friday, David Needle of InternetNews published an article with the provocative title, “Google Exec Disses Google’s In-House Search“. The essence of the article: Udi Manber, the Google VP of Engineering who is responsible for core search, evaluated Google’s internal search tools less than enthusiastically, saying “It’s not that good — I’m complaining about it”.
The article is a bit short on details. It quotes Nitin Mangtani describing recent updates to the Google Search Appliance to enable clustering of search results. But the most telling snippet is towards the end of the article, when Manber expresses his views on user interfaces:
While the search giant is constantly tinkering with new user interfaces, Manber said the simplicity of its standard, bare bones design remains tough to beat.
“Google has been very successful by being very minimal,” Manber said. “We’re doing hundreds of experiments with user interfaces; I see two to three new ones everyday.”
He added that Google might offer users the option of different views on its main search page, similar to the way it does so already on its personalized iGoogle page.
“Otherwise, I expect very incremental changes.” He said advanced users appreciate things like 3D and interfaces that offer more detailed views, but for the vast majority, “what happens now works. You type in two words, click and you’re done. You can’t beat that.”
To borrow a popular political slogan, yes we can. In fact, as IDC analyst Sue Feldman (also quoted in the article) said the other day, “One of the problems we have with search is that people ask such lousy questions…anytime tools hand people clues, it helps.”
Google’s success on the consumer web affords them the luxury of hiding their heads in the sand when it comes to enterprise information access. And I understand the appeal that Google’s computer scientists (and others) feel in approaching the information seeking problem as one of optimizing relevance ranking.
It’s not just Manber. Here are some quotes from Google Enterprise Product Manager Cyrus Mistry at a recent presentation:
- “[the ideal search engine] knows exactly what you meant, gives you exactly what you want.”
- “If you think tagging is the way to go, good luck. See me in 10 years.”
- “We’ll decide where to show it” (an explanation of the value proposition of Google’s universal search, which blends results from multiple sources into a single ranking)
It’s easy for Google to be cocky when they’re making $1.35B in quarterly profits. But it doesn’t make them right, especially when it comes to an area that accounts for about 1% of their business. Mind reading may not be impossible; some of my colleagues at CMU are working on it as we speak.
In the mean time, the only practical means our systems have for determining user intent is their input. And, as has been widely reported, the average search query contains 1.7 words. Perhaps the entropy of web search makes it possible to reliably infer intent from such a small signal. But enterprise search–which is to say information seeking in the enterprise–is harder.
At Endeca, we use our own technology in house. Our solution isn’t perfect, and we’re constantly working to improve it. But, most importantly, we’re going after the right problem. To respond to Mistry’s comments:
- The information access tool does not presume to know exactly what you meant or what you want, but instead works with you to establish this understanding through dialogue.
- Tagging can be a very effective way to bring in human expertise, especially when it is distributed across a broad population of users. But the tagging mechanism has to be easy for users, and the system needs to be smart about extrapolating from those tags to fill in the gaps.
- Often the best way to present diverse results is not by blending them into a single ranking but rather exposing that diversity to users in the form of a progressive refinement dialogue.
Google aspires to “organize the world’s information” but admits that its approach falls short when it comes to organizing the information inside their firewall. I commend Manber for his candor. But I hope he and his fellow Googlers take the next step and recognize that they have to think outside the search box.
6 replies on “Google Exec Udi Manber: In-House Search is “Not That Good””
I agree that sites, both behind-the-firewall and customer-facing, can get a lot more mileage out of tagging beyond simple navigation and the tag cloud (which, if there is any justice in the world, will go the way of the dodo).
For example, some interesting work has been done by Marti Hearst as well as within Endeca to create hierarchy for navigation from tags, though I haven’t really seen it applied anywhere visible. Anyone have more info on applied usage of this kind of trick?
Rob, stay tuned for the HCIR ’08 proceedings, which should be available online and include some work that might interest you. The workshop itself is taking place this Thursday, October 23rd.
We’re doing hundreds of experiments with user interfaces; I see two to three new ones everyday.”
I would be quite interested.. not in the two or three new UIs that they’re doing every single day.. but in the techniques that they are using to evaluate the quality of a change.
If there is any one thing that I’ve learned in my years as a researcher, it is that evaluation drives innovation. Your choice of what to evaluate drives, in very large part, both the definition of the problem, as well as the likely range of solutions.
If you change the evaluation, you change the problem, and therefore also change the solution.
So if all these new UIs are failing, my first question is: On what evaluation metric are they failing? And if hundreds of new UIs, every single year, continue to fail, might it be possible that you’ve got the wrong evaluation metric, rather than the wrong new UI?
It’s a serious, honest question.
I agree, it’s a great question. And this is the closest thing I can find to an answer: http://googleblog.blogspot.com/2008/09/search-evaluation-at-google.html
[…] The next reaction came from Daniel Tunkelang of Endeca who had a thoughtful response worth reading in its entirety regarding Google’s quandary. I found his statement—“the information access tool does not presume to know exactly what you meant or what you want, but instead works with you to establish this understanding through dialogue”—to be particularly relevant to the issue of enterprise search user satisfaction that Vivisimo also has been working hard to address. […]
[…] exactly been gushing about Google’s enterprise solutions–then again, neither have their executives. Still, I thought it fair and balanced to point to an article that Google Enteprise Product […]