Comments on: Google Exec Udi Manber: In-House Search is “Not That Good” http://thenoisychannel.com/2008/10/19/google-exec-udi-manber-in-house-search-is-not-that-good/ Wed, 10 Mar 2010 18:38:54 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: Google Defends Its Appliance | The Noisy Channel http://thenoisychannel.com/2008/10/19/google-exec-udi-manber-in-house-search-is-not-that-good/comment-page-1/#comment-614 Google Defends Its Appliance | The Noisy Channel Sat, 25 Oct 2008 13:39:04 +0000 http://thenoisychannel.com/?p=642#comment-614 [...] exactly been gushing about Google’s enterprise solutions–then again, neither have their executives. Still, I thought it fair and balanced to point to an article that Google Enteprise Product [...] [...] exactly been gushing about Google’s enterprise solutions–then again, neither have their executives. Still, I thought it fair and balanced to point to an article that Google Enteprise Product [...]

]]> By: Search Done Right » Blog Archive » Guessing About Search Intent http://thenoisychannel.com/2008/10/19/google-exec-udi-manber-in-house-search-is-not-that-good/comment-page-1/#comment-598 Search Done Right » Blog Archive » Guessing About Search Intent Wed, 22 Oct 2008 20:03:59 +0000 http://thenoisychannel.com/?p=642#comment-598 [...] The next reaction came from Daniel Tunkelang of Endeca who had a thoughtful response worth reading in its entirety regarding Google’s quandary. I found his statement—“the information access tool does not presume to know exactly what you meant or what you want, but instead works with you to establish this understanding through dialogue”—to be particularly relevant to the issue of enterprise search user satisfaction that Vivisimo also has been working hard to address. [...] [...] The next reaction came from Daniel Tunkelang of Endeca who had a thoughtful response worth reading in its entirety regarding Google’s quandary. I found his statement—“the information access tool does not presume to know exactly what you meant or what you want, but instead works with you to establish this understanding through dialogue”—to be particularly relevant to the issue of enterprise search user satisfaction that Vivisimo also has been working hard to address. [...]

]]> By: Daniel Tunkelang http://thenoisychannel.com/2008/10/19/google-exec-udi-manber-in-house-search-is-not-that-good/comment-page-1/#comment-594 Daniel Tunkelang Mon, 20 Oct 2008 19:18:56 +0000 http://thenoisychannel.com/?p=642#comment-594 I agree, it's a great question. And this is the closest thing I can find to an answer: http://googleblog.blogspot.com/2008/09/search-evaluation-at-google.html I agree, it’s a great question. And this is the closest thing I can find to an answer: http://googleblog.blogspot.com/2008/09/search-evaluation-at-google.html

]]> By: jeremy http://thenoisychannel.com/2008/10/19/google-exec-udi-manber-in-house-search-is-not-that-good/comment-page-1/#comment-587 jeremy Mon, 20 Oct 2008 01:46:49 +0000 http://thenoisychannel.com/?p=642#comment-587 <i>We’re doing hundreds of experiments with user interfaces; I see two to three new ones everyday.”</i> I would be quite interested.. not in the two or three new UIs that they're doing every single day.. but in the techniques that they are using to evaluate the quality of a change. If there is any one thing that I've learned in my years as a researcher, it is that evaluation drives innovation. Your choice of what to evaluate drives, in very large part, both the definition of the problem, as well as the likely range of solutions. If you change the evaluation, you change the problem, and therefore also change the solution. So if all these new UIs are failing, my first question is: On what evaluation metric are they failing? And if hundreds of new UIs, every single year, continue to fail, might it be possible that you've got the wrong evaluation metric, rather than the wrong new UI? It's a serious, honest question. We’re doing hundreds of experiments with user interfaces; I see two to three new ones everyday.”

I would be quite interested.. not in the two or three new UIs that they’re doing every single day.. but in the techniques that they are using to evaluate the quality of a change.

If there is any one thing that I’ve learned in my years as a researcher, it is that evaluation drives innovation. Your choice of what to evaluate drives, in very large part, both the definition of the problem, as well as the likely range of solutions.

If you change the evaluation, you change the problem, and therefore also change the solution.

So if all these new UIs are failing, my first question is: On what evaluation metric are they failing? And if hundreds of new UIs, every single year, continue to fail, might it be possible that you’ve got the wrong evaluation metric, rather than the wrong new UI?

It’s a serious, honest question.

]]> By: Daniel Tunkelang http://thenoisychannel.com/2008/10/19/google-exec-udi-manber-in-house-search-is-not-that-good/comment-page-1/#comment-583 Daniel Tunkelang Sun, 19 Oct 2008 21:12:37 +0000 http://thenoisychannel.com/?p=642#comment-583 Rob, stay tuned for the HCIR '08 proceedings, which should be available online and include some work that might interest you. The workshop itself is taking place this Thursday, October 23rd. http://research.microsoft.com/~ryenw/hcir2008/ Rob, stay tuned for the HCIR ‘08 proceedings, which should be available online and include some work that might interest you. The workshop itself is taking place this Thursday, October 23rd.

http://research.microsoft.com/~ryenw/hcir2008/

]]> By: Rob http://thenoisychannel.com/2008/10/19/google-exec-udi-manber-in-house-search-is-not-that-good/comment-page-1/#comment-582 Rob Sun, 19 Oct 2008 20:56:42 +0000 http://thenoisychannel.com/?p=642#comment-582 I agree that sites, both behind-the-firewall and customer-facing, can get a lot more mileage out of tagging beyond simple navigation and the tag cloud (which, if there is any justice in the world, will go the way of the dodo). For example, some interesting work has been done by Marti Hearst as well as within Endeca to create hierarchy for navigation from tags, though I haven't really seen it applied anywhere visible. Anyone have more info on applied usage of this kind of trick? I agree that sites, both behind-the-firewall and customer-facing, can get a lot more mileage out of tagging beyond simple navigation and the tag cloud (which, if there is any justice in the world, will go the way of the dodo).

For example, some interesting work has been done by Marti Hearst as well as within Endeca to create hierarchy for navigation from tags, though I haven’t really seen it applied anywhere visible. Anyone have more info on applied usage of this kind of trick?

]]>