I guess the Google Enterprise Search folks just chose to put their effort elsewhere.

That’s actually another way to try to hack it but it is not a viable solution in most cases… Without counts this only amounts to some static shortcuts to do new queries, with the counts (which at least would allow you to know that a link is useless and would lead to no results) it’s extremely inefficient as explained in the documentation:

“The code will then submit an additional query per facet and obviously multiply the load to the GSA by the number of facets you have defined. ”

For the parametric hack, going to 1000 results just means that you need to retrieve 1000 extra results per query making it again very inefficient…

I just wrote a blog post about this.

Actually, GSA does offer faceted search (at least through the front end xslt). Dunno if it’s accessible through the api & the facets are static. The Google team points back to parametric for dynamic facets but it’s limited to 1000, not 100 reults; which eliminates any chance for deep discovery.

http://code.google.com/p/gsa-faceted-search/

So the answer to the question may well simply be: because Google has yet to develop the capability… It’s actually not as simple as it seems. Google does a lot of post-processing work on the search results (to remove duplicates, group by sites, increase diversity etc) which makes it very hard to have meaningful counts in the facets.

The adversarial nature of web search is a real challenge. But I think web search engines make that problem even worse by reducing everything to one-dimensional ranking. Giving users more control through interfaces like faceted search makes it harder for a page author to execute an effective spamming strategy–it diffuses the adversary.

Yes, enterprises (at least most of them) do have the advantage of not having to deal with the partially adversarial dynamics of web search. But that’s not excuse for web search developers to rest on their laurels, taking advantage of the scale advantages of the web and the low expectations of users. It may be a great short-term strategy, but it does provide an opportunity for disruption.

I’m sure that Google, Microsoft, and others know this. The question is what anyone can and will do about it.

Web search indexers can’t rely on page metadata, because it’s usually missing, duplicate, wrong, or deliberately misleading. They can’t rely on entity extraction because of language ambiguity (ruby on rails = red train on a track?). They can’t rely on any kind of content review process because the scope of their corpus is too big — even with crowdsourcing.

Dirty data is bad enough when enterprises own it and are able to clean it up for facets. But when the content is not just unreliable but also actively tuned to reverse-engineer search algorithms… I just can’t see it.

This situation does give enterprise search a nice space for improvement. I’m sure we’ve all pointed to facets when someone complains that the search is ‘not as good as Google’.

Avi

and..

Sure, buy isn’t that just grouping records into aggregates based on their point-wise metadata? I thought Jeremy had in mind facets where the documents whose the facet values were implicitly rather than explicitly represented, e.g., a facet derived from point-wise metadata.

Ah, it’s either fantastic that you “get” exactly where I was headed with that line of reasoning — because we’re on the same wavelength — or terrible, because my line of reasoning is so immediately transparent and naive

Either way, yes. This is what I was after.

Computation aside, I see those types of facets as still useful, but also something that advertisers would have a horrendous time trying to bid against. Given their implicit, user-driven nature, it would be difficult.

Otherwise, don’t we live in a “cloud” world these days? Isn’t one of the benefits of the cloud the fact that we can now start to do some of this higher-order computation?

Surely you can, OLAP-style, do a rollup. For example, given a “city” facet, you could roll-up to the “state” facet and then a country facet…

I’ll go order your book.

As for my book, I do offer what I hope to be useful advice in my book, much as Marti Hearst and others do elsewhere–much of the advice about search and even web usability in general applies to faceted search. But you’re right, there’s no magic wand. If I had one, I’d certainly charge more for the book!

Jeremy, I would not say that faceted search is defined exclusively by point-wise meta-data. But I don’t really explore alternatives in the book (unless you count the use of multiple entity types, e.g., refining books by facets of their authors). In theory, a facet doesn’t have to be explicitly represented in the data at all, e.g., a facet could be derived from other data elements. In practice, however, no one seems to take such approaches–probably because the query-time computation becomes a major challenge.

Is faceted search defined exclusively by pointwise metadata? Or are there other ways of defining facets, in your book?

Alas, with tools like DBLP, I am unimpressed by the practical result. It just does not “flow”. It does not do what I’d expect. (And it is not nearly as hard a problem was web search, you’ll agree, I’m sure.)

ACM Digital Library integrated some faceted search (though I’m not sure you would called it faceted search) and it is terrible. I tried to refine a search today and end up closing the browser out of frustration.

No doubt, Endecca must have gotten it right. (Some of the time at least!) But clearly it is difficult.

I have not bought your book yet (sorry), but from what I could see in the reviews, you don’t seem to have some kind of magic wand to “make it work”.

Some maybe we need to do more research…

Ah! But how to get started?

But doesn’t the more targeted advertising also require input from the advertisers, too. And that’s where it gets complex. Advertisers already have to manage all different sorts of keywording options (exact match, subset match, fuzzy match, etc.) Once they have to start deciding which facets and subfacets they want to bid competitively for, it will become unmanageable for them. So while Google might be able to make web facets work, ad facets are even harder. So perhaps it still is the advertising that is holding them back.

I think the bigger issue is that faceted search on the web is hard, and offers many opportunities to mismanage user expectations. Google has incredible customer satisfaction. There’s a strong case for not risking that through innovation in the user interface. I think it’s not surprising that Microsoft is approaching such innovation more aggressively–though still not aggressively enough for my tastes.

The more time people spend exploring facets, the less time people spend clicking ads. Isn’t it as simple as that?

Speaking of Christensen, see also: http://irgupf.com/2009/03/19/long-term-versus-evolutionary-thinking-part-2-of-2/