Author: Daniel Tunkelang

Jeff Jarvis wrote a post today entitled “Google bigotry“, in which he asserts that:

Google has an image problem – not a PR problem (that is, not with the public) but a press problem (with whining old media people).

He then goes on to launch a tirade against a Le Monde journalist whose offense was to say she was writing “an article about Google facing a rising tide of discontent concerning privacy and monopoly.” He proceeds to stereotype the French as having “national insanity” of Google bigotry. I’ll leave analysis of irony as an exercise to the reader.

But the true irony is that Jarvis has a point. While I haven’t done a rigorous analysis, my impression is that there has been a sensationalist press overreaction against Google, singling out Google for behavior for which all other companies get a pass. As even one of the most vocal Google critics admits, “Google’s [privacy] policies are essentially no different than the policies of Microsoft, Yahoo, Alexa and Amazon.” Moreover, some of the newspapers criticizing Google as parasitic are the same ones who once turned–and still turn–to Google with open arms as a source of traffic–when they could easily cut Google off by configuring robots.txt. Granted, the newspapers are now locked into a prisoner’s dilemma, but they should at least take some responsibility for putting themselves in that position.

That said, there are lots of legitimate reasons to criticize Google, specifically concerning privacy and monopoly. While Google may not have engaged in any illegal or unethical practices to get there, it now holds a position as the primary gatekeeper to the internet for a substantial majority of Americans, as well as much of the western world. On the content creation side, site owners don’t ask “What Would Google Do?“–rather they ask how Google will index their sites. Meanwhile, on the consumption side, the broadening scope of Google’s role in ordinary people’s lives is legitimate cause for concern about privacy. It’s not insane or bigoted to raise these issues.

Moreover, Google claims to hold itself to a higher standard than other companies, so it’s not that surprising that people actually do hold them to it and criticize it when it flls short. Still, that’s no excuse for exaggeration or outright hallucination.

As I commented on Jarvis’s blog, I don’t think he’s the most credible judge of Google’s critics. He responded in kind. Touché. I accept that exchanging personal attacks doesn’t advance the argument. Perhaps more detached voices can chime in.

I’m a big fan of using machine learning and automated information extraction to improve search performance and generally support information seeking. I’ve had some very good experiences with both supervised (e.g., classification) and unsupervised (e.g., terminology extraction) learning approaches, and I think that anyone today who is developing an application to help people access text documents should at least give serious consideration to both kinds of algorithmic approaches. Sometimes automatic techniques work like magic!

But sometimes they don’t. Netbase‘s recent experience with HealthBase is, unfortunately, a case study in why you shouldn’t have too much faith in magic. As Jeff Dalton noted, the “semantic search” is hit-or-miss. The hits are great, but it’s the misses that generate headlines like this one in TechCrunch: “Netbase Thinks You Can Get Rid Of Jews With Alcohol And Salt”. Ouch.

It seems unfair to single out Netbase for a problem endemic to fully automated approaches, but they did invite the publicity. It would be easy to dig up a host of other purely automated approaches that are just as embarassing, if less publicized.

Dave Kellogg put it well (if a bit melodramatically) when he characterized this experience as a “tragicomedy” that reveals the perils of magic. His argument, in a nutshell, is that you don’t want to be completely dependent on an approach for which 80% accuracy is considered good enough. As he says, the problem with magic is that it can fail in truly spectacular ways.

Granted, there’s a lot more nuance to using automated content enrichment approaches. Some techniques (or implementations of general techniques) optimize for precision (i.e., minimizing false positives), while others optimize for recall (i.e., minimizing false negatives). Supervised techniques are generally more conservative than unsupervised ones: you might incorrectly assert that a document is about disease, but that’s less dramatic a failure than adding the word “Jews” to an automatically extracted medical vocabulary. In general, the more human input into the process, the more opportunity to improve the effectiveness and avoid embarassing mistakes.

Of course, the whole point of automation is to reduce the need for human input. Human labor is a lot more expensive that machine labor! But there’s a big difference between the mirage of eliminating human labor and the realistic aspiration to make its use more efficient and effective. That what human-computer information retrieval (HCIR) is all about, and all of the evidence I’ve encountered confirms that it’s the right way to crack this nut. Look for yourselves at the proceedings of HCIR ’07 and ’08. Having just read through all of the submissions to HCIR ’09, I can tell you that the state of the art keeps getting better.

Interestingly, even Google CEO Eric Schmidt may be getting around to drinking the kool-aid. In an interview published today in TechCrunch, he says: “We have to get from the sort of casual use of asking, querying…to ‘what did you mean?’.” Unfortunately, he then goes into science-fiction-AI land and seems to end up suggesting a natural language question-answering approach like Wolfram Alpha. Still, at least his heart is in the right place.

Anyway, as they say, experience is the best teacher. Hopefully Netbase can recover from what could generously be called a public relations hiccup. But, as the aphorism continues, it is only the fool that can learn from no other. Let’s not be fools–and instead take away the moral of this story: instead of trying to automate everything, optimize the division of labor between human and machine. HCIR.

Thanks to Tony Hollingsworth for alerting me to a post by Alex Campbell entitled “Stark realisation: I no longer depend on Google to find stuff“. The title is provocative link bait, but the take-away is very down to earth: Google is primarily useful for locating information than for discovering it.

Library scientists make a distinction between known-item and exploratory search. The former is about locating information: as an information seeker, you know the information exists, and you can even characterize it unambiguously; but the challenge is to convert that description into a location that allows you to retrieve the information. The latter is about discovery: you don’t know that the information you seek exists, and you may be sure of how to characterize what you are looking for–or even know what exactly you want until you’ve learned something about what is available.

These are extreme points on the information seeking spectrum, and most real-world tasks are in the middle, or combine subtasks of both types. For example, in physical libraries (yes, I’m that old!), I remember finding a book in the stacks and then browsing the nearby books in the hopes of serendipitous discovery. These days, I’d be more likely to scan its bibliography–or to look at the books and articles citing it. A known item can be an excellent entry point for exploration. Conversely, exploration can lead you to discover the existence of information that you then simply need to retrieve.

In common use, words like searching and finding cover this entire spectrum of information seeking activity. This breadth of meaning causes a lot of confusion. I’ve blogged about this before: “What is (Not) Search?“:

At the very least, I propose that we distinguish “search” as a problem from “search” as a solution. By the former, I mean the problem of information seeking, which is traditionally the domain of library and information scientists. By the latter, I mean the approach most commonly associated with information retrieval, in which a user enters a query into the system (typically as free text) and the system returns a set of objects that match the query, perhaps with different degrees of relevancy.

Back to Campbell’s article. His main points:

• Social networks have dramatically expanded our network of contacts.
• Search engine optimization (SEO) experts have killed their own game.
• The flow of information has changed: information now comes to us, rather than us having to go out and find it.

I like the spirit of the post, but I think he overstates his case. SEO isn’t all bad–in fact, it’s probably a key factor in Google’s effectiveness. And, while social networks enable social search in theory, and information does come to us; we are experiencing filter failure (Clay Shirky’s term) in a big way.

My conclusion: I agree with him about Google’s limitations–Google is primarily a locating tool, not a discovery tool. Unfortunately, I’m not persuaded that social networks and our theoretical ability to construct an ideal in-flow of information have actually delivered on the promise of more efficient information access. But I’m optimistic that we’ll eventually get there.