Categories
Uncategorized

SIGIR ’09 Industry Track: The Details You’ve Been Waiting For

Several weeks ago, I announced that I’d be organizing the Industry Track at SIGIR ‘09. Some of you offered helpful suggestions, and many people expressed excitement at this opportunity to bring together the too-often separate worlds of research and practice.

Today, I am proud to share more details about the Industry Track. It will take place during the regular conference program, on Wednesday, July 22nd. Everyone who signs up for the full conference program (July 19th – 23rd) will be allowed to attend the Industry Track sessions at no extra charge. There will also be a one-day registration option.

But the reason you’ll want to be there is the incredible set of people who will be presenting during this full-day event:

We will also have a pair of panels devoted to enterprise search:

For researchers, the Industry Track provides an opportunity to learn about the industry side of information retrieval from some of its leading lights. In particular, it brings together the major players from both web search and enterprise search.

For industry practitioners who may never have heard of SIGIR before, let alone attended one of its conferences, the Industry Track offers an unprecedented opportunity to learn about the science and technology of information retrieval in a vendor-neutral, analyst-neutral setting. All without paying an extortionary registration fee.

Finally, for everyone in the space, this is an incredible professional networking opportunity. I look forward to seeing you there!

Categories
Uncategorized

Is Google Serious about Exploratory Search?

I just noticed this post by Sarah Perez on ReadWriteWeb: “Google: “We’re Not Doing a Good Job with Structured Data“. Here is the excerpt that caught my attention:

The company wants to be able to separate exploratory queries (e.g., “Vietnam travel”) from ones where a user is in search of a particular fact (“Vietnam population”). The former query should deliver information about visa requirements, weather and tour packages, etc.

While I’m not sure how important it is to automatically distinguish between “exploratory queries” and fact-finding ones, I absolutely agree that neither Google nor its web search rivals have delivered much in the way of exploratory search capabilities to users.

Alon Halevy, who is leading up Google’s “deep web” efforts, seems to understand the problem, even if I might quibble on some of the details. The question is: how does he plan to solve it? And how will his solution integrate with Google’s decidedly non-exploratory approach to information seeking?

Categories
General

Is Google Evil? The Great Debate

This Thursday, I’m attending an evening with blogger Jeff Jarvis that is part of his book tour for “What Would Google Do?” I took the opportunity to review an Intelligence Squared debate, hosted last November by the Rosencranz Foundation, about whether Google violates its “don’t be evil” motto. You can listen to the debate on the NPR web site or read the transcript here. But, for those who can’t spare the hour and a half to listen or who don’t care to read through a 74-page transcript, I’ll attempt to summarize here, and then offer my own thoughts on both sides of the question.

Six people participated in this debate, not including the moderator, ABC News correspondent John Donvan.

Arguing for the motion that Google violates its motto:

  • Harry Lewis, Harvard, Professor of Computer Science
  • Randal Picker, University of Chicago, Professor of commercial law and senior fellow at the Computation Institute
  • Siva Vaidhyanathan, University of Virginia, Associate Professor of Media Studies and Law

Arguing against the motion that Google violates its motto:

  • Esther Dyson, journalist and entrepreneur
  • Jim Harper, director of Information Policy Studios at the Cato Institute
  • Jeff Jarvis, blogger; associate professor and director of interactive journalism program at the City University of New York

Google was invited to participate, but chose not to do so.

First, since the game is over, let’s take a look at the score, based on polling the debate’s live audience.

Before the debate:

  • 21% were for the motion that Google violates its motto.
  • 31% were against the motion that Google violates its motto.
  • 48% were undecided.

After the debate:

  • 47% were for the motion that Google violates its motto.
  • 47% were against the motion that Google violates its motto.
  • 6% were undecided.

While this may seem like a tie, it was clearly a victory for those who changed the most minds–namely, those arguing that Google does violate its motto.

But we at The Noisy Channel are not sheep who uncritically accept the purported wisdom of crowds. Let’s look at the actual arguments. I will try to distill the main arguments made by each side.

Those arguing in favor of the motion, i.e., that Google does violate its “don’t be evil” motto, made what amount to four  points:

  • Google’s China policy is evil (Harry Lewis): Google’s collaboration with the Chinese censors makes them complicit in the brainwashing and thought control of the Chinese people. Google’s motives cannot be equated to those of, say, the Voice of America in Bulgaria, even if both resulted in oppressed people experiencing partial access to the truth. The VOA was subverting Bulgaria’s regime, while Google made the deliberate choice to do business with a country whose local laws would require them to do evil. Google chose to collaborate with China’s censorship regime. That deliberate act violates its motto.
  • Google is evil by its own standard (Harry Lewis): Google surely intended a higher standard by “don’t be evil” than “don’t be as evil as Hitler”. Moreover, aspiration to not be evil isn’t enough. Google doesn’t live up to its aspirational promise. Google should be held to their literal motto, not the standard of whether they are evil in their hearts.
  • Google does evil by monopolistic abuse of market power (Randal Picker): Google has designed its auctions in a way that takes advantage of its market power. When faced with a conflict between what’s good for the world and its own interest, Google favors its own interests. Google acts as a monopoly, and its monopoly power is at the heart of its business model. They create evil that they do not need to create.
  • Google is guilty of hubris (Siva Vaidhyanathan): Google sees no limits to its power, and thus commits the deadliest sin of hubris. Aggressive competitiveness doesn’t make Google evil, but its holier-than-thou attitude is hypocritical hubris. Comparing Google to peers is irrelevant: Google has set its own standard and has not met it. Google’s motto reflects its hubris and is a cynical marketing ploy, a promise it has not kept.

Those arguing against the motion made these five points:

  • Google’s China policy virally spreads democracy (Esther Dyson): Google spreads democracy virally by its very presence, by engaging rather than succumbing to apathy. Google is not collaborating with the Chinese censors so much as infiltrating the regime, exposing people to the virtues of knowledge, inciting change for good. Google is bringing goodness to the world, eroding power structures and the abuse of power by others.
  • Google is not like other greedy corporations (Esther Dyson): Google’s leaders do not hold themselves directly accountable to shareholders, but rather pursue their own agenda that they believe maximizes long term value to shareholders. Google has set its own standard and has met it by aspiring to do good.
  • Much of the evil attributed to Google is more correctly attributed to governments (Esther Dyson): The real danger is not Google, it’s the government. Governments have power which can easily be abused, while Google is constrained by law, by competition, and by its users. Those constraints actually help prevent Google from being evil, but beyond that Google doesn’t want to be evil. In any case, don’t blame Google for bad things it does in order to comply with the law.
  • Google is no more evil than the internet as a whole (Jim Harper):  Google is neither evil like Hitler, nor does the fact of its aggressive competitiveness make it even in the Enron sense of being driven by corporate greed. Google is essentially good, and its stated aspiration to not be evil argues in its favor.
  • If Google is evil, then humanity is evil (Jeff Jarvis): Google is good and virtuous, and therefore not evil. Its very aspiration to not be evil is evidence in its favor. If Google is evil, then the standard is too high and we are all evil.  Why Google acts is more important than how Google acts, and Google acts out of the belief that they are trying to do good. If that is not enough the standard is God-like perfection. Google’s only crime is its success.

These are interesting arguments. But let me try to offer two arguments of my own to each side that, in my opinion, strengthen their positions:

For those arguing that Google is evil:

  • Google’s collaboration with the censors in China is comparable–albeit to a much lesser degree–to the behavior of companies that conducted business with the Nazis or supplied technology to the apartheid regime in South Africa. Google provided technology that the Chinese government wanted–perhaps even needed–in order to ensure China’s global competitiveness. But even if China did not need Google’s help (after all, they have Baidu), Google still was evil to collaborate with a regime that oppresses its people by censorship and widely documented abuses of human rights. Imagine walking in on a mob of people beating someone to death. Your contribution might have no effect on the outcome of the hapless victim. But it is still evil to participate.
  • Google is a parasite, abusing its market power to dominate the retail, media, and advertising industries. Two specific examples: the “secondary search” feature that Google imposed on retail and media sites, undermining those site owners’ control of their users’ experience; and the lack of transparency in Google’s auction model for advertising. Google’s successful acquisition of DoubleClick and its abortive attempt to partner with Yahoo! show that it only seeks to reinforce this market dominance.

For those arguing that Google is not evil:

  • If Google’s collaboration with the censors in China is evil, it is at most a lesser evil. China already has Baidu, which offers comparable technology and competitive advantage to China as a nation. Google at least offers a wedge that might open up China to viral democratization. As Google senior policy counsel Andrew McLaughlin said, Google made the decision that provides the greatest access to information to the greatest number of people.
  • Google may be not be winning the love of the retail, media, and advertising industries, but it is serving the interests of its the constituency that matters: users. Google has given users–ordinary people, the little guy–unprecedented access to information, and has unlocked the control that retailers and media companies have held over this content. Furthermore, Google has democratized advertising, making it possible for smaller players to operate in a market previously dominated by big, mass-market brands.

And where do I stand personally? I think it’s silly to evaluate a company in terms of good and evil, even if Google has invited us to do so. There are the rare examples of companies being evil (IG Farben in the traditional sense of the word; Enron in the more modern sense of being the caricature of a morally bankrupt corporation), but these are the exceptions. I find it more useful to ask whether Google is making the world a better place.

For the most part, I think the answer is yes. I remember the pre-Google world, and I’m much happier living in this one. Nonetheless, I believe that Google, by resting on its laurels, is dragging the rest of the world down it its complacency. Do I blame Google for its complacency? Yes, I do, and I’ve told them as much.

But I also blame the rest of us. We continue using Google by choice–perhaps pressured by network externalities, but certainly not by coercion. It’s up to us, as users and as technologists, to set the bar higher.

Categories
General

Amazon: Customers Who Bought Related Items Also Bought

Amazon: Customers Who Bought Related Items Also Bought

Perhaps Amazon has had this feature for a while, but today, for the first time I noticed a section labeled “Customers Who Bought Related Items Also Bought” as seen in the screen shot above. I was looking at an unreleased book, which might explain why they couldn’t show me information based on customers who actually bought the item.

Has anyone else noticed this? Am I just late to the party? I tried to find more information online, but nothing showed up. I assume they are using some item similarity measure to assemble a set of related items, and then are basing collaborative filtering on the purchase history associated with that set.

I’m very curious to hear more from anyone who is familiar with this functionality.

Categories
General

Opting Out of Ads

I’m a long-time fan of ad blocking software, from the Siemens Webwasher plug-in in the early days to the Adblock Plus and CustomizeGoogle Firefox add-ons today. I know that some people view the use of ad-blocking software on ad-supported sites as anti-social or unethical. My personal view is that it is no different from physically obscuring the ads, or muting a television set during an advertising break. In any case, the technology allows it, and I’m am a very satisfied customer.

But I’m delighted to see that mainstream sites are finally starting to understand that advertising should not be coercive. Check out a post by Marisa Taylor in the Wall Street Journal entitled “Turning the Ads Off“. There she notes that some high-traffic sites, including wikiHow, AboutUs and, Kayak.com, are allowing users to opt out of ads.

Most users apparently aren’t opting out. According to Jack Herrick, founder of wikiHow:

“‘Opt-out’ ads are the good netizen thing to do for users,” he said. “It doesn’t actually hurt revenues that much anyway. And users love it. So why not do it?”

The other day, I asked a friend at Google if, given the option, she’d opt out of Google’s ads. Since she’s an employee, I imagine she might take the site’s terms of service seriously, or at least have more moral qualms about violating them. But she said that she found value in the ads, and wouldn’t opt out of them. Indeed, Google claims that the ads are valuable to users, not just advertisers.

I’d love to see Google put its money where its mission is, and make it easy for users to opt out of ads. That would show true leadership, as well as a confidence in its most sacred principle: “Focus on the user and all else will follow.” But I’m not holding my breath.

Categories
General

Cyber Pollution: Not a Victimless Crime

According to Wikipedia:

Pollution is the introduction of contaminants into an environment that causes instability, disorder, harm or discomfort to the physical systems or living organisms.

Wikipedia does not supply a definition for “cyber pollution”, but Ragy Thomas offers one in a recent post: “the waste of time and energy created by the inconsiderate use of commercial and personal electronic communication.” Thomas is no stranger to the subject: he has held advisory and leadership positions at Epsilon Interactive and Goodmail Systems, which makes him familiar with both the creation and prevention of cyber pollution.

I’ve advocated for attention bond mechanisms (ABM) in the past, and I still feel they are the most promising weapon against cyber pollution. In fact, Goodmail is a step in the direction towards implementing an mainstream ABM. But Thomas cites a service that is truly an ABM: Rupeemail. As per the Rupeemail FAQ:

Merchants and advertisers send RupeeMail to recipients like you seeking your attention. RupeeMail can only be sent to recipients who have explicitly agreed to receive RupeeMail by either registering at the RupeeMail website or permitting a specific sender to send email to them.

Merchants and advertisers send RupeeMail either to a list of willing recipients selected from the RupeeMail data base or to their own list of customers that have opted to receive email from them. If the recipient opens the the RupeeMail, then the recipient collects the value of the stamp attached to the RupeeMail. The sender (advertiser) decides the value of the stamp paid to recipient as a gesture of appreciation for recipient to open the RupeeMail and hopefully read their message (advertisement, news letter, survey etc.)

I’m thrilled to see people employing approaches like these in the war against attention terrorism. Spam filtering, despite its successes, is neither sufficient nor necessary. What we really need are communcation mechanisms that reflect true attention scarcity and ensure the negotiaton of attention costs between the sender and the recipeint is mutually satisfying. Otherwise, as with all pollution, we have a tragedy of the commons that buries us all in crap.

Categories
General

Reconsidering Relevance: Now on YouTube!

What happens to a stream deferred? Does it dry up like a raisin in the sun? No, it eventually makes it on to YouTube! Despite some early difficulties, Google has managed to make my recent Tech Talk, “Reconsidering Relevance”, available on YouTube. My apologies to all for the delay. Slides available on SlideShare.

Categories
Uncategorized

Search-Related Conferences: Where’s The Beef?

The other day, Stephen Arnold published a post entitled “Conference Spam or Conference Prime Rib” bemoaning “an increasing amount of conference spam” in the enterprise search space. Sharing his frustration with the marketing and hype that passes for technical discussion in this field, I posted a comment extolling the upcoming SIGIR Industry Track as an opportunity to bring some substance to the conversation.

To my delight, Arnold included the comment in a follow-up blog entry today. While I don’t take this as an explicit endorsement, I find his arguments very consonant with the case I made in my call to action several months ago. Moreover, Arnold doesn’t mince words when it comes to criticizing trade shows in which “the real losers are the attendees who spend money and invest time to hear lousy speakers or sales pitches advertised as original, substantive talks.”

Over the next months, I’m attending the Enterprise Search Summit, presenting at the Infonortics Search Engine Meeting, and organizing the SIGIR Industry Track. I’m also presenting at Discover, Endeca’s annual user conference. At all of these events, I expect substance, not warmed-over sales pitches. Hopefully these two posts on Arnold’s widely read blog will help inspire such an outcome, or at least will serve to shame some of the worst offenders.

Categories
Uncategorized

Wikipedia Embracing Information Accountability?

Given the recent debate on this blog over the merits of Wikipedia, I’m tickled to see that Jimbo Wales, Wikipedia’s controversial founder, is coming out in favor of requiring anonymous edits to be approved. (via Matthew Webber at UID Teatime Blog). I’m stoked, since this is the one point on which I think Knol beats Wikipedia. Read my past rants on the subject here.

Categories
Uncategorized

No Privacy Through Difficulty

I’ve blogged in the past about the futility of increasing futility of pursuing privacy through difficulty, and generally advocate an approach of “when in doubt, make it public”.

But, if you have any doubt about the current state of personal privacy, read what Robert Mitchell’s article in Computerworld entitles “What the Web knows about you“. As he says, “Social Security numbers are just the beginning.”