The Noisy Channel

 

What Would Google Do? / What Does Google Do?

February 5th, 2009 · 15 Comments · General

This evening, I had the opportunity to hear Jeff Jarvis talk about his recently published book, “What Would Google Do?“. That opportunity was briefly in doubt: 277 people signed up for the event at the Daylife office, which had planned for a capacity of 150. Fortunately for me, my friend Ken Ellis let me in early, and I was not turned away at the door. Which is fortunate for you, since it means I have something to blog about!

Jarvis was entertaining, as expected. He is an excellent speaker, both when he’s delivering prepared material and when he’s put on the spot by aggressive audience members, who were in no short supply.

I perhaps deserve credit (responsibility?) for inciting the mob by asking the first question, suggesting that Google was the opposite of transparent (one of the most “Googly” qualities in his enumeration) and that, if we were to learn anything from Google, it was that success is best achieved through benign dictatorship. In fact, I told Jarvis that I thought he’d already seen the light on this issue.

Jarvis didn’t even flinch. First, he made clear that he was more interested in “the idea of Google” than the company itself. Second, he argued that Google has made the world transparent, even if Google isn’t always transparent itself. Finally, he suggested that being in continuous beta was a form of transparency. I didn’t have a chance to follow up after that, but others did, and I was happy to see that the crowd, on the whole, seemed unpersuaded by the culty premise of “the idea of Google”.

But what I enjoyed far more that the advertised event was the heated conversation I had, following Jarvis’s presentation. Bob Wyman, an engineer at Google, offered a full-throated defense of as many of Google’s technology and business decisions as I could question. While we ultimately agreed to disagree, I credit him for making a serious case. Unfortunately, I couldn’t take notes and uphold my side of the argument at the same time, so I’ll apologize in advance for any details I’ve lost or garbled in my good-faith recollection.

Here is what I recollect from our discussion:

  • Our biggest point of contention was about the black-box nature of Google’s approach to relevance. Bob was quite familiar with the analogy to security through obscurity, but he objected that, before the discovery of public key cryptography, security through obscurity was the best game in town. In other words, the folks working on relevance ranking algorithms are still waiting for the equivalent of Diffie-Hellman or RSA.
  • He rejected attention bond mechanisms as gameable, though I don’t recall any explanation as to why. It’s possible that he simply wasn’t familiar with them, and that my explanation didn’t do justice to the concept.
  • He insisted that I wasn’t giving enough credit to Google for its experimentation–specifically, that I underestimated how much variation there was in result ranking  based on the collection of simultaneous experiments running at any given time.
  • He felt I was being unreasonable to expect Google to disclose more about its retrieval approach, not only because it would help spammers, but also because it would unnecessarily give users more to think about.
  • Finally, he felt that almost any clever idea would break down because of the combined constraints imposed by the hordes of spammers, the scale of the data, and the challenges of freshness.

Bob defended Google well, and I can’t say that either of us “won” the fight. Indeed, so much hinges on whether you can believe, as he does, that Google is only limited by what technology makes possible and what its engineers can implement. He rejects my assertion that Google’s has crippled the user experience because of some philosophical predilection towards black box approaches. In fact, he maintains that Google is incredibly open for a company of its size.

Unfortunately, the truth resides in the ultimate black box:  I can’t evaluate Google’s motivations from the outside. Bob invited me to work at Google to help solve the problem from the inside (I don’t think he meant that as a literal offer), but that not here or there. I think it’s only fair to judge a company from the outside. If Google wants to fix the misimpressions that I and others hold, it can certainly do so by providing more information. Absent such information from the source, I have to fall back on the public data and my powers of reasoning.

But let me say this clearly: I believe that most–perhaps all–Googlers mean well. Google’s China policy notwithstanding, I don’t think that Google is an evil company. On the whole, Google has done much more good than bad, and I believe that doing the right thing is a core company value, even if Google does not always live up to its aspirations.

My only problem is that, in the one area I’m most passionate about, it seems that Google is holding the world back. Google has become the legacy system that HCIR has to beat.

A parting joke, courtesy of Ken Ellis:

Q: What would Google do if they were a restaurant?

A: They would build a search engine and an internet ad auction system.

15 responses so far ↓

  • 1 Jeff Jarvis talks about “What Would Google Do?” « NP-Harder // Feb 6, 2009 at 9:59 am

    […] Daniel Tunkelang posted another good review of the talk at http://thenoisychannel.com/2009/02/05/what-would-google-do-what-does-google-do/ […]

  • 2 JasonB // Feb 6, 2009 at 11:36 am

    That “idea of Google” supposition is creepy. A brand, among other things, is the interaction of a product with its customers. Any cognitive dissonance between what the product says and what the customers reflect back creates dissatisfaction with the product. That Jarvis was using that as an argument suggests he’s shilling. Not good for his “personal brand” either.

  • 3 Daniel Tunkelang // Feb 6, 2009 at 12:40 pm

    JasonB, I hear you loud and clear. I don’t think Jarvis was shilling Google (do they really need his help?), but I do find his approach problematic.

  • 4 jeremy // Feb 6, 2009 at 2:01 pm

    I also find myself scratching my head about Jarvis’ odd distinction between “Google” and “The Idea of Google”.

    Unfortunately, I don’t even agree that Google successfully executes on “The Idea of Google”. I think they’re very good at scratching the surface of “the world’s information”. But they completely fail to execute on any deeper involvement or exploration or contextualization of information. As we’ve often discussed.

    But to me, the deeper information is what makes “transparent” information ultimately useful and world-changing. Helping people discover the location of the nearest Starbucks or what time a movie starts, what the URL some someone’s homepage is, or getting a copy of a research paper is nice. But at the end of the day it doesn’t really change anything.

    What changes things, where transparency really becomes valuable, is when you can find hitherto unseen connections in disparate pieces of information, or when you can find revealing details that no one else in the crowd seems to be aware of. But this is the very sort of thing that Google *does not* help you with. Rather, it “pageranks” you into popular, common information and sources.

    So even if we look past this disconnect between Google and the idea of Google, I don’t think Google has really made the world all that more transparent. We can find movie times easier. But that is not real transparency.

  • 5 jeremy // Feb 6, 2009 at 2:11 pm

    He insisted that I wasn’t giving enough credit to Google for its experimentation–specifically, that I underestimated how much variation there was in result ranking based on the collection of simultaneous experiments running at any given time.

    Does Jarvis have inside information on this? How does he know so much about all the experimentation? Would he care to elaborate, or is all his knowledge behind an NDA firewall?

    Because I’m sure that Google does experiment. Lots. But the field of information retrieval, experimentation tends to be guided primarily by one’s fitness function, by one’s measure of “goodness”. So the bigger question to me is, what is Google’s measure of goodness?

    Because it doesn’t matter how many experiments you do, if you are still only trying to optimize a single kind of fitness function.

    By listening to statements from Google’s senior management, I have formed my own picture, my own belief, in what their fitness function is. And it’s a very limited function. But my own view is clouded, again, by the black box.

    So Jarvis, or Google, really needs to come out and explain what it is that they’re trying to optimize. Certainly they can be transparent about that, can’t they? They can give away the fitness function, without giving away any information about how they’re actually solving/meeting that function, can’t they?

    Or if they can’t even discuss what it is they’re trying to optimize, then that’s strike 2 for the “idea of Google”. Because a transparent company should at least be able to explain what service they’re trying to provide, i.e. what it is that they’re trying to help their customers do better. Right?

  • 6 Daniel Tunkelang // Feb 6, 2009 at 4:04 pm

    The “he” is Bob Wyman, not Jeff Jarvis, and he does have inside information. I’d love to hear him elaborate and continue our discussion in this venue–or anywhere else. I have no idea if he knows (or cares) that I’m blogging about our conversation.

  • 7 jeremy // Feb 6, 2009 at 4:22 pm

    This guy? http://www.linkedin.com/in/bobwyman

  • 8 Daniel Tunkelang // Feb 6, 2009 at 9:28 pm

    Yup, that one. I wish he’d attended my talk at Google, since he would have made the Q&A much more intense!

  • 9 jeremy // Feb 7, 2009 at 12:34 am

    I pinged him, invited him to join. We’ll see.

  • 10 Edward Vielmetti // Feb 7, 2009 at 11:49 pm

    Jeremy – I don’t think you can blame Google for pageranking you into the topmost popular hits; that’s where the ad revenue is.

    I almost always find the most interestingness in Google queries when I don’t put in the obvious search – when I restrict a search to site:.gov , or when I toss in some extra words to narrow down the search space, or when I re-run queries that took people to my own blog and see what they saw that made them click on my site.

    None of those techniques are ever going to generate Google any ad revenue worth measuring, and there are so many possible variants that it’s hard to do SEO to screw up the results for commercial gain.

  • 11 jeremy // Feb 8, 2009 at 12:50 am

    I don’t think you can blame Google for pageranking you into the topmost popular hits; that’s where the ad revenue is.

    Edward, I’m not trying to be dense, but.. I don’t quite get what you’re saying.. you’re saying that I can’t blame Google for how they construct the organic results to their search engine.. because that’s where all the ad revenue is? Do you mean that Google purposely constructs a certain type of both result content and result interface, in order to drive more traffic to their paid listings?

    I just want to make clear.. is this what you are saying?

  • 12 FXPAL Blog » Blog Archive » Google marketing vs. Google journalism // Feb 24, 2009 at 11:11 am

    […] is a lot of discussion on the web about Google and its proclamations, policies, and practices. One recent episode involves […]

  • 13 Jeff Jarvis Comes Clean | The Noisy Channel // Apr 2, 2009 at 6:40 pm

    […] “What Would Google Do?” I came back and assembled my reactions in a post entitled “What Would Google Do? / What Does Google Do?“. One of my strongest objections to Jarvis’s shtick is that he extols transparency as a […]

  • 14 I’m No Google Fan Boy, But… | The Noisy Channel // Apr 5, 2009 at 12:12 pm

    […] may not be a Google fan boy (start with this post if you’re new here), but the recent column in the Guardian (which, by the way, is one of my […]

  • 15 Search User Interfaces and Data Quality | The Noisy Channel // Dec 3, 2009 at 12:50 am

    […] working on the inside has also help me appreciate what Bob Wyman tried to tell me months ago–that Google has no philosophical predilection towards black box approaches, but […]

Clicky Web Analytics