Category: General

I just came back from the monthly NY Tech Meetup, whose theme this evening was “Built on Twitter“. While the meeting was well organized (a testament to Nate Westheimer, who received the torch from Meetup CEO Scott Heiferman, I had mixed feelings about the demos. Everyone is capitalizing on Twitter’s buzz, but so few people seem to be creating anything valuable on top of it.

But, by luck, Daniel Lemire sent me a link to Sylvie Noël’s post about a paper by HP Labs on “Twitter: Social Networks that Matter: Twitter under the microscope” by Bernardo A. Huberman, Daniel M. Romero and Fang Wu. She also pointed to an executive summary by Forrester analyst Jeremiah Owyang.

The paper is insightful. The authors practically had me at hello–this is the paper’s third paragraph:

While the standard definition of a social network embodies the notion of all the people with whom one shares a social relationship, in reality people interact with very few of those “listed” as part of their network. One important reason behind this fact is that attention is the scarce resource in the age of the web. Users faced with many daily tasks and large number of social links default to interacting with those few that matter and that reciprocate their attention. For example, a recent study of Facebook showed that users only poke and message a small number of people while they have a large number of declared friends. And a casual search through recent calls made through any mobile phone usually reveals that a small percentage of the contacts stored in the phone are frequently contacted by the user.

They then define a user’s “friend” as a person to whom that user has specifically directed at least two posts and show that the a user’s number of friends is a better predictor of the user’s activity (number of posts) than the user’s number of followers. Having thus validated the number of friends as a more important input variable than the number of followers, they explore the friend graph, which turns out to be much sparser than the follower graph.

Their conclusion:

Many people, including scholars, advertisers and political activists, see online social networks as an opportunity to study the propagation of ideas, the formation of social bonds and viral marketing, among others. This view should be tempered by our findings that a link between any two people does not necessarily imply an interaction between them. As we showed in the case of Twitter, most of the links declared within Twitter were meaningless from an interaction point of view. Thus the need to find the hidden social network; the one that matters when trying to rely on word of mouth to spread an idea, a belief, or a trend.

I urge you to read the whole paper, as my abbreviated version hardly does it justice. And then, if you’re practically minded, think about ways to build applications on Twitter than leverage this real social network that is hidden in plain sight.

I further suspect that the authors result generalize beyond Twitter to other social networks where the cost of connecting is far lower than the cost of actually investing in the connection. It doesn’t seem hard to identify the hidden social network, and by doing so we can unlock its value.

Of course, Twitter has the virtue that its network is mostly available to the public, not hidden behind a walled garden like LinkedIn or Facebook. As a result, I expect that Twitter will drive both research and innovation in the social network space, at least in the near term.

I was going through my alerts this morning for enterprise search and found this nugget in an interview with an enterprise search executive (the interview is easy to find on the web):

What is the basis of your firm’s technical approach?

XXX provides a highly scalable and manageable information access platform built on open standards. XXX transforms raw data, whatever its nature, into actionable intelligence through best of breed indexing, extraction and classification technologies.

The term “information access platform” has become standard enough, but it tells us nothing about the company’s technical approach. “Highly scalable and manageable”? Again, these are neither specific nor informative about the approach. In fact, the only thing we learn about the company’s technical approach is that it uses open standards, and we don’t even hear which ones. And perhaps it is unfair to single out this company, since I’ve seen similar content-free descriptions across the industry.

But compare that to how I answered Ron Miller in a recent one-on-one:

What is the differentiator for Endeca search?

Endeca combines a set-oriented retrieval approach with user interaction to create an interactive dialogue, offering next steps or refinements to help guide users to the results most relevant for their unique needs. An Endeca-powered application responds to a query with not just relevant results, but with an overview of the user’s current context and an organized set of options for incremental exploration.

I’m not claiming that my answer is complete–it is hard to go deep in a short interview. But I at least tried to explain *what* Endeca does, rather than just concatenate buzz words. A litmus test is that you could not pick another enteprise search vendor name out of a hat and substitute it into that paragraph.

As I posted the other day, the enterprise search market is beset by marketing and hype. There are lots of folks to blame for this set of affairs, but we who are selling enterprise search technology have a particular responsiblity for emphasizing reality over hype. Let’s keep it real.

According to Wikipedia:

Click-through rate or CTR is a way of measuring the success of an online advertising campaign. A CTR is obtained by dividing the number of users who clicked on an ad on a web page by the number of times the ad was delivered (impressions). For example, if a banner ad was delivered 100 times (impressions delivered) and one person clicked on it (clicks recorded), then the resulting CTR would be 1 percent.

The click-through rate measure is the key enabler for the pay-per-click (PPC) advertising model, where advertisers only pay when a user actually clicks on an advertisement to visit the advertisers’ website. A testament to the success of the PPC model is that it accounts for the overwhelming majority of Google’s $20B+ annual revenue.

Most of the attention to CTR has been ad-centric. The quality of an ad–or, rather, of how well an ad is targeted–is largely measured based on its click-through rate, average over all of the users to whom it is presented.

Google, in particular, requires advertisers with a low CTR to place a higher bid per click. The relative ranking of ads reflects a product of the bid and the CTR. This product can be interpreted in one of two ways: either combination of the advertister’s and users’ interest, or as the expected revenue that the ad will generate for Google.

But there is a different way to look at CTR. Instead of looking at the aggregate behavior for an ad across all users, why not look at the aggregate behavior for an user across all ads?

For example, consider a user who never clicks on ads. Perhaps the user is using an ad blocker that the search engine cannot detect, or the user may simply be ignoring the ads. At the other extreme, there are users who click on ads at higher than average rates (though some may be bots committing click fraud).

Of course, all user behavior is averaged  in calculating the CTR for an ad. But a user-centric view suggests a a couple of advertising personalization strategies:

1.  Don’t bother showing ads to a user who never clicks on them, since there is no value in doing so. If ad display alone is valuable in influencing users, then there should be a cost-per-impression component, though that would require a reliable way to determine that the user actually sees the ad.
    
2. Calibrate the threshold for ad quality (i.e., the minimum CTR across all users) to a user’s propensity to click on ads. Doing so could reduce the annoyance of people with high thresholds while increasing the ad revenue from people with low ones.

It’s clear that enough people click on ads to keep search engines in business, and that the easy availability of ad blockers (AdBlock for display ads, CustomizeGoogle for Google’s PPC ads) has not made a dent in this revenue stream. Nonetheless, showing ads to users who don’t click on them degrades user experience without generating revenue for anyone–a lose/lose.

Personalizing advertising based on a user’s demonstrated inclination to click on ads feels like a no-brainer. Has anyone tried it?

Two posts that I read recently raise the question of whether the information economy has become too efficient.

In “The Implications of the Information Economy” (via Oscar Berg), Rachel Happe argues:

The information economy has created a better informed but often impulsive consumer and it has created a networked information effect that ripples quickly through society allowing small pieces of information to set off a disproportionately large reaction – some positive and some negative. Like never before we can also act on information immediately creating economic whiplash.

In “Are We Losing the Narrative Self?“, Michael Goldhaber discusses how we are losing the concept of defining ourselves through our histories in favor of a more reductionist approach where:

You are your current set of interests, contacts, Twitter postings, Facebook postings, blog postings, listserv controversies, your latest images, and YouTube videos — trapped in an eternal but changing present that gives no sense of birth or death or growing up, or even growing at all.

I’m hardly a Luddite, but I sympathize with both arguments. I applaud the technical advances that have made information flow so efficient, but I also recognize that our cultural evolution has lagged this technological progress.

We’re in similar situations with regard to trust, privacy and connectedness. Our ability to transmit data has far outstripped out ability to comprehend information, and our initial reflexes tend to promote the superficial.

Historically, technical advances that improve the flow of information have done far more good than harm. Just consider the evolution from scribes to the printing press to digital libraries as an example. Yes, we also ended up with reality TV shows and Perez Hilton, but overall we’ve made progress.

Perhaps technical advances will help us overcome the same superficiality they have promoted. If not, it will be up to our own cultural evolution to catch up.

Today, when I was looking at recent Twitter updates from the people I follow, I saw one that read:

Just started using http://twply.com/ to get my @replies via email. Neat stuff!

Curious, and perhaps a bit groggy from last night’s festivities, I signed up for a free account. I then found that twply sent out a tweet on my behalf with the same “neat stuff” message. I deleted the message immediately (to the limited extent that one can delete anything on Twitter), but I was quite annoyed. And I’m not the only one.

I’m a big fan of viral marketing, and I think it may ultimately supplant advertising as the main way we find out about new goods and services. But what twply is doing is hardly viral marketing. Rather, it is sending out spam intended to simulate endorsement. And their approach clearly even brings short-term results, perhaps even enough to offset the cost of the backlash against their spammy approach. Even Facebook learned its lesson when it had to scale back its Beacon system, that would have taken a similar approach on a larger scale.

The other day, Ben Kunz wrote a satirical piece about a business model of selling our opinions. At least in his “modest proposal”, I imagined I’d be a willing and compensated participant. Now I find that I unwittingly sold my opinion for nothing!

I hope the backlash against twply discourages other companies from pursuing this approach. Personal endorsements and recommendations are an extremely important source of information, especially in a world of information and advertising overload. Undermining their integrity undermines this oasis in a sea of untrustworthy sources, and it’s an oasis we have to protect.

Curt Monash recently shared his views of the text analytics market through his blog and a slide presentation that he’s made available online. The presentation is refreshingly hype-free, and I recommend you take a look.

His observations about the web search market are spot-on: the current attention is on transactional queries (see Andrei Broder‘s classic paper on the taxonomy of web search for an explanation of navigational, informational, and transactional queries), and web search generally is dominated by the dynamics of adversarial information retrieval. Depressing (to me, at least), but accurate. He does see potential future with better interfaces, but he asks, “how good does the technology have to get before people care?” My sentiments exactly.

On to the enterprise market, which is more interesting. Here it’s harder to summarize Monash’s thoughts, except to say that he sees the current landscape of enterprise search offerings as hopelessly confused.

Monash divides the enterprise market into public-facing site search, which he further divides between e-commerce and “general”; and “true” enterprise search which seems to mostly denote intranet search; and custom publishing. While I’m not entirely comfortable with his taxonomy of the space, I do give him credit for laying one out.

He then goes on to explain how “one-size-fits-all” approaches have failed and how the enterprise search market landscape is “bollixed”. He lists a number of technical challenges, all of which I agree with.

But I’d add one: the need for content enrichment techniques and interfaces that support interaction, exploration, and discovery. Yes, we’ve seen these as buzzwords in vendor hype, but that doesn’t make them any less real. There’s been too much emphasis on best-first, known-item search, and not enough on the other use cases that comprise enterprise search and information access.

I think that exploratory search will eventually be important for web search too, but the complacency with current approaches kills any sense of urgency. There is no imminent threat to Google’s reign.

In the enterprise search market, however, there is a justified dissatisfaction with the status quo. And, in my belief and experience, that is because too many people (vendors and enterprises) are trying to treat the enterprise like a microcosm of the web, where the only major differences are the connectors to acquire content and the ranking algorithm to sort results. Getting these right is necessary but not sufficient. Interaction, exploration, and discovery–in short, HCIR–are not just nice-to-have features, but rather are essential to making search work in the enterprise.

I must have been asleep at the RSS reader a couple of weeks ago, because I missed this gem in  “Lost in the Crowd“, David Brooks’s review of Malcolm Gladwell’s new book Outliers in the New York Times:

Control of attention is the ultimate individual power. People who can do that are not prisoners of the stimuli around them. They can choose from the patterns in the world and lengthen their time horizons. This individual power leads to others. It leads to self-control, the ability to formulate strategies in order to resist impulses. If forced to choose, we would all rather our children be poor with self-control than rich without it.

Fortunately, Mike Elgan called it to my attention in a column entitled “Work Ethic 2.0: Attention Control“. And, like me, Elgan reacts much more strongly to Brooks’s comment about control of attention than to anything he actually says about Gladwell’s book.

But  Elgan’s concern is with what he sees as the “distraction virus” of the internet in general, and of social media in particular. I don’t dispute his observations, but I have a different take on Brooks’s point.

It’s true that self-control gives us individual power, and we knew all this long before we had to contend with the online demands on our attention. For example, Herman Hesse wrote in Siddhartha:

“I can think. I can wait. I can fast.”

“That’s everything?”

“I believe, that’s everything!”

What’s new is that attention is becoming a currency to rival the tangible goods we’ve usually thought of as scarce and valuable. But, as Brooks and Elgan note, it differs from money in its far more subjective nature.

A growing economy revolving around something money can’t buy, and which is subject to the vicissitudes of individual control. That certainly makes things interesting.

One of the mistakes we often make in our quest for economic reductionism is to assume that all value can be monentized. But this model often breaks down in the context of online communities. As Manila Austin, a psychologist who heads up research at Communispace, put it, “People want the validation that they are being heard.”

In a BusinessWeek article entitled “Will Work for Praise: The Web’s Free-Labor Economy“, Stephen Baker tells some stories about the unpaid volunteers who invest their energy in helping others online, and the companies who try to monetize their efforts. Of course, the volunteers are only unpaid in financial terms; they are very much incented by one thing money can’t always buy: attention.

These days, there’s a lot of concern about what business models will sustain social media–particularly blogs and Twitter. It’s clear from stories like Baker’s that many participants in online communities are sufficiently motivated to invest their own time–and possibly even their own money–in order to reap the non-financial reward of attention.

As it is, most bloggers and tweeters are unpaid for their efforts. Perhaps this model will ultimately sustain the blogosphere, and attention will trump money as the currency of communication.

A few days ago, Jon Pareles wrote a New York Times article entitled “Songs From the Heart of a Marketing Plan“, describing the increased licensing of new music for commercials, video games and soundtracks.

He comments:

Selling recordings to consumers as inexpensive artworks to be appreciated for their own sake is a much-diminished enterprise now that free copies multiply across the Web.

While people still love music enough to track it down, collect it, argue over it and judge their Facebook friends by it, many see no reason to pay for it. The emerging practical solution is to let music sell something else: a concert, a T-shirt, Web-site pop-up ads or a brand.

Licensing music to marketers is hardly new, but there is an urgent pressure from the decline of CD sales and the industry’s inability to make up the difference in digital music sales. The music industry isn’t about to go down without a fight, and licensing music to marketers is immune to digital piracy.

Rather, it ties the fate of the music industry, at least in part, to that of the advertising industry. While no industry is recession-proof, advertising seems better placed than most.

But consider the irony. Historically, people either have bought music or have listened to it for free on ad-supported media like radio. Is the future one where music is distributed for free and embedded in the very ads that historically subsidized its distribution?

Moreover, if this change is successful, will we see it extend to other digital media–like movies, books, or even news? Could the entire world of publication revolve around advertising?

I hope this is just a paranoid fantasy. It wasn’t that long ago that we were proclaiming “content is king”. Perhaps Andrew Odlyzko was right to challenge this assertion in his article “Content is not king“–though he argued the primacy of communication over content. I just hope that communication does not devolve to advertising.