Author: Daniel Tunkelang

You’d think that, after my less than flattering coverage of the Wolfram Alpha pre-launch hype, they would have blocked my IP address even after the public launch. I certainly thought so.

But I was wrong. Last week, someone from their business development group contacted me to arrange a preview of the system. He spent an hour with me today, discussing the system and showing me how it worked. I couldn’t type into the search box myself, but he did let me suggest queries and then entered them for me. He also encouraged me to speak freely about what I’d seen–they’re actually concerned that the hype and its propagation through the echo chamber are setting inappropriate expectations for the product, and eager for cooler heads to prevail or at least temper the exaggeration.

While I stand by my critique of the marketing, I am persuaded that Wolfram Alpha has built something interesting. It is unfortunate that it’s invited so much comparison to Powerset and is being hyped as a potential Google killer.

As Nova Spivack and others correctly pointed out, Wolfram Alpha isn’t really competing with Google, but rather with Wikipedia and Freebase. Wolfram Alpha offers two major core competencies. The first is a vast amount of curated knowledge: they claim to have roughly 20 trillion “facts” as raw inputs, not the results of any kind of inference process. The second is the capability to relate those facts programmatically by exploiting their formal structure and to generate inferences from them.

Yes, Wolfram Alpha also has a natural language component, but that aspect of their offering is, by their own admission, its weakest point. Having a natural language interface is a necessary evil to provide a way for human beings to access this structured data. But unfortunately its the aspect that people have fixated on most–and will likely continue to do so.

In my view, a more productive way to think of Wolfram Alpha is as a highly structured content repository that offers an API in order to get at the content programatically. I saw example queries like (population of china) + (population of japan) / (population of USA). By itself, that’s a parlor trick that feels a bit like Google Calculator. But the ability to include expressions like is_a (china, country) and population (capital_city (china)) in an application (e.g., Excel) would create real value. Note: this is my fanciful syntax, not theirs.

In May, Wolfram Alpha plans to open up their web site to the public, which means that people will have access to the natural language interface to their content. While the launch will surely generate a lot of press, I suspect that most people will focus on the natural language interface, or on the inability to handle subjective information needs.

That’s a pity. The technology has challenges, but I think the questions people should be asking are if, when and how they will be able to integrate it with business applications. Wolfram Alpha may have value as a stand-alone alternative to Wikipedia for objective data, but its real potential is as a service to use in other applications.

Let me start by saying that I really like LinkedIn. I use it as everything from a self-updating address book to a gateway to professional communities like the enterprise search professionals group. I am delighted by the information LinkedIn has assembled about companies just by aggregating user profiles. In short, I take LinkedIn quite seriously as a professional networking tool.

With that preamble out of the way, I’d like to vent a bit about LinkedIn’s approach to search. Directories are a poster-child domain for faceted search. LinkedIn specifically has high-quality semi-structured data, since users are personally incented to optimize their own findability. Moreover, the process doesn’t even seem adversarial–I haven’t seen any Joe the Scammer claiming to be the CEO of a Fortune 500 company (oops, bad example). LinkedIn has done the best job I’ve seen of aggregating high-quality data about people’s professional history–in a volume that is not only unprecedented but more importantly is large enough to be broadly useful. And the site designers clearly care about search: they still proclaim above the search box, “New Improved Search!” (see my earlier review here).

Why is faceted search so valuable for directories? Because finding someone is often a task that calls for exploratory search. Unless you’re looking someone up by name (and hopefully by a unique name), you’re not performing known-item search. Rather, you’re looking for a potential employee, employer, business partner, or expert. You may not even know what you want until you have a sense of what’s out there–the different companies in the space, the different relevant job titles, etc. Moreover, now that LinkedIn has a significant amount of text associated with its  users (e.g., the Q&A section and forums), it could do a much better job of linking people to the content they produce.

I understand that uniting the social network functionality of LinkedIn with search is hard enough, and that introducing faceted search makes the problem that much harder. But it’s not impossible, and the value of such an application more than justifies the effort. So far, LinkedIn has benefited from having the best data. But users have no incentive to give LinkedIn exclusive access to that data. LinkedIn knows this, and its increasing emphasis on community will surely make it harder for someone to compete just by offering better information seeking support.

Still, the core value proposition of LinkedIn is tightly bound to the site’s search functionality, and LinkedIn would do well to take a more modern approach. Doing so would increase the site’s value dramatically, and I’m certain LinkedIn would find ways to monetize that value.

Today’s New York Times features an essay by Noam Cohen entitled “Wikipedia: Exploring Fact City“, in which Cohen explores the metaphor of Wikipedia as an information metropolis, repete with dead-end streets and industrial districts. It’s a nice read, even if Cohen gets a bit carried away with his artistic license.

But Nick Carr reacts strongly to the notion that Wikipedia offers “sidewalk-like transparency and collective responsibility”. In his post, entitled “Potemkinpedia“, Carr argues:

Wikipedia has imposed editorial controls on those articles, restricting who can edit them. Wikipedia has, to extend Cohen’s metaphor, erected a whole lot of police barricades, cordoning off large areas of the site and requiring would-be editors to show their government-issued ID cards before passing through.

Carr is right that that Wikipedia exerts editorial controls, and it’s true that not everyone agrees with Wikipedia’s means of doing so. Still, I feel his tone seems unnecessarily hostile, and he doesn’t point out that the amount of top-down control exerted to maintain Wikipedia is extremely low compared to the value it creates. Indeed, he cites “hundreds” of pages being protected without noting that Wikipedia contains 2,817,176 articles in English alone.

One of Wikipedia’s founding principles is “Assume good faith.” Perhaps I should do that myself, yet I can’t help but wonder if Carr’s relationship with Brittanica predisposes him against Wikipedia. In fairness to Carr, he has pointed out that relationship in some of his articles (like this one). But I actually hunted down a reference on his Wikipedia entry.

Personally, I give Wikipedia high marks for transparency. What other publication exposes the full revision history of its content and provides a public discussion forum for both its content and editorial process? I’ve had my disagreements with that process, but I can’t fault its openness. Indeed, my main criticism of Wikipedia is that it doesn’t require such transparency of authors, but instead allows authors to contribute anonymously.

Perhaps Carr sincerely wants Wikipedia to be more like Brittanica, or at least wants everyone to know that Wikipedia is not 100% unlike Brittanica. And perhaps Cohen offered an oversimplified vision of Wikipedia’s governance that some could see as a utopian vision of anarchy. But I think Carr is being a bit disingenuous. After all, even a metropolis has a mayor.

Curt Monash recently published a pair of sweeping posts about the future of the information ecosystems:

• The grand discussion on the future of journalism
• Where I think the information ecosystem is headed

They’re nice posts, and Monash does a great job of playing by the rules of the link economy (the one time I agree with Jeff Jarvis) and bringing together thinking from around the blogosphere.

But I do have some differences. I posted a comment on Monash’s post about where the information ecosystem is headed, which I’ll reproduce here:

I think you may be underestimating the challenge of transitioning from an ecosystem dominated independent information providers to one dominated by vendors or analysts motivated by self-interest.

Yes, everyone has self-interest–I’m not trying to suggest otherwise. But, much as I’m sure anyone who reads my blog takes anything I say about enterprise search with a grain of salt, I’m sure anyone who reads your blog maintains a healthy skepticism towards you say about the vendors you do business with–or about their competitors. I think there will always be a market for information that comes without conflicts of interest.

The question of course, is how large that market will be, and where consumers’ willingness to pay will intersect the cost of production, especially if independent information providers are competing with vendors or analysts who provide information for free in order to market their products and services.

And, while I’m specifically thinking about technology news / analysis, the same goes for other arenas, like politics. Do we want all of the reporting to come from activist organizations? Some would argue that’s already the case, but it could be much, much worse.

It’s all about who you know, and this week I was lucky to meet one of the investors in Hunch, a decision-making tool from Flickr founder Caterina Fake that “gives you its best hunch” of what you would like when you’re feeling indecisive. It made a big splash today and is the top story on Techmeme as I write.

It’s an intriguing concept. I played with it for a bit, and I have to confess I’m not indecisive enough to fully appreciate it. But I can see the appeal for folks who would like to crowd-source minor decisions and thus introduce a bit of randomness and novelty into their lives. Especially for decisions like what to eat or where to go to college.

In any case, my new acquaintance was kind enough to not only give me an account, but also to provide me with 10 invites to distribute to incorrigibly curious.

As always, it’s first come, first serve.