Category: General

Back on my 30th birthday, my wife gave me a copy of Logan’s Run, with a card ensuring me that I’d found sanctuary. The joke is probably lost on those who haven’t seen this wonderful sci-fi B-movie, as is the title of this post, but you can crib from the script here.

But I’ll get to the point of this post, I just read in the New York Times that Equifax, one of the larger consumer credit reporting agencies in the United States,  is developing an “i-card” service that will let you create and then assert an online identity, backed up by them. Yes, they’re hardly the first to offer some kind of online identity validation, but their being a major offline player may make them different than OpenID or similar services. Then again, the article suggests that the service is complex to use, so it might just fall under its own weight.

In any case, I hope that the blogosphere takes these efforts seriously. As I’ve noted in the past (e.g., here), it strikes me as oddly antisocial that anonymous publishing is the norm in social media, at least for commenters. Yes, anonymity makes sense for whistle blowers, political dissidents, and anyone else who fears retribution. But it is hardly necessary for your average TechCrunch commenter. Instead, it makes it easy for people to post vitriol–or just nonsense–without any risk to personal reputation. I don’t see the social value.

Moreover, just imagine how easy it would be for someone who didn’t like you do start posting embarrassing comments and signing them with your name. Or perhaps someone might pursue a more subtle strategy, such as posting reasonable-sounding comments in order to advance an agenda. Less speculatively, we’ve seen how anonymity can be troublesome for the integrity of Wikipedia editing.

Given the growing role of social media, we’re going to have to cross this information accountability bridge sooner or later. I hope it’s sooner. Would it be nice if we developed a cultural norm that people stood proudly behind their online words?

One of the things I love about being in the enterprise search /  information access business is that there is always new blood keeping us old-timers on our toes and maintaining the pressure to innovate. While the competitive landscape is brutal (ask any analyst who has covered it over the past decade!), it apparently doesn’t dissuade entrepreneurs from making their own attempts to tackle the fundamental problems of making information accessible and useful.

Two of the higher profile newcomers to the scene are Attivio and Digital Reef. Attivio seems to be everywhere these days: sending its CTO to giving talks; sponsoring conferences and dinners;  and even winning awards. Digital Reef is a bit less gregarious, but they made a lot of press in March when they emerged from stealth mode after two years. Just today, they announced a partnership with FAST, the enterprise search subsidiary of Microsoft.

I’ve interacted with a couple of people at Attivio, and I’ve read some of the Digital Reef blog posts. Both companies intrigue me. But what intrigues me more is that they say almost nothing about their customers. As far as I can tell, Attivio has only announced two customers (Thumbplay.com, Intralinks) and Digital Reef hasn’t announced any. There’s nothing wrong with ramping up (I still remember the early years myself), but I’m struck by the discrepancy between the highly visible marketing and the seemingly invisible customers.

If anyone here knows more about these companies (including representatives from the companies themselves), I’d love to hear your perspectives.

Otis just wrote a post in which he cited the Open Relevance Project, an embryonic effort by the Lucene project to build a free, public information retrieval evaluation framework analogous to the TREC conference. Not surprisingly, he sees this as an opportunity for Lucene to prove that it is just as good as the commercial enterprise search engines.

On one hand, I’m delighted to see an attempt to make a TREC-like infrastructure more widely accessible. While the Linguistic Data Consortium and the University of Glasgow may only be charging enough to cover their costs, perhaps there are more efficient ways to manage corpora today. Indeed, other alternatives include publishing torrents and Amazon’s public data sets. If the bottleneck is licensing costs, then perhaps there should be a call to donate data–or to assemble collections from public domain sources.

On the other hand, if the goal of this project is to help companies evaluate competing search offerings, then I think its proponents are chasing the wrong problem. Lest you think I’m biased because of my affiliation with one of those commercial search vendors Otis taunts in his post, I encourage you to check out a post that Jeff Dalton (who is certainly pro-Lucene) wrote a year ago, entitled “Open Source Search Engine Evaluation: Test Collections“. In it, he raises a number of issues that go beyond the issue of data availability. One of the issues he brings up is the evaluation of interactive retrieval, an area for where even TREC has struggled.

I understand the desire for Lucene advocates to prove that Lucene is just as good as or better than the commercial search engines–it’s not that different from the desire every vendor has to make competitive claims about his or her own technology. To Otis’s credit, he recognizes that relevance isn’t the only criterion worthy of assessment–he also suggests extending the Open Relevance Project to include  the non-functional metrics of efficiency and scalability. But he still seems to accept an evaluation framework that would treat search engines as out-of-the-box relevance ranking engines.

I dare say I have a little bit of experience with how companies make decisions about search technnology, so let me offer my perspective. Companies build search applications to support specific tasks and information needs. For example, ecommerce sites want to help users find what they are looking for, as well as to target those users with their marketing strategies. Manufacturing companies want to optimize their own part reuse, as well as to make sense of their supply chains. Staffing agencies want to optimize utlization of their consultants and minimize their own costs. Etc.

All of the above rely on search applications to meet their needs. But I don’t think they’d be swayed by a TREC-style relevance bake-off. That’s why companies (and vendors) trumpet success in the form of metrics that reflect task performance (and there are often standard key performance indicators for the various application areas) rather than information retrieval performance. Yes, non-functional requirements like efficiency and scalability matter too–but they presume the functional requirements. If an application can’t meet the functional needs, it really doesn’t matter how quickly it processes queries, or how many documents it can index. Moreover, many companies ask for a proof of concept as part of the sales process. Why? Because they recognize that their needs are idiosyncratic, and they are even skeptical of vendors who have built similar solutions in their space. They see success stories and satisfied customers as positive–but not definitive–evidence.

To summarize: the quest to open up TREC may be of great interest to information retrieval researchers, but I’m highly skeptical that it will create a practically useful framework for comparing search technologies. I think it would be more useful to set up public frameworks where applications (both vendor-sponsored and open-source) can compete on how effectively they help users complete information seeking tasks that are representative of practical applications. I’d love to see a framework like Luis Von Ahn’s “games with a purpose” used for such an endeavor. I would happily participate in such an effort myself, and I’m pretty sure I could drag my employer into it.

I just read in The Observer that the New York Times is considering two plans to charge online users:

One includes a “meter system,” in which the reader can roam freely on the Web site until hitting a predetermined limit of word-count or pageviews, after which a meter will start running and the reader is charged for movement on the site thereafter…the second proposal [is] a “membership” system. In this model, readers pledge money to the site and are invited into a “New York Times community.”

Here is my free advice: ditch option one, and embrace option 2. It’s not that I don’t believe in charging for content, but rather that nobody else does, and it’s quixotic for even the New York Times to think it can buck the trend solo. OK, not quite solo, but the article cites New York Times executive editor Bill Keller as saying that the Times makes significantly more money  from digital advertising  than The Wall Street Journal makes from its subscription-based pay model. Of course, past performance isn’t necessarily a great predictor of the future, but it’s probably indicative of the near term.

I wrote a couple of months ago that “Community = Copy Protection“. It may also equal business model protection. Of course, the New York Times would have to put serious thought and effort into offering a community worth paying for (I hope the “baseball cap or a T-shirt” suggestion in the article is a joke). But I do believe it’s a vision they should pursue.

If you’re curious about Wolfram Alpha but tired of reading second-hand reports and third-hand hysteria about it, then be assured that your wait for first-hand access is almost over. Their blog reports that they will launch today.

UPDATE: The Wolfram Alpha site says:

Watch a live webcast of the Wolfram|Alpha system being brought online for the first time. Friday, May 15, beginning at 7pm CST

It’s been an interesting pre-launch hype cycle, particularly since I’ve gotten to watch it from a pretty good seat–a perk of being an obsessive a prolific blogger.

The initial marketing and buzz offered a level of hyperbole comparable to the hype that surrounded the publication of Wolfram’s New Kind of Science (appreciated NKS by Wolfram and fans) seven years ago. Regulars may recall that I responded by calling it “A New Kind of Marketing (NKM)“.

Apparently sensitive to the dangers of being hyped up (and thus set up to fail) as a “Google killer”, Wolfram Alpha’s marketing team reached out to influencers (and a few little people like me), offering demos and explanations. In fact, I think they were doing an admirable job of mitigating the original damage–up until April 28th. That day, as Wolfram himself was giving a public demonstration of Wolfram Alpha at Harvard, Google releasing Google Public Data to the general public. Ouch. To add insult to injury, Google’s Matt Cutts says the timing was a coincidence–an accidental upstaging!

In any case, the release of Google Public Data amplified the pressure on Wolfram Alpha. A week later, the latter was offering preview access to reporters and bloggers, presumably knowing that the testers would compare the two offerings side by side. Meanwhile, Google has continued raising the stakes through announcements like this week’s preview of Google Squared. I doubt the two companies are as focused on one another as the blogosphere makes them out to be, but it’s certainly an entertaining David vs. Goliath story (albeit where the David has a Goliath-sized ego).

And today is public launch day for Wolfram Alpha. I wish them luck! But I’m pretty sure that many of the people who’ve been waiting for this access will be disillusioned as they struggle with the  NLP interface. None of the marketing team’s attempts at expectation management can mitigate the frustration of an undocumented, brittle interface. Ah well. I did try to tell them. I hope they can make it past that initial blow and then reconsider their approach to the interface.

‘Tis the season of search conferences, at least for me, and I spent the last couple of days attending the Enterprise Search Summit in New York. I enjoyed it immensely–it’s one of the better networking events in the industry. It was great to catch up with analysts, consultants, information architects, and even competitors–or, as Nate put it, “respected foes”. I’m grateful to Michelle Manafy for putting it all together, and to Will Evans for getting me included on a social search panel.

I found the informal conversations and the final discussion session to be the most valuable activities during the two days. But I also appreciated a number of the presentations, particularly Jared Spool‘s keynote on “Search, Scent, and the Happiness of Pursuit”. Perhaps unsurprisingly, I found a few of the talks disappointingly shallow or salesy. I wish that more vendors and consultants realized that people expect substance from a 45-minute talk, and that the best sales pitch is to deliver that substance. I’m determined to see that all the talks at the SIGIR ’09 Industry Track are substantive; I hope I’ve recruited a line-up that will deliver!

One thing I also liked about the conference is that that the various participating vendors (and there were a lot of us!) were very collegial. There was, however, an exception that I feel compelled to point out. An article on internetnews.com sums it up: “Google Talks Enterprise Search, Bashes Microsoft“, in which Nitin Mangtani, lead product manager for Google enterprise search, said:

One way of doing enterprise search would be to start something in 2001 that didn’t work. You could then do a complete overhaul in 2003, which also didn’t work. In 2007, you could launch a rip-and-replace system and then … you could acquire a large, random, non-integrated system. I’m not going to name any specific company.

I thought that was catty, especially since the reference was obvious to an audience of enterprise search professionals. Moreover, Mangtani’s characterization of the enterprise search space was, to put it diplomatically, interesting. He described Google’s approach to enterprise search as being distinctive because of their attention to structured and semi-structured data. If you’re familiar with both Google‘s and Endeca‘s offerings (or FAST‘s for that matter), I think you’ll share my surprise at this particular characterization. I don’t want to commit the same offense I’m criticizing. I’ve had cordial exchanges with a few of the Google enterprise folks (including Mangtani), and I think Google has a respectable offering in the enterprise space. But they should be big enough (and mindful enough of Google’s corporate reputation) to treat their competitors with respect–and they might do a bit more homework on competitive analysis.

In any case, I can’t complain about Endeca’s visibility among participants and speakers. It seemed that I was hearing Endeca mentioned in every other talk attended–quite a feat considering that none of the talks were by Endeca partisans and that, until recently, Endeca’s marketing department studiously avoided using the term “enterprise search”! Even now, the official corporate positioning is that Endeca enables search applications. I understand the distinction, and in any case I probably shouldn’t use my blog to second-guess my colleagues in marketing. Still, it’s clear that many people looking for what they consider to be enterprise search want what Endeca has to offer, and I’m not one to let a vocabulary problem get in the way of selling software and meeting customers’ needs!

All in all, two days well spent, though I’m glad to get back to my blog–and to my day job. At least I get a small break before the the Text Analytics Summit on June 1-2!

Last week, I stopped by the Hunch office to learn more about what they’re doing, as well as to contribute my own thoughts about socially enhanced decision making. I consider Hunch, like Aardvark, to be an example of social search, but I recognize that I use the term in a broad sense. Perhaps, as Jeremy suggests, it’s better to think of social search and collaborative search being different aspects of multi-person search.

In any case, Hunch is doing some interesting things. Their mission, roughly speaking, is to become a Wikipedia for decision making. They are inspired by human computation success stories like 20Q.net and presumably the ESP Game. Their general approach is to learn about people by asking them multiple-choice questions that help cluster them demographically (“Teach Hunch About You”), and then to create customized decision trees to help people find their own answers to questions. The questions themselves are crowd-sourced from users (though now they are vetted first in a “workshop”).

They’re learning as they go along. For example, they’ve recognized that it’s important to distinguish between objective questions (e.g., concerning the price of a product) and questions of taste (e.g., what is art?). They’re also experimenting with interface tweaks, including giving users more control over what information their algorithms use to rank potential answers, and allowing users to short-circuit the decision tree at any time by skipping to the end.

Perhaps of particular interest to readers here, they’ve made an API available, which you can also play with in a widget on their blog.

As I told my friend at Hunch, I’m still skeptical about decision trees. Maybe I’m a bit too biased toward faceted search, but I don’t like having such a rigid decision making process. Apparently they’re not wedded to decision trees, but they are understandably concerned about creating a richer interface that might turn off or  intimidates ordinary users. I can’t deny that decision trees are simple to use, and I can’t argue with their 77% success rate.

Still, the rigidity of a decision tree leaves me a bit cold. Even if it leads me to the right choice, it doesn’t give me the necessary faith in that choice. Transparency helps, and I like that you can click on “Why did Hunch pick this?” to see what in your question-specific or personal profile led Hunch to recommend that answer. But I’d like more freedom and less hand-holding.

I still have a handful of invites; let me know if you’re interested. As usual, first come, first serve.

There has been a flurry of reports about Twitter search–whether about Twitter’s plans to improve their search functionality or about alternative ways to search Twitter. But Danny Sullivan makes a great point in a recent post about Google:

Ironically, Google gets a taste of its own medicine with Twitter. It still can’t access the “firehose” of information that Twitter has, in order to build a decent real-time search service. If it can’t strike a deal, expect to hear the company start pushing on how “real-time data” should be open.

Of course, that logic applies not only to Google, but also to anyone with aspirations to build a better mousetrap for Twitter search. As things stand, applications can’t do much better than post-processing native Twitter search results–which makes it hard to offer any noticeable improvement on them. If Twitter offered full Boolean set retrieval (e.g., if a search for star trek returned the set of all tweets containing both words), then applications could implement lots of interesting algorithms and interfaces  on top of their API. I’d love to work on exploratory search applications myself! But the trickle that Twitter returns is hardly enough.

I believe this limitation is by design–that Twitter knows the value of such access and isn’t about to give it away. I just hope Twitter will figure out a way to provide this access for a price, and that an ecology of information access providers develops around it. Of course, if Google or Microsoft buys Twitter first, that probably won’t happen.

I’ll be at the Enterprise Search Summit in New York next week, participating on a panel Tuesday morning to discuss “Emergent Social Search Experience”. Our game plan as a panel is to discuss what social search is, why it matters, and how to implement it.

Obviously these are broad questions, but here are my rough notes:

WHAT: Social search means many things, but they have one common thread: improving information seeking through the knowledge and efforts other people. Back in the mid 90s, researchers distinguished between semantic and social navigation as the ability to explore information based on its objective, semantic structure, versus choosing a perspective based on the activity of another person or group of people. Perhaps the earliest instance of social search was collaborative filtering, still popular today as driver for product recommendations on sites like Amazon. But social search is much more than collaborative filtering. Building on the 90s vision of social navigation, we can give users full control over a social lens through which to view information, e.g., show me the local restaurants where women in my mom’s demographic like to eat brunch. Social search also includes explicit and implicit collaborative approaches, such as finding an expert to help you with a search, or building shared knowledge management artifacts that increase the collective efficiency of information seeking.

WHY: The “why” of social search depends on the specific aspect of social search that we’re discussing. But the common theme is this: we all know that, for a large swath of information needs, we prefer to turn to a person than to ask a machine. Sometimes that’s appropriate, and it’s a question of finding the right person to ask. But often we have no need to bother any one; we just want to borrow someone else’s perspective—or to assemble a composite perspective. There’s an efficiency gain of not reinventing the wheel, as well as an upside of discovering people (or information by way of those people) that may be valuable to you in ways you didn’t anticipate.

HOW: Again, it depends on the aspect of social search. We need rich knowledge representations that treat both information and people as first-class objects, and interfaces that let people seamlessly use both. Endeca does this by supporting record relationship navigation for multiple entity types (e.g., documents, people), as do interfaces like David Huynh’s Freebase Parallax. To facilitate collective knowledge management, we need to make contribution both easy and rewarding: the reason people don’t contribute to such systems today is that they are onerous and don’t work. Some of the work Endeca has done with folksonomies is encouraging: we found that we can productively recycle folksonomies (or even search logs) in combination with automatic text mining techniques. Finally, we need to rethink our attitudes toward privacy, anonymity, and reputation. Consumer social networks like Facebook and Twitter have shown us that users are willing to forgo privacy in order to gain social benefits. Wikipedia has shown us that a group of strangers can assemble a valuable collective knowledge store. But Wikipedia, product reviews, blog comments, etc. have shown us that the default of anonymity can undermine the trust we have in these socially constructed artifacts. As we evolve these tools—and as we work to apply them within the enterprise, we need to simultaneously work to evolve our social norms.

Those are my thoughts. But, in the spirt of social search, I’d love to reach out to experts here for ideas. If you were attending a panel about social search, specifically in the context of an event target to enterprise search practitioners, what would you want to hear about? For that matter, if you were participating on such a panel, what would you talk about? Bear in mind that the audience will consist of practitioners, not researchers, and I’ll only have one third of a 45-minute session–some of that reserved for Q&A.

Woo hoo, I have preview access to Wolfram Alpha! I’ve only had a short time to play with it, but I can already report that my experience confirms my previously expressed expectations: the NLP is very brittle, but there’s great potential for structured queries on quantitative data. Here is an example use case that, in my view, shows Wolfram Alpha’s strengths:

This bit of analysis tells a great story: Microsoft has almost three times as much revenue as Google, but Google has about 50% higher revenue per employee. Meanwhile, Yahoo is in third place on revenue,  number of employees, and revenue per employee. Ouch.

As I said, this query shows Wolfram Alpha favorably. What you don’t see are the false starts it took me to get this query to work. The NLP interface, in my view, is a really bad idea. Instead, Wolfram Alpha should be helping users generate good structured queries–and, better yet, helping other businesses build such queries through APIs. Wolfram Alpha could deliver an excellent plug-in for Excel, if they can expose a workable query API. I have no idea whether the company is able or willing to go down this path, but I hope someone there is listening to this free advice.

I can’t share my account, but I’m willing to take suggestions for queries through the comment thread, and I’ll try my best to share what I learn.