Category: General

Dina Bass from Bloomberg reports today that Steve Ballmer is talking smack about Google:

“Google does have to be all things to all people,” Ballmer said yesterday in an interview in New York. “Our search does not need to be all things to all people.”

That is an interesting take on the search market–an attempt to turn a bug (Microsoft’s 8% market share vs. Google’s 63%–at least in the United States) into a feature. Ballmer essentially claims that Google’s near-monopoly stifles its innovation.

I agree that Google hasn’t been particularly innovative when it comes to search interfaces, but I’m not persuaded by Ballmer’s hand-waving “innovator’s dilemma” reasoning. Besides, as Chris Lake at Econsultancy notes, “it’s hard to know exactly what Microsoft is trying to be, in terms of search.”

Lake goes on to make an excellent point:

Search has always been about intent. That’s essentially what a search query is: an indicator of intent. You want something, you need something, you mean to purchase something, you’re going to do something.

But most queries do not reveal the exact nature of intent.

And here is the money shot:

The search engines might be able to determine intent automatically… But for me nothing works as well as asking the question, and seeking out some explicit data. Ask the question!

Yes, it’s like Feynman said: you just ask them! So much effort in the search industy aims at coming up with more clever ways to divine the user’s intent automatically, and so little focuses on building better tools to work *with* the user. Yes, I’m just beating the HCIR drum again–it’s what I do here. 🙂

But I can’t let a moment like this pass without pointing it out. If Microsoft wants a serious shot at Google, it should invest less in bribing users and more in HCIR.

Sadly, as Silicon Alley Insider’s Eric Krangel learned from a discussion with Microsoft Search director Stefan Weitz, “Microsoft doesn’t want to scare off users by introducing any dramatic changes to what people expect from the search engine experience.” Or, as Krangel summarized it pithily, what we can expect are “tweaks.”

I understand that Microsoft can’t ignore the fact that users have been trained on Google. But Microsoft is in a market where it has little to lose and everything to gain. This is the time to be bold, not conservative. Moreover, some the best HCIR researchers are working for their own research division! Want to build a better search engine? Just ask them!

Regular readers may not be surprised that my commentary on the Wolfram Alpha pre-launch publicity didn’t earn me a hands-on preview–though it did earn me a surprisingly positive email from their PR department. But fortunately I have my sources, and one of them was kind enough to share reactions to demo of the system.

His impressions in brief:

• He’s impressed with the technology, though dubious that they have a business model.
• Their knowledge base incorporates 10 trillion “facts” (RDF triples) derived from curated sources.
• They focus on factual and numerically oriented queries, as opposed to fuzzier semantic ones.
• Their engine is based on the approach described in NKS.
• Their presentation interface reminds him of Wikipedia’s infoboxes.

My reaction: still intrigued, still skeptical. It sounds like a great toy, but a toy nonetheless. But I’ll try to keep an open mind until I get to play with it myself.

Now that the cat is out of the bag, I’m proud to tell readers here about an effort I’ve been involved with over the past few months. As reported in TechCrunch and Search Engine Land,  the Financial Times just launched Newssift, a semantic search engine, powered by Endeca, that sifts through business news. Regular readers may recognize the application from an example I used in my presentation on exploring semantic means.

I’m very excited about the Financial Times embracing exploratory search, and I think Erik Schonfeld from TechCrunch gets it right when he notes that the site “employs several subtle navigational techniques that make it more of a discovery engine than a search engine.”

Sorry to keep you guys in the dark about this for so long, but I hope it was worth the wait.

Here is some of the news coverage of the launch (yes, I’m shamelessly plugging this):

• AltSearchEngines: Announcing News Search Engine Newssift
• eWeek: Search Engine for Business News Rolled Out
• Financial Times: FT Group launches ‘semantic search’ tool
• Information Week: New Search Site Debuts For Business Users
• paidContent:UK: FT Launches Semantic News Aggregator Newssift
• Reuters: Searching for business with the Financial Times
• ResourceShelf: Faceted Browsing + Search: Financial Times Launches Newssift …
• Search Engine Journal: Why Search Is Broken, and How Semantic Search Fixes It
• Search Engine Land: Financial Times Launches Newssift, A Business News Search Engine
• Silicon Alley Insider: Pearson Launches A Financial Times-Branded News Search Engine
• TechCrunch: The Financial Times Launches Its Own Business News Search Engine (Newssift)

I’ve always bristled at the legal principle, at least in the United States, that a judge has absolute power to decide what evidence is admissible in a trial. I recognize how this power is can prevent abuse by prosecutors or other government officials (cf. 4th amendment, 5th amendment). I also realize that judges are supposed to ensure that the law constrains jurors, and that there are important historical precedents for having judges enforce the law in the face of juror prejudice.

Nonetheless, I  find that this power puts jurors in an awkward position. As luck would have it, I’ve only served as a juror once, in a civil case. When the plaintiff  asked a witness questions following a line of inquiry that seemed highly relevant to the case, the judge sustained objections by the defendant. The case was settled mid-trial, but my fellow jurors and I questioned whether the judge was right to sustain the objections. In fact, we had become increasingly sympathetic to the plaintiff, since we had come to our own conclusions about the answers to the suppressed questions, and those presumed answers made a strong case for the plaintiff.

That brings us to today’s New York Times story: “Mistrial by iPhone: Jurors’ Web Forays Are Upending Trials“. A juror in a federal drug trial admitted to the judge that he had researching the case on the Internet. The judge questioned the rest of the jury, only to find out that eight other jurors had been doing the same thing. The  judge declared a mistrial.

The article quotes Douglas L. Keene, president of the American Society of Trial Consultants: “There are people who feel they can’t serve justice if they don’t find the answers to certain questions.”  The article continues by quoting University of Texas law professor Olin Guy Wellborn III: “the rules of evidence, developed over hundreds of years of jurisprudence, are there to ensure that the facts that go before a jury have been subjected to scrutiny and challenge from both sides.”

Some rules make a lot of sense to me. For example, I’m not amused by the juror who recently used Twitter to provide a running commentary on the proceedings of a trial. That was not only illegal but just plain dumb.

But preventing jurors from doing their own research is a hard rule for me to stomach as someone who has not only devoted his professional life to the process of uncovering truth, but who is actually building machinery with the goal of improving that process. There is something that feels intellectually dishonest about even temporarily giving up that pursuit, especially given the potential stakes of a legal proceedings. Trusting in the infallibility of a judge just doesn’t come naturally to a skeptical inquirer.

Nonetheless, I don’t expect the advent of mobile browsers to up-end centuries of legal tradition; it is more likely that people simply won’t be allowed to bring them into court houses. If nothing else, the story serves as a reminder that some of our bottlenecks in working with information aren’t technical.

This month’s issue of IEEE Computer is a special issue featuring information seeking support systems, edited by Gary Marchionini and Ryen White. You can read their introduction for free here; unfortunately, the articles, while available online, are only free for IEEE Xplore subscribers.

What I can share is a 500-word sidebar I wrote that appears on p. 39, in an article by Peter Pirolli entitled “Powers of 10: Modeling Complex Information-Seeking Systems at Multiple Scales“.

Precision and Recall

Information retrieval (IR) research today emphasizes precision at the expense of recall. Precision is the number of relevant documents a search retrieves divided by the total number of documents retrieved, while recall is the number of relevant documents retrieved divided by the total number of existing relevant documents that should have been retrieved.

These measures were originally intended for set retrieval, but most current research assumes a ranked retrieval model, in which the search returns results in order of their estimated likelihood of relevance to a search query. Popular measures like mean average precision (MAP) and normalized discounted cumulative gain (NDCG) [1] mostly reflect precision for the highest-ranked results.

For the most difficult and valuable information-seeking problems, however, recall is at least as important as precision. In particular, for tasks that involve exploration or progressive elaboration of the user’s needs, a user’s progress depends on understanding the breadth and organization of available content related to those needs. Techniques designed for interactive retrieval, particularly those that support iterative query refinement, rely on communicating to the user the properties of large sets of documents and thus benefit from a retrieval approach with a high degree of recall [2].

The extreme case for the importance of recall is the problem of information availability, where the seeker faces uncertainty as to whether the information of interest is available at all. Instances of this problem include some of the highest-value information tasks, such as those facing national security and legal/patent professionals, who might spend hours or days searching to determine whether the desired information exists.

The IR community would do well to develop benchmarks for systems that consider recall at least as important as precision. Perhaps researchers should revive the set retrieval models and measures such as the F1 score, which is the harmonic mean of precision and recall.

Meanwhile, information scientists could use information availability problems as realistic tests for user studies of exploratory search systems, or interactive retrieval approaches in general. The effectiveness of such systems would be measured in terms of the correctness of the outcome (does the user correctly conclude whether the information of interest is available?); user confidence in the outcome, which admittedly may be hard to quantify; and efficiency—the user’s time or labor expenditure.

Precision will always be an important performance measure, particularly for tasks like known-item search and navigational search. For more challenging information-seeking tasks, however, recall is as or more important, and it is critical that the evaluation of information-seeking support systems take recall into account.

References

1. K. Järvelin and J. Kekäläinen, “Cumulated Gain-Based Evaluation of IR Techniques,” ACM Trans. Information Systems, Oct. 2002, pp. 422-446.
2. R. Rao et al., “Rich Interaction in the Digital Library,” Comm. ACM, Apr. 1995, pp. 29-39.

The NPD Group, a market research firm specializing in the entertainment industry, reported that in 2008, compared to 2007:

• The number of Internet users paying for digital music increased to 36 million from 28 million.
• Purchases of online digital music downloads increased by 29 percent and now account for a third of all music tracks purchased in the United States.
• The number of people buying CDs decreased by 17 million.
• The number of people buying music at all decreased by 13 million.
• Among internet users, the number purchasing  CDs or digital music decreased from 65% to 58%.

You can see more coverage on ZDNet, MediaMemo, and paidContent.org.

Of course, it’s the last two of these statistics that should be freaking out the music industry. Not only are they trading analog dollars for digital dimes, but the swap may be worse than one for one! Moreover, perfectly legal distribution channels like MySpace Music and Pandora may be significant factors in the decline of music purchases.

I don’t think any of this should come as a big surprise to anyone. But I suspect it will break the last strongholds of denial for some. Stay tuned for when video starts to suffer the same fate.

In academic publishing, there is a concept known as the  least publishable unit (LPU). As per the Wikipedia entry, an LPU is “the smallest amount of information that can generate a publication in a peer-reviewed journal.”

Until recently, I thought the concept was unique to academia. But today I read a post by TIME White House correspondent Michael Scherer entitled “The Politico Is Transforming Our Approach To News” that explained the emergence of the LPU in mainstream online media:

Once upon a time, the incentive of a print reporter at a major news organization was to create a comprehensive, incisive account of an event like Cheney’s provocative interview on CNN….That account would then be packaged into a container (a newspaper, a magazine, a 30-minute network news broadcast) and sold to the consumer. In the Internet-age, by contrast, what matters is not the container, but the news nugget, the blurb, the linkable atom of information. That nugget is not packaged (since the newspapers, magazine, broadcast television structure do not really apply online), but rather sent out into the ether, seeking out links, search engine ranking and as many hits as possible. A click is a click, after all, whether it’s to a paragraph-length blog post or a 2,000 word magazine piece. News, in other words, is increasingly no longer consumed in the context of a full article, or even a full accounting of an event, but rather as Twitter-sized feeds, of the sort provided by the Huffington Post, The Page, and The Drudge Report. Each quote gets its own headline. Context and analysis are minimized for space.

I’ve certainly noticed the steady deterioration of news quality in all media over the last several years, but I’d ascribed it to news providers’ inability to sell quality news for the cost of producing it, coupled with the popular obsession with 24-hour breaking news that is superficially reported and then immediately forgotten.

It had never occurred to me that news providers were also trying to slice each story into as many slivers as possible. In fact, such an approach strikes me as involving more work than writing one article per story–since the immediacy-driven deadlines are the same whether it’s one article or a dozen. But I can certainly see the logic of maximizing the number of distinct vehicles for selling ads.

I’m not sure whether to take Scherer at face value (perhaps others here who are journalism experts can comment), but I find the prospect of LPUs or “salami publication” depressing. It’s not that I object to precision-targeted articles addressing specific issues. Nor do I object to brevity–it is the soul of wit, even if Polonius lacked it. What I don’t want to see is a reductionist approach to news that destroys the context of a story by slicing it up too thin for adult-sized single servings.

But is that where the ad-supported model is pushing us?

One of the recurring objections to exploratory search is that it can’t work for the web. For example, some folks at Microsoft Research presented a poster at HCIR ’08 explaining what they see as daunting challenges to implement faceted search for the web. I’ve often plugged Duck Duck Go as an successful example of exploratory search for the web, though it does not go as far as to offer faceted search.

Neither does Kosmix,  but I’m impressed nonetheless. Kosmix has been around for a few years, and Ken Ellis has blogged about it a fair amount at NP-Harder. But, perhaps because of a underwhelming first impression, I forgot about it. Today, I was reminded by a New York Times article entitled “Just Don’t Compare Kosmix to Google“. Intrigued, I decided to give Kosmix another whirl.

Here are some of the searches I tried (yes, of course I start with vanity searches):

• tunkelang
• endeca
• exploratory search
• hcir
• new york karaoke
• renaissance polyphony

Verdict: I’m impressed. When there’s a Wikpedia entry, it piggy-backs on it, but most of those queries are far enough into the “long tail” to have their own Wikipedia entries. I slightly prefer how Duck Duck Go handles ambiguity (e.g., compare a query for sigir on Kosmix vs. Duck Duck Go), but I have to say that. on the whole, Kosmix is delivering on its promise of offering exploratory search for the web, and the functionality is the richest I’ve seen so far.

My only complaint is that it returns a lot of content for every query. I’d prefer a more progressive experience that returns a bit less initially but simply shows me the directions for further exploration–perhaps even including them in the initially returned page, but tucked away for a cleaner presentation. And of course I wonder how well it will work for the research tasks that most demand exploratory search. There’s still a lot of work to do.

It’s clear that my blog + Twitter vs. Aardvark challenge yesterday wasn’t much of a challenge, so I’m trying again. This time, I believe the questions are much harder. And, for what it’s worth, I don’t know the answers to any of them. I’m posting them all to Aardvark and here on The Noisy Channel at the same time, modulo my speed of typing.

1. What is the origin of my name, Tunkelang? To the best of my knowledge the only people alive in the world who bear this name are myself (Daniel), my brother, and my daughter. My father, Ben, died last year.
2. I used to play a text-based game called “Kill” over a dial-up teletype terminal in the 1970s. Is this game still around in any form?
3. Is there a karaoke bar in New York or Boston that has the song “Mercy Street” by Peter Gabriel?
4. Are there bacon and sour cream potato chips available for sale in the United States, like the ones Ruffles used to make? No bacon ranch Pringles or cheddar and bacon anything, please!
5. Where can I find the proceedings of the 1987 International Conference on The Business of CD-ROM?

Beyond being genuinely interested in the answers to these questions, I’d like to see how “social search” via Aardvark compares to social search via my homegrown social network.

The other day, I explained why, as far as I can tell, Twitter’s existing search functionality isn’t that hard to implement. In a subsequent post, I argued that Twitter is not a search engine, an opinion that seems to place me in a minority of the blogosphere, albeit a substantial one.

Today I see that Twitter is have some trouble with what strikes me as a core constituency, SXSW attendees.

Daniel Terdiman at CNET writes that “At SXSW, attendees confront Twitter saturation“:

At SXSW, the standard is for everyone to include the tag “#sxsw” in their tweets. For example, on Friday, I was looking for sources for a different story and tweeted, “If you are launching an iPhone app at #sxsw, or know someone who is, please let me know. Thanks!”

That’s a great convention because it allows anyone wanting to know what’s going on to search Twitter for posts using any search term important to them.

…

I did a search for the “#sxsw” tag on Saturday afternoon and found that there had been 392 tweets with the term in just the previous 10 minutes. That number mushroomed to more than 1,500 in the previous hour.

Large volumes of results wouldn’t be such a problem if users had a way to summarize, navigate, and explore them. But that will take more than a search engine that offers more than reverse-date ordering of Boolean queries.

I wonder if Abdur Chowdhury and his team are working on this problem. Perhaps if Twitter is willing to make some of the historical logs available for download (they’re already public, just not easily downloaded), some of us HCIR wonks could implement interfaces on top of it to explore the possibilities. Abdur, if you read this and are interested, please let us know!