Comments on: HCIR: Better Than Magic!

By: Don Byrd

Don Byrd — Tue, 15 Sep 2009 18:23:54 +0000

Daniel, thanks for your quick response; I’m pleased but not at all surprised you’re a GWAP fan. And Josh, I like your analysis of the Slate visualization; they have an environment that could add so much more HCI to the IR!

By: Picture this! The news graphed. « Networked News

Picture this! The news graphed. « Networked News — Tue, 15 Sep 2009 15:48:00 +0000

[...] There’s no navigation, no browsing. There’s no search—and especially none involving interaction between the human and computer. There’s no news judgment beyond what newspaper editors originally add. And the corpus is [...]

By: Daniel Tunkelang

Daniel Tunkelang — Tue, 15 Sep 2009 14:47:55 +0000

Better late than never! I hadn't heard about that example, but I'm not surprised, as I've heard similar stories about other automated tools (including those in earlier versions of Microsoft Word). Here's a link to the Croft et al. paper. As for Luis Von Ahn's GWAP approach, I'm a big fan. Check out some of my earlier posts on the subject, or stay tuned for my HCIR '09 position paper! In any case, I'll amend my statement: reduce the need for human labor and / or make it more fun.

By: Don Byrd

Don Byrd — Tue, 15 Sep 2009 13:21:50 +0000

A belated comment on this. An early example of a high-profile gaffe by an IR system appeared in the Boston Globe in 1995. An article criticized the then-new THOMAS system for legislative information, pointing out that its top match for “elderly black Americans” was a bill on “black bears”! The problem is discussed in the well-known (and easily found) paper by Bruce Croft et al, Providing Government Information on the Internet: Experiences with THOMAS.

One other thing. You say “the whole point of automation is to reduce the need for human input. Human labor is a lot more expensive that machine labor! But there’s a big difference between the mirage of eliminating human labor and the realistic aspiration to make its use more efficient and effective. That’s what HCIR is all about, and all of the evidence I’ve encountered confirms that it’s the right way to crack this nut.” I mostly agree. I’d agree completely except for Luis Von Ahn’s amazing discovery of “games with a purpose”, a way to make human labor _free_! Free, that is, except for the overhead of developing the game, and collecting and analyzing the information. And, of course, this offline method of adding intelligence to an IR system has severe limitations.

By: Daniel Tunkelang

Daniel Tunkelang — Mon, 07 Sep 2009 16:06:29 +0000

Borja, thanks for the comment and link. To the credit of automated text processing tools, Google’s translation of your post is readable.

In short, I think you’re right that the start of the art in alerting is way ahead of the state of the art in extracting objective meaning, let alone subjective opinion.

By: Borja Ramirez

Borja Ramirez — Mon, 07 Sep 2009 15:57:29 +0000

there´s a lot of talk about sentiment analysis these days, following a few recent articles from NYT, EL Pais, etc…are we really capable of extracting “sentiment”, lets just leave it at “meaning” out of unstructures information, in this case social media tools suchas as blogs, twitter, FB, etc? Talking about sentiments is way too ambitous based on my experience in classification and search projects, but hey, it sells terribly well to government and enterprise marketing folks. Everybody wants to know, what do people think about you?

Technology can monitor and spot comments about us, but what about interpreting them and extraction meaning or a conclusion? Very complicated I think…

We´re having an interesting debate here..sorry it´s in Spanish:
http://www.busquedaempresarial.com/blog/2009/09/07/¿podemos-extraer-el-feeling-de-un-pedazo-de-texto-sin-leerlo/

By: Avi Rappoport / SearchTools.com

Avi Rappoport / SearchTools.com — Sat, 05 Sep 2009 03:24:14 +0000

I agree with you, automation can only get so far, that being the 80/20 split. I’ve seen some systemst that combined techniques to get to to 85% or 90%, but it’s that last few that are so wrong and embarrassing. There’s a lot to be said for human editorial judgment.

By: Daniel Tunkelang

Daniel Tunkelang — Fri, 04 Sep 2009 17:57:46 +0000

Maybe I'm too optimistic--or too quick to give him the benefit of the doubt. But I do want to clarify that I'm not anti-semantic.

By: jeremy

jeremy — Fri, 04 Sep 2009 17:50:07 +0000

Interestingly, even Google CEO Eric Schmidt may be getting around to drinking the kool-aid. In an interview published today in TechCrunch, he says: “We have to get from the sort of casual use of asking, querying…to ‘what did you mean?’.”

I don’t see this quote as a sign that Schmidt is finally coming to his senses and seeing the value of human input in the search process/dialogue.

I see it as a sign that he believes that more machine learning will be able to automatically infer “what you mean”.