And I do like how the WInquery interface in the D-Lib paper communicates the contribution of the various terms to each result. What I’m curious about is whether users understand this information. But in general I like their framework.

Finally, as to whether enterprise content being more structured than web content, there is some truth to that, though I think the bigger issue with web content is that the adversarial nature of web search prevents search engines from trusting what structure the document authors offer. We have had Meta tags for a while.

Could it be that “enterprise” data is more likely to be structured or have some semantic tags around it as the sample set in Cramer et al. paper as opposed to the Web which is less likely to be structured.

http://www.dlib.org/dlib/january97/retrieval/01shneiderman.html

See how you get a bar that represents both the scale of the retrieval score, as well as a proportionalized distribution of the query terms in that document?

http://www.sigchi.org/chi95/Electronic/documnts/papers/mah_bdy.htm

I’m also vaguely remembering other work, though I don’t know from who.. only that it was from 10-15 years ago, where the distribution of query terms in the document was visually represented as a normalized, stacked bar. I.e. imagine a horizontal bar.. with 34% of the bar in blue, the next 20% of the bar in green, and the final 46% of the bar in red. Green, blue, and red correspond to the 3 terms of your 3-word query. (obviously, 2 colors if a 2-word query, etc.)

That way, the user could see the relative proportion of their query terms in the document.

I think you’re making a few points, which I’ll try to restate and address.

First, transparency isn’t a panacea for the problems of information seeking, and systems that offer transparency often have worse performance than systems that don’t. That does not mean that transparency is to blame, but it may be that systems that offer transparency have had to make other design decisions to do so, such as limiting their efforts to those they can easily explain to users.

Second, if the conclusions of the Cramer et al. paper are generalizable, then transparency matters most when users focus on recall and least when users focus on precision. That may be related to transparency being more valued in the enterprise than on the web. I’ll have to stew on that one.

Third, as I read the CHI paper, it actually supports my case that users benefit from transparency–although in this case, the context is training a classifier rather than retrieving search results. Because users don’t think like machine learning algorithms, they don’t necessarily supply the best training examples, and can actually confuse the models because of their lack of visibility into the process.

Finally, I do like search UIs like Wordbars that offer transparency. I’m looking to see a lot more of these at HCIR!

Im not saying it wont be helpful for search. Wordbars is a search UI that i find interesting for adding transparency.

Interesting posts!