Author: Daniel Tunkelang

Thanks to David Fauth for calling my attention to this news release from the U.S. Office of the Director of National Intelligence: New Policy Makes Information Sharing a Factor in Employees’ Performance Reviews.

It’s certainly a step in the right direction. But what will determine the success of this effort is the implementation of Information Sharing Environment to connect federal, state, local, and tribal governments, the private sector, and foreign allies. The security challenges alone are daunting, but I’m just as concerned about the challenges of getting the right information to the right people. Anyone who thinks that search is 90% solved should think about problems like these!

I read this morning that Ask had revamped their search engine. I tried it out, since I’ve appreciated how Ask has been more willing than its larger brethren to experiement with interfaces beyond conventional ranked lists. Unfortunately, I didn’t see anything to write home about. But perhaps I’m just not asking the right questions. I’d appreciate commentary from the readership.

Among the various facets of my identity is that I spent a fair amount of my student days at IBM Research. While I’ve kept in touch with IBM colleagues over the years, I’ve recently reconnected with IBM as an institution through their Greater IBM Connection initative. I’ll be occasionally blogging there, starting with this post entitled “Friending vs. Making Friends“.

An unfortunate indicator of this blog’s success is that I’m seeing an increased volume of spam comments. I’ve installed the Akismet plug-in; hopefully that blocks spam without too many false positivies. Please let me know if you experience any problems.

I’ve decided to eliminate tags and will instead use categories for annotating posts. I’ve occasionally posted links that refer to tags; unfortunately, these will break. I’ve eliminated any internal links that referred to tags, in some cases replacing them with links to Wikipedia pages (some of which reflect the efforts of the readership here!).

Eventually I’d like to use some of the capabilities I’ve helped develop at Endeca to tag posts. But I’m not quite ready to figure out how to install and maintain an Endeca instance for this blog. But it’s something to think about as I approach a thousand posts.

My colleague Herb Roitblat at Orcatec recently launched a vertical search engine called Truevert to demo Orcatec’s semantic search capabilities. From the about page:

This version of Truevert is focused on green, environmental awareness. All searches are done from the point of view of environmental and social concern. The results are obtained from YAHOO BOSS. They are then organized and clustered by Truevert. If you search for the word “carbon” for example, it knows that you want information about carbon’s impact on climate change, not its physical chemistry. What’s most surprising, is that the Truevert search engine learned all this in less than one hour on a single server with no ontology, taxonomy, or thesaurus.

Check it out! Not only is this an interesting applicaiton of Yahoo BOSS, but it also uses performs theme extraction to support limited query refinement. I’d like to see a truly exploratory interface, but I think this is a step in the right direction for vertical search.

Thanks to Bob Gourley for calling my attention (via Twitter) to this series of posts on Government 2.0 by Mark Drapeau, an Associate Research Fellow at the U.S. National Defense University. He asks (and tries to answer: “How can the government acknowledge, assess, and embrace social software?”

A choice quote:

Internal governmental information sharing means different things to different people. The most commonly-stated objection to the incorporation of social software into national security operations is that malware could be implanted or the social tools could otherwise provide access into government systems, thereby reducing network integrity. However, a good deal of government information, while perhaps private, is not necessarily so much so that off-the-shelf Web 2.0 sites cannot be utilized – they do typically come with some security features.

Much as the web has transformed our notions of publication and information seeking, social media are transforming our notions of information sharing. I’m glad to see earnest efforts to exploit these technologies to make government agencies more effective.

Thanks to Dave Kellogg for calling my attention to CIA Director Michael Hayden’s remarks at the DNI Open Source Conference 2008.

Here’s a choice excerpt:

Secret information isn’t always the brass ring. In fact, there’s something special about solving a problem or answering a tough question with information that others are dumb enough to leave out in the open.

Given my academic training and exposure to the culture of peer review, I’ve always been skeptical of folks who rely on secret information to justify their conclusions. Perhaps I’ll feel differently when I have security clearance.