Categories
General

LinkedIn Signal = Exploratory Search for Twitter

I like Twitter. Yes, I know that a lot of its content is noise. But I’ve found Twitter to be a useful professional tool for both publishing and consuming information. Publishing to Twitter is the easy part: I publish links to my blog posts and occasionally engage in public conversations.

Consuming information from Twitter is more of a challenge. I follow 100 people, which is about the limit of my attention budget. I use saved searches to track long-term interests (much as I use web and news alerts), and I perform ad hoc searches when I am interested in finding out what people are saying about a particular topic.

But Twitter search is not a great fit for analysis or exploration–unless you count trending topics as analysis. Originally, the search results were simply the tweets that contained the matching tweets in order of recency. The current system sometimes promotes a few “top tweets” to the top of the results. Still, if you’d like to get a summary view, slice and dice the results, or perform any other sort of HCIR task, you’re out of luck.

Until now.

The LinkedIn Search, Network, and Analytics team–the same folks that built LinkedIn’s faceted search system and developed open-source search tools Zoie and Bobo–just introduced a service called Signal that is squarely aimed at folks like me who use Twitter as a professional tool. It is still in its infancy (in private beta, in fact), but I think it has the potential to dramatically change how people like me use Twitter. You can learn more about its architecture and implementation details here.

Signal joins the often cacophonous Twitter stream to the high-quality structured data that LinkedIn knows about its own users. For example, when I post a tweet, LinkedIn knows that I am in the software industry, work at Google, and live in New York. LinkedIn can only make this connection for people who include Twitter ids in their LinkedIn profiles, but that’s a substantial and growing population.

Signal then lets you use this structured information to satisfy analytic and exploratory information needs. For example, I can see which companies’ employees are tweeting about software patents (top two are Google and Red Hat).

Or compare what Microsoft employees are saying about Android…

…to what Google employees are saying about Android.

As you can see on the right-hand side, Signal also mines shared links to identify popular ones relative to given search–and allows you to see who has shared a particular link. This functionality is similar to Topsy, but with the advantage of allowing structured searches. Like Topsy, it wrangles the mass of retweeted links into a useful and user-friendly summary.

Signal is still very much in beta. An amusing bug that I encountered earlier today was that, due to some legacy issues in how Linkedin standardized institution names, the system decided that I was an alumnus of the Longy School of Music rather than of MIT. Fortunately, that’s fixed now (thanks, John!)–I love karaoke, but I’m not ready to quit my day job!

Also, Signal only exposes a handful of LinkedIn’s facets, which limits the breadth of analysis and exploration. I’d love to see it add a past company facet, making it possible to drill down into what a company’s ex-employees are saying about a particular topic (e.g., their ex-employer).

Finally, while Signal offers Twitter hashtags as a facet, these are hardly a substitute for a topic facet. In order to provide such a facet, LinkedIn needs to implement some kind of concept extraction to provide a useful topic facet (something I’d also love to see for their regular people search). This is a challenging information extraction problem, especially for the open web, but I also know from experience that it is tractable within a domain. Given LinkedIn’s professional focus, I believe this is a problem they can and should tackle.

Of course, Linkedin also needs to convince more of its users to join their LinkedIn accounts to their Twitter accounts–since that is their input source. But I suspect it’s mostly a matter of time and education–and hopefully the buzz around Signal will help raise awareness.

All in all, I see LinkedIn Signal as a great innovation and a big step forward for exploratory search and for Twitter. Congratulations to John Wang, Igor Perisic, and the rest of the LinkedIn search team on the launch!

By Daniel Tunkelang

High-Class Consultant.

11 replies on “LinkedIn Signal = Exploratory Search for Twitter”

Daniel, what does your experience say about faceted discovery in streams of information versus explicit search?

While we’ve been working on extracting relevant categories based on streams of content at nuggetize.com, we’ve found quality to change across the following use cases:

i) Topic search: Results in a set of documents extremely well connected to each other. All normally have a strong context (the search query). The categories we mine out end up being of high quality. Eg: http://nuggetize.com/nuggetize?q=exploratory%20search&type=topic

ii) RSS feeds: The focus loses out a bit here, since articles can float around an agenda. Our categories have been good on very focused blogs, but tend to pick up noise on blogs that are loose. However, a blog still has an agenda and largely sticks to it.
http://nuggetize.com/nuggetize?q=http://feeds.feedburner.com/LinkedInBlog&type=feed

iii) Twitter feeds: Twitter feeds (especially incoming) are typically all over the place. Your social contacts have no agenda to keep. Not only does a person tweet about very diverse things, you’d also subscribe to very diverse people. Do you think exploratory search spreads too thin in such scenarios – and will not be able to surface up enough important categories? (We’ll soon launch a twitter interface for nuggetize – so the quality with feeds and topic searches can be compared)

Quality of results apart, is there a need for exploratory search in these life streams?

a) When I search, I know my intent very well. When I see a facet or category, I build a mapping between the concept i see and my search intent. I know what to expect when I click on a facet or category. When I search, the domain (and unit) of my interest is not just one article, but all the articles – since I want to know everything thats relevant to me across them.

b) When I am consuming an RSS feed, I know what type of content to expect. So I have a map (which is looser than the previous case) between the concepts written in the blog and the categories and facets. The unit of interest has narrowed down here – largely to one article. I do not expect different articles from the same feed to be connected in some form that I need to slice and dice across them.

c) What can I expect when I see facets around tweeted out links? They could be just about anything. Since a context is not set, I do not know what to expect when I click on a facet. Does this hurt exploratory search? In my trials on our twitter interface, I was actively seeking the actual tweet that brought a link, and the person it came from. Knowing the categories helped far lesser. The unit of information I wanted was just the sender, the tweet and the actual link.

Thoughts?

Like

Bharath, thanks for the comments–lots to respond to!

The short answer is that, whether I’m looking at recent or archival Twitter content, I like the use of facets in order both for elaborating my initial information need and as guidance for exploration.

I’ll write a longer answer soon–perhaps as a blog post rather than in the comment thread.

Like

By Post Author

Bharath, let me answer at least another part of your question here. A lot of people see Twitter primarily as a real-time streaming data source. That makes sense when you’re tracking an event as it happens.

But, for many other use cases, Twitter is more of an archival resource. Time matters–but for retrospective rather than real-time analysis. As I wrote last year, it’s about “over time” rather than real-time.

For these cases, exploratory search is useful–and faceted search is a great tool to enable such exploration.

Like

By Post Author

Daniel, we launched http://intweetion.com. While this is still an early iteration, and we can still massively improve usability, you get the “topic facets” you were mentioning in your article. Please take a look, we’d love your feedback.

On a related note, also try, http://bit.ly/dmAr0w
This drill’s into this page, and gives you a great way to mine references.

Like

Comments are closed.