Just letting readers know that I’ll be on vacation for the next week. If you are starved for reading materials, check out some of the blogs I read.

Today is the last day of WSDM 2010, and I unfortunately spent it at home drinking chicken soup. But I’ve been following the conference via the proceedings and tweets.

The day started with a short session on temporal interaction. Topics included clustering social media documents (e.g., Flickr photos) based on their association with events, statistical tests for early identification of popular social media content, and analysis of answers sites (like Yahoo! Answers) as evolving two-sided economic markets.

The next session focused on advertising. Two papers focused on click prediction: one proposing an Bayesian inference model to better predict click-throughs in the tail of the ad distribution; the other presenting a framework for personalized click models. Another paper addressed the closely related problem of predicting ad relevance. The remaining papers discussed other aspects of search advertising: one on estimating the value per click for channels like Google AdSense, where ad inventory is supplied by a third party; the other proposing an algorithmic approach to automate online ad campaigns based onlanding page content.

The following session was on systems and efficiency, a popular topic given the immense data and traffic associated with web search. Two papers proposed approaches to help short-circuit ranking computations: one by optimizing the organizations of inverted index entries to consider both the static ranks of documents and the upper bounds of term scores for all terms contained in each document; the other using early-exit strategies to optimize ensemble-based machine learning algorithms. Another used machine learning to mine rules for de-duplicating web pages based on URL string patterns. Another focused on compression, showing that web content is at least an order of magnitude more compressible that what can be achieved by gzip. The last paper proposed a method to perform efficient distance queries on graph (i.e., web graphs or social graphs) by pre-computing a collection of node-centered subgraphs.

The last session of the conference discussed various topics in web mining. One presented a system for identifying distributed search bot attacks. Another proposed an image search method using a combination of entity information and visual similarity. The final paper showed that shallow text features can be used for low-cost detection of boilerplate text in web documents.

All in all, WSDM 2010 was an excellent conference, and I’m sad to not to have been able to attend more of it in person. I’m delighted to see an even mix of academic and industry representatives sharing ideas and working to make the web a better place for information access.

Unfortunately, I woke up this morning rather under the weather, so I’m having to resort to remotely reporting on the second day of WSDM 2010 conference, based on the published proceedings and the tweet stream.

The day started with a keynote from Harvard economist Susan Athey. Her research focuses on the design of auction-based markets, a topic core to the business of search which largely relies on auction-based advertising models (cf. Google AdWords). Then came a session focused on learning and optimization. One paper proposed a method to learn ranking functions and query categorization simultaneously, reflecting that different categories of queries leads users to have different expectations about ranking. Another combined traditional list-based ranking with pair-wise comparisons between results to separate the results into tiers reflecting grades of relevance. An intriguing approach to query recommendation treated it as an optimization problem, perturbing users’ query-reformulation path to maximize the expected value of a utility function over the search session. Another paper looked not at ranking per se, but rather at improving the quality of training data for using machine learning for ranking. The final paper of the session, which earned a best-paper nomination, modeled document relevance based not on click-through behavior, but rather on post-click user behavior.

The next session was about users and measurement. It opened with another best-paper nominee: a analysis of over a hundred million users to understand how they re-find web content. Another offered a rigorous analysis of the often sloppily presented “long-tail” hypothesis: it found that light users disproportionately prefer content at the head of distribution while heavy users disproportionately prefer the tail. Another log-analysis paper analyzed search logs using a partially observable Markov model, a variant of thehidden Markov model in which not all of the hidden state transitions emit observable events–and compared the latent variables with eye-tracking studies. An intriguing study demonstrated that user behavior models are more predictive of goal success than models based on document relevance. The final paper of the session proposed methods for quantifying the reusability of the test collections that lie at the heart of information retrieval evaluation.

The last session of the day focused on social aspects of search. Two of the papers were concerned with modeling authority and influence in social networks, a problem in which I take a deep personal interest. Another inferred attributes of social network users based on those of other users in their communities (cg. MIT’s Project Gaydar). Another analyzed Flickr and Last.fm user logs to show that users’ semantic similarity based on their tagging behavior is predictive of social links. The final paper tackled the sparsity of social media tags by inferring latent topics from shared tags and spatial information.

Not surprisingly, a disproportionate number of contributors to the conference work at major web search companies, who have both the motivation to improve results and the access to data that is needed for such research. One of the ongoing research challenges for the field is to find ways to make this data available to others while respecting the business concerns of search engine companies and the privacy concerns of their users.

Today was the first day of the Third ACM International Conference on Web Search and Data Mining (WSDM 2010), held at the Polytechnic Institute of NYU in Brooklyn, NY. WSDM is a young conference that has already become a top-tier publication venue for research in these areas. In contrast to some of the larger conferences, WSDM is single-track and feels more intimate and coherent–even with over 200 attendees.

The day started with an ambitious keynote by Soumen Chakrabarti (IIT Bombay): “Bridging the Structured Un-Structured Gap”. He described a soup-to-nuts architecture to annotate web documents and perform complex reasoning on them using a structured query language. But perhaps this ambitious approach is a practical one: it uses the web we have–as opposed to waiting for the semantic web to emerge–and there is a prototype using half a billion documents.

The first paper session focused on web search. Of the five papers, two emphasized temporal aspects of content, one considered social media recommendation, and one focused on identifying concepts in multi-word queries. The last paper of the session proposed using anchor text as a more widely available input than query logs to support the query reformulation process. It also attracted the most audience attention–whileinteraction is often a niche at information retrieval conferences, it always elicits strong interest and opinions.

The following session focused on tags and recommendations. Some take-aways: users produce tags similar to the topics designed by experts; individual “personomies” can be translated into aggregated folksonomies; matrix factorization methods can produce interpretable recommendations.

The last session of the day covered information extraction. One of the papers used pattern-based information extraction approaches, demonstrating how far we’ve come since Marti Hearst’s seminal work on the subject. Another offered a SQL-like system for typed-entity search, complete with a live, publicly accessible prototype. The final paper addressed an issue the came up repeatedly at the SSM workshop: the problem of distilling the truth from a collection of inconsistent sources.

It is my pleasure to report on the 3rd Annual Workshop on Search in Social Media (SSM 2010), a gathering of information retrieval and social media researchers and practitioners in an area that has captured the interest of computer scientists, social scientists, and even the broader public. The one-day workshop took place at the Polytechnic Institute of NYU in Brooklyn, NY, co-located with the ACM Conference on Web Search and Data Mining (WSDM 2010). The quality of the presenters, the overbooked registration, and the hundreds of live tweets with the #ssm2010 hashtag all attest to the success of this event.

The workshop opened with a warm welcome from Ian Soboroff (NIST), immediately followed by a keynote from Jan Pedersen, Chief Scientist of Bing Search. Jan established a clear business case for search in social media: the opportunity to deliver content that is fresh, local, and under-served by general web search. He drilled into particular types of content where social media search is most useful: expert opinions, breaking news, and tail content. The benefits of social media search include trust and personal interaction (as compared to web content that is often soulless and of uncertain provenance), low latency (though perhaps at the cost of accuracy), and access to niche or ephemeral information that web search rarely surfaces. But delivering social media results to searchers creates its own variety of challenges, such as weighing freshness against accuracy and relevance, coping with loss of social content’s conversational context, managing low update latency when search engines have not been optimized for it, and fighting new kinds of spam. Despite these challenges, it is clear that the major web search engines have embraced the brave new world of real-time social content.

Eugene Agitchein (Emory University) then moderated a panel representing the world’s leading search engines: Jeremy Hylton (Google), Matthew Hurst (Microsoft), Sihem Amer-Yahia (Yahoo!), and William Chang (Baidu). Jeremy justified the universal interface approach, pointing out that users don’t want to have to figure out what kind of search site to use for their queries, and that they expect a familiar interface. He also noted that Google has made great strides on update latency: it can index the Twitter firehose in the same amount of time as serving a query. Matthew offered various analyses of the social search problem, based on whether the information signal resides in content (e.g., web) or attention (e.g., Twitter), or whether the information need is expressed in an explicit search query or inferred from the user’s context. Sihem offered a counter-point to Jeremy, arguing that social media search queries often represent broad or vague information needs, and thus call for a more browsing-oriented interface than web search, which is optimized for highly specific needs. William noted that the biggest competitive threat he sees for web search engines comes from social media players–and he credits much of Baidu’s success to its surfacing of social media content.

Then came a flurry of questions, perhaps the most interesting of which was how to address identity management. William argued that people prefer interacting with real-named (or pseudonymous) people to whom they are directly connected. Sihem offered the counter-example of obtaining recommendations through community aggregation. Matthew noted the incongruity of there being no economic relationship between social network companies that maintain proprietary social graphs and people whose identities and relationships those graph represent. Jeremy pointed out that users benefit if the data is as open as possible.

Given the almost even split between academic and industry participation in the workshop, the panelists were also asked to present research challenges to academia. Jeremy posed the problem of determining when social media results are actually true. Matthew wants to see more interdisciplinary work between computer scientists and social scientists. Sihem offered two challenge problems:  scalable community discovery and evaluation of collaborative recommendation systems. William wants to see a rigorous axiomatization of social media search behavior.

After lunch, Jeremy Pickens (FXPAL) moderated a panel representing social media / networking companies: Hilary Mason (bit.ly), Igor Perisic (LinkedIn), and David Hendi (MySpace). Hilary noted that, while bit.ly does not have access to an explicit social graph, it captures implicit connections from user behavior that may not be represented in the graph. Jeremy asked the panelists how much a person’s extended network matters; David and Igor pointed out research indicating correlations of mood and even medical conditions between people and their third-degree connections. Again, the audience was full of questions, especially for Igor. As a fan of faceted search, I was glad to see him touting LinkedIn’s success in making faceted search the primary means of performing people search on the site. For an in-depth view, I recommend “LinkedIn Search: A Look Beneath the Hood“.

The afternoon continued with a poster / demo session emphasizing work in progress: tools, interfaces, research studies, and position papers. I particularly enjoyed listening to the stream of interaction between academic researchers and industry practitioners.

The final panel session assembled academic researchers to discuss their views of the challenges in social media. Gene Golovchinsky (FXPAL) moderated a panel comprised of Meena Nagarajan (Wright State University), Liangjie Hong (Lehigh University),Richard McCreadie (University of Glasgow), Jonathan Elsas (CMU), and Mor Naaman (Rutgers University). Meena highlighted the need to build up meta-data to describe the context around social utteracnces. Liahjie took a position similar to William Cheng’s, calling for a framework to model the tasks and behavior of users who interact with social media. Richard focused on the intersection of social media and news search, and noted that some of the most useful information is private and proprietary (e.g., search and chat logs). Jonathan offered a variety of challenges: determining the right retrieval granularity, managing multiple axes of organization, aggregating author behavior, and multidimensional indexing of social media content. Finally, Mor noted that we’re moving from a world of email to a “social awareness stream”, in which the content we directed content at a group and have lower expectations of readership than email. As with all of the panels, there were countless questions from the moderator and audience, particularly about determining the truthfulness of social media content and delivering social content in an effective user interface.

The final conference session was a conference was a full-group discussion that dived into the various topics addressed throughout the day. But Gene Golovchinsky provided the “one more thing” at the end, showing us a glimpse of a faceted search interface to explore a Twitter stream. It was an elegant finish to a day filled with informative and engaging discussion, and I look forward to seeing many of the participants in the WSDM conference over the next few days.

Of course, I’ll cross-post here. I also encourage folks to follow the live tweet streams at #ssm2010 and #wsdm2010, as well as Gene and Jeremy’s posts at the FXPAL blog.

To those attending: see you all tomorrow through Saturday! To everyone else: I will try my best to communicate the substance and spirit of the conference.

But I’m pleased to report that Endeca now has is its own blog, aptly entitled Search Facets. I’m not usually a fan of corporate blogs, but I like the approach Endeca is taking to this one. The folks who have posted so far are Adam Ferrari (CTO), Vladimir Zelevinsky (Research Scientist), and Pete Bell (Co-Founder)–an indication that the blog will contain substance, rather than warmed-over press releases.

Indeed, the posts so far are nice and meaty. I particularly like Adam’s post about “Vertical stores for vertical web search?“–it’s nice to see read intelligent analysis from someone who understand the strengths of both MapReduce and column-oriented relational databases.

Anyway, I’m delighted that my former co-workers have taken to the blogosphere, and I look forward to reading what they have to say!

Last week, I had the good fortune to attend a presentation by John Wang, search architect at LinkedIn. You may have read my earlier posts about LinkedIn introducing faceted search and celebrating the interface from a user perspective. John’s presentation at the SDForum took a developer’s perspective, discussing the challenges of combining faceted search and social networking at scale.

John was kind enough to publish his slides, and I’ve embedded them above. Unfortunately, there’s no recording of the extensive Q&A (which included various attempts to get John to reveal the precise details of LinkedIn’s data volume), but the slides are quite meaty.

Personally, I learned two surprising things from the talk.

First, I was surprised that LinkedIn dismisses index/cache warming as “cheating”, instead computing almost everything in real time. Specifically, I would have expected LinkedIn to cache information like a user’s set of degree-two connections: these are expensive to compute at query time, especially when the social graph is distributed and sharded by user. I did ask John whether LinkedIn recomputes a user’s degree-two network during a session, and he admitted that LinkedIn is sensible enough to “cheat” and not perform this expensive but almost useless re-computation.

Second, I learned about reference search, a feature I may have missed because it is only available for premium LinkedIn accounts. It’s a nice feature, allowing you to search against company + date range pairs. People who are familiar with implementing faceted search may recognize the preservation of such associations between facet values as a gnarly implementation challenge.

All in all, it was a treat to get this look under the hood, as well as to finally meet John in person. I also ran into Gene Golovchinsky there–so much for my spending a few days on the west coast incognito!

In any case, I’m looking forward to seeing Gene, some of John’s colleagues, and many more interesting people at the Search and Social Media Workshop (SSM 2010) on Wednesday. My apologies to those who aren’t able to attend this oversubscribed event. I promise to blog about it!

It features a keynote from Jan Pedersen, Chief Scientist for Core Search at Microsoft, as well as an impressive set of posters and panel sessions. Other participants include:

• Sihem Amer-Yahia, Yahoo!
• Jon Elsas, CMU
• Gene Golovchinksky, FXPAL
• David Hendi, MySpace
• LiangJie Hong, Lehigh U.
• Jeremy Hylton, Google
• Matthew Hurst, Microsoft
• Hilary Mason, bit.ly
• Richard McCreadie, U. of Glasgow
• Mor Naaman, Rutgers U.
• Meena Nagarajan, Wright State U.
• Igor Perisic, LinkedIn
• Jeremy Pickens, FXPAL

There’s still time to register if you’re interested!

As my wife, daughter, and I were walking home from a playground, we noticed a large number of fire trucks congregating a block away from our house. A quick search on Twitter explained what was going on, particularly by pointing us to this post on Gothamist–which as of this writing seems to be the only reporting about this incident.

I think this example tells us a lot about the utility of real-time search. Most of us don’t need real-time search to tell us about the news in Haiti, since a critical mass of major news providers is covering the story around the clock. Where real-time search matters most is at the personal level–specifically, when our personal urgency to obtain information is higher than that of the general population. In such situations, we’re willing to accept less polished–and even risk less accurate–information, particularly if the alternative is to wait until if and when news providers cover the story. At least to some extent, urgency trumps authority.

Yes, there are other use cases for conversational media like Facebook and Twitter, such as sharing the experience of watching a live event, or simply chatting with friends and strangers about arbitrary topics. But I wouldn’t consider such use of these media to be search. Real-time search, in my view, is about helping users obtain the latest information available–in accordance with their personal needs. Twitter and Google served me well today, and I’m grateful that real-time search gave me real-time peace of mind.

Earlier this week, Peter Morville and Mark Burrell presented a UIE virtual seminar on “Leveraging Search & Discovery Patterns For Great Online Experiences“. It sold out! And I thought Pete Bell and I had done well with our seminar on faceted search!

But I’m hardly surprised. Although I wasn’t able to attend it myself, I gather from Twitter and the blogosphere that it was a great presentation. I enjoyed serving as a reviewer for Peter’s new book on Search Patterns, and I contributed a bit to Endeca’s UI Design Pattern Library while I was there and Mark’s team was developing it.

In reading reactions to the seminar, I was particularly intrigued by a post entitled “Search and Browse” by Livia Labate on her fantastically named blog, “I think, therefore IA“. She raised a question that I think needs to be asked more often: when is (or isn’t) faceted search appropriate?

Her conversation with readers in a comment thread offered some possible answers:

• Faceted search helps users who think in terms of attribute specifications as filtering criteria.
• Faceted search supports search by exclusion, as opposed to by discovery.
• Faceted search requires a set of useful facets that is neither too small nor too large.

I’d like to propose my own answers. Here are the conditions for which I see faceted search being most useful:

• Faceted search supports exploratory use cases, in contrast to known-item search. For known-item search, users are better served by a search box to specify an item by name, or a non-faceted hierarchy to locate it. In contrast, faceted search optimizes for cases where users are either unsure of what they want or of how to specify it.
• Faceted search helps users who need or want to learn about the search space as they execute the search process. Facets educate users about different ways to characterize items in a collection. If users do not need or want this education, they may be frustrated by an interface that makes them do more work.
• The search space is classified using accurate, understandable facets that relate to the users’ information needs. As I’ve discussed before, data quality is often the bottleneck in designing search interfaces. Offering users facets that are either unreliable or unrelated to their needs is worse than providing no facets at all.

Given the above criteria, it’s not surprising that faceted search has been a huge success in online retail: shopping is often an exploratory learning experience, and retailers tend to have good data.

But the success of faceted search in retail overshadows other domains where faceted search may be even more valuable. My favorite example is faceted people search, most recently demonstrated by LinkedIn. I would love to see other entities (locations, businesses, etc.) receive similar treatment, at least in contexts where exploration is a common use case.

I think Livia is right to be skeptical about any interface that introduces complexity–and facets do introduce complexity. I hope that my guidelines help answer her question as to when that complexity is worthwhile and perhaps even necessary to help users satisfy their information needs.

Weren’t we just talking about what’s different about mobile search use cases and about how to make web search more exploratory? I may be biased, but I think that Google’s recently launched “near me now” button is a step in the right direction (no pun intended!) on both of these fronts.

I’m curious to hear unbiased feedback from iPhone and Android users who have gotten to play with it.

For me, 2009 marked the end of a decade-long run at Endeca, where I focused on bringing HCIR to enterprises. I’m particularly proud of two professional accomplishments: writing a book on faceted search, and organizing the SIGIR 2009 Industry Track.

But past is prologue. I spent the last several weeks of 2009 as a Noogler, and I launch into 2010 living and breathing search on the open web.

What’s on my mind? Here are some top-of-mind questions to which I hope to have better answers by this time next year:

• Exploratory Search: how should we determine that users want a more exploratory search experience, rather than one that minimizes time to a best-effort result? How should we respond to queries that clearly don’t have a single best answers, such as queries of the form [category] or [category location]?

• Mobile Search: should it be just like non-mobile search with a few tweaks to accommodate the device form factor? Or does / should mobile search fundamentally change the way we interact with information?

• Real-Time Search: is it more than real-time indexing plus emphasizing recency as a query-independent relevance factor? What are the use cases, and how should we be addressing them?

• Social / Collaborative Search: should we be looking to microblogging or other social media signals to augment (or even supplant!) link-based citations as authority cues? Should we be supporting mediated search by linking people to people, rather than directly to information?

• Transparency: is it possible to offer more transparency in relevance ranking without losing ground in the battle against spam and black-hat SEO?

To be clear, these are simply the questions that are on my mind–I’m speaking as an individual and not as a Google employee. That said, a great thing about being at Google is that there are people working on all of these areas. So I expect 2010 to be an exciting year!

Curious to hear what problems are on other people’s minds as we enter the new year. Comment away!

committed to providing a framework for any company big or small, rich or poor to do a deal with us to get access to the Firehose in the same way we did deals with Google and Microsoft. We want everyone to have the opportunity — terms will vary based on a number of variables but we want a two-person startup in a  garage to have the same opportunity to build great things with the full feed that someone with a billion dollar market cap does. There are still a lot of details to be fleshed out and communicated, but this a top priority for us and we look forward to what types of companies and products get built on top of this unique and rich stream.

That and some other details, like raising the API rate limit from 150 requests per hour to 1500,  may well bring on what Marshall Kirkpatrick of ReadWriteWeb calls “Twitter 2.0“. But it was something else in Kirkpatrick’s write up that caught my attention–this quote from Wow.ly co-founder Kevin Marshall:

The more I do with and around social data, the less interested I seem to become in ‘realtime’ and the more interested I become in ‘over time.’ When I first started hacking on Twitter (and Facebook) apps, I was in love with the idea of parsing and analyzing data in real-time and I was very link/content focused. But the more I build and use these tools, the more I see the value in the history and the trails of the data set.

I couldn’t have said it better! Not that I haven’t tried: you look back at my post about Topsy, you’ll see where real-time and over time meet. Recency matters, but the signal is far too sparse without some way to aggregate and analyze over time.

I’m thrilled that Twitter plans to open up its platform in a way that could enable analysis over semantic, social, and temporal dimensions. Now I’m curious to see what that access will look like, and what everyone has been clamoring for that access will do with it.

I believe there’s a clearly established business case for faceted search inside the enterprise, for site search (especially for retail and media / publishing sites), even for vertical search on the open web. In all of these cases, relevance-ranked results are insufficient to meet a large subset of users’ more exploratory information needs, and HCIR approaches like faceted search are an easy sell.

But it seems much harder to make this case for general web search. The track record of startups in this space isn’t very encouraging. That could be because no one has done it right, but Clayton Christensen’s theory of disruptive innovation would suggest that a successful entrant wouldn’t have to have parity across the board, but would simply need to win on an underserved market segment. Perhaps the increasing use of faceted search for vertical search is how this process is playing out, and faceted search for general web search may end up being a slow agglomeration of verticals.

I’m curious if others have been pursuing efforts like Eric’s. Are the available APIs powerful enough to prototype your own faceted web search engine? If they aren’t, then is this a potential business opportunity for one of the major (or non-major) search engines to promote innovation by offering an open system? Or, if Yahoo! BOSS already offers such an open system, what should we make of the scale of its impact?

Long-time readers may recall my post about visual search startup Modista last November, or this guest post by one of its principals. Unfortunately, the story has a sad ending. I hope that both this technology and its developers find a good home.

One of my favorite resources for learning about user experience is the UXmatters blog. This group blog boasts a set of authors that represent a diverse collection of industry practitioners (and one academic) and offer concrete case studies and recommendations.

For example, in “Best Practices for Designing Faceted Search Filters“, Greg Nudelman offers a constructive critique of the Office Depot search user interface. Some of his material will be familiar to those who have read my faceted search book (particularly the chapter on front-end concerns), but the focus on a single example makes for a compelling read. I also liked Greg’s most recent post, entitled “Cameras, Music, and Mattresses: Designing Query Disambiguation Solutions for the Real World“. I was amused that he and I use the same “canonical” example for the need to offer clarification before refinement.

Here are a few more posts from other authors to give you a taste for the blog:

• “First, Do No Harm” by Pabini Gabriel-Petit
• “The Five Competencies of User Experience Design” by Steve Psomas
• “Beyond Usability: Designing Web Sites for Persuasion, Emotion, and Trust” by Eric Schaffer

If you are a user experience professional, in name or in deed, then you should be reading the the UXmatters blog — or perhaps even contributing to it. Of course, you’re always welcome to contribute a guest post here too.

LinkedIn started rolling out a beta version of faceted people search back in July. Now it’s officially out of beta, as announced on their blog. I’ve re-posted the video above in case you missed it in July.

Interestingly, LinkedIn developed its own tool to support the combination of faceted search with social network search: Bobo-Browse (Otis mentioned it in our recent presentation to the New York CTO Club). I helped develop similar functionality when I was at Endeca, so I know how hard this problem is. LinkedIn has done an impressive job–and has applied it to one of the most valuable data sets on the web. Bravo!

But I can’t help asking for just one more thing. LinkedIn has great semi-structured data about its 50+ million members. I’d love to be able to explore that data using more facets–in particular, facets relating to people’s job skills and expertise. I hope that’s something they’re working on. Perhaps a good topic of conversation at the upcoming Workshop on Search and Social Media!

Most karaoke venues provide printed song books, typically sorted by title and by artist. This approach is certainly adequate for very limited selections, but it doesn’t scale gracefully. Indeed, one of my favorite karaoke bars, the Courtside in Cambridge, MA, has a fantastic song selection that is only accessible through printed books. Kinda frustrating for a search guy, even though the staff is very helpful!

My regular karaoke venue in New York, Second on Second, is a bit more technologically advanced: it provides computers with dedicated software that allows patrons to search through their song catalog. Aside from being faster than thumbing through books, the software makes it possible to find songs when you only remember words that are in the middle of song or artist names.

But even such a system only addresses known-item search–in this case, looking for a song or artist by name when you know precisely what you are looking for. There’s room for incremental improvement here, e.g., searching for songs based on the lyrics you remember. For example, many people remember a famous David Bowie song based on its protagonist “Major Tom” rather than its title “Space Oddity“; fortunately, tools like Google’s music search are happy to make such connections.

But none of the karaoke search technology I’ve see to date supports exploration. Specifically, I’d love to go into a karaoke bar and have a procedure for finding songs I know that is better than trial and error. For example, I’d like to be able to see my options for hard rock 80s songs with male vocals. Or to find out which downtempo bands, if any, are on the menu. A little faceted search would go a long way towards making the song-finding experience more pleasant and efficient.

But why stop there? I’d really like a system that suggests songs based on what it knows about me. For example, knowing that I like to sing Scorpions songs is a reasonable basis to suggest similar artists like Def Leppard and Guns N’ Roses. Or perhaps to suggest 80s songs in general–after all, karaoke roulette notwithstanding, most people sing songs they know (or at least think they know), and their song knowledge tends to have some temporal locality. I’m sure you can imagine far more sophisticated personalization–and such personalization could be accomplished with complete transparency to the user.

Even if you aren’t into karaoke (and yet have managed to read this far!), I hope you can appreciate the universality of the information needs I’m describing. Exploratory search is everywhere. But I think it’s easiest to demonstrate its practical importance by working through concrete use cases. As an HCIR advocate, I’ve repeatedly learned the lesson that such demonstrations are critical in order to successfully evangelize this worldview.

Otis Gospodnetic and I recently gave a talk at the New York CTO Club on faceted search. The club is a group of senior technologists who meet monthly in midtown Manhattan to host breakfast presentations and to share ideas and expertise. Those of you who have read my book or attended the UIE virtual seminar a few months ago that I gave with Pete Bell (whom I worked with for 10 years at Endeca) might recognize some of my material. Otis focused on the specifics of implementing faceted search using the open-source Solr platform.

Here were the major take-aways:

• Think about what users are trying to do, not just how they search.
• Facets get polluted with bad result sets, so offer clarification before refinement.
• Don’t just move the information overload problem to the facets! Show less, not more.
• Look at the potential data facets you already have, you will be surprised.
• Facets can come from new data, e.g. sentiment.

• BIA / Kelsey Blog
  • By The Kelsey Group, a division of BIA Advisory Services that provides data and analysis on directories and local media.
• Exploring Local
  • By Mike Dobson, President of TeleMapics, a company that provides consulting services focused on local search.
• Local SEO Guide
  • Andrew Shotland’s blog on local search optimization, small business marketing & search engine optimization strategy.
• Localsearchdatabase
  • By Gib Olander, Director of Business Development for Localeze, an online content management company serving businesses, local search engines and consumers.
• Mihmorandum
  • David Mihm’s blog on local search engine optimization and marketing.
• Screenwerk
  • Greg Sterling’s thoughts on online and offline media. Sterling used to run The Kelsey Group’s Interactive Local Media program.
• SEO Igloo Blog
  • By Solas Web Design, which specializes in web design and search engine optimization for small businesses.
• Understanding Google Maps & Local Search
  • By Mike Blumenthal, whose company offers consulting services and market research advice relating to maps and local search.

Not surprisingly, these blogs offers me a critical perspective on how Google and other search engines serve the local space.  Granted, everyone has their own motives–and it’s hard to avoid some tension in a space with the competitive dynamics of local search. But now that I’m no longer an outsider myself, I appreciate having others to help keep me honest as I work to make local search better for users and businesses.

But working on the inside has also help me appreciate what Bob Wyman tried to tell me months ago–that Google has no philosophical predilection towards black box approaches, but rather is only limited by what technology makes possible and what its engineers can implement. I’d qualify that slightly by saying that I perceive an additional constraint: Google does have a strong predilection towards data-driven decisions. Some folks have found that approach objectionable in the context of interface design.

Anyway, if you’re a regular here, then you’re probably predisposed towards HCIR and exploratory search. In that case, I’d like to take a moment to help you appreciate the challenge I face on a day-to-day basis.

Which one of these two statements do you most agree with?

1. We need better data quality in order to support richer search user interfaces.
2. Richer search user interfaces allow us to overcome data quality limitations.

On one hand, consider two search engines whose interfaces are designed to support exploratory search: Cuil and Kosmix. Sometimes they’re great, e.g., [michael jackson] on Cuil and [iraq] on Kosmix. But look what can happen for queries that are further out in the tail, e.g. [faceted search] on Cuil [real time search] on Kosmix. Yes, the kinds of queries I make. I don’t mean to knock these guys–they’re trying, and their efforts are admirable. Moreover, both generally return respectable search results on the first pages (in Kosmix’s case, through federation). But the search refinements can be way off, and that undermine the overall experience. I strongly suspect that the problem is one of data quality, along the lines of what others have argued.

On the other hand, some of the work that I did with colleagues at Endeca (e.g., work presented at HCIR 2008 on “Supporting Exploratory Search for the ACM Digital Library”) at least dangles the possibility that the second statement holds–namely, a richer user interface could help overcome data quality limitations. Interaction draws more of the information need out of the user, and the process may be able to mask imperfection in the data. For example, it’s clear to users–and clear from the search refinements–that [michael jackson beer] and [michael jackson -beer] are about different people. If we can just get that incremental information from the user, we don’t have to achieve perfection in named entity recognition and disambiguation.

I think there’s some truth in both arguments. Data quality is a major bottleneck for effectively delivering an exploratory search experience, and data quantity, much as it helps, is not a guarantee of quality. Richer interfaces offer the enticing possibility of leveraging human computation, but they also introduce the risk of disappointing and alienating users. Even for an HCIR zealot like me, the constraints of reality are sobering.

And yes, speed and computational cost matter too. But hey, it wouldn’t be a grand challenge if it were easy!

Google returns standard results for such searches:

Bing is generous to a fault, saving you a click if you choose to use one of its leading competitors:

Finally Yahoo, whose CEO claims “we have never been a search company,” seems quite eager to keep searchers from going elsewhere:

It’s easy to dismiss these queries as corner cases, but the logs show that they really happen. And, as browsers increasingly blur the line between an address bar and a search box, it’s not unreasonable to consider that switches between search engines are likely to commence with such queries.

Earlier this week, Marti Hearst gave a Tech Talk at Google about her recently published book, Search User Interfaces. Fortunately for those of us who missed (myself included!), it is now available on YouTube. Enjoy! (via Jon Elsas)

Consider MusicLab, an “experimental study of self-fulfilling prophecies in an artificial cultural market“. In this study, sociologists Matt Salganik, Peter Dodds, and Duncan Watts manipulated the social information available to consumers (specifically teens) regarding their peers’ musical tastes. The experimenters’ goal was to empirically validate a quantitative model of social contagion.

But we can look at this study another way: by isolating the social factors that influence musical taste, the experimenters were also isolating the non-social signal–in theory, how popular a song would be in the absence of social signaling. Indeed, they found that, if they measured a song’s quality by isolating out the social factor, “the best songs never do very badly, and the worst songs never do extremely well, but almost any other result is possible”.

It’s interesting–interesting to me, at least!–to ask if search engines can do the same for search. One of the frequent objections to link-based authority measures like PageRank is that they make the rich get richer. “Real-time” variants like re-tweet frequency (and even TunkRank) suffer from the same weakness. Unchecked, these measures can cause authority / influence market has to resemble a winner-take-all market.

It strikes me as interesting to learn from cases where searchers swim upstream against the social signals to find information. Of course, you may already see the contradiction–this is just another kind of social signaling! Still, it seems like it might be a way to hedge our bets and against the weaknesses of positive feedback and spammers. In a similar vein, we might look at how users find information that suffers from poor accessibility or retrievability.

I don’t have answers about how to pursue such an approach, or whether it would even be feasible to do so. But I hope you agree with me that it’s an interesting question.

Google’s recently released Image Swirl is slick. But I’ve been struggling to figure out whether it’s useful or simply a showcase for cool technology.

And that’s prompted me to think about the overloaded term “exploratory search“. A while back, I tried to define exploratory search based on what it is not. This time, let me aim to positively characterize what I see as its two primary use cases:

1. I know what I want, but I don’t know how to describe it.
2. I don’t know what I want, but I hope to figure it out once I see what’s out there.

The first use case cries out for tools that support query refinement or elaboration. Existing tools span a range from suggesting spelling corrections (aka “did you mean”) to offering semantically or statistically related searches that hopefully provide the user with at least a step in the right direction. One of my favorite approaches, faceted search, is primarily used to support query refinement through progressive narrowing of an initial search query.

The second “I don’t know what I want” use case is fuzzier. In the language of machine learning, this use case is unsupervised, while the previous one is supervised. In general, it’s a lot harder to define or evaluate outcomes for unsupervised scenarios. Indeed, Hal Daume has argued that we should only do unsupervised learning if we do not have a trustworthy automatic evaluation metric. That’s a strong position, and you can see some of the counterarguments in his comment thread. But, going back to our scenario, it’s really hard to judge the effectiveness of tools like similarity browsing when they support exploration in the absence of any concrete goal.

With that in mind, I’ll reserve judgment on the utility of tools like Image Swirl. To the extent that it aims at the first use case, clustering images for a particular search, I’m ambivalent. I’d prefer a more transparent interface, in which I have more of a sense of control over the navigational experience. I suspect it is more aimed at the second use case, offering a compact visualization of what is out there.

Besides, as some folks have brought up at the HCIR workshops, it’s important that we make information seeking fun. And Swirl certainly scores on that front.

So reads the first claim from a patent application that Apple recently filed (with Steve Jobs as first inventor, no less!) for technology to deliver a rather compelling ad-supported business model. Or perhaps the better word is compulsory. You can read an analysis by Randall Stross in the New York Times.

I agree with Stross that it’s hard to imagine Apple ever implementing the technology described by the patent application–indeed, Apple has been one of the few success stories for paid digital content models. That said, the approach does feel like at least one endpoint for the ad-supported model–it guarantees the advertisers the attention that they are paying for by subsidizing content or services.

The advertising business is a bit more top of mind for me, now that it pays my salary. Google’s approach, however, follows the aphorism that honey catches more flies than vinegar: it tries to target ads well enough that users want to click on them, rather than to simply endure them as a cost of subsidizing free services. Google’s revenue (and the popularity of PPC models in general) is a testament to the success of this approach, my occasional rant notwithstanding.

In general, the industry seems to have found a compromise in how aggressively to push ads at users. Users can safely ignore (or even block) sponsored links, but few people do.  Pre-roll ads on video sites (i.e., advertising before a video starts)  are more invasive, but a number of sites let users skip them. You can read why the YouTube folks are testing this approach. Advertisers–or at least ad-supported services–seem to recognize that they can’t cross the line between pursuing users’ attention and annoying users to the point of alienation.

Still, technology like Apple’s patent application describes shows that it is possible for the ad-supported model to take a more more aggressive approach. Part of me wonders if more aggressive ad-supported models would revitalize paid content models, as users would stop perceiving the former as free. But I suspect that the gentler ad-supported model is here to stay, and that it will continue to strive toward the point of optimal effectiveness.

The deadline for proposal submission is November 30th–you only have to submit a 250-word abstract.

Here is the call for proposals:

We seek dynamic speakers who can talk knowledgeably about detailed aspects of how to implement and maximize search within an organization. Search can no longer be viewed as a stand-alone application. It is increasingly part of everything we do and has become the de facto gateway to information in the enterprise. This year’s Summit will examine the ways to leverage search tools, information architecture, classification, and other strategies and technologies to deliver meaningful results—not just in terms of information, but to the bottom line.

Ours is a well-informed, tech-savvy audience, so proposals should be specific and detailed. Consider topic such as:

• Integrating search into enterprise systems and workflow
• Customizing your search solution/ Task-specific search
• Compliance, records management, and eDiscovery with effective search
• Migrating your search engine
• Social search and social tagging strategies & solutions
• Search-enabled decision making
• Business intelligence, data mining
• Search as the gateway to enterprise information
• Optimizing the interface and user experience
• Navigational tools—context, facets, entity extraction, clustering, and visualization
• Emerging trends, the future of search
• Overcoming information overload
• Categorization techniques
• Semantic Search
• Query Federation & Federated Search
• Enhancing an existing solution

If you represent a company that has an enterprise search software product, your best bet to be on our program is to collaborate with a customer to submit a case study to be presented by them, following the guidelines above.

If you need more information–or more time–I encourage you to reach out directly to Michelle Manafy, the conference chair.

As I finish my first week, I can sum the experience in a word:
overwhelming. The tools for accessing internal information are better
than I expected, but both the volume of baseline knowledge–technical
and cultural–and the relentlessness of the update stream are
daunting.

Indeed, the internal ecosystem is so rich that it’s easy to forget
there is a world outside it–ironic given Google’s enormous role in the world outside it! Then again, this is just my first week–it will take me
some time to pop up the stack from the build system to the surface.

Will I continue blogging at The Noisy Channel?

Absolutely! I’m committed to posting at least weekly, and I’ll try to do better than that once I’m settled into my new environment.

Will I participate in scholarly conferences and workshops?

Of course! I’m co-organizing SSM 2010, which will be held in conjunction with WSDM 2010 in February, and of course HCIR 2010, which will be held in conjunction with IIiX 2010 in August. You probably won’t see me at vendor fests, but I do hope to continue bringing industry practitioners and academic researchers together.

Will I blog about Google?

I certainly won’t disclose any confidential information–people get fired for that–or worse. And, given how much access I will have to such information, I will err on the side of caution, only discussing information that I’m sure Google has released to the general public. Beyond that, I’ll exercise common sense. I don’t want to either come across as a shill for my employer or to spar with my new colleagues in public. Subject to those constraints, however, I can and will blog about Google.

Can I get you a job at Google?

I can advise you and connect you to a recruiter, but that’s the limitof my power. The hiring process here is specifically designed to prevent any individual from manipulating it–even me!

Will I talk about what I’m working on?

See above regarding confidential information. I’ll be delighted talk about anything I’m working on that Google has decided to disclose publicly.

Does Google know about my karaoke habit?

Too late, they’ve already signed the offer letter.

This is my last week at Endeca. The decision to leave has been a heart-wrenching one: not only have the past ten years been the best of my life, but my experiences at Endeca have defined me professionally. Moreover, Endeca is riding a wave of success with recent advances in our products, new relationships with key partners, and fascinating new deployments.  (You can read Endeca’s latest announcements in our newsroom).

Ironically, it is this very success that compels me to move on. In the past several years, I have developed an increasing passion for search on the open web–an interest only furthered by the explosion of social media.

That is why I’ve decided to accept an opportunity at Google’s New York office. Readers here know that I’ve been a very public critic of Google’s simplistic approach to user interaction on the open web. I’m being offered an opportunity to help fix that approach–and it is an offer I can’t refuse. My mission is to apply my passion for human-computer information retrieval (HCIR), an approach that Endeca has pioneered in the enterprise, to the world’s largest information problems–and where better to do that than at the company that aspires to organize the world’s information.

This moment is bittersweet: I am excited about the new experiences that await me, but I have a heavy heart as I turn in my badge and part with a world-class team that has succeeded against incredible odds.

Given my role and tenure at Endeca, I want to say explicitly that this move is about my personal ambition. My passion for web search and social media, which have grown exponentially over the past couple of years, simply doesn’t align with Endeca’s focus in the enterprise.

Also, I want to make clear: Google hired me because of my values, and not in spite of them. I know that some folks will find it difficult to reconcile my criticisms of Google with my decision to join. That’s why there’s an opt-out village! Seriously, though, I take my values with me. Google is offering me the opportunity to channel my passion for HCIR into action, on the world’s largest stage. I’m well aware of the magnitude of the challenge, but hey, I’m feeling lucky.

In “Using Twitter Lists To Judge Influence“, Todd Zeigler of the Bivings Report writes:

I think Twitter Lists will end up helping separate the men from the boys when it comes to influence.  In addition to seeing a Twitter users follower count, we can now see the number of other Twitter users who have added them to lists (example to the right).  I would argue that getting added to a list is a bigger deal than simply getting someone to follow you.

I’m certainly intrigued by Twitter Lists, but I’m skeptical that counting how many lists someone is on will prove that much more useful than follower count. For example, I currently have 1159 followers, am on 33 lists, and have a TunkRank of 24.1. For grins, here’s a handful of people who have similar stats:

• Evan Sandhaus: 796 followers, 21 lists, TunkRank = 17.2
• Josh Young: 801 followers, 25 lists, TunkRank = 14.3
• Chris Ahearn: 1108 followers, 14 lists, TunkRank = 30.1
• Brynn Evans: 1303 followers, 33 lists, TunkRank = 18.9
• Eric Andersen: 1543 followers, 37 lists, TunkRank = 3.1

While I can’t generalize from a few arbitrarily selected data points (though Gladwell seems to have no trouble doing so in Outliers), my suspicion is that list count will be highly correlated to follower count–and may actually be a noisier signal because the numbers are so much smaller.

Of course, there’s no reason we should use raw list counts–any more than we should use raw follower counts. Just as TunkRank aspires to model attention scarcity and recognizes that not all followers are created equal, an effective measure of how lists contribute to influence must recognize that not all list memberships are created equal either.

I’ve been chatting with Chris Langreiter, who is working on enhancements to TunkRank to address some of the oversimplifications of its model, as well as with Jonathan Glick and Ken Reisman at TLists. I’d like to see online influence–on Twitter and in general–measured more effectively. It will be great if lists can help, but we can’t make the same naive mistakes as those who were quick to embrace follower count as a measure of authority.

With all of the activity around e-books last week, you might think that the online world wasn’t paying attention to the media category most transformed by the Internet music. But a week is a lifetime in the ADD-addled technology press, and today’s top story is that Google is “making search more musical“. From the official blog post:

Now, when you enter a music-related query — like the name of a song, artist or album — your search results will include links to an audio preview of those songs provided by our music search partners MySpace (which just acquired iLike) or Lala. When you click the result you’ll be able to listen to an audio preview of the song directly from one of those partners.

As with most Google features, this one is being rolled out gradually. If you’re impatient (like me), you can try it directly from this page. Or you can watch the video above.

My first impression: this is great feature to improve known-item search, and it’s nice that they’ve partnered with folks that often let you hear whole songs, rather than 30-second snippets. The selection seems limited, but it could be that my tastes are a bit obscure. I’m curious if others share my sense that the catalog is much smaller than the ones on iTunes or Amazon.

But, as music IR specialist and fellow HCIR advocate Jeremy Pickens points out, Google is “doing to music what they did to the web“. I’m not as concerned as Jeremy is about the prospect of musical tastes being homogenized through the “rich get richer” effect of ranking–perhaps because we’re already there. Not only is pop music self-perpetuating (see this great study by my friend (and Princeton sociologist) Matt Salganik and his former advisor Duncan Watts), but even recommendation engines quash diversity. Google really can’t make things that much worse.

Besides, much as Google’s default search leads many searchers to Wikipedia, a great starting point for exploratory search, the new music search leads users to Pandora, which is probably the leading engine for exploratory music search offers users a more exploratory user experience (though it would be great if they also linked to last.fm) (thanks Jeremy!). OK, maybe “leads” is a strong word for a “listen on” link below the search result, but it’s there for people in the know.

I’d love to see Google embrace HCIR. But I appreciate the improvements to known-item search too, especially if they can delegate the HCIR functionality to others that focus on it.

The slides for Ben Shneiderman’s HCIR 2009 keynote on “The Future of Information Discovery” are now available on the workshop web site. I’ve also taken the liberty to upload them to SlideShare and embed them here. The slides don’t do justice to Ben’s presentation style, but hopefully they at least communicate a taste of the material he covered and his vision of where HCIR needs to go as a field and community.

Google may be an also-ran in the social networking market with its Brazil-centric Orkut service, but that hasn’t stopped the search giant from adding social features to its products. A post at the (unofficial) Google Operating System blog recounts the history of Google Reader’s social evolution, up to but not including its latest update last week. SearchWiki, though not a social search feature per se, allows users to share personal annotations of their search results, as does the more recently introduced Sidewiki. And, like Bing, Google has established a partnership with Twitter in order to surface “social” results.

But the feature announced today, which Google is actually calling “Social Search“, is a much bigger step, even if it is tucked away as an experiment on Google Labs. From the official blog post:

With Social Search, Google finds relevant public content from your friends and contacts and highlights it for you at the bottom of your search results. When I do a simple query for [new york], Google Social Search includes my friend’s blog on the results page under the heading “Results from people in your social circle for New York.” I can also filter my results to see only content from my social circle by clicking “Show options” on the results page and clicking “Social.”

I gave it a whirl, search for “noisy channel” and then restricting the search to content from what Google considers my social circle. The results are as promised, and could further refine to results by author name, selecting from a familiar list of Neal Richter, Jason Adams, Daniel Lemire. Ken Ellis, and Joshua Young (though for some reason Josh’s link didn’t work). Cool! Except that there are a lot of names missing (check out the bloggers in The Noisy Community) and, more importantly, I can’t further refine or even sort the search results. Indeed, the ordering of search results seems quite arbitrary–a phenomenon I’ve noticed more generally for search engine ranking of social media content.

In short, Google Social Search is a welcome initiative, but there’s a lot more work to do before I would find a productive use for it. Given the mismatch between social search and black-box relevance ranking, a little bit of HCIR would go a long way towards making this feature practically useful.

But this year was different. We were overloaded with strong submissions from all corners, and we had to turn people away for lack of capacity! While we didn’t relish saying no to prospective participants, these are great problems to have! And, thanks to Nick Belkin and Diane Kelly, we’ve arranged to greatly increase that capacity at HCIR 2010–more on that in a moment.

Max Wilson has already written up an excellent summary of the workshop, which I encourage you to read. You can also see the live tweet stream at #hcir09. Rather than duplicate these efforts, let me add my personal reflections as an organizer and participant.

Ben Shneiderman’s keynote address was sweeping and inspiring. I expected him to talk about information visualization, the area where he is most known for his contributions. He did present some examples of his group’s work on visualization-centric interfaces to support medical research, but his overall presentation took the much more ambitious approach of discussing the past, present, and possible future of HCIR. Specifically, he urged us to link our work to societal goals, such as the United Nations Millennium Development Goals. His challenge may seem impossibly idealistic, but I agree with his assertion that it is a practical one: we will do our best research by grounding ourselves firmly in the real and pressing problems of our age. Last year’s keynote speaker went on to win the Gerard Salton Award; I can only hope that Ben receives comparable accolades for his past accomplishments and future contributions to HCIR.

A new feature for this year’s workshop was having a “poster boaster” session, in which each of the presenters in the poster session had one minute to pitch his or her work.  For those of you unfamiliar with this format, I highly recommend it. The compressed format forces presenters to distill the essence of their contributions–a useful exercise in general. And the audience doesn’t get bored: if you decide halfway into a presentation that you aren’t interested, then you only have to wait 30 seconds until the next one! Not the we had that problem: the posters were consistently interesting, as the submissions were unusually strong this year. You can download the full workshop proceedings here.

Even the full presentations weren’t that long. The five speakers were each allotted ten minutes, with a healthy amount of time reserved for a panel-style Q&A sessions. The papers in this session were, by design, some of the more controversial ones. In particular, Ellen Voorhees delivered a full-throated defense of Cranfield / TREC-style evaluation: “I Come Not to Bury Cranfield, but to Praise It” (similar to her presentation at the 2006 Workshop on Adaptive Information Retrieval that I discussed on this blog last year). Her reminder of HCIR’s challenges on the evaluation front surely ruffled some feathers, but all of us HCIR avocates need to address these challenges if we want researchers (and practitioners) outside our community to drink our kool-aid.

The above format was already quite interactive (as befits a workshop about interaction), but the second half of the day was explicitly designed to facilitate discussion. We had lunch on site, followed by a one-hour poster session.  We then had two one-hour guided discussion sessions to address the theoretical and practical concerns of HCIR. As organizers, we seeded both sessions with questions, but we also incorporated concerns that had come up during earlier discussions.

Finally, I am grateful to our sponsors. Catholic University was a gracious host and sponsor, providing the workshop with a great space and very helpful student volunteers. Between that and the financial contributions of Endeca and Microsoft Research, we were able to continue our tradition of not charging attendees for the workshop. I can’t promise that will continue indefinitely, but I am glad that our insistence on emphasizing substance over frivolous amenities has helped us deliver what I believe to be some of the best bang-for-buck in the scholarly community.

I’m already excited about HCIR 2010. Unlike the past three workshops, which have been held as independent events, next year’s workshop will be co-located with the Information Interaction in Context Symposium (IIiX’10) in New Brunswick, New Jersey. The workshop will take place on August 22nd, breaking our unintended tradition of holding the workshop on October 23rd. Nick Belkin assures us that there will be lots of space, so hopefully we’ll be able to accommodate everyone who is interested. We’ll also be soliciting sponsors for both the workshop and the broader symposium.

But there’s more to HCIR than enjoying each other’s company at workshops. We must spend the remaining 364 days of the year fleshing out our vision, and relating that vision not only to the disciplines HCIR explicitly integrates, but to pressing social concerns. It is up to us all to make our work relevant.

When my daughter was born almost two years ago, I wondered if she’d grow up reading books. After all, I do most of my reading online, and increasingly find myself reading short articles rather than whole books. Needless to say, she’s loved books so far, even if she’s shredded a few.

But the bigger surprise for me is that books–specifically e-books–have become such a hot industry. When I briefly worked for a consulting firm after grad school in 1999, my first assignment was to evaluate the e-book market. The readers then consisted of the Rocket ebook and SoftBook Reader. Needless to say, I correctly predicted at the time that the ebook-market wasn’t ready for prime time.

But fast forward to the present. Amazon has given the e-book market some credibility: Citigroup says they sold 500K Kindles in 2008, and Forrester predicted they will sell 1.8M units this year.

But the last days (and even the last 24 hours!) of news show that the e-book market is only starting to open up:

• In May, Sony, whose e-reader sales have lagged behind the Kindle, announced a partnership with Google in May in order to make copyright-free books available for free.
• Google just announced a service called Editions that it plans to launch in 2010 (by when it will have presumably settled the Google Books Settlement Agreement).
• The Internet Archive just announced the Bookserver project as “a growing open architecture for vending and lending digital books over the Internet”.
• Spring Design just announced Alex, an e-book reader based on Google’s Android operating system.
• Barnes & Noble is expected to announce an e-reader that competes directly with the Kindle and has generated lots of buzz through leaked photos.

I grew up on books, and I’m excited to see that, a decade after the initial market failures, e-books (like touchscreens) are a mainstream reality. I still worry about who will buy them, especially considering that the marginal cost of distributing a typical e-book is even less than that of distributing a 5-minute song. A quick scan of a popular file-sharing site reveals that the pdf version of bestseller The Lost Symbol takes up less than 3MB.

Still, I’ll take a moment to celebrate the progress of technology. I’ve always known that reading was cool, but now we have the gadgets to prove it!

Who will buy this wonderful morning?
Such a sky you never did see!
Who will tie it up with a ribbon
And put it in a box for me?

Of course, the trope that the best things in life are free predates musical theater, let alone the web. But recent years have witnessed dramatic changes in our price sensitivities in every genre of digital (or digitizable) content, and I’m curious (sometimes morbidly so) about where it goes from here.

I won’t make you suffer through a rant about the malaise of the music and news industries–those topics, important as they are, have been overplayed in the blogosphere. If you need a refresher, I suggest Lawrence Lessig and the Nieman Journalism Lab as some of the more rational voices contributing to the discussion.

But it’s not just news and music that are experiencing the effects of the “information wants to be free” movement. Consider these industries:

• Books. Many publishers worry that the Kindle has been setting a consumer expectation that a book should only cost $10. Indeed, a recent price war between Amazon and Wal-Mart drove some of those prices down to $8.99. Is this a boon for consumers, or a body blow to the publishing industry? It’s easy to evoke the $0.99 / per song expectation set by iTunes–but that change was more about disaggregating albums than about changing the per-unit cost. Besides, books have not yet had to confront the scale of unauthorized distribution that we see in the music industry. Legal or not, free is a potent source of price pressure.
• Software. Wolfram Alpha just made headlines by releasing a $50 iPhone app. Many have reacted that such a high price is outrageous and will doom the application to failure. They may be right on that latter point–the market will vote with its clicks soon enough. But I’m old enough to remember $50 as being in the ballpark of what it cost to purchase a new consumer software application. Even then, unauthorized distribution was an issue–remember the “don’t copy that floppy” campaign? Today, my impression is that few people consciously purchase consumer software–a trend that I at least date to Microsoft’s strategy of bundling its software into PC purchases. The most noted exceptions are console games (which are impressive holdouts in the consumer software space) and iPhone apps–with the caveat that only a tiny minority of apps make enough money for the creators to live on. (Update: just saw this note about how EA Sports President Peter Moore sees the current console game business model of cartridges and discs as a “burning platform”.)
• Television. Between Boxee and Netflix, there is a real chance that digital content’s cash cow, cable television, will see its regional monopolies disrupted. I can’t imagine that anyone will shed a tear for the cable companies. And yet I can’t help but wonder what happens as the notion of premium content is subsumed by an expectation that video content should be free. Are we heading towards a proliferation of cheaply produced reality TV, contests, and game shows–all sponsored by rampant product placement?

If we are to believe Mike Masnick, then the price of content is driven to its marginal cost. It’s pretty clear that the marginal cost of distributing most digital content is, while not free, close enough to be a rounding error. Should we be looking forward to a world where no one can charge consumers for content? Folks like Jeff Jarvis and Chris Anderson are cheerleading such a world as not only inevitable but a good thing–though both of them have had the sense to make some money on non-free books while the going is good.

Yes, there are and will always be business models to support content creators. In particular, one-time content (live events, consulting services) has some degree of insulation from the inexorable trend toward free. But what an inefficient turn of events, if people are rewarded for creating one-time content but not for creating far more valuable content that is useful to a broad audience of consumers!

I know that there are non-financial incentives that drive scholars, open-source developers, and activists to create free content. Indeed, I personally write this blog without any direct financial incentive. Perhaps these incentives will be the driving forces for content creation in the 21st century. One way or another, I hope we find a way to fund the things we value, rather than devolving into a locally optimal rut where value creation isn’t economic for the creators.

p.s. You can find the lyrics to Oliver for free online, and you can easily view an free (unauthorized) copy of a performance of “Who Will Buy?” on YouTube. Or you can buy the song for $0.99.

Here’s the key information from the workshop web site:

Overview

Social applications are the fastest growing segment of the web. They establish new forums for content creation, allow people to connect to each other and share information, and permit novel applications at the intersection of people and information. However, to date, social media has been primarily popular for connecting people, not for finding information. While there has been progress on searching particular kinds of social media, such as blogs, search in others (e.g., Facebook, Myspace, of flickr) are not as well understood.

The purpose of the 3rd Annual Workshop on Search in Social Media (SSM 2010), is to bring together information retrieval and social media researchers to consider the following questions: How should we search in social media? What are the needs of users, and models of those needs, specific to social media search? What models make the most sense? How does search interact with existing uses of social media? How can social media search complement traditional web search?  What new search paradigms for information finding can be facilitated by social media?

SSM 2010 follows up on the highly successful SSM 2009 and SSM 2008 workshops held at SIGIR 2009 and CIKM 2008 respectively. We are looking forward to an equally exciting workshop at WSDM 2010 in New York!

Format and Topics

We are planning for a full-day workshop consisting of invited speakers, organized in both plenary and panel sessions, and a contributed poster/demo session.

We solicit short (under 2 pages) position papers, posters or demo proposals to be presented as part of a poster session, describing late-breaking and novel research results or demonstrations of prototypes or working systems. All topics at the intersection of information finding and social media are of interest, including, but not limited to:

• Searching blogs, tweets, and other textual social media.
• Searching within social networks, including expert finding.
• Searching Wikipedia discussions and revision histories.
• Searching online discussions, mailing lists, forums, and community question answering sites.
• The role of human-powered and community question answering.
• Novel models of information finding and new search applications for social media.
• The role of timeliness, authority, and accuracy in social media search.
• Interaction between traditional web search and social media search.
• User needs assessments and task analysis for social media search.
• Interactions between searching and browsing in social media.
• Searching and exploiting folksonomies, tags, and tagged data.
• Spam and adversarial interactions in social media.

Ideal papers may include late-breaking and novel research results, position and vision papers discussing the role of search in social media, and demonstrations of prototypes or working systems. Note that the workshop proceedings will not be archived or considered as formal publication, to encourage the informal atmosphere and to allow the authors to publish expanded versions of the work elsewhere.

The poster/demo proposals should be in standard ACM SIG format, more details to be posted soon.

Submissions are due on December 15th. I hope to see some of you there! Meanwhile, feel free to suggest ideas for invited speakers who have done interesting work at the intersection of social media and search, and I’ll share your suggestions with my co-organizers.

But apparently the Huffington Post is blazing new trails in this area. The Nieman Journalism Lab reports that:

The Huffington Post applies A/B testing to some of its headlines. Readers are randomly shown one of two headlines for the same story. After five minutes, which is enough time for such a high-traffic site, the version with the most clicks becomes the wood that everyone sees.

NJL also reports that Huffington Post social media editor–and long-time Noisy Channel reader–Josh Young uses Twitter to help crowd-source better headlines.

I’m sure this approach must rattle some old-school journalists. And there is a real danger of optimizing for the wrong outcome. For example, including the word “sex” in this message might improve its traffic (the popularity of this post attests to that), but to what end?

Still, I don’t see this use of technology as cramping anyone’s style. Most of us write to be read–especially those in the media industry who are trying to monetize their audiences. Measurable success matters, and there’s no harm in trying to maximize it.

Recurring Tweets are a violation no matter how they are done, including whether or not someone pays you to have a special privilege. We don’t want to see any duplicate tweets whatsoever- They pollute Twitter, and tools shouldn’t be given to enable people to break the rules. Spinnable text seems to just be a way to bypass the rules against duplicate updates and essentially provides the same problems.

Hence, from Thursday, October 15th, 2009, 00:00 AM CST we will prevent the entry of recurring tweets on Twitter accounts within the SocialOomph system. Existing recurring tweets on Twitter accounts will all be placed in paused state at that time, so that the content of the tweet text is still accessible to you, but no publishing to Twitter of those tweets will take place.

Not everyone is thrilled with this new feature. My friend (and Noisy Channel reader) Eric Andersen notes: “this doesn’t make a lot of sense to me – many highly regarded Twitter users (e.g. @GuyKawasaki) regularly re-post tweets…primarily because of the “dip” model: re-posting the same tweet means more people will see, especially with an int’l audience.”

On one hand, I loathe inefficient communication, and I see repeated tweets as exposing the inefficiency of the dip model. We won’t get into my differences of opinion with Guy Kawasaki. If Twitter offered better search and control to users, then I think it would make sense for them to consider duplicate tweets as a spam issue.

On the other hand, Twitter search is crude. And the dip model, much as it may raise my personal hackles, is, in fact, what many users embrace. Twitter takes pride in letting users drive innovation, and I think they should be cautious about being too autocratic. Surely many of the people who post duplicate tweets do so with unspammy intentions.

Let’s face it: Twitter is going through growing pains, even if it just inherited the mother of all trust funds. They really do have to address spam. But they might consider doing so in a less heavy-handed way. I suspect that duplicate tweets are mainly a problem because they affect the statistics for Trending Topics–a problem they could easily address without prohibiting the tweets themselves. Better search would make it users to take charge of the user experience–a small dose of HCIR would go a long way.

I think Twitter has the best of intentions, and that it is confronting a real problem. I hope they work harder to find the right solution.

If you’re into search startups, then today’s a great day to check out what a couple of them are up to.

TheFind just launched (or relaunched?) a “buying engine” that aspires “to help every shopper find exactly what they want to buy, and to help every merchant, large and small, to reach those shoppers.” It has some nice interface elements, but I can’t say I’m sold on the overall user experience.

Meanwhile, Aardvark just launched a web-based version of its social search application. The site urges users to “ask any question in plain English, and Aardvark will discover the perfect person in your network to answer…in under 5 minutes!” As I’ve commented before, I think they need to embrace the philosophy of “when in doubt, make it public“. But hey, they made the Time’s top 50 websites for 2009, so perhaps they are right to ignore my advice.

Freebase. Wolfram Alpha. Google Squared. I hesitate to declare a trend, but there does seem to be a growing interest in more structured approaches to information seeking.

The latest entry is Factual, launched today by Gil Elbaz. Elbaz is no slouch: in 1998, he and Adam Weissman co-founded Applied Semantics (originally known as Oingo) and built a word sense disambiguation engine based on WordNet. In 2003, they sold the company to Google for $102M, where it became the bases of their very lucrative AdSense offering.

According to Factual’s website:

Factual is a platform where anyone can share and mash open data on any subject.  For example, you might find a comprehensive directory of restaurants along with dozens of searchable attributes, a huge database of published books, or a list of every video game and their cheat codes.  We provide smart tools to help the community build and maintain a trusted source of structured data.

Factual’s key product, the Factual Table, provides a unique way to view and work with structured data.  Information in Factual Tables comes from the wisdom of the community and from our powerful data mining tools, and the result is rich, dynamic, and transparent data.

You can read more detailed coverage in Search Engine Land, TechCrunch, ReadWriteWeb, GigaOM, and VentureBeat.

To me, Factual sounds like a hybrid between Freebase and Many Eyes. And, like both, it’s free (as in free beer). Free cuts both ways: the Factual site states clearly that “There is currently no way for us to help you monetize these tables.” As with many companies at this stage, the business model is TBD.

I have mixed feelings. I like the increasing interest by startups in structured search. It’s a step in the right direction, since structure is a key enabler for interaction. But we already have one Freebase (and even Google Base), and it’s not clear that we need yet another company to enable crowd-sourced submission of structured data. Perhaps what we need is a way to incent the sort of behavior that has made Wikipedia so successful. As my colleague Rob Gonzalez (who is rumored to have a blog in the works) is always happy to point out, structured data repositories are a public good that no one is ever willing to pay for. The current best hope seems to be the Linked Data initiative, which sounds great in theory–though I think the jury is still out on whether it will succeed in practice.

My ambivalence aside, I am excited that some of the greatest minds in computer science are focused on bringing more structure to the information seeking progress. Even if some of these efforts prove to be false starts, we’re going in the right direction. Structured search is on the table.

As some of you may remember, I’m excited about Google Squared, a project I see as a great first step toward exploratory search at a web scale. Yes, I know that Duck Duck Go, Kosmix and others are already taking on this challenge, but it makes a difference to see Google throw its weight behind such an ungoogley initiative. Plus Google Squared is ambitious, to say the least–the input is free-form text and the output is highly structured.

Since I’ve beaten up Wolfram Alpha for is overreliance on NLP, I can’t give Google a free pass. It would be nice to be able to give Google Square more structured guidance (yes, I’m still an HCIR fanatic). But Google Squared seems to achieve far more robust query interpretation than Wolfram Alpha’s–perhaps because supporting exploratory search is less brittle than question answering.

The quality of the tables that Google Square produces as results is still spotty, but it is a major improvement from the initial release. To those who wrote off Google Squared in June, I suggest you take a second look.

But the trickle that Twitter returns is hardly enough.

I believe this limitation is by design–that Twitter knows the value of such access and isn’t about to give it away. I just hope Twitter will figure out a way to provide this access for a price, and that an ecology of information access providers develops around it. Of course, if Google or Microsoft buys Twitter first, that probably won’t happen.

Now that Twitter has raised $100M at a valuation of $1B, I doubt any acquisition will happen anytime soon. But, according to Kara Swisher’s unnamed sources:

Twitter is in advanced talks with Microsoft and Google separately about striking data-mining deals, in which the companies would license a full feed from the microblogging service that could then be integrated into the results of their competing search engines.

If so, then it’s about time! How much either Microsoft or Google would pay for this feed is an interesting question. It’s probably not a coincidence that Twitter raised its last round of funding before pursuing this path–the revenue they obtain this way could be significant, but is unlikely to justify a $1B valuation.

In any case, I’m excited as a consumer that Twitter may finally allow Google and Microsoft to better expose the value of its content. But I’m also curious what my friends on the Twitter Search team think of the potential competition from the web search titans. Until now, no one has been able compete effectively with Twitter’s native search because of  lacking access to the firehose. Having such access would give Google and Microsoft more than a fighting chance. Given the centrality of search to Twitter’s user experience, it’s an interesting corporate strategy.

Here’s an excerpt (via TechCrunch) to pique your interest:

Q: Do you think Bing is something different or a rebranding?

Sergey Brin: I don’t want to speak about our competitors.

Eric Schmidt: Better for you to judge. We like to focus on our customers.

More coverage at:

• Danny Sullivan (Search Engine Land)
• Peter Kafka (AllThingsD)
• Erick Schonfeld (TechCrunch)

For anyone who’s ever wondered what it would be like to see me live on CNN, this is your chance! Sorry that it isn’t my most telegenic moment. Still, it was a nice opportunity to share my perspective on the new FTC regulations facing bloggers.

There have been a number of reactions across the blogosphere, but I’d like to hone in on two opposing views: those of What Would Google Do author (and blogger) Jeff Jarvis and Googler Matt Cutts.

Jarvis describes the regulations as “a monument to unintended consequence, hidden dangers, and dangerous assumptions…the greatest myth embedded within the FTC’s rules [is] that the government can and should sanitize the internet for our protection.”

Commenting on Jarvis’s post, Cutts replies:

As a Google engineer who has seen the damage done by fake blogs, sock puppets, and endless scams on the internet, I’m happy to take the opposite position: I think the FTC guidelines will make the web more useful and more trustworthy for consumers. Consumers don’t want to be shilled and they don’t want payola; they want a web that they can trust. The FTC guidelines just say that material connections should be disclosed. From having dealt with these issues over several years, I believe that will be a good thing for the web.

It’s a fascinating debate, and I can see merit in both sides. Like the folks at Reason, I lean libertarian (at least on issues of freedom of expression) and am not eager to see more government regulation of online speech. That said, I see the value of laws requiring truth in advertising, and I don’t see why pay-for-play bloggers should get a free pass if they are acting as advertisers. Interestingly, Jarvis’s response to Cutts is: “I trust you to regulate spam more than the FTC. You are better at it and have more impact.” That’s probably true today, but wouldn’t want to invest that responsibility in a company that makes 99% of its revenue from advertising.

Everyone in this discussion sees the value of transparency–the question is whether it should be a legal norm enforced through FTC regulation or a social norm enforced by the marketplace. Despite my general skepticism about regulation of expression, I temper my libertarianism with a dose of pragmatism. For example, I’m glad that the Food and Drug Administration (FDA) at least tries to regulate health claims–its efforts may not eliminate quackery, but they surely reduce the problem.

Do we need FTC regulation in order to tame the jungle of social media? For that matter, will regulations have a positive effect, or will sploggers and other scammers simply ignore them–and perhaps even more offshore? I share Jarvis’s fear that the regulation will cause more harm than good–perhaps even having chilling effects on would-be bloggers. Certainly the FTC will have to use its new power wisely–both to avoid trampling the existing blogosphere and to not scare off newcomers. Still, if the FTC shows that it is only out to get true scammers, it may help establish, in Cutts’s words, a web we can trust.

I’m Daniel Tunkelang, and I endorse this blog post.

With that preamble out of the way, let me tell you a true story. The other day, I received a phone call from a friend who has been building a kick-ass startup. That friend had been contacted by a much larger competitor with what amounted to an ultimatum: shut down and come work for us, or we’ll crush you with a patent infringement suit. My friend’s startup didn’t cave in–in fact, my friend even went through the trouble of sharing a pile of incontrovertible prior art with the competitor. The competitor was unimpressed, and my friend’s startup is now facing a potentially ruinous lawsuit.

If you know any of the characters in this story, I beg you to keep that information to yourself–at least for now. I’d like my friend to have a chance of getting his company out of this predicament, and premature publicity might hurt his case.

But back to the case: let me give you an idea of how a story like this can play out. At a high level, the startup can choose to fight or not fight.

Not fighting means the entrepreneurs writing off their startup, but it allows them to move on and try something new. It might be the best career move for the entrepreneurs, but it means that the world loses a promising startup, and the surrender rewards bad behavior, reinforcing a regime where innovators can’t afford to compete with more established players.

Fighting means  mounting a non-infringement defense, an invalidation defense, or both.

A non-infringement argument asserts that, regardless of the validity of the patent, its claims don’t cover what the startup is doing. Since patents carry a presumption of validity, the non-infringement route is appealing–there’s no need to slog through the much longer invalidation process. Leaving a bad patent alive may be a worse outcome for the rest of the world, but entrepreneurs don’t have the luxury of taking the weight of the world onto their own shoulders.

Unfortunately, the very characteristics of a bad patent make it hard for an accused infringer to succeed in a non-infringement argument. If a patent is overly broad, then it’s more likely that the infringement argument will be valid (but not sound, since the patent itself is–or should be–invalid). Vaguely worded claims are also a problem–while a patent examiner may have granted a patent based on one interpretation of the claim language, the patent holder may now be asserting infringement under a different (and typically broader) interpretation of that same language.

As a result, a non-infringement argument often depends almost entirely on the result of a Markman hearing, more formally known as a claim construction hearing. In such a hearing, a judge decides how to interpret any language in the claim whose meaning is contested by the opposing parties in the suit. Such a hearing is often a crap shoot for the accused infringer. An unfavorable result which supports the infringement accusation may ultimately help invalidate the patent, but the results are likely to come too late–justice delayed for a startup is often an extreme case of justice denied.

Which brings us to the invalidation route. In theory, invalidation is the right approach to take when confronted with an invalid patent. Ideally, the accused infringer presents prior art to the patent office to reexamine the patent, resulting in the patent either being invalidated or rewritten to have a much narrower scope. In practice, however, this approach requires significant effort, time,  money–especially if you depend on lawyers to do the heavy lifting–and luck. The best hope is to rapidly request and obtain a reexamination, and then to request and obtain a stay of the infringement suit pending reexamination. Needless to say, the patent holder will fight tooth and nail to avoid this outcome.

I don’t know how my friend’s story will end. But, as the above analysis should make clear, he’s between a rock and a hard place. Whether or not you believe that there should be software patents–and there is room for reasonable people to debate this question–I hope you agree that the situation my friend is facing amounts to legalized extortion. I understand that no system is perfect, and that our legal system requires compromises that have inevitable casualties.

Nonetheless, my friend’s story does not feel like an isolated incident, but rather evidence of a systemic problem. There are a lot of software patents floating around right now of dubious validity, many of them granted to companies that have since folded and have unloaded their assets in fire sales. It would be sad for this supply of ersatz intellectual property to impede the real innovation that the patent system was intended to protect.

Update: this post has been picked up by Y Combinator’s Hacker News.

Google announced today that its Search Options feature, which allows users to progressively refine search results, now includes new refinement options: past hour, specific date range, more shopping sites, fewer shopping sites, visited pages, not yet visited, books, blogs and news. Of course, you could do some of this already with clever hackery. In any case, it’s great to see Google slouching towards HCIR on its most visible search property. Perhaps I was too quick to write off their interest in faceted search. Meanwhile, I’m staying tuned for Bing 2.0.

Seven years ago, Paul Graham famously stated:

“I think it’s possible to stop spam, and that content-based filters are the way to do it.”

Well, seven years of innovation and research have brought about some great advances in the field of text classification, so perhaps it’s time to raise the stakes a little. In short, we think it’s possible to stop abusiveness in user-generated content, and that content-based filters are the way to do it.

The Problem with UGC

Publishers these days are in a tight spot with user-generated content (UGC). The promise of UGC in terms of engagement and overall stickiness is hard to pass up, but along with the benefits come some headaches as well. Comment spam is less of an issue than it once was, thanks to services such as Akismet, but the problem of trolling and outright abuse is as bad as it ever was. Any publisher venturing into UGC is stuck with the question of how to keep comments in line with their editorial standards while at the same time avoiding accusations of censorship. The solution employed thus far has mainly been a combination of keyword filters and human moderators. Unfortunately for publishers, there are serious problems with both of those, so let’s look at that more closely.

The main problem with human moderators is the cost involved. They’re expensive, hard to outsource, and they don’t scale. The average human has a maximum capacity of about 250 comments per hour, which is a generous estimate. At minimum wage this works out to about $0.03 per comment, which seems reasonable until you consider that a typical online publisher like the Huffington Post receives about 2 million comments per month site-wide. Add in overhead costs like hiring, training, auditing, etc and it quickly starts to get out of control. On top of this is the issue of moderator bias. Is it possible that your Democratic moderator is simply deleting every post that disagrees with President Obama, regardless of content?

To mitigate the costs involved, many publishers add in a layer of non-human filtering, such as a keyword list. While this may seem like a good idea at first, all it really does is offer you the worst of both worlds. Now you have an expensive, non-scalable solution that also gives bad results. Keyword lists can be easily beaten by the simplest obfuscation, such breaking up bad words or simply replacing a letter with a symbol. In addition, it is impossible for keyword filters to catch anything but the crudest type of abusiveness. A great example is the recent Facebook poll “Should Obama Be Killed?” which would likely pass right through a keyword filter but is quite obviously abusive content.

The Solution:  Sentiment Classifiers

The idea of using a machine-learning classifier to identify text-based semantics is not a new one. Vladimir Vapnik introduced the original theory of support vector machines (SVMs) in 1995, and in 1998 Thorsten Joachims argued that the algorithm was perfectly suited for textual data. Finally, in 2002 Lillian Lee and colleagues showed that not only are SVMs well suited for identifying sentiment, but they also dominate keyword-based filters consistently. When applied to the problem of comment moderation, SVMs can mimic human moderation decisions with an accuracy of about 85%. That raises the question, is 85% good enough? How can we push the accuracy higher?

We have some proprietary answers to that question over at Adaptive Semantics, but a less controversial one arises from a well-documented property of classifier output known as the hyperplane distance (labeled v^k in the diagram below).

If the separating hyperplane can be viewed as the dividing line between abusive and non-abusive content, the hyperplane distance of any individual test comment can be interpreted as the classifier’s confidence in its own answer. If a test comment turns out to be very far from the dividing line, we can say that it lies deeply in “abusive space” (or in “non-abusive space” depending on the polarity). Now let’s imagine an SVM pre-filter that only makes auto-publish or auto-delete decisions on comments that have a large hyperplane distance, and sends all other comments to the human moderator staff. Such a classifier would have a guaranteed accuracy above 85%, and would progressively reduce the reliance on human moderators as it is re-trained over time. Even a conservatively tuned model can reduce the human moderation load by about 50% while keeping comment quality roughly the same. That’s a pretty good start.

In addition to high accuracy, a content-based classifier does not have the same limitation of a keyword filter in terms of vocabulary. Since the classifier is trained by feeding it thousands of real-world examples, it will learn to identify all of the typical types of obfuscation such as broken words, netspeak, slang, euphemisms, etc. And since the entire content of the comment is used as an input, the classifier will implicitly take account of context. So the comment “Should Obama Be Killed?” would likely be flagged for deletion, but a comment like “A defeat on healthcare may kill Obama’s chances at re-election.” would be left alone.

So is the abusiveness problem licked? Not quite yet, but the use of linear classifiers would be a huge step in the right direction. You could imagine further advances such as aggregating comment scores by user to quickly identify trolls, and maybe even using those scores as input for another classifier. Or how about training more classifiers to identify quality submissions and pick out the experts in your community? The possibilities are definitely exciting, and they raise another question: why are publishers not using these techniques? That one we don’t have a good answer for, so we founded a company in response.

Mathematics illuminates the patterns that abound in our world. The Math Factory strives to enhance public understanding and perception of mathematics. Its dynamic exhibits and programs will stimulate inquiry, spark curiosity, and reveal the wonders of mathematics. The museum’s activities will lead a broad and diverse audience to understand the evolving, creative, human, and aesthetic nature of mathematics.

The above is the mission statement of The Math Factory, an organization headed by former Renaissance Technologies analyst (and CTY alumnus) Glen Whitney that aspires to build a national museum of mathematics in New York. The effort is well underway–the organization has raised $4M to date, attracted an impressive group of trustees and advisors, and has obtained quite a bit of enthusiastic press coverage. No wonder–the Math Midway it exhibited at the World Science Festival this past June was a wild success. I’d gone there to offer moral support, only to find that I was lucky to get close enough to see the exhibits!

Last night, I was fortunate enough to attend a gala at the Urban Academy and actually play with the exhibits–from riding a tricycle with square wheels to walking through a maze without making left turns. It was a blast! And, while I’ll admit to being favorably predisposed towards math, the exhibits hardly required such a predisposition–any more than the Exploratorium in San Francisco requires a predisposition towards science. Rather, experiences like these create excitement, overcoming the negative preconceptions that too many children (and adults!) have about this subjects.

While I suspect that many Noisy Channel readers are already sold on both the enjoyment and core societal value of mathematics, I encourage you to think about how much better a world we would have if this appreciation were more widely shared. For those who have to think about large numbers just to manage their assets, I encourage you to think of The Math Factory as worthy of your philanthropy. I encourage everyone to contribute your ideas and endorsements to this visionary effort.

Another topic in that conversation was pseudonymity. Someone pointed to a 2000 USENIX paper entitled “Can Pseudonymity Really Guarantee Privacy?” The challenges of implementing pseudonymity have, of course, received lots of attention in the past few years. The most notorious example is the AOL search data scandal, which made the front page of the New York Times. But there’s also the work co-authored by my friend Vitaly Shmatikov on de-anonymizing Netflix data. Indeed, some have expressed concern that the new Netflix competition is a privacy lawsuit waiting to happen.

Finally, danah boyd’s master’s thesis on “faceted id/entity: managing representation in a digital world” also came up–and I recently discovered by way of Robert Scoble that she’ll be keynoting at SXSW next year. Now I feel even more proud that I convinced her to speak at the SIGIR Industry Track this year. But I digress.

What does any of this have to do with copyright? Watch Lessig’s presentation–it’s long, but I promise you it’s worthwhile and entertaining to boot. Besides, I’ve made it easy by embedding it for you! He makes an analogy–rather, he makes fair use of Jonathan Zittrain’s analogy–between privacy rights and copyright.

The executive (and overgeneralized) summary is that both privacy-holders (“consumers”) and copyright-holders (“industry”) have complained that technology has undermined their rights, and both have sought out legal remedies. Consumers push back on industry, frustrated with legal strategies to enforce copyright at the expense of consumer freedom, preferring instead to let technology dictate policy; industry pushes back on consumers, frustrated with their legal strategies to enforce privacy rights at the expense of industry freedom, in this case preferring instead to let technology dictate policy. The analogy may not be perfect, but it is close enough to be compelling.

But I’d like to stretch the analogy further than Lessig and Zittrain to consider pseudonymity and derivative works. The pseudonymity challenge (e.g., the recent reports about Project Gaydar) remind us that privacy isn’t binary, and that we have to accept at least some loss of privacy if we are going to live in a social world. Similarly, provisions like fair use exist because copyright is an inherent trade-off between protecting creators’ rights and embracing the value of creation in a social context.

As I said, I find the Zittrain’s analogy and Lessig’s presentation compelling. While it may not answer any of society’s urgent questions about privacy and copyright, it may at least further the conversation. At the very least, I hope the topic is intellectually stimulating.

A few days ago, I was exchanging email with serial entrepreneur Chris Dixon about human-computer information retrieval (HCIR). He’d just looked through the accepted submissions list for HCIR 2009 and said, if I may paraphrase: this is great stuff, but it needs to be better communicated for broader consumption. I quickly shot back a reaction that I’ll excerpt here (when in doubt, make it public!):

At some level it’s blindingly obvious: to err is human, to really screw up takes a computer. The HealthBase fiasco isn’t a shocker: lots of people are skeptical of pure AI approaches.

What people don’t get is that you can work to optimize the division of labor. I’m evangelizing it in places like Technology Review–a bit more mainstream than my blog. But ultimately the message has to resonate with entrepreneurs and investors who will make that vision a reality. Endeca is all about HCIR. Bing is a step in the right direction for the open web. But there’s a long way to go.

His response: that’s a lot more consumable that any other description of HCIR he’d seen to date (and he’s a regular reader here!). Having just finished reading Steve Blank’s Four Steps to the Epiphany, I appreciate his point: in a new market, the most critical priority is educating the potential customers.

As a number of us prepare for the HCIR 2009 workshop, that’s something to keep in mind. There’s a natural tension between rigorous scholarship and mass communication, but some have the greatest scholars (e.g., Richard Feynman and Linus Pauling) have shown the way for us mere mortals. Indeed, in a field as cross-disciplinary as HCIR, we would do well to make our work and vision as broadly consumable as possible, albeit without oversimplifying it to the point that it is vapid or even misleading.

Generally speaking, I blog in order to convince people that some of the esoteric ideas I encounter–and the occasional ideas I am fortunate enough to conceive–are worthy of broader consideration. I started blogging in order to bring greater visibility to HCIR–to convince people that the choice between human and machine responsibility is a false dichtomy in almost every aspect of the information seeking process.

In grade school, I learned that division of labor is the cornerstone of civilization–perhaps and our adaptive process of allocating effort our greatest achievement as a species. As machines play an increasingly important role in our lives–and serve as the lenses through which seek and consume almost all information–it is key that we not forget our roots. Let us be neither Luddites nor passive participants, but rather let us help computers help us.

Instead of an actual physical space, in IR, we are predominately concerned with accessing information within a collection of documents (i.e., information space), and instead of a transportation system, we have an Information Access System (i.e., a means by which we can access the information in the collection, like a query mechanism, a browsing mechanism, etc). The accessibility of a document is indicative of the likelihood or opportunity of it being retrieved by the user in this information space given such a mechanism.

After reading a pre-print of my HCIR 2009 position paper about the information availability problem, Vinay pointed me at follow-up work he’d done with Leif on information retrievability. I agree with his observation that, while I look at information availability from a user-centric perspective; they consider retrievability from  a document- or system-centric perspective. The approaches are complementary, and both add to a growing body of work that advocates a holistic model of how users access information, rather than a narrow focus on reductionist measures like precision and recall at the level of individual queries.

To be clear, those reductionist measures still have their place. In fact, I’m looking forward to NIST’s Ellen Voorhees defending Cranfield next month to an HCIR crowd that is, for the most part, deeply suspicious of it.

By the way, a chapter of my own book on faceted search is also available for free online, as is Marti Hearst’s entire book on search user interfaces.

Maggie Fox, founder and CEO of Social Media Group, will moderate a panel composed of Augie Ray, who blogs at Experience: The Blog) and is Managing Director of Experiential Marketing at interactive and social media agency Fullhouse; John Jantsch, who blogs at Duct Tape Marketing and is a marketing and digital technology coach; and yours truly. It’s free to attend; just register here.

Panel Presentations

• Usefulness as the Criterion for Evaluation of Interactive Information Retrieval
  Michael Cole, Jingjing Liu, Nicholas Belkin, Ralf Bierig, Jacek Gwizdka, Chang Liu, Jun Zhang and Xiangmin Zhang (Rutgers University)
• Modeling Searcher Frustration
  Henry Feild and James Allan (University of Massachusetts Amherst)
• Query Suggestions as Idea Tactics for Information Search
  Diane Kelly (University of North Carolina at Chapel Hill)
• I Come Not to Bury Cranfield, but to Praise It
  Ellen Voorhees (National Institute of Standards and Technology)
• Search Tasks and Their Role in Studies of Search Behaviors
  Barbara Wildemuth (University of North Carolina at Chapel Hill) and Luanne Freund (University of British Columbia)

Posters and Demonstrations

• Visual Interaction for Personalized Information Retrieval
  Jae-wook Ahn and Peter Brusilovsky (University of Pittsburgh)
• PuppyIR: Designing an Open Source Framework for Interactive Information Services for Children
  Leif Azzopardi (University of Glasgow), Richard Glassey (University of Glasgow), Mounia Lalmas (University of Glasgow), Tamara Polajnar (University of Glasgow) and Ian Ruthven (University of Strathclyde)
• Designing an Interactive Automatic Document Classification System
  Kirk Baker (Collexis)
• The HCI Browser Tool for Studying Web Search Behavior
  Robert Capra (University of North Carolina at Chapel Hill)
• A Graphic User Interface for Content and Structure Queries in XML Retrieval
  Juan M. Fernández-Luna, Luis M. de Campos, Juan F. Huete and Carlos J. Martin-Dancausa (University of Granada)
• Improving Search-Driven Development with Collaborative Information Retrieval Techniques
  Juan M. Fernández-Luna (University of Granada), Juan F. Huete (University of Granada), Ramiro Pérez-Vázquez (Universidad Central de Las Villas) and Julio C. Rodríguez-Cano (Universidad de Holguín)
• A visualization interface for interactive search refinement
  Fernando Figueira Filho (State University of Campinas), João Porto de Albuquerque (University of Sao Paulo), André Resende (State University of Campinas), Paulo Lício de Geus (State University of Campinas) and Gary Olson (University of California, Irvine)
• Cognitive Dimensions Analysis of Interfaces for Information Seeking
  Gene Golovchinsky (FX Palo Alto Laboratory, Inc.)
• Cognitive Load and Web Search Tasks
  Jacek Gwizdka (Rutgers University)
• Visualising Digital Video Libraries for TV Broadcasting Industry: A User-Centred Approach
  Mieke Haesen, Jan Meskens and Karin Coninx (Hasselt University)
• Log Based Analysis of How Faceted and Text Based Searching Interact in a Library Catalog Interface
  Bradley Hemminger (University of North Carolina), Xi Niu (University of North Carolina) and Cory Lown (NC State Libraries)
• Freebase Cubed: Text-based Collection Queries for Large, Richly Interconnected Data Sets
  David Huynh (Metaweb Technologies, Inc.)
• System Controlled Assistance for Improving Search Performance
  Bernard Jansen (Pennsylvania State University)
• Designing for Enterprise Search in a Global Organization
  Maria Johansson and Lina Westerling (Findwise AB)
• Cultural Differences in Information Behavior
  Anita Komlodi (University of Maryland Baltimore County) and Karoly Hercegfi (Budapest University of Technology and Economics)
• Adapting an Information Visualization Tool for Mobile Information Retrieval
  Sherry Koshman and Jae-wook Ahn (University of Pittsburgh)
• A Theoretical Framework for Subjective Relevance
  Katrina Muller and Diane Kelly (University of North Carolina)
• Query Reuse in Exploratory Search Tasks
  Chirag Shah and Gary Marchionini (University of North Carolina at Chapel Hill)
• Augmenting Cranfield-Style Evaluation with GOMS to Obtain Timed Predictions of User Performance
  Mark Smucker (Waterloo University)
• Text-To-Query: Suggesting Structured Analytics to Illustrate Textual Content
  Raphael Thollot (SAP Business Objects) and Marie-Aude Aufaure (Ecole Centrale Paris)
• The Information Availability Problem
  Daniel Tunkelang (Endeca)
• Exploratory Search Over Temporal Event Sequences: Novel Requirements, Operations, and a Process Model
  Taowei Wang, Krist Wongsuphasawat, Catherine Plaisant and Ben Shneiderman (University of Maryland)
• Keyword Search: Quite Exploratory Actually
  Max Wilson (Swansea University)
• Using Twitter to Assess Information Needs: Early Results
  Max Wilson (Swansea University)
• Integrating User-generated Content Description to Search Interface Design
  Kyunghye Yoon (SUNY Oswego)
• Ambiguity and Context-Aware Query Reformulation
  Hui Zhang (Indiana University)

I don’t know how much the audience took heed of his call, but he certainly followed his own advice. He and Endeca alum Mark Watkins just launched Goby, a vertical search engine that exhorts you to “create your own adventure”.  It’s fun–a sort of exploratory search for explorers. And it uses a deep web crawl to populate its index with semi-structured data.

Anyway, try it out! I’ve been in the private beta, but haven’t had the chance to see what they’ve been up to in the final stretch leading to the launch. You can also read more on Search Engine Land or CNET.

The morning kicked off with a series of 18 lighting talks in 90 minutes–that was 5 minutes apiece, with a ruthless gong for anyone who went overtime. The presentations were consistently intense, and I had the misfortune to follow one of the best talks–a very passionate presentation about crowd-sourced translation by IBM’s Uyi Stewart. Other notable presenters included design ninja Alexis Lloyd from the New York Times R&D Lab, Karrie Karahalios from the University of Illinois talking about the experimental WeMeddle Twitter client,  MIT Media Lab professor and Berkman Fellow Judith Donath showing a stunning gallery of “data portraits”, and Dragon Systems co-founder Janet Baker explaining how the brain recognizes speech–with an skull as a prop! The session was incredible, and I hope other conferences adopt this model.

After the coffee break, there was a session on Text Analysis in the Large, featuring Dan Gruhl (IBM), Gary King (Harvard), and David Ferrucci (IBM). Dan Gruhl talked about web-scale text analysis–a topic up his alley, considering his role in architecting the IBM WebFountain project. Gary King gave a fascinating talk about using ensemble methods to improve on existing clustering methods–the idea is to synthesize a collection of derived clusterings and place them in an explorable metric space. You can read the full paper here. But the winner for this session was definitely David Ferrucci, who described the work IBM Research is doing to develop a machine Jeopardy player. He spent much of the talk building a case for the difficulty of the problem–and then delivered the punchline: In less then three years of research, they’ve developed a machine player whose performance is comparable to that or jeopardy winners. Hopefully they’ll be competing on live television by next year!

After lunch, there was a session on Investigation, featuring MAPLight Research Director Emily Calhoun, UC Berkeley law professor Kevin Quinn, and Guardian news editor Simon Rogers. Emily Calhoun showed how MAPLight illuminates the connections between money and politics–it was great seeing data to correlate who supports and opposes bills with the associated campaign contributions from interest groups. Kevin Quinn’s presentation was a bit more technical, but his work reminds me a lot of Miles Efron’s work on estimating political orientation in web documents–but Quinn’s work is more general and goes beyond co-citation analysis to analyze the actual language of the documents. Great application of topic modeling! But my favorite presentation in this session was the one from Simon Rogers: he told the story of how the Guardian successfully crowd-sourced a project to investigate the expenses of UK Parliament members.

The final session was a panel discussion about how visualization might elevate or advance the debate over health care policy. The panelists were Ben Fry, Marti Hearst, Gary King, and Simon Rogers; Fernanda Viégas and Martin Wattenberg moderated. Unfortunately, the overwhelming sentiment from the panel was pessimism that anything we could do might actually lead to improved outcomes. Nonetheless, it’s clear that a lot of people are going to try.

Again, I want to thank Fernanda, Martin, Irene Greif, and everyone at IBM for organizing this fantastic event–and for inviting me to attend! I am impressed that anyone could manage to assemble such an impressive set of speakers in one place, and I appreciate the effort that everyone put into making the past two days so worthwhile. I look forward to seeing the videos available online, and I hope those who weren’t able to attend take the opportunity to watch some of them. I also encourage you to check out the live Twitter stream at #tt09 while it’s still available.

I’ll start with Deputy U.S. CTO Beth Noveck’s keynote about the Open Government Initiative. First, the very existence of such an initiative is incredible, given the culture of secrecy traditionally associated with Washington. Second, I like the top priority of releasing raw data so that other people can work on analyzing it, visualizing it, and generally making it more accessible either to the general public or to particular interest groups. This is very much what I had in mind in January when I posted “Information Sharing We Can Believe In” and I’m glad to see tangible progress. I was never a big fan of faith-based initiatives.

The next session was a group of talks about watchdogs and accountability–people looking at how to ensure government transparency from the outside. New York Times editor Aron Pilhofer and software developer Jeremy Ashkenas talked about DocumentCloud, an ambitious project to enable exploratory search for news documents on the open web. Sunlight Foundation co-founder and executive director Ellen Miller offered a particularly compelling example of the power of visualization: a graph correlating the campaign contributions and earmark associated with a congressman under investigation. But my favorite presenter in this section was ProPublica’s Amanda Michel, whose thoughts about a “human test of transparency” are worth a talk in themselves. For now, I recommend you look at the two projects she discussed: Stimulus Spot Check and Off the Bus.

After lunch, we shifted gears from government transparency to more of a focus on text. The first of the two afternoon sessions was entitled “Analyzing the Written Record” and featured Matthew Gray from Google Books, Tom Tague from Open Calais (a free text annotation service that almost all of the previous speakers raved about), and Ethan Zuckerman from Harvard’s Berkman Center. All of the talks were solid, but Ethan’s was outstanding. I blogged about his Media Cloud project back in March, but it’s come a long was in the past six months and is doing something I’ve been waiting years to see someone do: comparing how different news organizations select and cover news.

The final session was about visualization. David Small offered a presentation about literally transparent text that was, in the words of Marian Dörk, “refreshingly non-utilitarian and visually stimulating”. Ben Fry showed the power of visualizing changes in a document over time–specifically, a project called “the preservation of favoured traces” that illustrates  the evolution of Darwin’s On the Origin of Species. But, as expected, IBM’s Many Eyes researchers Fernanda Viégas and Martin Wattenberg stole the show with an incredibly informative and entertaining presentation about the visualization of repetition in text. No summary can do it justice, so I urge you to watch the video when it is available.

After all that, we enjoyed a nice reception at the IBM Center for Social Software. I’m incredibly grateful to IBM for organizing and sponsoring this event, and to Martin Wattenberg for being so kind as to invite me. I’ll try to earn my keep in my 5 minutes at the “Ignite-style” session tomorrow morning.

I’m glad to finally see a real world example of the issues I blogged about last year in a post entitled “Privacy and Information Theory“:

The mainstream debates treat information privacy as binary. Even when people discuss gradations of privacy, they tend to think in terms of each particular disclosure (e.g., age, favorite flavor of ice cream) as binary. But, if we take an information-theoretic look at disclosure, we immediately see that this binary view of disclosure is illusory.

I’m curious to see if this project advances the conversation. At the very least, I’m gratified to see my abstract ramblings validated by a real-world example!

But I can certainly see the appeal of delivering faceted search for the web to support exploratory information seeking. It’s the dream that’s been driving Bing, Freebase, not to mention smaller efforts like Kosmix. It’s hard, to be sure. But, as Sarah Lacy tells us, startups are supposed to be changing the world–and established companies can play too.

The demo video is appealing, but I’ll believe it when I can off-road on it–and on more than just recipes and restaurants, two highly structured domains that already well covered by sites like Food Network and Yelp. Twine doesn’t necessarily have to cover all domains to be useful–perhaps a “short snout” approach like Bing’s will be good enough to drive adoption.

In any case, I’m impressed with Twine’s ambition. But ambition isn’t enough–especially given the increasing number of people and companies who share it. If Nova really wants to build a “World Wide Database“, then he’ll have to do more than swing for the fences and miss. I’ll be waiting for a beta invite, and I’ll let you know what I find out.

The Transparent Text symposium is a free event that will focus on ways to make large collections of documents understandable to laypeople and experts alike. We are interested in approaches that shed light on unstructured text, ranging from novel statistical techniques to web-based crowdsourcing.

The speaker list is impressive, ranging from familiar (at least to me) interface experts Ben Fry and Marti Hearst to social scientist Gary King and Sunlight Foundation Executive Director Ellen Miller. IBM also contributed some of its own researchers to the program, including David Ferrucci, who has been leading the Jeopardy project. There’s even an “Ignite-style” session where all attendees will have the opportunity to give five-minute presentations.

I’m looking forward to the eclectic mix of speakers and attendees. As Chris Dixon recently reminded us, it’s important to introduce some randomization into our intellectual diets so that we don’t get stuck in a rut of local optimization. While an event with a theme of transparency and interacting with textual information is hardly a detour for me, I am excited about the opportunity to hear a diversity of new perspectives on this topic. There will be videos of the speakers posted after the event, as well as a  Twitter stream at #tt09.

Of course, I’ll blog about what I learn and recycle it in the discussion activities at the HCIR workshop next month.

But sometimes I like to get away from all that and turn back to my roots inside the ivory tower. And that leads me to one of my favorite university blogs: the Haystack Blog.

The Haystack Blog is published by faculty and grad students in the MIT Computer Science and AI Lab (CSAIL)–specifically those in the Haystack group. Principal Investigator (and occasional dance instructor) David Karger is its most prolific blogger–you might have read some of his SIGIR 2009 posts or his debate with Stefano Mazzocchi about how to properly use RDF. But other people’s posts are just as interesting–check out the most recent post by Eirik Bakke about bridging the gap between spreadsheets and relational databases.

I wish that more universities and departments would encourage their faculty and students to blog. As Daniel Lemire has pointed out, it’s a great way for academic researchers to get their ideas out and build up their reputations and networks. He should know–he leads by example. Likewise, Haystack is setting a great example for university blogs, and is a credit to MIT and CSAIL.

Those of you who don’t live and breathe the software startup scene might be oblivious that a substantial fraction of Silicon Valley is following TechCrunch50, an annual competition hosted by TechCrunch. As if it weren’t enough to have A-list judges like Marissa Mayer and Paul Graham, there’s even the fortuitous timing of Intuit acquiring 2007 TC50 winner Mint for a respectable $170M.

Here in New York, I have to confess that I haven’t had my eyes glued to the proceedings. But I have been looking at some of the entries, and one that at least stands out as distinctive is Udorse (and no, I’m not just biased because they’re local). Their premise is simple: democratize product placement through “visual endorsement”. Everyone who shares photos can embed a “udorsement” and can either pocket the advertising revenue or donate it to charity. More details from TechCrunch (naturally) and VentureBeat.

Perhaps your reaction is like mine, uncertain whether to be awed or horrified by this simple concept. Indeed, given my penchant for using ad blockers, you might think I’d be ideologically against product placement.

But I’m not, as long as it’s transparent–and, as far as I can tell, Udorse passes that test. In theory, this is advertising done right: content creators monetizing their own content by advertising goods and services they believe in–and putting their own credibility on the line to do so.

Of course, it might turn out very differently in practice. Any way of making money online brings out the worst in people, and I’m sure we’ll see lots of people try to game this service if it takes off. Meanwhile, people like me will probably block the “udorsements” like any other ads.

Or maybe not. I certainly don’t block emails from friends recommending the products they like, and I actually wish it were easier to benefit from their sincere opinions. If Udorse succeeds in a way that feels like word-of-mouth marketing, I’ll be thrilled. I think it’s a long shot, but I’m at least intrigued by their approach.

ps. No, I wasn’t payed to write this post, nor do I have any stake in Udorse. I at least have to keep my record clean for the Ethics of Blogging panel next week!

Bing launched a Visual Search beta today that is fun to play with. The name may be a bit misleading–this isn’t an image search engine, let alone one that allows you to find images based on visual similarity. Rather, it’s a graphically intensive (don’t forget to install Silverlight!) way to explore a small data collection.

I agree with Elisabeth Osmeloski at Search Engine Land that the galleries included with this beta launch emphasize novelty over utility. Still, it’s nice to see a visual faceted search application for exploring the periodic table. And it’s an interesting example of micro-IR.

First, it’s an interesting counterpoint to the conventional wisdom that search (if not the future of business as we know it) is all about the “long tail“. But second and more importantly, it’s an intriguing claim about Bing’s strategy for differentiating itself from Google.

Needleman goes on to say:

I’d wager that this is how Bing is making its gains in market share. Latest Nielsen data says Bing gained 22 percent month-over-month in August, taking it to 10.7 percent of all U.S. searches. People probably try Bing for a travel or product search (where there’s also a cash-back financial kicker) and remember their good experience, and then they try it for more obscure searches and find it good enough. It highlights, I believe, an important flaw in Google’s historic strategy of indexing the entire Web equally well and making the user interface fast and consistent above all, as opposed to specializing as dictated by the query.

While I’ve never heard this claim about Bing before, it is consistent with something I’ve noticed–and which Nick Craswell said when he talked about Bing at SIGIR 2009. In the upper left area that Bing calls the table of contents (TOC), Bing selectively presents a refinement interface based on the entity type it infers for the search query. For example, a search for Argentina returns options that include Argentina Map, Argentina Tourism, and Argentina Culture; while a search for Abraham Lincoln returns options that include Abraham Lincoln Speeches and Abraham Lincoln Facts.

It’s a nifty feature, even if marketers and reporters have struggled to label it. But, as Needleman says, it does indeed focus on the short snout. For example, there are no TOC options when you search for faceted search, since the technical term doesn’t match a recognized entity type. Searches for names of auto companies, such as Toyota, yield a rich set of options, while those for scooter companies like Vespa do not. Similarly, searches for celebrities receive VIP treatment, as compared to searches for ordinary people that just return a list of search results.

All in all, I’m inclined to agree with Needleman that Bing is focusing on the short snout–and I love that phrase to describe it. The open question is whether he’s right that users “remember their good experience, and then they try it for more obscure searches and find it good enough”. It would be great to see data to confirm or refute that hypothesis.

1. In ad hoc (text) IR a principal intellectual challenge lies in modeling ‘aboutness.’  In micro-IR settings, the creativity comes into play in posing a useful (and tractable) question to answer.  The engineering comes easily after that.
2. The constrained nature of micro-IR applications leads to a lightweight articulation of information need.  There is a tight coupling here between task, query, and the unit of retrieval, a dynamic that I think is compelling.  Pushing this a bit farther, we might consider the simple act of choosing to use a particular application from those apps on a user’s palette as part of the information need expression.
3. The tight coupling of task to data to ‘query’ enables a strong contextual element to inform the interaction.  Context constitutes the foreground of the micro-IR interaction.

He then asks: “is micro-IR something at all?  Is it actually related to IR?” Fernando Diaz answers that “the only difference between micro and macro IR is text.” Jinyoung Kim adds that in micro-IR “the context (searcher goal) is known, with domain-specific notion of relevance (goodness) and similarity measures.”

I hadn’t thought of making this particular distinction, but I like it. While I prefer to think about distinguishing the needs of information seekers–rather than the characteristics of search applications–I would be the first to argue that a well-designed search application caters to particular user needs. Indeed, I think the definition of a good micro-IR application implies that it addresses a highly constrained space of information needs. Just as importantly, micro-IR applications can often assume that their users are highly familiar with the information space the applications address, and thus that those users need less of the basic orienteering support that can be critical for success using macro-IR systems. That said, micro-IR users have (or should have) higher expectations of support for more sophisticated information seeking.

The other day, I speculated about why Google holds back on faceted search. I feel that the distinction between macro- and micro-IR is in the same vein: micro-IR settings (e.g., site search, enterprise search,vertical search) drive needs for more richer interfaces and support for interaction, while macro-IR application developers (e.g., general web search) worry mostly about producing a reasonable answer for the query–and often lead users to micro-IR destinations that offer their own support for information seeking within their constrained domains.

In short, it’s a nice way to think about the IR application space, and it’s increasingly relevant (no pun intended!) as we see a proliferation of micro-IR applications. And it’s great to see activity on the Probably Irrelevant blog after all these months of radio silence!

In any case, Yahoo researchers haven’t stopped thinking about search. As Jeff Dalton reports, Yahoo recently issued a press release about its Key Scientific Challenges Summit. Jeff was kind enough to post Henry Feild’s notes about the presentations by Andrew Tomkins on search and by Sathiya Keerthi Selvaraj on machine learning. I’d love to hear more detail about how they perceive (and hope to address) the search challenges of optimizing task-aware relevance and measuring / predicting generating user engagement.

Regardless of Yahoo’s fate, I’m certainly glad that there are still people at Yahoo working on these big problems. I hope they find a way to develop solutions and bring those solutions to the users who need them.

I’ve been invited to participate in a webinar on the ethics of blogging that will take place Thursday, September 24th at 1 PM EST. It’s free to attend; just register online here.

Maggie Fox, founder and CEO of Social Media Group, will moderate. My two co-panelists are Augie Ray, who blogs at Experience: The Blog) and is Managing Director of Experiential Marketing at interactive and social media agency Fullhouse, and John Jantsch, who blogs at Duct Tape Marketing and is a marketing and digital technology coach.

Among the topics to be discussed:

• Transparency: How and when should a blogger reveal revenue sources?
• Pay for play: Blog posts, tweets, and more as marketing tools
• Online privacy
• Astroturfing: Organizations creating artificial “grassroots” campaigns
• Compliance and Legal: What should a corporate blog policy look like? What are a blogger’s legal obligations?

I hope some of you will be able to attend! Regardless, please use the comment thread make suggestions here about topics you’d like me to cover or concerns you’d like to see me address. I know that a lot of you have thought hard about these issues, and I’d like to ethically exploit your collective wisdom.

Anyway, thanks to Jeff Dalton for alerting me that the CIKM 2009 accepted papers list is now available. I don’t plan to make it to Hong Kong this November, but I hope that those who do are kind enough to blog about it!

Also, I see mention of an industry track, but not of an Industry Event like the widely acclaimed one held at CIKM 2008–which inspired my own organization of the SIGIR 2009 Industry Track. I’m curious whether such industry events will prove to be one-time phenomena or will become a staple of these  conferences. I hope for the latter, but I am admittedly biased, given my industry-centric perspective.

Google has an image problem – not a PR problem (that is, not with the public) but a press problem (with whining old media people).

He then goes on to launch a tirade against a Le Monde journalist whose offense was to say she was writing “an article about Google facing a rising tide of discontent concerning privacy and monopoly.” He proceeds to stereotype the French as having “national insanity” of Google bigotry. I’ll leave analysis of irony as an exercise to the reader.

But the true irony is that Jarvis has a point. While I haven’t done a rigorous analysis, my impression is that there has been a sensationalist press overreaction against Google, singling out Google for behavior for which all other companies get a pass. As even one of the most vocal Google critics admits, “Google’s [privacy] policies are essentially no different than the policies of Microsoft, Yahoo, Alexa and Amazon.” Moreover, some of the newspapers criticizing Google as parasitic are the same ones who once turned–and still turn–to Google with open arms as a source of traffic–when they could easily cut Google off by configuring robots.txt. Granted, the newspapers are now locked into a prisoner’s dilemma, but they should at least take some responsibility for putting themselves in that position.

That said, there are lots of legitimate reasons to criticize Google, specifically concerning privacy and monopoly. While Google may not have engaged in any illegal or unethical practices to get there, it now holds a position as the primary gatekeeper to the internet for a substantial majority of Americans, as well as much of the western world. On the content creation side, site owners don’t ask “What Would Google Do?“–rather they ask how Google will index their sites. Meanwhile, on the consumption side, the broadening scope of Google’s role in ordinary people’s lives is legitimate cause for concern about privacy. It’s not insane or bigoted to raise these issues.

Moreover, Google claims to hold itself to a higher standard than other companies, so it’s not that surprising that people actually do hold them to it and criticize it when it flls short. Still, that’s no excuse for exaggeration or outright hallucination.

As I commented on Jarvis’s blog, I don’t think he’s the most credible judge of Google’s critics. He responded in kind. Touché. I accept that exchanging personal attacks doesn’t advance the argument. Perhaps more detached voices can chime in.

But sometimes they don’t. Netbase’s recent experience with HealthBase is, unfortunately, a case study in why you shouldn’t have too much faith in magic. As Jeff Dalton noted, the “semantic search” is hit-or-miss. The hits are great, but it’s the misses that generate headlines like this one in TechCrunch: “Netbase Thinks You Can Get Rid Of Jews With Alcohol And Salt”. Ouch.

It seems unfair to single out Netbase for a problem endemic to fully automated approaches, but they did invite the publicity. It would be easy to dig up a host of other purely automated approaches that are just as embarassing, if less publicized.

Dave Kellogg put it well (if a bit melodramatically) when he characterized this experience as a “tragicomedy” that reveals the perils of magic. His argument, in a nutshell, is that you don’t want to be completely dependent on an approach for which 80% accuracy is considered good enough. As he says, the problem with magic is that it can fail in truly spectacular ways.

Granted, there’s a lot more nuance to using automated content enrichment approaches. Some techniques (or implementations of general techniques) optimize for precision (i.e., minimizing false positives), while others optimize for recall (i.e., minimizing false negatives). Supervised techniques are generally more conservative than unsupervised ones: you might incorrectly assert that a document is about disease, but that’s less dramatic a failure than adding the word “Jews” to an automatically extracted medical vocabulary. In general, the more human input into the process, the more opportunity to improve the effectiveness and avoid embarassing mistakes.

Of course, the whole point of automation is to reduce the need for human input. Human labor is a lot more expensive that machine labor! But there’s a big difference between the mirage of eliminating human labor and the realistic aspiration to make its use more efficient and effective. That what human-computer information retrieval (HCIR) is all about, and all of the evidence I’ve encountered confirms that it’s the right way to crack this nut. Look for yourselves at the proceedings of HCIR ‘07 and ‘08. Having just read through all of the submissions to HCIR ‘09, I can tell you that the state of the art keeps getting better.

Interestingly, even Google CEO Eric Schmidt may be getting around to drinking the kool-aid. In an interview published today in TechCrunch, he says: “We have to get from the sort of casual use of asking, querying…to ‘what did you mean?’.” Unfortunately, he then goes into science-fiction-AI land and seems to end up suggesting a natural language question-answering approach like Wolfram Alpha. Still, at least his heart is in the right place.

Anyway, as they say, experience is the best teacher. Hopefully Netbase can recover from what could generously be called a public relations hiccup. But, as the aphorism continues, it is only the fool that can learn from no other. Let’s not be fools–and instead take away the moral of this story: instead of trying to automate everything, optimize the division of labor between human and machine. HCIR.

Their approach strikes me as a generalization of measuring retweets, but perhaps I’m giving it too cursory a read. I did compare their results to TunkRank: we at least agree that mashable is more influential than CNN–though even as simple a measure as follower count would confirm that judgment.

Anyway, I am delighted to see serious researchers looking at this problem. I’m still hoping to investigate hypotheses regarding TunkRank and friend:follower ratios.

Here are the posts:

• What a tangled MeSH we weave
• Open-source queries
• Have queries, want answers

Even if you’re not specifically interested in medical literature search, I recommend you check these posts out. Much of the interesting work on information seeking is taking place in specialized domains like this one, where the value of getting it right offers far more promising returns than incremental improvements to general web search.

Library scientists make a distinction between known-item and exploratory search. The former is about locating information: as an information seeker, you know the information exists, and you can even characterize it unambiguously; but the challenge is to convert that description into a location that allows you to retrieve the information. The latter is about discovery: you don’t know that the information you seek exists, and you may be sure of how to characterize what you are looking for–or even know what exactly you want until you’ve learned something about what is available.

These are extreme points on the information seeking spectrum, and most real-world tasks are in the middle, or combine subtasks of both types. For example, in physical libraries (yes, I’m that old!), I remember finding a book in the stacks and then browsing the nearby books in the hopes of serendipitous discovery. These days, I’d be more likely to scan its bibliography–or to look at the books and articles citing it. A known item can be an excellent entry point for exploration. Conversely, exploration can lead you to discover the existence of information that you then simply need to retrieve.

In common use, words like searching and finding cover this entire spectrum of information seeking activity. This breadth of meaning causes a lot of confusion. I’ve blogged about this before: “What is (Not) Search?“:

At the very least, I propose that we distinguish “search” as a problem from “search” as a solution. By the former, I mean the problem of information seeking, which is traditionally the domain of library and information scientists. By the latter, I mean the approach most commonly associated with information retrieval, in which a user enters a query into the system (typically as free text) and the system returns a set of objects that match the query, perhaps with different degrees of relevancy.

Back to Campbell’s article. His main points:

• Social networks have dramatically expanded our network of contacts.
• Search engine optimization (SEO) experts have killed their own game.
• The flow of information has changed: information now comes to us, rather than us having to go out and find it.

I like the spirit of the post, but I think he overstates his case. SEO isn’t all bad–in fact, it’s probably a key factor in Google’s effectiveness. And, while social networks enable social search in theory, and information does come to us; we are experiencing filter failure (Clay Shirky’s term) in a big way.

My conclusion: I agree with him about Google’s limitations–Google is primarily a locating tool, not a discovery tool. Unfortunately, I’m not persuaded that social networks and our theoretical ability to construct an ideal in-flow of information have actually delivered on the promise of more efficient information access. But I’m optimistic that we’ll eventually get there.

Chris has an interesting history that includes heading R&D at a hedge fund, co-founding SiteAdvisor, investing in a number of technology companies (including Skype and Postini), and most recently co-founding Hunch (which I’ve blogged about here a few times). As a karaoke junkie, I can’t help noting that he developed the software that became MySpace Karaoke.

Not surprisingly, Chris brings the combined perspective of an investor and a technologist to his blog. Here are some examples of recent posts that illustrate his range.

Thoughts on machine learning:

• To make smarter systems, it’s all about the data
• Machine learning is really good at partially solving just about any problem

Career advice for entrepreneurs:

• The worst time to join a startup is right after it gets initial VC financing
• Why you shouldn’t keep your startup idea secret

And of course he occasionally blogs about Hunch, his current venture.

Chris has a strong personality that comes through as a blogger. I think that’s critical for making a blog both informative and entertaining, and I try to channel my own personality (which I’m told, for better or worse, is quite distinctive) through this blog.

In short, check out cdixon.org if you’re interested in the perspective of a practical (and successful) technologist-entrepreneur.

And I’ve dabbled a bit on the theoretical side myself. The TunkRank measure (I’m indebted to Jason Adams for his implementing it on a live site!) attempts to take the most basic assumption about our social behavior–the constraint that we have a finite attention budget–and explore its implications for influence over social networks. I have a few unexplored hypotheses queued up for when I can find the spare time to try validate them empirically!

But why settle for theory? We live in an age where social networks compete with web search (and perhaps complement search) as the hottest online technologies. If we’re not reading about Google vs. Bing, we’re reading about Facebook vs. Twitter, with LinkedIn offering a third way that seems to co-exist with its more storied peers. In this post, I’d like to focus on LinkedIn.

LinkedIn, despite its feature creep, is still fairly old-school: its raison d’être is for users to build, maintain, and exploit their professional networks. In theory, connections on LinkedIn represent present or past working relationships that become the basis for referrals–whether the goal is employment, sales, or partnership. LinkedIn is not the only professionally oriented social network, but at this point it’s certainly the dominant one.

But I’ve found at least two additional ways to use LinkedIn that I’d like to share:

Intelligence gathering. For reasons I don’t yet claim to understand, people share far more information about themselves–and in a much cleaner, structured form–on LinkedIn than in perhaps any other online medium. Most people’s resumes are not available online, but their LinkedIn profiles are tantamount to resumes. Moreover, their structured format makes it possible for LinkedIn to assemble aggregate profiles of companies, revealing composite pictures that must drive some of those companies’ legal and HR departments batty! At a higher level, LinkedIn also works well as a discovery tool–much more so now they’ve enabled faceted search. It’s still a bit tricky to explore people and companies by topic, but far more effective using LinkedIn than using any other tool I’m aware of.

Meeting new people. Cold-calling, spamming–pick your poison. In short, LinkedIn doesn’t have to only be about connecting with people you already know. But there’s an art to sending unsolicited messages: you have to pass the moral equivalent of a CAPTCHA by proving that your communication strategy isn’t indiscriminate. Let me use a personal example (that Maisha Walker was nice enough to write up in her Inc. magazine column). I decided that I wanted to find everyone on LinkedIn who might be interested in HCIR ‘09. So I searched for everyone whose profiles indicated interests in both IR and HCI and sent out a targeted message (in fact, a invite with personalized message–a feature I recently feared they’d killed). The results were overwhelmingly positive. I’m not sure how many of the people I contacted will attend, but I raised awareness without inflicting annoyance. Better yet, one of the people I contacted then discovered I was looking for volunteers to review the draft of my book–and I thus obtained hours of help of someone who, just a day before, had never heard of me!

What intrigues me about LinkedIn (and other social networks) is the extent to which I am exploiting attention market inefficiencies (as LinkedIn may be doing as well). For example, LinkedIn makes it easy to send unsolicited invitations to anyone. Granted, you can lose this privilege by even having a couple of people respond to invitations with “I don’t know this person”. There’s also the question of why people’s social norms around disclosure are so different on LinkedIn than anywhere else–people not only post the content of their resumes, but go through the effort of providing it to LinkedIn in a structured form! Meanwhile, LinkedIn keeps tightfisted control over the information it aggregates–understandably, they recognize that this content is their most valuable asset.

People are still getting used to the idea of social networks. It will be interesting to see how their use evolves, particularly in term of information and attention market efficiency.

Well, that was child’s play compared to what PR firm Reverb Communications has allegedly been doing for it clients. According to Gagan Biyani at  TechCrunch, Reverb hired interns to post positive review to Apple’s App Store for clients. Indeed, TechCrunch posted documentation obtained through an anonymous tipster, including the following:

Reverb employs a small team of interns who are focused on managing online message boards, writing influential game reviews, and keeping a gauge on the online communities. Reverb uses the interns as a sounding board to understand the new mediums where consumers are learning about products, hearing about hot new games and listen to the thoughts of our targeted audience. Reverb will use these interns on Developer Y products to post game reviews (written by Reverb staff members) ensuring the majority of the reviews will have the key messaging and talking points developed by the Reverb PR/marketing team.

What makes this story especially newsworthy is that Reverb’s client list includes some big names, such as Harmonix (i.e., Guitar Hero and Rock Band) and MTV Games.

Apparently the reviewer system isn’t entirely anonymous, so Biyani was able to look for patterns:

iTunes allows you to see other reviews posted by the same reviewer. So, we clicked on the reviewer “Vegas Bound” (iTunes link) and started to look at his reviews. He reviewed 7 applications, and gave each one of them 5 stars. Each review was short and sweet, and extremely positive. These reviews represented 6 different developers. A quick Google search revealed an infuriating truth: every single one of these developers was a client of one PR firm: Reverb Communications.

I can only hope that scandals like these will cause people to be more skeptical of reviews (or opinions in general) that come from anonymous or obfuscated sources. While most reviews are probably sincere, it doesn’t take much to erode public trust. Moreover, a few shill reviews can attract attention to a product, thus leading legitimate reviews to follow afterward. Where’s the harm? Products without those shill reviews are starved of the attention they might deserve. Money substitutes for authentic endorsement.

Our brave new world of social media makes it possible to truly democratize the sharing of knowledge and opinions. But gaming the system like this erodes the trust that is essential for this process to work–and thus devalues all of the information available to us online. The key enabler of such gaming is anonymity. Fortunately the miscreants do get caught on occasion. Hopefully we will learn from this experience and build more robust systems that aren’t so easily gamed. Transparency or FAIL.

Since we didn’t manage to get to all of the questions (over 40–possibly over 50 counting the activity on Twitter!), we’re going to do a follow-up podcast that will be available even to people who didn’t attend the seminar. And, since even that might not be enough, I’m saving all of the questions as blog fodder.

To all who attended–and to Jared, Adam, and all the folks of UIE–thanks from me and Pete for giving us this great opportunity to connect with folks interested in faceted search and user experience.

But I was wrong. Don’t take it from me–watch the video below (or read this blog post) and listen to what Cyrus Mistry,  the product manager for the Google Search Appliance has to say. I might give him a pass on his dubious conflation all features other than ranked retrieval with “advanced search”. But here’s a direct quote: “users care about one thing: the right result coming to the top”.

Sigh. I don’t dismiss the value of relevance ranking. Some search queries are easy and clearly point to single documents as answers–and any search engine should do well on them. But lots of queries in site search and enterprise search environments (more so than on the web) don’t have a single best answer. That’s why we have faceted search and interfaces that offer useful information scent to users.

I understand that Google is, on the whole HCIR-averse. But I expect more from their enterprise division. To be clear, the “side by side” feature that Mistry touts is nice. It reminds me of Blind Search (built by a Microsoft employee in his spare time), and of a relevance ranking evaluator that Endeca customers have been using for years.

But there’s more to search results than ten blue links. Even the Google web folks seem to be slouching towards accepting the importance of interaction. Their enterprise team should be leading, not lagging.

But I noticed on Twitter today that Mark Williams (aka @Mr_LinkedIn) had noticed the same change and followed up on it with LinkedIn’s customer service department. I never assume any site behavior on a freely provided service is permanent, but it is starting to look like this is a deliberate decision and not a transient bug.

If so, it’s an annoying change, though I can see the merits. I’ve made heavy use of the connection message, especially when inviting someone I don’t know all that well–or don’t know at all. A personal message can be what distinguishes a welcome cold call from spam. But I’m guessing that others have abused that capability, filling it with spam or worse. Still, I feel like LinkedIn may be throwing the baby out with the bathwater. Will follow up if / when I hear more.

UPDATE: Just saw this message on the LinkedIn site via Twitter:

Unable to Personalize Invitation Message

Why can’t I personalize the message in my Invitation?

We are aware of an issue preventing some members from customizing their Invitation messages. There is no need to contact Customer Service as our team is reviewing the issue to determine the best overall solution.

As a temporary workaround, the following message (with your name in the signature) is being sent when you click on the ‘Send Invitation’ button: ‘I’d like to add you to my professional network on LinkedIn.’

As long as you approve of this message, you may continue to take advantage of this feature. If you prefer a more customized message to be sent, you may delay sending your Invitations until the functionality has been restored.

UPDATE #2: Looks like the problem is resolved.

Google in particular has been researching the predictability of search trends (just to be fair and balanced, so have Bing and Yahoo). Yossi Matias, Niv Efron, and Yair Shimshoni at Google Labs Israel have made some fascinating observations based on Google Trends, including the following:

• Over half of the most popular Google search queries are predictable in a 12 month ahead forecast, with a mean absolute prediction error of about 12%.
• Nearly half of the most popular queries are not predictable (with respect to the model we have used).
• Some categories have particularly high fraction of predictable queries; for instance, Health (74%), Food & Drink (67%) and Travel (65%).
• Some categories have particularly low fraction of predictable queries; for instance, Entertainment (35%) and Social Networks & Online Communities (27%).
• The trends of aggregated queries per categories are much more predictable: 88% of the aggregated category search trends of over 600 categories in Insights for Search are predictable, with a mean absolute prediction error of of less than 6%.

You can read their full 32-page paper here.

I’m not surprised at the predictability of human search behavior, especially for stable topics or even for unstable ones viewed as aggregates–one could argue the celebrities and scandals du jour are unpredictable but interchangeable. What I’m curious about is what we can do with this predictability.

In the SIGIR ‘09 session on Interactive Search, Peter Bailey talked about “Predicting User Interests from Contextual Information“, analyzing the predictive performance of contextual information sources (interaction, task, collection, social, historic) for different temporal durations. Max Van Kleek wrote a nice summary of the talk at the Haystack blog. The paper doesn’t investigate seasonality (perhaps because they only looked at four months of data), but I’d imagine they would subsume it under the broader categories of historic and social context. But they do set a clear goal:

Postquery navigation and general browsing behaviors far outweigh direct search engine interaction as an information-gathering activity…Designers of Website suggestion systems can use our findings to provide improved support for post-query navigation and general browsing behaviors.

I hope Google is following a similar agenda. If you’re going to go through the trouble of predicting the future, then help make it a better one for users!

Jared Spool, who invited us to give this seminar, will be moderating. Indepedendent of the seminar, you you check out his work (and the UIE site) if you are interested in web usability.

The regular price is $129, but Noisy Channel readers who are interested in attending can get a $30 discount by using TUNKELANG (yes, all caps) as a promo code. Attendees also receive a free copy of my book, Faceted Search. That’s a a total value of over $150 for just $99! And it slices and dices!

Arnon Mishkin:

The vast majority of the value gets captured by aggregators linking and scraping rather than by the news organizations that get linked and scraped.

Jeff Jarvis:

Links are worth what the recipient makes of them.

Mike Masnick:

It’s not the link alone that has value or the story alone that has value, but the overall process of building a community.

Erick Schonfeld:

If a news site or a blog can say enough interesting things enough times that news aggregators (or other sites) keep linking to them, then they can build up their brand and reader loyalty.

Sigh. I thought the health care debate was bad enough, but I suppose that almost all impassioned debates come down to opposing sides exchanging half-truths.

In Mishkin’s defense: news organizations are in a catch-22. Many have suggested that if a news organization doesn’t want its content showing up on aggregators’ sites, it simply has to modify robots.txt accordingly. But news organizations can only do so individually–which puts them in a prisoner’s dilemma. Anti-trust law prevents news organizations from collectively bargaining with those who aggregate their content. For all intensive purposes, they are forced to abide by the status quo.

In Jarvis’s defense (yes, I’m actually defending Jeff Jarvis!): there isn’t much point in producing content for which most of the value is captured in a teaser so small as to be covered under fair use rights. As he’s said elsewhere, newspapers are inefficient, and the industry will have to shrink a lot to be healthy.

In Masnick’s defense: I cite my own blog post (also inspired by one of his posts) about monetizing community because participation is inherently uncopiable. It’s hard for me to agree with him more strongly than that!

In Schonfeld’s defense: his argument sounds a lot like the “freemium” strategy, which has a respectable track record. In order to build a loyal customer base, you often need to give away free trials as teasers–and that’s effectively what happens when media sites make some of their content available through aggregators. And, as in the freemium model, the actual product has to be significantly more interesting that the free teaser to earn the consumer’s investment–whether that investment is in the form of money, attention, or loyalty.

So, do I agree with them all? Not exactly. Mishkin’s first prescription to news organization should probably be to cut investment in undifferentiated content. Jarvis should acknowledge that the inability of news organizations to collectively bargain is unfair to them. Masnick–well, I basically do agree with him on the limited point he’s making. I suppose the strongest objection would be that not all media sites should be forced to become communities just because they’re hobbled in their ability to negotiate the monetization of the content they produce. And Schonfeld’s argument assumes the current link economy as a given–and one of the biggest points of contention is whether news organizations should be allowed to try to change that economy.

Sadly, I don’t see any of these guys giving the other an inch, which is why this discussion will probably continue unchanged for the foreseeable future. Hopefully the passion of the debate helps sell, um, papers.

Let’s say I know almost nothing about wines/digital cameras/cars and a search site offers me “options” to drill down. However, I can’t use those effectively and eventually it comes down to availability and price for me. My questions are what are your thoughts on these kinds of situations and is there a scientific explanation/theory on this case?

This may be why Google does not endorse faceted search except for experimental projects.

It’s a great question. There’s been a lot of research on how people make decisions when they have to manage trade-offs among multiple attributes, and the increasing interest in behavioral economics since Daniel Kahneman won the Nobel Prize in 2002 has helped some of that research has even percolated into the mainstream thanks to bestsellers like Freakonomics and Dan Ariely’s Predictable Irrationality.

The short answer is that there’s no point in offering users options that they can’t (or won’t) use effectively. Choice overload is certainly a problem, and our reaction to it is to satisfice, typically resorting to “fast and frugal” heuristics that throw out most of the potential decision criteria and instead focus on one or two attributes, e.g., price and availability.

But that’s no reason to dumb down the data we make available to decision makers. We make hard choices all the time, and fast and frugal can be horrendously suboptimal. We don’t hire employees based solely on their price and availability–or at least good employers don’t! For that matter, I don’t think most people pick wines that way, given that even Trader Joe has to diversify beyond “Two Buck Chuck“. And, while there’s probably more of a market for cheap cameras and cars, I’m pretty sure you’re an extreme outlier if you completely ignore other criteria.

That said, there are some caveats about exposing options to users. Faceted search is hard, especially on the open web. Take it from the folks at Microsoft Research–but I’m sure Googlers would be the first to agree, especially given their experience with projects like Google Squared that, while promising, are nowhere near ready for prime time.

I appreciate that Google is conservative about embracing faceted search–and HCIR in general. I’m actually impressed by the steadily improving quality of their related terms for search queries–even if they do hide them behind two clicks (show options -> related searches). Perhaps they’re feeling some pressure from Bing. But I think they’re largely following the dictum of “if it ain’t broke, don’t fix it”. Google is an extremely successful company. And, as Clayton Christensen argues, successful companies are great at incremental innovation and bad at disruptive innovation. As far as I can tell, faceted search is very disruptive to their model.

On this data issue, people keep talking about how more data gives you a bigger advantage. But when you look at data, there’s a small statistical point that the accuracy with which you can measure things as they go up is the square root of the sample size. So there’s a kind of natural diminishing returns to scale just because of statistics: you have to have four times as big a sample to get twice as good an estimate.

Another point that I think is very important to remember…query traffic is growing at over 40 percent a year. If you have something that is growing at 40 percent a year, that means it doubles in two years.

So the amount of traffic that Yahoo, say, has now is about what Google had two years ago. So where’s this scale business? I mean, this is kind of crazy.

The other thing is, when we do improvements at Google, everything we do essentially is tested on a 1 percent or 0.5 percent experiment to see whether it’s really offering an improvement. So, if you’re half the size, well, you run a 2 percent experiment.

For those unfamiliar with statistics, I encourage you to look at the Wikipedia entry on standard deviation. Varian is obviously reducing the argument to a sound bite, but the sound bite rings true. More is better, but there’s a dramatically diminishing return at the scale of either Microsoft or Google.

However, I do think there’s a big difference when you start talking about running lots of experiments on small subsets of your users. The ability to run twice as many simultaneous tests without noticeably disrupting overall user experience is a major competitive advantage. But even there quality trumps quantity–how you choose what to test matters a lot more than how many tests you run.

What does strike me as ironic is that the moral here is a great counterpoint to the Varian’s colleagues’ arguments about the “unreasonable effectiveness of data“. Granted, it’s apples and oranges–Alon Halevy, Peter Norvig, and Fernando Pereira are talking about data scale, not user scale. Still, the same arguments apply. Sampling is sampling.

ps. Also check out Nick Carr’s commentary here.

Whether or not you can attend, I do encourage you to check out the UIE site. It’s got a lot of free, useful content, and Jared Spool is definitely someone worth following if you are interested in web usability.