SIGIR 2009: Day 3, Industry Track: Analyst Panel

The morning sessions of the SIGIR 2009 Industry Track consisted of five individual presentations; the afternoon consisted of two panels. The requirement to synchronize with the research talks led to the allocation of 90 minutes for each panel–which was a bit more than I’d originally planned on (and this change, like many, occurred in the two weeks before the conference). James Allan, one of the SIGIR 2009 co-chairs, suggested that we add an academic responder to each of the panels to account for the additional time, and we went with that approach.

The first of the two afternoon panels consisted of industry analysts: Whit Andrews (Gartner), Sue Feldman (IDC), and Theresa Regli (CMS Watch). I moderated the panel–or, more accurately, attempted to moderate it. Marti Hearst served as the academic responder.

The panel opened with each of the three panelists making an opening statement, sharing their perspectives about the key business concerns and trends in the search industry. I asked them to talk about enterprise search in the broadest sense of the term–search applications that companies buy or build)–rather than in the narrow sense of no-frills intranet search.

It became immediately clear from their opening statements that the panelists had wildly different perspectives and styles. While I think the term “food fight” that I heard bandied around afterward is a bit of an exaggeration, they certainly engaged in a heated debate.

One topic that attracted particular controversy was how enterprise search applications should assign relevance to search results. Whit suggested that, in an enterprise setting, the main objective function is the profitability of the enterprise, and that relevance should essentially be money driven. Sue and Theresa disagreed sharply, mainly arguing that relevance should be user-controlled.

I’m probably oversimplifying their arguments, and in any case shouldn’t take sides in a debate among analysts! Still, Whit probably won’t be surprised that my sympathies generally lie with the users. That said, Whit is right that enterprise search companies sell to enterprises, not directly to customers, and those enterprises (like web search companies that sell to advertisers) may have interests that aren’t always aligned with those of users. At Endeca, we advise our customers on how to configure and communicate a relevance ranking strategy, but ultimately our customers make their own decisions. After all, it’s their site and their money.

And that leads to the other topic that caught my attention and came up during Marti’s responder session: the question of how analyst firms make money. All three of the panelists were open about how their employers make money, whether from enterprise buyers, vendors, or some combination thereof. My personal preference would be that analysts make money primarily from enterprise buyers–but of course I work for a frugal vendor. I’ve heard from a variety of sources that Endeca “doesn’t spend enough” on analyst services–or on corporate marketing in general. Since neither vendors nor analyst firms open up their books, I can only speculate. Fortunately, it’s clear the analysts on the panel not only have integrity, but also have strong enough views that they can’t probably couldn’t be swayed by money or pressure.

As organizer of the track, I indended for the panel to offer an audience of mostly academic types a chance to see people whose opinions influence tens (if not hundreds) of millions of dollars in purchasing decisions. I hope I accomplished that. I’m especially grateful to these highly billable analysts for freely sharing their time and ideas. Neither SIGIR nor I could possibly have afforded their market rates!

For other perspectives, take a look at Theresa’s blog post, “Know Your Relevance“, or Mary McKenna’s summary post.

By Daniel Tunkelang

High-Class Consultant.

2 replies on “SIGIR 2009: Day 3, Industry Track: Analyst Panel”

Sustainable companies should be about the best products and services possible, and from this, indirectly, money should flow.

Early and constant focus on “monetization” seems to be a short-term solution. You may get lucky and make it big quickly, but statistically, if you are too focused on money, and not enough on offering the best services and products, you will eventually go down the toilet.

When I was younger, the best and largest company in the world was easily General Motors. It was the most powerful corporation in the world. It was so powerful that they decided to make bad cars on purpose. It was so powerful that they could ignore foreign competitors. Yet, look at where they are now? Without government support, they’d be bankrupt.


The key is “too focused”. Web search engines and publishers rely on ads to make money, even though few would argue that the ads improve the products they provide. In enterprise settings, search engines have many reasons to exist. It’s hard to imagine circumstances where user satisfaction isn’t a major consideration, but that doesn’t mean it’s only about the user. Stores want to push excess inventory. Companies want to implement policies that their employees may resist. Media providers may be depending on partners or sponsors to provide revenue. Business is a negotiation. Screwing over your customer is, among other things, a bad business strategy. But so is ignoring your own interests.


Comments are closed.