Comments on: Q&A with Amit Singhal

By: SIGIR 2009: Day 3, Industry Track: Matt Cutts | The Noisy Channel

SIGIR 2009: Day 3, Industry Track: Matt Cutts | The Noisy Channel — Wed, 29 Jul 2009 04:06:41 +0000

[...] I’d heard Amit speak before: he delivered one of the keynotes at ECIR 2008 (and inspired one of my first blog posts!). So I decided to aim for Matt Cutts, despite having no way to contact him (the head of [...]

By: Age-Old Questions about BWBX « Network(ed)News

Age-Old Questions about BWBX « Network(ed)News — Mon, 26 Jan 2009 22:11:03 +0000

[...] So if we’re still getting it wrong, why? And if we’re getting it right, why can’t we be more transparent about it? We know how pagerank is the beating heart of google’s effort to out-engineer spam, and some argue that’s not even enough. [...]

By: Disincenting Spam | The Noisy Channel

Disincenting Spam | The Noisy Channel — Mon, 20 Oct 2008 18:55:12 +0000

[...] Greg’s argument reminds me of one of the first posts I wrote on this blog. I was criticizing Google’s approach of keeping its relevance approach [...]

By: Aaswath

Aaswath — Tue, 15 Apr 2008 04:50:00 +0000

This was also a topic of discussion while I was at MS — we did in fact publish some papers on spam and adversarial IR based on collaborations with MSR, but we were well aware that blackhats/spammers out there were reading these papers. When spam proves to be an existential threat to the relevance of search engines, things like transparency are sometimes hard to justify.

However, Google (and Yahoo and Live) have done a good job of providing at least some transparency to siteowners with their webmaster tools (ie: are you being hit with spam filters, etc). These didn’t exist until relatively recently, but I’d say everyone’s now come around to seeing the positive benefits of engaging with legitimate siteowners and offering them information in return for their registering themselves, their sites and sitemaps in a formal way. This is a sea-change from a previously adversarial relationship to all siteowners to one that tries to engage with normal/”good” sites.

Some other thoughts:
1) Search engines are always looking for proxies for relevance (like links) and aspects of an attention economics-approach to this are in play already. Unless I’m misunderstanding however, this too is currently prone to be gamed as well — for example, botnets can be frighteningly effective.

2) Can there ever be a truly (or even quasi-) objective definition of relevance across the web? Obviously engines have ways of measuring their effectiveness, but those definitions are ultimately subjective ones. Would an “open” standard of relevance achieve this?

I’m inclined to think web relevance will remain subjective by virtue of the nature of the dataset, and because the stakes are high, monetarily speaking for all involved. I’ll also say given the amount of money involved, spammers will try very, very hard to game whatever system is put out there. There’s a few billion too many involved, and thus, unsurprisingly, a large number of smart folks working on spamming. Perhaps I have a dimmer view of human nature after dealing with this for a while, but I’m skeptical we can ever end the arms race with spammers unless the monetary incentive decreases in some way

Another thought is that an ancillary beneficiary to spam succeeding is often the search engine’s ad wing itself in terms of fees. This isn’t to suggest any conspiracy, just an example of the weird dynamics often at play vis-a-vis spam.

By: Mark

Mark — Mon, 14 Apr 2008 12:35:00 +0000

I might be wrong, but if we built a ranking system based on user visitation data, won’t that be gamed as well? – instead of “link farms”, won’t we get “traffic farms” that artificially inflate the user visitation values of some sites by directing extra (fake) traffic to them?

By: fd

fd — Sat, 12 Apr 2008 19:11:00 +0000

Back in the day PageRank succeeded because it was a baseline approximation for user data. With richer user visitation data (through e.g. toolbars) PageRank becomes moot. See here.

By: Mark

Mark — Fri, 11 Apr 2008 21:30:00 +0000

Well back in the day Page Rank succeeded because it worked better than existing approaches, as well as being harder to game.

But yes the “attention economics” approach would be interesting, if there were some way to measure that, that did not incent people to mount DoS attacks to simulate attention 8).

By: Daniel Tunkelang

Daniel Tunkelang — Fri, 11 Apr 2008 19:07:00 +0000

Well, the initial success of PageRank, at least as far as I can tell, came from it being harder to game than the IR measures that other search engines were using at the time. Since then, of course, it's been an arms race.

I'd really love to see the relevance arms race replace with a principled approach based on attention economics.

By: Mark Watkins

Mark Watkins — Thu, 10 Apr 2008 18:35:00 +0000

From the "loyal opposition" (the author of
How Pagerank wrecked the web founded a new search company) - essentially arguing that Google's ranking algorithms (effective, but, ultimately, arbitrary in some sense) are in fact the origin of the arms race. There must be some "less gamable" ranking system out there....