Categories
General

Banging on Bing: A Bummer

So, Bing is out early. Yes, an early release from Microsoft! And it’s snappy, attractive, and offers decent quality. If I needed to use Bing as my main search engine for the web (yes, readers, imagine a world without Google and Yahoo as search options). I’d survive.

But I can’t say I’d be thrilled. I’ve only had a short time to play with Bing, but I’m not overwhelmed. In fact, I’m quite disappointed, given their big talk about deliver a “decision engine“, I expected at least a little bit of innovation in the user experience. No such luck, The focus is still on the ranked list, and their ranking is, at least to my taste, perceptibly inferior to Google’s. I could live with that small difference if the interface offered real opportunities for interaction. But there isn’t anything new there. You can refine by result type (Web, Images, Videos, Shopping, News, Maps, Local, Travel), but search engines have been doing that for years.

The only novelty is “xRank”, which lets you “see who and what everyone’s searching for most”. It’s intriguing, but it seems half-baked, and I suspect that others are further ahead on crowd-sourcing relevance through the social stream.

I take no pleasure in throwing cold water on the queue of challengers that attempt to provide competition for Google in web search. Perhaps Bing is truly in beta, and will prove itself a more formidable challenger in the future. But it’s surely not there now.

By Daniel Tunkelang

High-Class Consultant.

23 replies on “Banging on Bing: A Bummer”

I’ve only done a couple of queries now. And as any good IR researcher knows, a couple of queries does not a proper evaluation make. But I haven’t found the results to be inferior at all. They seems very much on par. I was impressed by the query [collaborative exploratory search]. I really liked the results for [dave alvin concert]. My query

was totally appropriate, though the related searches were, well, wrong 🙂 I then tried [sigir industry track], and that looked great — even has a link to Daniel Tunkelang at rank 6. (Same link on Google isn’t found until result #23.)

I went to the Maker Faire this past weekend, and my [maker faire] query on bing looks essentially equivalent to Google. Complete with the out-of-date 2007 and 2008 links. (i.e. both Google and Bing make the same mistakes — Google is not better.)

Yes, I know all the arguments about “it can’t be equal to Google — it has to be better, or users won’t switch”. But right now, I do find it equal to Google, at least with these half-dozen queries. And while I find the exploratory aspect of Bing sorely lacking at the moment, I’m encouraged by the fact that they’re at least willing to pay some lip service to the idea. Google is not. So that makes me hopeful about where Bing might go next — it’s as good as Google now, but with a better attitude. Does that not count for something? 🙂

Like

I agree with Jeremy, but I actually think their (tiny) step towards exploratory search is significant. The push towards enabling more efficient query reformulations will make a big impact on some queries, particularly ambiguous ones. It is a simple but effective way of opening up the dialog between the user & the system and taking a step away from the single ad-hoc query-click feedback loop. Yahoo does this a little, google somewhat less. But, initially anyway, I think the Bing implementation and presentation is superior.

Like

I have tried it a few times and its ok, to be honest it seems that all search engines are about the same, each one has its own things I like but for the most part I use google and yahoo. I will though try this one out for a while before I say one way or another as to if I like it or not.

Like

Ooh, I just discovered one thing that I really like: The “advanced” search options are inline. On Yahoo and Google, the “advanced search” link takes me to a separate page. To me, this has the psychological effect of taking me away from the search that I am doing.

On Bing, the advanced search options expand inline, and preserve the ranked list that I am already seeing.

I can see that being extremely useful to on-the-fly results refactoring, in the future. Already, Bing looks poised to offer more interactivity.

Like

I’ll cede for the sake of argument that relevance is comparable–in any case, I don’t think the difference is large enough be a deciding factor for anyone. And status quo obviously favors Google.

So the real question is whether Bing is a step forward when it comes to interaction. I didn’t experience that, but maybe I haven’t played with it enough yet. The “related searches” struck me as no better than those offered by Ask. The search previews for results without clicking off the page (which I just noticed) are cute, but I’m not sure they add much over the snippets. And the integrated travel site is again nice, but I’m not persuaded that I’d use it instead of Kayak.

I feel like I’m a wet blanket–maybe I’m just disappointed because I expected more. Jeremy, Jon: do either of you have examples of tasks that you feel Bing helps you complete more effectively than Google or the default sites people would use for those tasks? That’s what I’m searching for (sorry, couldn’t resist), and not finding.

Like

By Post Author

Jeremy, Jon: do either of you have examples of tasks that you feel Bing helps you complete more effectively than Google or the default sites people would use for those tasks?

If you’re talking about the difference between process and result, then no, at this moment Bing doesn’t feel like it significantly offers any different (for better or worse) process than Y, G, Ask, etc. As I said, I do like that the advanced search options are inline, and that feels like the beginning of a new and better process.

But currently, no, it’s not that different.

I will say again, though, that I think it’s a big deal to be at a starting point that is as least as good as Google, and then add to that the willingness and attitude to at least want to start playing with more interactivity.

I’m definitely adding Bing to my rotation of search engines. (I don’t use only one.)

Like

I do think there are a few tasks that Bing is better at, but these aren’t necessarily new, just unified into a single point of access. For example, Live has been better than Google at local searches for a while. Compare:

Bing: http://www.bing.com/search?q=pizza+15218&go=&form=QBLH

Google: http://www.google.com/search?client=safari&rls=en-us&q=pizza+15218&ie=UTF-8&oe=UTF-8

First of all, the map display on Bing is much easier to read, with some intelligence behind marker placement and much less clutter.

Then, clicking on a listing from each. Bing has more sources for reviews, and I like the layout a little better (although neither are great). The one-click directions at Bing from several different starting points are also extremely useful — I’ve used this feature of Live maps several times when I’m sending directions out to multiple people or I don’t particularly care to enter my current location as a starting point.

Another superior task on Bing is airfare search. Compare

Bing: http://www.bing.com/search?q=LGA+to+SEA&go=&form=QBRE

Google: http://www.google.com/search?client=safari&rls=en-us&q=LGA+to+SEA&ie=UTF-8&oe=UTF-8

Sure, google doesn’t have a property like farecast to integrate into their results, but even the query suggestion is totally off.

Finally, to illustrate the power of query suggestions for ambiguous searches, consider [python]:

Bing: http://www.bing.com/search?q=python&go=&form=QBRE

Google: http://www.google.com/search?hl=en&client=safari&rls=en-us&q=python&aq=f&oq=&aqi=g10

Both engines’ first page of results are mostly dominated by the programming language, with the exception of two pages, one on Monty python and one Wikipedia page referring to snakes. On Google these three concepts aren’t covered until position 8, but on Bing they’re covered at position 4 — all above the fold.

The real differentiator is in the query reformulations (and related organization of results), which Bing shows right at the top, and Google has at the very bottom of the page. This makes a big difference — it gives some idea of what else is in the index about this topic, and gives the user an easy way to redirect the search engine into something more appropriate for their information need.

If someone doesn’t know about the programming language, and is really interested in snakes, I think Bing wins.

Like

Point taken about Bing’s decision to put related searches up top (like Ask), while Google pushes them below the fold–not a technical differentiator, but I suppose it is a statement.

As for the travel, I’d say that Google isn’t really a contender, but that the real question is whether people will use Bing instead of going straight to Kayak or some other travel site. I doubt anyone will switch to Bing for travel (Kayak is at least comparable), but perhaps it will help retain Bing users.

I’ll add Bing to my rotation as well–maybe it will take some time for me to fully appreciate it. I don’t think it’s embarrassing–just that parity isn’t enough to make a dent in Google’s dominant market share.

Like

By Post Author

its pretty! heh.

My favourite part of the interaction, though, is the little popups you get from the dots on the right hand side. with additional snippets of information.

on the related searches to the side vs below the fold. talking to daniel russell, he indicated that they have done extensive multi-server million user studies on that sort of thing.. and at the bottom gets the highest click-thru. maybe its just us search enthusiasts that think it should be above the fold. perhaps the majority of users dont care.

it always amazes me at the default pitch-level that companies like orange and virginmedia take when you speak to them on the phone. they assume im a techno-idiot. i have to regularly remind myself to the massive extent that i (we here) dont represent the majority for things like this.

dont get me wrong, just because google gets a higher clickthru down there, doesnt mean it would be better in the long run for it to be above the fold.

Like

talking to daniel russell, he indicated that they have done extensive multi-server million user studies on that sort of thing.. and at the bottom gets the highest click-thru. …dont get me wrong, just because google gets a higher clickthru down there, doesnt mean it would be better in the long run for it to be above the fold.

This is what I was trying to say the other day, when I wrote a couple of paragraphs about machine learning and log analysis:

http://irgupf.com/2009/05/26/machine-learning-and-search-action-or-reaction/

I would really like to understand how it is that the believe that more clicks = better interface came into being. I, for one, often find that just having the peripheral awareness of available side-topics, even if I never use/click them, is quite useful for helping me understand how well the search engine understood what I was (and wasn’t) looking for.

I’ll never get that peripheral awareness if I have to scroll down to the bottom for every single query that I do.

So I don’t care if there are multi-million user studies.. if all we know from those studies is where the click were. What I need to know is *why* it is we (or Google or Yahoo or whoever) believes that a click is better than no click. Or are they simply making that assumption, and have no real million-user data on how useful (or not) peripheral awareness data is on the information seeking task?

Like

jeremy – actually i mentioned your views in my talk at the sensemaking workshop at CHI09 (referencing one of your previous comments on location of facets in different interfaces in thenoisychannel blog). people found it interesting, especially Dagobert at UMD.

Like

Oh, cool 🙂

Were there any thoughts/comments on what the answer might be?

(BTW, please pay no attention to the half dozen grammatical mistakes in my previous comment.. it’s early. I really can conjugate verbs, and know the difference between verbs and nouns.. belief me!) 😉

Like

unfortunately there wasnt any specific discussion on the topic. i did see dan russell making lots of notes heh, but then he was doing that anyway throughout, since he was running it. im stopping in DC on the way to JCDL, but dagobert will already be at JCDL while im there. Im sure i’ll chat more with him, and others hopefully, at JCDL. maybe a BOF is in order.

interestingly ed cuttrell, i think it was, asked a question on a paper during the main CHI09 proceedings. the paper was about tag cloud structuring. the question, well a comment actually, was that tag clouds were more for contextual sensemaking, rather than for aiding retrieval. it got a fairly blank response, given the paper was about aiding retrieval. but it appears to be a topic in the minds of many people.

the only empirical stuff i know about it so far is the paper i sent you before about backward highlighting, where we saw people retaining the labels of items in facets without actually clicking on them. whether that == the larger process of sensemaking was not studied directly. i should have a 3rd year undergrad looking at this sort of topic tho from sept. whether, at that stage in his education, he’ll pull up decent work I dont know. I’m gonna shoot for a higher-degree student next round too.

Like

I think Bing is doing a good job being a beta site. I like the exploratory search features that they have. They are limited, of course, but I’m sure they have a number of things in the pipeline that they plan to release later.

I personally don’t care about Bing vs. Google. I want alternatives to search the web and I want *different* ones.

Like

While I suspect that the readership here is a somewhat unrepresentative of the web searching population, I recognize that I am in a minority in my less than enthusiastic reaction. I hope that bodes well for Bing, as I would like to see it succeed. And perhaps even I’ll warm up to it in the next weeks.

Like

By Post Author

It seems to me like it does the job ok, but like it’s been said many times already, doesn’t it need to be a step change in order to increase its user base. It seems to me that it is just delivering a similar result using a slightly different algorithm to deliver it. Where is the sizzle?

Like

I’m inclined to agree. I’m giving it a 2-week trial, using it exclusively. The test for me is whether, at the end, I’ll want to continue using it for a significant fraction of my information seeking needs.

Like

By Post Author

Comments are closed.