Comments on: Google Similar Images: A Glitch?

By: ichimako

ichimako — Tue, 13 Oct 2009 17:47:54 +0000

How to submit my own image?

By: dinesh vadhia

dinesh vadhia — Sun, 26 Apr 2009 08:00:48 +0000

Sorry, but the first demo is now on my laptop only. We are working on rolling out a few live services after which we want to enhance the image service with a significantly larger data set. We can consider offering both query by text and by image.

By: Daniel Tunkelang

Daniel Tunkelang — Sat, 25 Apr 2009 19:58:27 +0000

Is that first demo still up? URL?

By: dinesh vadhia

dinesh vadhia — Sat, 25 Apr 2009 19:53:16 +0000

Both are possible. Indexed images will be faster to return results than non-indexed images. This can be mitigated with appropriate use of backend hardware. Starting with an image retrieved through (text) search isn’t an issue and in fact, the first demo we built showed this.

Note also that we support multiple images per query.

By: Daniel Tunkelang

Daniel Tunkelang — Sat, 25 Apr 2009 18:54:21 +0000

Sorry for losing a digit there! In any case, it would be nice if, as a user, I could supply my own image as input or, as in Google Similar Images, use an image I retrieve through search as a starting point for exploration. I imagine the latter is more plausible than the former, if you need to index all images in the collection.

By: dinesh vadhia

dinesh vadhia — Sat, 25 Apr 2009 18:07:40 +0000

Daniel, Thanks for checking out the image search demo at Xyggy. There are in fact 32,000 images to choose from.

Image features can range from the simple (eg. color and texture) to the complex (edge, sift, and so on). The feature vector could also include, if available and required, text tags. This means there is a lot of flexibility in the kinds of images services that can be built.

Given a query of image items, it is possible to cluster the results by specific features (eg. color, texture etc.).

For large-scale data sets, query results would likely contain many similar images. Depending on a users needs you want them to be able to cluster the results between very similar and not very similar. We can do this.

The (beta) Xyggy image search service is one example of “item search” where the query contains items (not keywords) and the results are items. An item could be a patent, a news article, a piece of music, a medical record and so on.

Wrt images, our next step is to use a much larger image set in the tens of millions if not larger. The ultimate goal is image search from any web page by simply dragging and dropping images into the search oval. To us this is how people do image search in real life and how we should do it in the digital world as it is natural and intuitive.

Dinesh

By: Daniel Tunkelang

Daniel Tunkelang — Sat, 25 Apr 2009 16:32:20 +0000

Dinesh, I’m not familiar enough with the state of the art to evaluate the Google Labs experiment relative to it. But I did take a look at Xyggy. It’s intriguing, but hard to evaluate without being able to pick from outside the 3,000 images in your set.

And of course I’d really like similarity browsing do clustering, with as much transparency as possible about what holds together a cluster. But I realize this may be hard to do for visual similarity.

By: dinesh vadhia

dinesh vadhia — Sat, 25 Apr 2009 14:12:40 +0000

Got to this story a bit late. It would be pretty accurate to say that the revised Google Lab’s image search prototype is not a huge leap from what went before. Techcrunch has a couple of posts to confirm this.

Have a look at our (beta) image search at:

http://www.xyggy.com/image.php

Drag ONE or MORE images from the left-hand set of images into the search oval. The results will appear on the right.

Add additional images to the search by dragging and dropping more images. Remove images from the query by clicking on the image. Click the search oval to clear the search and start a new one.

Isn’t this what image search should be all about ie. dealing with an image as an item? Imagine being on any web page and dragging and dropping images from that page into the search oval to find similar images.

Dinesh

By: Daniel Tunkelang

Daniel Tunkelang — Thu, 23 Apr 2009 19:50:24 +0000

That’s pretty funny!

By: Eric Rumsey

Eric Rumsey — Thu, 23 Apr 2009 19:47:05 +0000

Here’s a funny case of the word associated overwhelming the image — A search for george retrieves a non-famous example of a person with that name — But clicking the accompanying “similar images” pulls up mostly pictures of GEORGE BUSH, instead of finding faces like the non-famous George.

By: jeremy

jeremy — Thu, 23 Apr 2009 18:42:09 +0000

But transparency is not “googly”. No matter what Jeff Jarvis says.

By: Daniel Tunkelang

Daniel Tunkelang — Thu, 23 Apr 2009 12:29:44 +0000

Indeed, I see nothing wrong with combining evidence. I just would have liked a clearer explanation upfront. Maybe there was one any I missed it. Not to beat a dead horse, but transparency goes a long way to build trust.

By: Max L. Wilson

Max L. Wilson — Thu, 23 Apr 2009 10:30:11 +0000

cueflik is great. Id love to see that sort of interaction on google image search. Again on your latest example, they are all mostly white background, circular, yellow and black. I guess they are all images under the search term of 'kiwi' though. You might notice as well that live search's similar image search has a similar but different result.

By: Daniel Tunkelang

Daniel Tunkelang — Thu, 23 Apr 2009 02:08:59 +0000

Here's another fun example where text overwhelms visual similarity. Again, it's consistent with my ESP Game intuition--basically, that humans would think the same word to as the best label for unrelated images.

By: Daniel Tunkelang

Daniel Tunkelang — Wed, 22 Apr 2009 17:32:28 +0000

Interesting. For those without ACM subscriptions:

http://www.cs.washington.edu/homes/jfogarty/publications/chi2008-cueflik.pdf

By: Vladimir

Vladimir — Wed, 22 Apr 2009 17:15:48 +0000

This reminds me of this:
http://portal.acm.org/citation.cfm?id=1357054.1357061

By: Daniel Tunkelang

Daniel Tunkelang — Wed, 22 Apr 2009 13:24:13 +0000

I’m guessing they are at least using image labeling from their version of the ESP Game (this is a gut feeling from trying a bunch of queries), and it just so happens that “blackberry” is such a high-information term that it tricked them.

By: Max L. Wilson

Max L. Wilson — Wed, 22 Apr 2009 08:39:46 +0000

Well hold on, lets look at the pictures for a second. they are all blackish blobs on a white background. they are all covered in bobbles (the buttons the cell phone ARE bobbly). even the icons on the screens add to a bobbly effect. and the variation in colour on the bobble buttons do look a little like the light reflection on the fruit bobbles. most of the phones are at the same slant as the fruit. I'd say that they look more like that piece of fruit than the other images i get for the search 'blackberry fruit'. maybe it is just by image alone?

By: jeremy

jeremy — Tue, 21 Apr 2009 19:07:26 +0000

Pay no attention to the man behind the curtain

By: Daniel Tunkelang

Daniel Tunkelang — Tue, 21 Apr 2009 17:14:54 +0000

That makes sense. I'm just surprised because, in a TechCrunch interview, Google engineering director Radhika Malpani said it was based on "analyzing the comment of the image". I guess that doesn't rule out use of text--and, at the end of the interview, she does say that Google generally approaches image search by all means necessary, including the use of textual metadata. Still, I feel a bit bait-and-switched here, and I think the general perception was that this new image search application was based solely on visual content.

By: ogrisel

ogrisel — Tue, 21 Apr 2009 17:04:17 +0000

It probably uses some kind of semantic hash of the pictures along with the fulltext match of the keywords in the query (in the html page context, image filename or text of the links pointing to the image from other pages):

Semantic hashing using Deep Belief Networks:

http://www.cs.toronto.edu/~rsalakhu/papers/semantic_final.pdf

Semantic Hashing using stacked autoencoders:

http://www.cs.nyu.edu/~ranzato/publications/ranzato-icml08.pdf

Semantic Hashing using Spectral Hashing:

http://people.csail.mit.edu/torralba/publications/spectralhashing.pdf

By: Jon

Jon — Tue, 21 Apr 2009 16:37:05 +0000

Great find — a fascinating failure.