Froogle and Googlebot-Image in your robots.txt

Are you going to allow?

[sun818]

A lack of images hurts the user experience, and makes the user less likely to click on your products. Are you going to allow Googlebot-Image? Up to now, I have disallowed the bot. But as I started a daily feed to Froogle, I am allowing Googlebot-Image to crawl my web store.

[pmac]

If you are using the data feed, Froogle already has the image url's to your products.

The image bot is not tied to Froogle but to Google's image search.

[sun818]

Hmmm, in a "fix it" e-mail that I received from Froogle (September 2003), they asked that I remove this from robots.txt:

User-agent: Googlebot-Image
Disallow: /

So, is Google just pulling my leg here? I prefer disallowing Googlebot-Image and still have my images included in Froogle.

[Chndru]

Do they have a seperate bot for feeds, like MediaBot (for Adsense)? If so, What's the UA?

[Robino]

In the FAQ section of their Advanced Feed Instructions, it says to make sure that robots.txt isn't prohibitiong their image crawler from grabbing images.

So, to be safe I would allow it.

-I guess you could ask Froogle directly.

[pmac]

>In the FAQ section of their Advanced Feed Instructions, it says to make sure that robots.txt isn't prohibitiong their image crawler from grabbing images. <

Got a link Robino?

From the merchant information page:

Merchants who meet Froogle's criteria and want to submit a data feed must allow Google to crawl their websites by removing any robots.txt files that disallow our Googlebot spider.

>I guess you could ask Froogle directly.<

I did, I'll let you know when I hear back.

[edited by: pmac at 8:31 pm (utc) on Dec. 8, 2003]

[Robino]

Sorry, no link. I got it from a .pdf attached to the email they sent me when I set up the feed.

[sun818]

If you look at the Froogle results page, each image is a 90x90 thumbnail of the original image. As far as I can tell, Froogle is hosting these thumbnails on their servers.

[pmac]

Confirmed. Blocking the image-bot via robots txt will result in your images not being shown in Froogle.

[ILLstyle]

I might not allow the image bot. So many sites steel my work as is. Don't want to make it easier.