Forum Moderators: open

Message Too Old, No Replies

Googlebot-Images

robots.txt disallow?

         

keyplyr

9:40 am on Apr 2, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



In the robots.txt: if I do not disallow Googlebot-Images but do disallow * in /images/ can the image finder at Google still post images found from insite links?

Example: <a href="mydomain.com/images/pic.jpg">click here</a>

mbauser2

11:05 am on Apr 2, 2002 (gmt 0)

10+ Year Member



No. robots.txt is an absolute block, regardless of how a robot finds a file.

wilderness

1:29 am on Apr 3, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I've had Google Images in my robots for some time.
They do honor that.

I have been steadly renaming all my images and moving to a folder which is also contained in the robots txt as "disallow all."

It has not stopped these new image names from being indexed by Google.

I have also implemented htaccess measures to prevent (not the correct term as there are exceptions) deep linking.
Google image file links are returned as 403 errors in my logs.

In summary Google images is not indexing my images and yet they are still appearing in Google Images :-(

TallTroll

10:16 am on Apr 3, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>> Google images is not indexing my images and yet they are still appearing in Google Images

Hmmm, so I guess standard Googlebot can scoop them and pass them over to the Images d/base?

Why a separate robot/UA for GG Images then? Seems like duplication of effort. Unless its designed to give less eagle-eyed webmasters the impression that they can keep GG out.

The other possibility is that someone else is ripping the images and it is the copies that are indexed. Might check your referrals for those images, and the site that GG Images found them on, see if anything screwy is happening

wilderness

10:39 am on Apr 3, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



<snip>see if anything screwy is happening>

Any image link from a Google image cache result in a 403 :-)
Per my htaccess.
They have however lessend as I rename image files and moved over to the folder which is disallowed.
IMO search engines should read html content and nothing else with images excluded.

<snip>guess standard Googlebot can scoop them and pass them over to the Images d/base?>

My point exactly.

keyplyr

12:56 am on Apr 4, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Well my question was: if I DO NOT disallow Googlebot-Images
but DO disallow all from the /images/ directory, will the Google Image Finder still list them.

I have disallowd all from the /images/ directory for about a month now, and Google Images keeps sending referrals so I guess the answer is "yes, Google is finding the images from my HTML"