Forum Moderators: open

Message Too Old, No Replies

Googlebot-Image/1.0

Off the deep-end

         

wilderness

6:54 pm on Aug 9, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



66.249.66.79 - - [09/Aug/2006:10:30:46 -0700] "GET /myfolder/myimgae.jpg HTTP/1.1" 403 - "-" "Googlebot-Image/1.0"

Not a massive crawl of images,

however NOT compling with robots text for Googlebot-Image
AND

even going into folders that are denied in robots.text

wilderness

4:15 pm on Aug 10, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Update:

Early yesterday the bot was sparsely grabbing images.

Last night it began a more extensive crawl.

I've contacted google and awaiting reply.

wilderness

10:40 pm on Aug 11, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



The good news is that I recieved a reply from Google.
It appears to be automated.

They use a special UA for this an accessed both websites inquired with that UA.
The requests have ceased at one site, however contine at the other. (eating 403's).

Possibly the bot is stuck in loop because it cannot read robots.txt (I'm using SetEnvIf).
The reply requested notification if spidering continues and I have notified Google and added the additional log lines, after the contact time.

In any event I'm not about to make changes on Google's behalf until the spidering attempts cease as I have no desire for my images in that SE.

Don