Forum Moderators: open

Message Too Old, No Replies

Blank UA and referer with googlebot IPs

all asking for image files

         

nancyb

9:44 pm on Nov 2, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I've been getting a lot of requests for image files with both blank UA and referrer but with googlebot IPs.

I've searched here and on G (site:WebmasterWorld) and can't find anything specific to this. I did see somewhere that a spoofed IP wouldn't get the response from the server so I don't understand unless there are a few googlebots that have a glitch and aren't sending their UA? This has been happening since mid September with these IPs:

216.239.33.5 proxy.google.com
64.68.87.21 crawler7.googlebot.com
64.68.87.23 crawler7.googlebot.com
64.68.87.29 crawler7.googlebot.com
64.68.87.48 crawler8.googlebot.com
64.68.87.50 crawler8.googlebot.com
64.68.87.51 crawler8.googlebot.com
64.68.87.59 crawler8.googlebot.com
64.68.87.61 crawler9.googlebot.com

I am also getting a lot of Googlebot-Image/ visits for the first time in almost 1 1/2 years. These appear to be related to Froogle because I am starting to see some of these images show up there. (I have never sent a feed to froogle - yet.)

any ideas on the blank UAs? Do you think google would answer if I ask them about these?

Thanks :)

BlueSky

6:56 am on Nov 3, 2003 (gmt 0)

10+ Year Member



Send a copy of the respective raw log entries with the empty UA's plus your URL to googlebot@google.com and ask them why their bot isn't identifying itself. They'll be able to tell you if it's really theirs. If it is, they'll fix the problem. Is the bot asking for your robots.txt and obeying it?

Googlebot-Image seems to be hitting a number of folks right now. He's been crawling my site too.