Googlebot requesting the same image file 20+ times a day

[shrirch]

I see googlebot visiting one of my domains which already has about 9000 pages indexed and requesting the same file day in and day out for the last week.

Any ideas on what could have gone wrong? The robots.txt file passes the validation test so I don't think thats it (besides its never been changed).

Here's a snippet from my log.

64.68.87.43 - - [22/Aug/2003:00:54:17 -0500] "GET /robots.txt HTTP/1.0" 200 820 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"
64.68.87.43 - - [22/Aug/2003:00:54:17 -0500] "GET /images/mint/dot.gif HTTP/1.0" 200 67 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"
64.68.82.7 - - [22/Aug/2003:02:03:34 -0500] "GET /robots.txt HTTP/1.0" 200 820 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"
64.68.82.7 - - [22/Aug/2003:02:03:40 -0500] "GET / HTTP/1.0" 200 34036 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"
64.68.86.79 - - [22/Aug/2003:02:11:47 -0500] "GET /robots.txt HTTP/1.0" 200 820 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"
64.68.86.79 - - [22/Aug/2003:02:11:47 -0500] "GET /images/mint/dot.gif HTTP/1.0" 200 67 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

[Dolemite]

/images/mint/dot.gif

Googlebot has halitosis? [rimshot]

Sorry, couldn't help it. ;)

[percentages]

Googlebot does the same thing to me.

I don't think you have anything wrong. I have several sites with 25,000+ pages indexed.....they don't change very often.....but Googlebot keeps coming back for them.

It grabs about 200 new pages per day from each site while visiting....so I don't have any real reason to complain. But I would prefer it to grab the new pages only and quit burning the data transfer for those that stay the same.

[RonPK]

Is the image related to the first hit on Google's SERP for the filename? In that case, make sure your CGI-script sends a correct header. Currently the page is being sent as text/plain. Maybe that causes your problem.

[shrirch]

Ron, Definately not anywhere on the first few pages of the SERPs if I search for dot.gif or other combinations.

Percentages, I'd be happy if it requested anything other than the gif file. Its just requesting that one file 20+ times a day.

[percentages]

So we have to live the fact that currently GoogleBot ain't the smartest creature on the planet I guess.

It costs me about $0.80 per day in data transfer.....and makes me about $1,800 per day in free SERPs....As a perfectionist I would prefer it got it right.....but I'm not exactly going to complain about the minor errors it makes right now :)

[shrirch]

One of these days I'll make that kinda money and stop complaining. :D