Googlebot and my robots.txt

[web_young]

I was looking through my log files and noticed on the 3rd and 4th of August the Googlebot accessed my robots.txt file between 2000 - 3000 times. Has anyone seen this happen before? Does anyone know why this would happen? It accessed very few files other than the robots.txt.

[Brett_Tabke]

Never seen that before. Are you sure it was gb? Verified the correct agent and ip?

[web_young]

I'm not 100% sure it was googlebot but here's what it says in my log file:

crawlerXX.googlebot.com - - [03/Aug/2003:21:02:25 -0600] "GET /robots.txt HTTP/1.0" 200 315 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

I was really excited to have so many hits from googlebot until I realized it was 60% robots.txt.

[bcolflesh]

Might be related to this thread:

[webmasterworld.com...]

[hthota]

what is the best way to deploy a robots.txt file? i made a robots.txt file, put it in the root directory... and in the robots.txt file, I said exclude certain URLs.. but not seen google yet do it.. though it's been around a week since I did it.. anything wrong I'm doing?

thanks for any help in this regard.. :)