Googlebot causing error

adds robots.txt to index.html

[keyplyr]

This is an ongoing problem. Googlebot adds the index.html onto [my_site.com,...] then adds robots.txt causing this error:

[Fri Jun 13 02:31:15 2003] [error] [client 64.68.82.45] File does not exist: /www/m/my_account/htdocs/index.htmlrobots.txt

The robots.txt file resides in the root directory (htdocs) along with the index.html, and nowhere on the website do I have a link to robots.txt that could be causing this error. Also, nowhere on the site do I call the index.html.

Any ideas?

BTW - My site gets crawled deep and often with no problems. And at least one of the other Googlebots reads the robots.txt correctly because my disallows are being respected.

[nancyb]

had a similar visit

64.68.82.31 - - [09/Jun/2003:03:51:09 -0500] "GET /folder/m HTTP/1.0" 404 7540 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

[Visit Thailand]

I get this a lot on some of the more important pages. Don't know if it is gb though.

http:*//www.mydomain.com/thispage.htm/Robots.txt

Can't remember if the slash is there at the end will have to wait till it happens again to check.