Forum Moderators: open

Message Too Old, No Replies

Strange Robot Request

Strange Robot Request

         

Updevision

5:42 pm on Oct 17, 2003 (gmt 0)

10+ Year Member



Yesterday GoogleBot 2.1 came round as it does every day and it requested my robots.txt and all is good. Then it normally requests my default.asp but today it requested some fugazzi IP address that has no relation to my site. Thus incuring a 404 error.

Anybody else seen this activity?

Coincedently today my site is missing from all keywords.

claus

11:10 pm on Oct 17, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>> it requested some fugazzi IP address that has no relation to my site. Thus incuring a 404 error.

If a 404 is recorded in your log files, then your server has served the Gbot a 404 message when that IP address was requested. This implies that the request containing the IP was related to your site. If it was not, then the 404 would not be served from your server or recorded in your log files.

Could you provide a little more details, eg. post the error line, replacing the IP with 127.0.0.0 and the domain with example.com?

dirkz

6:19 pm on Oct 18, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Could you post the log entry (with your domain specifics erased please)?

JoeyBall

7:53 pm on Oct 18, 2003 (gmt 0)

10+ Year Member



I am seeing about 40 robots.txt requests a day. dont know why so many

dmorison

11:24 pm on Oct 18, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



G'bot can't get enough robots.txt at the moment. Either something's broken or...