Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

Googlebot GET /robots.txt HTTP/1.0" 404

Googlebot is the only bot getting 404

10:52 pm on Apr 2, 2004 (gmt 0)

New User

10+ Year Member

joined:Apr 29, 2003
votes: 0

I have validated the robot file numerous times and can't find anything wrong with it. Googlebot is the only bot getting the 404 error.

I have also noticed that Googlebot will request robots.txt than request a page and then request robots.txt again. It than proceeds to repeat this cycle over and over.

Any clues what is going on?


Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members